Granted Patent B2

US 12,650,889 · App. 18/939,189 · Granted Jun 9, 2026

Inventors: Wenjie Li (Beijing, CN); Yaohua Li (Beijing, CN); Wenchi Zhang (Beijing, CN); Chao Cui (Beijing, CN); Hongbin Liu (Beijing, CN)

Assignee: Beijing Volcano Engine Technology Co., Ltd.

G06F11/008

Abstract

Claims (92)

1 . A method for assessing a risk caused by improper use of a cluster data resource, applied to a scenario of resource allocation of a department, wherein the method comprises:

dividing, according to an impact degree of a risk caused by improper use of a data resource on a service, risk levels in a use process of the cluster data resource into three levels: a low-risk level, a high-risk level, and an accident level;

allocating the cluster data resource to the department;

obtaining an assessment dimension for assessing the risk of the cluster data resource, wherein the cluster data resource comprises a data storage resource and a data computation resource, and the assessment dimension comprises a storage assessment dimension and a computation assessment dimension;

obtaining at least one high-risk diagnosis indicator in the storage assessment dimension and at least one high-risk diagnosis indicator in the computation assessment dimension, wherein the high-risk diagnosis indicator in the storage assessment dimension, indicated for the storage assessment dimension, comprises an indicator for representing that a data resource usage amount of the data storage resource exceeds a first data storage resource threshold and/or an indicator for representing that the data storage resource does not comply with a first data storage resource usage rule, and the high-risk diagnosis indicator in the computation assessment dimension, indicated for the computation assessment dimension, comprises an indicator for representing that a data resource usage amount of the data computation resource exceeds a first data computation resource threshold;

counting a high-risk indicator trigger quantity in the storage assessment dimension and a high-risk indicator trigger quantity in the computation assessment dimension, wherein the high-risk indicator trigger quantity in the storage assessment dimension is an amount of high-risk diagnosis indicators in the storage assessment dimension that are satisfied in a use process of the data storage resource, and the high-risk indicator trigger quantity in the computation assessment dimension is an amount of high-risk diagnosis indicators in the computation assessment dimension that are satisfied in a use process of the data computation resource; and

assessing a risk caused by improper use of the data storage resource based on the high-risk indicator trigger quantity in the storage assessment dimension, and assessing a risk caused by improper use of the data computation resource based on the high-risk indicator trigger quantity in the computation assessment dimension.

2 . The method according to claim 1 , wherein counting the high-risk indicator trigger quantity in the storage assessment dimension and the high-risk indicator trigger quantity in the computation assessment dimension comprises:

determining indicator values corresponding to high-risk diagnosis indicators in the storage assessment dimension based on a usage situation of the data storage resource, and determining the high-risk indicator trigger quantity in the storage assessment dimension based on the indicator values corresponding to the high-risk diagnosis indicators in the storage assessment dimension; and

determining indicator values corresponding to high-risk diagnosis indicators in the computation assessment dimension based on a usage situation of the data computation resource, and determining the high-risk indicator trigger quantity in the computation assessment dimension based on the indicator values corresponding to the high-risk diagnosis indicators in the computation assessment dimension.

3 . The method according to claim 2 , wherein the high-risk diagnosis indicator in the storage assessment dimension comprises one or more of following:

a remaining storage usage duration being less than or equal to a first number of days, a storage amount increment day-on-day ratio being greater than a first proportion, a proportion of small files being greater than a first ratio, a proportion of disk bad blocks being greater than a first bad block proportion, a write forbidding trigger duration being greater than a third duration, a storage usage rate exceeding a first usage rate, or a set lifetime being less than a recommended value; and

the high-risk diagnosis indicator in the computation assessment dimension comprises one or more of following:

a queue blocking duration exceeding a first duration, a scheduling task failure rate being greater than a first failure rate, a scheduling task running duration being greater than a first running duration, a queue usage computation time rate exceeding a first computation time rate, or a service level agreement task breach.

4 . The method according to claim 3 , further comprising:

obtaining at least one low-risk diagnosis indicator in the storage assessment dimension and at least one low-risk diagnosis indicator in the computation assessment dimension, wherein the low-risk diagnosis indicator in the storage assessment dimension comprises an indicator for representing that the data resource usage amount of the data storage resource exceeds a second data storage resource threshold and/or an indicator for representing that the data storage resource does not comply with a second data storage resource usage rule, the low-risk diagnosis indicator in the computation assessment dimension comprises an indicator for representing that the data resource usage amount of the data computation resource exceeds a second data computation resource threshold, the second data storage resource threshold is less than the first data storage resource threshold, and the second data computation resource threshold is less than the first data computation resource threshold;

obtaining a low-risk indicator trigger quantity in the storage assessment dimension and a low-risk indicator trigger quantity in the computation assessment dimension, wherein the low-risk indicator trigger quantity in the storage assessment dimension is an amount of low-risk diagnosis indicators in the storage assessment dimension that are satisfied in the use process of the data storage resource, and the low-risk indicator trigger quantity in the computation assessment dimension is an amount of low-risk diagnosis indicators in the computation assessment dimension that are satisfied in the use process of the data computation resource; and

assessing the risk caused by improper use of the data storage resource based on the low-risk indicator trigger quantity in the storage assessment dimension, and assessing the risk caused by improper use of the data computation resource based on the low-risk indicator trigger quantity in the computation assessment dimension.

5 . The method according to claim 4 , wherein obtaining the low-risk indicator trigger quantity in the storage assessment dimension and the low-risk indicator trigger quantity in the computation assessment dimension comprises:

determining indicator values corresponding to low-risk diagnosis indicators in the storage assessment dimension based on the usage situation of the data storage resource, and determining the low-risk indicator trigger quantity in the storage assessment dimension based on the indicator values corresponding to the low-risk diagnosis indicators in the storage assessment dimension; and

determining indicator values corresponding to low-risk diagnosis indicators in the computation assessment dimension based on the usage situation of the data computation resource, and determining the low-risk indicator trigger quantity in the computation assessment dimension based on the indicator values corresponding to the low-risk diagnosis indicators in the computation assessment dimension.

6 . The method according to claim 5 , wherein the low-risk diagnosis indicator in the storage assessment dimension comprises one or more of following:

the remaining storage usage duration being less than or equal to a second number of days, the storage amount increment day-on-day ratio being greater than a second proportion, the proportion of small files being greater than a second ratio, or the proportion of disk bad blocks being greater than a second bad block proportion, wherein the second number of days is greater than the first number of days, the second proportion is less than the first proportion, the second ratio is less than the first ratio, and the second bad block proportion is less than the first bad block proportion; and

the low-risk diagnosis indicator in the computation assessment dimension comprises one or more of following:

the queue blocking duration exceeding a second duration, a queue overflow duration exceeding a first overflow duration, the scheduling task failure rate being greater than a second failure rate, the scheduling task running duration being greater than a second running duration, a waiting-to-pass concurrent tasks ratio exceeding a third proportion, the queue usage computation time rate exceeding a second computation time rate, or a day-level task being completed across days, wherein the second duration is greater than the first duration, the second failure rate is less than the first failure rate, the second running duration is less than the first running duration, and the second computation time rate is less than the first computation time rate.

7 . The method according to claim 1 , further comprising:

obtaining a risk observation indicator in the storage assessment dimension and a risk observation indicator in the computation assessment dimension; and

performing risk assessment on the data storage resource and the data computation resource based on the risk observation indicator in the storage assessment dimension and the risk observation indicator in the computation assessment dimension, wherein

the risk observation indicator in the storage assessment dimension comprises one or more of following:

a small file ranking result, a remaining storage amount, an average storage increment in a time period, a number of remaining storage available days, a daily additional storage amount, or a total number of small files; and

the risk observation indicator in the computation assessment dimension comprises one or more of following:

daily usage computation time, a daily usage computation time rate, a daily usage computation time day-on-day ratio, a scheduling task failure amount, a proportion of unfinished instances on a current day, a T+1 unfinished rate, a service level agreement task breach amount, a one-time success rate of task scheduling, a sum of task suspension durations in a cycle, or a number of queue tasks in the cycle.

8 . An electronic device, applied to a scenario of resource allocation of a department, wherein the electronic device comprises:

one or more processors; and

a storage apparatus storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement a method for assessing a risk caused by improper use of a cluster data resource, the method comprising:

dividing, according to an impact degree of a risk caused by improper use of a data resource on a service, risk levels in a use process of the cluster data resource into three levels: a low-risk level, a high-risk level, and an accident level;

allocating the cluster data resource to the department;

obtaining an assessment dimension for assessing the risk of the cluster data resource, wherein the cluster data resource comprises a data storage resource and a data computation resource, and the assessment dimension comprises a storage assessment dimension and a computation assessment dimension;

obtaining at least one high-risk diagnosis indicator in the storage assessment dimension and at least one high-risk diagnosis indicator in the computation assessment dimension, wherein the high-risk diagnosis indicator in the storage assessment dimension, indicated for the storage assessment dimension, comprises an indicator for representing that a data resource usage amount of the data storage resource exceeds a first data storage resource threshold and/or an indicator for representing that the data storage resource does not comply with a first data storage resource usage rule, and the high-risk diagnosis indicator in the computation assessment dimension, indicated for the computation assessment dimension, comprises an indicator for representing that a data resource usage amount of the data computation resource exceeds a first data computation resource threshold;

counting a high-risk indicator trigger quantity in the storage assessment dimension and a high-risk indicator trigger quantity in the computation assessment dimension, wherein the high-risk indicator trigger quantity in the storage assessment dimension is an amount of high-risk diagnosis indicators in the storage assessment dimension that are satisfied in a use process of the data storage resource, and the high-risk indicator trigger quantity in the computation assessment dimension is an amount of high-risk diagnosis indicators in the computation assessment dimension that are satisfied in a use process of the data computation resource; and

assessing a risk caused by improper use of the data storage resource based on the high-risk indicator trigger quantity in the storage assessment dimension, and assessing a risk caused by improper use of the data computation resource based on the high-risk indicator trigger quantity in the computation assessment dimension.

9 . The electronic device according to claim 8 , wherein counting the high-risk indicator trigger quantity in the storage assessment dimension and the high-risk indicator trigger quantity in the computation assessment dimension comprises:

determining indicator values corresponding to high-risk diagnosis indicators in the storage assessment dimension based on a usage situation of the data storage resource, and determining the high-risk indicator trigger quantity in the storage assessment dimension based on the indicator values corresponding to the high-risk diagnosis indicators in the storage assessment dimension; and

determining indicator values corresponding to high-risk diagnosis indicators in the computation assessment dimension based on a usage situation of the data computation resource, and determining the high-risk indicator trigger quantity in the computation assessment dimension based on the indicator values corresponding to the high-risk diagnosis indicators in the computation assessment dimension.

10 . The electronic device according to claim 9 , wherein the high-risk diagnosis indicator in the storage assessment dimension comprises one or more of following:

a remaining storage usage duration being less than or equal to a first number of days, a storage amount increment day-on-day ratio being greater than a first proportion, a proportion of small files being greater than a first ratio, a proportion of disk bad blocks being greater than a first bad block proportion, a write forbidding trigger duration being greater than a third duration, a storage usage rate exceeding a first usage rate, or a set lifetime being less than a recommended value; and

the high-risk diagnosis indicator in the computation assessment dimension comprises one or more of following:

a queue blocking duration exceeding a first duration, a scheduling task failure rate being greater than a first failure rate, a scheduling task running duration being greater than a first running duration, a queue usage computation time rate exceeding a first computation time rate, or a service level agreement task breach.

11 . The electronic device according to claim 10 , wherein the method further comprises:

obtaining at least one low-risk diagnosis indicator in the storage assessment dimension and at least one low-risk diagnosis indicator in the computation assessment dimension, wherein the low-risk diagnosis indicator in the storage assessment dimension comprises an indicator for representing that the data resource usage amount of the data storage resource exceeds a second data storage resource threshold and/or an indicator for representing that the data storage resource does not comply with a second data storage resource usage rule, the low-risk diagnosis indicator in the computation assessment dimension comprises an indicator for representing that the data resource usage amount of the data computation resource exceeds a second data computation resource threshold, the second data storage resource threshold is less than the first data storage resource threshold, and the second data computation resource threshold is less than the first data computation resource threshold;

obtaining a low-risk indicator trigger quantity in the storage assessment dimension and a low-risk indicator trigger quantity in the computation assessment dimension, wherein the low-risk indicator trigger quantity in the storage assessment dimension is an amount of low-risk diagnosis indicators in the storage assessment dimension that are satisfied in the use process of the data storage resource, and the low-risk indicator trigger quantity in the computation assessment dimension is an amount of low-risk diagnosis indicators in the computation assessment dimension that are satisfied in the use process of the data computation resource; and

assessing the risk caused by improper use of the data storage resource based on the low-risk indicator trigger quantity in the storage assessment dimension, and assessing the risk caused by improper use of the data computation resource based on the low-risk indicator trigger quantity in the computation assessment dimension.

12 . The electronic device according to claim 11 , wherein obtaining the low-risk indicator trigger quantity in the storage assessment dimension and the low-risk indicator trigger quantity in the computation assessment dimension comprises:

determining indicator values corresponding to low-risk diagnosis indicators in the storage assessment dimension based on the usage situation of the data storage resource, and determining the low-risk indicator trigger quantity in the storage assessment dimension based on the indicator values corresponding to the low-risk diagnosis indicators in the storage assessment dimension; and

determining indicator values corresponding to low-risk diagnosis indicators in the computation assessment dimension based on the usage situation of the data computation resource, and determining the low-risk indicator trigger quantity in the computation assessment dimension based on the indicator values corresponding to the low-risk diagnosis indicators in the computation assessment dimension.

13 . The electronic device according to claim 12 , wherein the low-risk diagnosis indicator in the storage assessment dimension comprises one or more of following:

the remaining storage usage duration being less than or equal to a second number of days, the storage amount increment day-on-day ratio being greater than a second proportion, the proportion of small files being greater than a second ratio, or the proportion of disk bad blocks being greater than a second bad block proportion, wherein the second number of days is greater than the first number of days, the second proportion is less than the first proportion, the second ratio is less than the first ratio, and the second bad block proportion is less than the first bad block proportion; and

the low-risk diagnosis indicator in the computation assessment dimension comprises one or more of following:

the queue blocking duration exceeding a second duration, a queue overflow duration exceeding a first overflow duration, the scheduling task failure rate being greater than a second failure rate, the scheduling task running duration being greater than a second running duration, a waiting-to-pass concurrent tasks ratio exceeding a third proportion, the queue usage computation time rate exceeding a second computation time rate, or a day-level task being completed across days, wherein the second duration is greater than the first duration, the second failure rate is less than the first failure rate, the second running duration is less than the first running duration, and the second computation time rate is less than the first computation time rate.

14 . The electronic device according to claim 8 , wherein the method further comprises:

obtaining a risk observation indicator in the storage assessment dimension and a risk observation indicator in the computation assessment dimension; and

performing risk assessment on the data storage resource and the data computation resource based on the risk observation indicator in the storage assessment dimension and the risk observation indicator in the computation assessment dimension, wherein

the risk observation indicator in the storage assessment dimension comprises one or more of following:

a small file ranking result, a remaining storage amount, an average storage increment in a time period, a number of remaining storage available days, a daily additional storage amount, or a total number of small files; and

the risk observation indicator in the computation assessment dimension comprises one or more of following:

daily usage computation time, a daily usage computation time rate, a daily usage computation time day-on-day ratio, a scheduling task failure amount, a proportion of unfinished instances on a current day, a T+1 unfinished rate, a service level agreement task breach amount, a one-time success rate of task scheduling, a sum of task suspension durations in a cycle, or a number of queue tasks in the cycle.

15 . A non-transitory computer-readable storage medium, storing a computer program, wherein the computer program, when executed by a processor, causes a method for assessing a risk caused by improper use of a cluster data resource to be implemented,

wherein the method is applied to a scenario of resource allocation of a department, and the method comprises:

dividing, according to an impact degree of a risk caused by improper use of a data resource on a service, risk levels in a use process of the cluster data resource into three levels: a low-risk level, a high-risk level, and an accident level;

allocating the cluster data resource to the department;

obtaining an assessment dimension for assessing the risk of the cluster data resource, wherein the cluster data resource comprises a data storage resource and a data computation resource, and the assessment dimension comprises a storage assessment dimension and a computation assessment dimension;

obtaining at least one high-risk diagnosis indicator in the storage assessment dimension and at least one high-risk diagnosis indicator in the computation assessment dimension, wherein the high-risk diagnosis indicator in the storage assessment dimension, indicated for the storage assessment dimension, comprises an indicator for representing that a data resource usage amount of the data storage resource exceeds a first data storage resource threshold and/or an indicator for representing that the data storage resource does not comply with a first data storage resource usage rule, and the high-risk diagnosis indicator in the computation assessment dimension, indicated for the computation assessment dimension, comprises an indicator for representing that a data resource usage amount of the data computation resource exceeds a first data computation resource threshold;

counting a high-risk indicator trigger quantity in the storage assessment dimension and a high-risk indicator trigger quantity in the computation assessment dimension, wherein the high-risk indicator trigger quantity in the storage assessment dimension is an amount of high-risk diagnosis indicators in the storage assessment dimension that are satisfied in a use process of the data storage resource, and the high-risk indicator trigger quantity in the computation assessment dimension is an amount of high-risk diagnosis indicators in the computation assessment dimension that are satisfied in a use process of the data computation resource; and

assessing a risk caused by improper use of the data storage resource based on the high-risk indicator trigger quantity in the storage assessment dimension, and assessing a risk caused by improper use of the data computation resource based on the high-risk indicator trigger quantity in the computation assessment dimension.

16 . The non-transitory computer-readable storage medium according to claim 15 , wherein counting the high-risk indicator trigger quantity in the storage assessment dimension and the high-risk indicator trigger quantity in the computation assessment dimension comprises:

determining indicator values corresponding to high-risk diagnosis indicators in the storage assessment dimension based on a usage situation of the data storage resource, and determining the high-risk indicator trigger quantity in the storage assessment dimension based on the indicator values corresponding to the high-risk diagnosis indicators in the storage assessment dimension; and

determining indicator values corresponding to high-risk diagnosis indicators in the computation assessment dimension based on a usage situation of the data computation resource, and determining the high-risk indicator trigger quantity in the computation assessment dimension based on the indicator values corresponding to the high-risk diagnosis indicators in the computation assessment dimension.

17 . The non-transitory computer-readable storage medium according to claim 16 , wherein the high-risk diagnosis indicator in the storage assessment dimension comprises one or more of following:

a remaining storage usage duration being less than or equal to a first number of days, a storage amount increment day-on-day ratio being greater than a first proportion, a proportion of small files being greater than a first ratio, a proportion of disk bad blocks being greater than a first bad block proportion, a write forbidding trigger duration being greater than a third duration, a storage usage rate exceeding a first usage rate, or a set lifetime being less than a recommended value; and

the high-risk diagnosis indicator in the computation assessment dimension comprises one or more of following:

a queue blocking duration exceeding a first duration, a scheduling task failure rate being greater than a first failure rate, a scheduling task running duration being greater than a first running duration, a queue usage computation time rate exceeding a first computation time rate, or a service level agreement task breach.

18 . The non-transitory computer-readable storage medium according to claim 17 , wherein the method further comprises:

obtaining at least one low-risk diagnosis indicator in the storage assessment dimension and at least one low-risk diagnosis indicator in the computation assessment dimension, wherein the low-risk diagnosis indicator in the storage assessment dimension comprises an indicator for representing that the data resource usage amount of the data storage resource exceeds a second data storage resource threshold and/or an indicator for representing that the data storage resource does not comply with a second data storage resource usage rule, the low-risk diagnosis indicator in the computation assessment dimension comprises an indicator for representing that the data resource usage amount of the data computation resource exceeds a second data computation resource threshold, the second data storage resource threshold is less than the first data storage resource threshold, and the second data computation resource threshold is less than the first data computation resource threshold;

obtaining a low-risk indicator trigger quantity in the storage assessment dimension and a low-risk indicator trigger quantity in the computation assessment dimension, wherein the low-risk indicator trigger quantity in the storage assessment dimension is an amount of low-risk diagnosis indicators in the storage assessment dimension that are satisfied in the use process of the data storage resource, and the low-risk indicator trigger quantity in the computation assessment dimension is an amount of low-risk diagnosis indicators in the computation assessment dimension that are satisfied in the use process of the data computation resource; and

assessing the risk caused by improper use of the data storage resource based on the low-risk indicator trigger quantity in the storage assessment dimension, and assessing the risk caused by improper use of the data computation resource based on the low-risk indicator trigger quantity in the computation assessment dimension.

19 . The non-transitory computer-readable storage medium according to claim 18 , wherein obtaining the low-risk indicator trigger quantity in the storage assessment dimension and the low-risk indicator trigger quantity in the computation assessment dimension comprises:

determining indicator values corresponding to low-risk diagnosis indicators in the storage assessment dimension based on the usage situation of the data storage resource, and determining the low-risk indicator trigger quantity in the storage assessment dimension based on the indicator values corresponding to the low-risk diagnosis indicators in the storage assessment dimension; and

determining indicator values corresponding to low-risk diagnosis indicators in the computation assessment dimension based on the usage situation of the data computation resource, and determining the low-risk indicator trigger quantity in the computation assessment dimension based on the indicator values corresponding to the low-risk diagnosis indicators in the computation assessment dimension.

20 . The non-transitory computer-readable storage medium according to claim 19 , wherein the low-risk diagnosis indicator in the storage assessment dimension comprises one or more of following:

the remaining storage usage duration being less than or equal to a second number of days, the storage amount increment day-on-day ratio being greater than a second proportion, the proportion of small files being greater than a second ratio, or the proportion of disk bad blocks being greater than a second bad block proportion, wherein the second number of days is greater than the first number of days, the second proportion is less than the first proportion, the second ratio is less than the first ratio, and the second bad block proportion is less than the first bad block proportion; and

the low-risk diagnosis indicator in the computation assessment dimension comprises one or more of following:

the queue blocking duration exceeding a second duration, a queue overflow duration exceeding a first overflow duration, the scheduling task failure rate being greater than a second failure rate, the scheduling task running duration being greater than a second running duration, a waiting-to-pass concurrent tasks ratio exceeding a third proportion, the queue usage computation time rate exceeding a second computation time rate, or a day-level task being completed across days, wherein the second duration is greater than the first duration, the second failure rate is less than the first failure rate, the second running duration is less than the first running duration, and the second computation time rate is less than the first computation time rate.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (11)