Methods, systems, apparatus, electronic devices, and storage media for evaluating work tasks

1 . A method for controlling performance of a work task, the method comprising:

acquiring point cloud data of a workpiece in the work task via a point cloud scanner and a three-dimensional image frame of a worker in the work task via three-dimensional camera equipment;

performing, using a CAD model from a database, machine vision recognition on the point cloud data of the workpiece to recognize attribute information of the workpiece, wherein the performing machine vision recognition comprises performing voxel grid down sampling and the attribute information of the workpiece comprises physical features, measurement parameters, spatial location, and surface state;

recognizing action information of the worker in a deep learning manner based on the three-dimensional image frame;

comparing combination information of the attribute information and the action information with a standard operating procedure of the work task, wherein the standard operating procedure comprising a standard attribute of the workpiece and a standard action of the worker in the work task;

evaluating the work task based on a comparing result; and

displaying an instruction for the worker as a result of the evaluating the work task;

wherein the instruction includes a message that includes at least one of: an alarm message if a type of workpiece used does not conform to a standard type of workpiece for the work task, a reminder message to remind the worker to perform a standard action for a future time point based on the standard operating procedure, and a prompt message to check the work tool.

2 . The method according to claim 1 , further comprising:

determining a real time point of the action information; and

determining a standard action for the next time point of the real time point based on the standard operating procedure.

3 . The method according to claim 1 , wherein:

the work task comprises a plurality of work phases;

the standard operating procedure comprises a plurality of sub-standard operating procedures respectively corresponding to the plurality of work phases; and

the comparing the combination information of the attribute information and the action information with the standard operating procedure of the work task comprises:

splitting the combination information into a plurality of sub-combination information based on respective comparisons with the respective starting actions of the plurality of work phases, wherein the plurality of sub-combination information respectively corresponding to the plurality of work phases; and

comparing each sub-combination information of the plurality of sub-combination information with the corresponding sub-standard operating procedure.

4 . The method according to claim 3 , wherein:

the plurality of sub-standard operating procedures have respective standard working hours; and

the comparing each sub-combination information of the plurality of sub-combination information with the corresponding sub-standard operating procedure comprises:

comparing actual working hours corresponding to each sub-combination information with standard working hours of the corresponding sub-standard operating procedure;

comparing attribute information in each sub-combination information with standard attribute of the corresponding sub-standard operating procedures; and

comparing action information in each of the sub-combination information with standard action of the corresponding sub-standard operating procedures.

5 . The method according to claim 4 , further comprising:

determining an overtime work phase, wherein actual working hours of the overtime work phase is greater than the corresponding standard working hours;

determining the ratio of the number of overtime work phases to the number of all work phases; and

determining a work tool used by the worker in the overtime work phase when the ratio is less than a predetermined threshold.

6 . The method according to claim 1 ,

wherein the evaluating the work task based on the comparing result comprises:

generating a first evaluation value when action information conforms to standard action and attribute information conforms to standard attribute;

generating a second evaluation value when action information conforms to standard action, attribute information does not conform to standard attribute, and attribute information is within a predetermined attribute interval;

generating a third evaluation value when action information conforms to standard action, attribute information does not conform to standard attribute, and the attribute information is not within the predetermined attribute interval; and

generating a fourth evaluation value when action information does not conform to standard action; and

wherein the first evaluation value is larger than the second evaluation value, the second evaluation value is larger than the third evaluation value, and the third evaluation value is larger than the fourth evaluation value.

7 . A system for evaluating a work task, the system comprising:

an acquisition module, which includes a point cloud scanner and a three-dimensional camera device, to acquire point cloud data of a workpiece in the work task via the point cloud scanner and a three-dimensional image frame of a worker in the work task via the three-dimensional camera device;

a server configured to:

perform, using a CAD model from a database, machine vision recognition on the point cloud data of the workpiece to recognize attribute information of the workpiece, wherein, to perform the machine vision recognition, the server is further configured to perform voxel grid down sampling and the attribute information of the workpiece comprises physical features, measurement parameters, spatial location, and surface state;

recognize action information of the worker in a deep learning manner based on the three-dimensional image frame;

compare combination information of the attribute information and the action information with a standard operating procedure of the work task, wherein the standard operating procedure comprising a standard attribute of the workpiece and a standard action of the worker in the work task; and

evaluate the work task based on a comparing result; and

a workstation to display an instruction for the worker as a result of evaluating the work task;

wherein the instruction includes a message that includes at least one of: an alarm message if a type of workpiece used does not conform to a standard type of workpiece for the work task, a reminder message to remind the worker to perform a standard action for a future time point based on the standard operating procedure, and a prompt message to check the work tool.

8 . The system according to claim 7 , wherein:

the attribute information comprises the type of the workpiece; and

the server is further configured to generate the alarm message when the type of the workpiece does not conform to the standard type of the workpiece in the work task.

9 . The system according to claim 7 , wherein the server is further configured to:

determine a real time point of the action information;

determine a standard action for the next time point of the real time point based on the standard operating procedure; and

issue a reminder message to remind the worker to perform the standard action for the next time point.

10 . The system according to claim 7 , wherein:

the work task comprises a plurality of work phases, and the standard operating procedure comprises a plurality of sub-standard operating procedures respectively corresponding to the plurality of work phases; and

the server is further configured to:

split the combination information into a plurality of sub-combination information based on respective comparisons with the respective starting actions of the plurality of work phases, wherein the plurality of sub-combination information respectively corresponding to the plurality of work phases; and

compare each sub-combination information with the corresponding sub-standard operating procedure.

11 . The system according to claim 10 , wherein:

the plurality of sub-standard operating procedures have respective standard working hours; and

the server is further configured to:

compare actual working hours corresponding to each sub-combination information with standard working hours of the corresponding sub-standard operating procedure; compare attribute information in each sub-combination information with standard attribute of the corresponding sub-standard operating procedures; and

compare action information in each of the sub-combination information with standard action of the corresponding sub-standard operating procedures.

12 . The system according to claim 11 , wherein the server is further configured to:

determine an overtime work phase, wherein actual working hours of the overtime work phase is greater than the corresponding standard working hours;

determine the ratio of the number of overtime work phases to the number of all work phases; and

determine a work tool used by the worker in the overtime work phase when the ratio is less than a predetermined threshold.

13 . An apparatus for evaluating a work task, the apparatus comprising:

an acquisition module, which includes a point cloud scanner and a three-dimensional camera device, configured to acquire point cloud data of a workpiece in the work task via the point cloud scanner and a three-dimensional image frame of a worker in the work task via the three-dimensional camera device;

a first recognition module configured to perform, using a CAD model from a database, machine vision recognition on the point cloud data of the workpiece to recognize attribute information of the workpiece, wherein, to perform machine vision recognition, the first recognition module is further configured to perform voxel grid down sampling and the attribute information of the workpiece comprises physical features, measurement parameters, spatial location, and surface state;

a second recognition module configured to recognize action information of the worker in a deep learning manner based on the three-dimensional image frame;

a comparison module configured to compare combination information of the attribute information and the action information with a standard operating procedure of the work task, wherein the standard operating procedure comprising a standard attribute of the workpiece and a standard action of the worker in the work task;

an evaluation module configured to evaluate the work task based on a comparing result; and

a workstation configured to display an instruction for the worker as a result of evaluating the work task;

wherein the instruction includes a message that includes at least one of: an alarm message if a type of workpiece used does not conform to a standard type of workpiece for the work task, a reminder message to remind the worker to perform a standard action for a future time point based on the standard operating procedure, and a prompt message to check the work tool.

14 . The method according to claim 1 ,

wherein the performing voxel grid down sampling obtains key points; and

wherein the performing the machine vision recognition on the point cloud data of the workpiece to recognize the attribute information of the workpiece comprises:

before the performing the voxel grid down sampling:

performing denoising and filtering of the point cloud data of the workpiece; and

calculating normal vectors for denoised point cloud data from the performing denoising and filtering of the point cloud data of the workpiece.

15 . The method according to claim 14 , further comprising:

for the key points, calculating a histogram feature descriptor of a normal direction.

16 . The method according to claim 15 , further comprising:

obtaining a SHOT descriptor from the calculating the histogram feature descriptor of the normal direction.

17 . The method according to claim 1 , wherein the performing the machine vision recognition on the point cloud data of the workpiece to recognize the attribute information of the workpiece comprises identifying physical characteristics of the workpiece, physical size measurement parameters of the workpiece, or spatial position information of the workpiece.

18 . The system according to claim 7 ,

wherein the voxel grid down sampling obtains key points; and

wherein, to perform the machine vision recognition on the point cloud data of the workpiece to recognize the attribute information of the workpiece, the server is further configured to:

before the voxel grid down sampling is performed:

perform denoising and filtering of the point cloud data of the workpiece; and

calculate normal vectors for denoised point cloud data from the denoising the performing denoising and filtering of the point cloud data of the workpiece.

19 . The system according to claim 18 , wherein the server is further configured to, for the key points, calculate a histogram feature descriptor of a normal direction.

20 . The apparatus according to claim 13 ,

wherein the voxel grid down sampling obtains key points; and

wherein, to perform the machine vision recognition on the point cloud data of the workpiece to recognize the attribute information of the workpiece, the first recognition module is further configured to:

before the voxel grid down sampling is performed:

perform denoising and filtering of the point cloud data of the workpiece; and

calculate normal vectors for denoised point cloud data from the denoising the performing denoising and filtering of the point cloud data of the workpiece.