IP Library Granted Patent US 11,763,592
Granted Patent B2
US 11,763,592 · App. 17/151,212 · Granted Sep 19, 2023

Video processing device

Inventor: Yuishi Takeno (Numazu Shizuoka, JP)
Assignee: Toshiba Tec Kabushiki Kaisha
G06V40/107G06V20/00G06V20/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,763,592
App. No.
17/151,212
Granted
Sep 19, 2023
Kind
B2
Abstract

An image processing device includes a camera interface connectable to a camera configured to capture a video of a worker in a work area, an output interface connectable to a display device that is viewable from the work area, a memory that stores task information about a task to be performed by the worker, and a processor configured to determine a position and a posture of the worker in the work area based on the captured video, determine whether the worker has performed the task properly based on the task information stored in the memory and the determined position and posture of the worker, upon determining that the worker has not performed the task properly, extract from the video a first part showing the task that has been performed by the worker, and control the output interface to output the first part to the display device.

Claims (55)

1. An image processing device, comprising:

a camera interface connectable to a camera;

an output interface connectable to a display device;

a memory that stores task information about a task to be performed; and

a processor configured to:

determine a position and a posture of a worker in a video captured by the camera,

determine whether the worker has performed the task properly based on a comparison of the determined position and posture with the task information stored in the memory,

upon determining that the worker has not performed the task properly, extract from the captured video a first video showing the worker who is performing the task, and

control the display device to display the first video.

2. The image processing device according to claim 1 , wherein the processor is further configured to:

based on the task information, determine a first position at which the task is to be performed by the worker and a first posture to be taken by the worker at the first position,

determine whether the worker has taken the first posture at the first position based on the position and the posture of the worker that have been determined from the captured video,

upon determining that the worker has taken the first posture at the first position, determine that the worker has performed the task properly, and

upon determining that the worker has not taken the first posture at the first position, determine that the worker has not performed the task properly.

3. The image processing device according to claim 2 , wherein when the worker has not performed the task properly, the processor controls the display device to display the first video such that each of the position of the worker that has been determined from the captured video and the first position at which the task is to be performed by the worker is indicated by a different marker.

4. The image processing device according to claim 3 , further comprising:

an input interface connectable to an input device, wherein

the task information is input via the input device.

5. The image processing device according to claim 3 , wherein the task information is displayed as text together with the marker.

6. The image processing device according to claim 2 , wherein the task is to take a commodity from a shelf or pack the commodity into a container, and the first position at which the task is to be performed by the worker is a position at which the commodity is taken from the shelf or a position at which the commodity is packed into the container.

7. The image processing device according to claim 6 , wherein the first posture is a posture taken by the worker to take the commodity from the shelf or a posture taken by the worker to pack the commodity into the container.

8. The image processing device according to claim 6 , wherein the camera is arranged to capture the video of a work area where the worker moves to take and pack the commodity.

9. The image processing device according to claim 1 , further comprising:

a network interface configured to communicate with an external device, wherein the task information is received from the external device.

10. The image processing device according to claim 1 , wherein a sound is output when the first video is displayed on the display device.

11. A method carried out by an image processing device, the method comprising:

storing task information in a memory;

capturing a video of a worker in a work area;

determining a position and a posture of the worker in the captured video;

determining whether the worker has performed the task properly based on the task information and the determined position and posture of the worker;

upon determining that the worker has not performed the task properly, extracting from the captured video a first video showing the worker who is performing the task; and

displaying the first video on a display device.

12. The method according to claim 11 , further comprising:

reading the task information and determining a first position at which the task is to be performed by the worker and a first posture to be taken by the worker at the first position;

determining whether the worker has taken the first posture at the first position based on the position and the posture of the worker that have been determined from the captured video;

upon determining that the worker has taken the first posture at the first position, determining that the worker has performed the task properly; and

upon determining that the worker has not taken the first posture at the first position, determining that the worker has not performed the task properly.

13. The method according to claim 12 , wherein the first video is displayed such that each of the position of the worker that has been determined from the captured video and the first position at which the task is to be performed by the worker is indicated by a different marker.

14. The method according to claim 13 , further comprising:

receiving the task information that has been input via an input device.

15. The method according to claim 13 , wherein the task information is displayed as text together with the marker.

16. The method according to claim 12 , wherein the task is to take a commodity from a shelf or pack the commodity into a container, and the first position at which the task is to be performed by the worker is a position at which the commodity is taken from the shelf or a position at which the commodity is packed into the container.

17. The method according to claim 16 , wherein the first posture is a posture taken by the worker to take the commodity from the shelf or a posture taken by the worker to pack the commodity into the container.

18. The method according to claim 16 , wherein the work area is an area where the worker moves to take and pack the commodity.

19. The method according to claim 11 , further comprising:

receiving the task information from an external device.

20. An image processing system, comprising:

a camera;

a display device;

a memory that stores task information about a task to be performed; and

a processor configured to:

determine a position and a posture of a worker in a video captured by the camera,

determine whether the worker has performed the task properly based on the task information stored in the memory and the determined position and posture of the worker,

upon determining that the worker has not performed the task properly, extract from the captured video a first video showing the worker who is performing the task, and

control the display device to display the first video.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2021
From: TAKENO, YUISHI
To: TOSHIBA TEC KABUSHIKI KAISHA
Reel/Frame 054940/0971 →
Priority Claims (1)
JP 2020-047813 · Mar 18, 2020 · national
Continuity (1)
Related Publication 20210295040A1 · Sep 23, 2021