Granted Patent B2

US 12,605,843 · App. 18/064,577 · Granted Apr 21, 2026

Inventors: Han Lin (Shenzhen, CN); Jui-chun Cheng (Shenzhen, CN)

Assignee: HAI ROBOTICS CO., LTD.

B25J9/1697B25J9/0009B25J9/1687B25J19/023G06T7/50G06T7/70

Abstract

Claims (42)

1 . A photographing method for picking or placing a target item, applied to a transport robot, comprising:

obtaining first multi-dimensional image information of a target position in a target shelf, wherein more than one storage positions are arranged in the target shelf along a depth direction of the target shelf, and the target position is a storage position in the target shelf from which the transport robot is to pick the target item up, or a storage position in the target shelf to which the transport robot is to place the target item;

determining, according to the first multi-dimensional image information, whether there is a first item in the target position; and

performing an operation of starting a solution, wherein the solution comprises at least one of stopping photographing, sending a warning signal, and reporting to a server to which the transport robot belongs;

wherein the target position is a shallow storage position at a first storage position in the target shelf along the depth direction of the shelf, a second storage position along the depth direction of the shelf, or a deep storage position behind the second storage position;

wherein the target position is the deep storage position, before the determining, according to the first multi-dimensional image information, whether there is a first item in the target position, the method further comprises:

determining, according to the first multi-dimensional image information, whether there is a second item in any storage position in front of the deep storage position; and

in response to determining that there is a second item in any storage position in front of the deep storage position, performing the operation of starting the solution.

2 . The photographing method according to claim 1 , wherein when the transport robot is performing a task of picking, the performing an operation of starting a solution further comprises:

in response to determining that there is the first item in the target position, performing an operation of picking; in response to determining that there is no first item in the target position, continuing photographing for a plurality of times; determining, for a plurality of times, whether there is the first item in the target position according to image information obtained by the plurality times of photographing; in response to number of continuing photographing reaching a preset photographing threshold and results of the plurality times of determining are that there is no first item in the target position, performing the operation of starting the solution; or

in response to determining that there is the first item in the target position, determining whether the first item is the target item according to the first multi-dimensional image information; in response to determining that the first item is the target item, performing the operation of picking; in response to determining that the first item is not the target item, continuing photographing for a plurality of times; determining, for a plurality of times, whether the first item is the target item according to image information obtained by the plurality times of photographing; in response to number of continuing photographing reaching the preset photographing threshold and results of the plurality times of determining are that the first item is not the target item, performing the operation of starting the solution; or

in response to determining that there is the first item in the target position, determining whether the first item is a target item according to the first multi-dimensional image information; in response to determining that the first item is the target item, determining whether pose information of the target item satisfies preset requirements according to the first multi-dimensional image information; in response to determining that the pose information of the target item satisfies the preset requirements, performing the operation of picking; in response to determining that the pose information of the target item does not satisfy the preset requirements, continuing photographing for a plurality of times; determining, for a plurality of times, whether the pose information of the target item satisfies the preset requirements according to image information obtained by the plurality times of photographing; in response to number of continuing photographing reaching the preset photographing threshold and results of the plurality times of determining are that the pose information of the target item does not satisfy the preset requirements, performing the operation of starting the solution.

3 . The photographing method according to claim 1 , wherein when the transport robot is performing a task of placing, the performing an operation of starting a solution further comprises:

in response to determining that there is no first item in the target position, performing the operation of placing; and

in response to determining that there is the first item in the target position, performing the operation of starting the solution.

4 . The photographing method according to claim 1 , wherein the first multi-dimensional image information comprises first depth information, and the determining, according to the first multi-dimensional image information, whether there is a first item in the target position, comprises: determining, according to the first depth information, whether there is the first item in the target position.

5 . The photographing method according to claim 1 , wherein the first multi-dimensional image information comprises first depth information, and whether there is the second item is determined by detecting whether the first depth information corresponds to depth information of the target position.

6 . A photographing system for picking or placing a target item, applied to a transport robot, comprising:

a memory, and a processor, wherein the memory stores instructions executable by the processor, and the processor is configured to execute the executable instructions and control the transport robot to perform operations of:

obtaining first multi-dimensional image information of a target position in a target shelf, wherein more than one storage positions are arranged in the target shelf along a depth direction of the target shelf, and the target position is a storage position in the target shelf from which the transport robot is to pick the target item up, or a storage position in the target shelf to which the transport robot is to place the target item;

determining, according to the first multi-dimensional image information, whether there is a first item in the target position; and

performing an operation of starting a solution, wherein the solution comprises at least one of stopping photographing, sending a warning signal, and reporting to a server to which the transport robot belongs;

wherein the target position is a shallow storage position at a first storage position in the target shelf along the depth direction of the shelf, a second storage position along the depth direction of the shelf, or a deep storage position behind the second storage position;

wherein the target position is the deep storage position, before the determining, according to the first multi-dimensional image information, whether there is a first item in the target position, the processor is further configured to control the transport robot to perform;

determining, according to the first multi-dimensional image information, whether there is a second item in any storage position in front of the deep storage position; and

in response to determining that there is a second item in any storage position in front of the deep storage position, performing the operation of starting the solution.

7 . A non-transitory computer-readable storage medium, storing a computer-executable instruction, wherein when the computer-executable instruction is executed by a processor, the computer-executable instruction is configured to control a transport robot to perform:

obtaining first multi-dimensional image information of a target position in a target shelf, wherein more than one storage positions are arranged in the target shelf along a depth direction of the target shelf, and the target position is a storage position in the target shelf from which the transport robot is to pick an item up, or a storage position in the target shelf to which the transport robot is to place an item;

determining, according to the first multi-dimensional image information, whether there is a first item in the target position; and

performing an operation of starting a solution, wherein the solution comprises at least one of stopping photographing, sending a warning signal, and reporting to a server to which the transport robot belongs;

wherein the target position is a shallow storage position at a first storage position in the target shelf along the depth direction of the shelf, a second storage position along the depth direction of the shelf, or a deep storage position behind the second storage position;

wherein the target position is the deep storage position, before the determining, according to the first multi-dimensional image information, whether there is a first item in the target position, the computer-executable instruction is further configured to control a transport robot to perform:

determining, according to the first multi-dimensional image information, whether there is a second item in any storage position in front of the deep storage position; and

in response to determining that there is a second item in any storage position in front of the deep storage position, performing the operation of starting the solution.

8 . A photographing method for picking or placing a target item, applied to a transport robot, comprising:

obtaining first multi-dimensional image information of a target position in a target shelf, wherein more than one storage positions are arranged in the target shelf along a depth direction of the target shelf, and the target position is a storage position in the target shelf from which the transport robot is to pick the target item up, or a storage position in the target shelf to which the transport robot is to place the target item;

determining, according to the first multi-dimensional image information, whether there is a first item in the target position; and

performing an operation of starting a solution, wherein the solution comprises at least one of stopping photographing, sending a warning signal, and reporting to a server to which the transport robot belongs;

wherein when the transport robot is performing a task of picking, the performing an operation of starting a solution further comprises:

in response to determining that there is the first item in the target position, performing an operation of picking; in response to determining that there is no first item in the target position, continuing photographing for a plurality of times; determining, for a plurality of times, whether there is the first item in the target position according to image information obtained by the plurality times of photographing; in response to number of continuing photographing reaching a preset photographing threshold and results of the plurality times of determining are that there is no first item in the target position, performing the operation of starting the solution; or

in response to determining that there is the first item in the target position, determining whether the first item is the target item according to the first multi-dimensional image information; in response to determining that the first item is the target item, performing the operation of picking; in response to determining that the first item is not the target item, continuing photographing for a plurality of times; determining, for a plurality of times, whether the first item is the target item according to image information obtained by the plurality times of photographing; in response to number of continuing photographing reaching the preset photographing threshold and results of the plurality times of determining are that the first item is not the target item, performing the operation of starting the solution; or

in response to determining that there is the first item in the target position, determining whether the first item is a target item according to the first multi-dimensional image information; in response to determining that the first item is the target item, determining whether pose information of the target item satisfies preset requirements according to the first multi-dimensional image information; in response to determining that the pose information of the target item satisfies the preset requirements, performing the operation of picking; in response to determining that the pose information of the target item does not satisfy the preset requirements, continuing photographing for a plurality of times; determining, for a plurality of times, whether the pose information of the target item satisfies the preset requirements according to image information obtained by the plurality times of photographing; in response to number of continuing photographing reaching the preset photographing threshold and results of the plurality times of determining are that the pose information of the target item does not satisfy the preset requirements, performing the operation of starting the solution.

Assignments (2)

Priority Claims (1)

Continuity (2)

References Cited (58)