Human-tracking method and robot apparatus for performing the same
Provided are a human-tracking method and a robot apparatus. The human-tracking method includes receiving an image frame including a color image and a depth image, determining whether user tracking was successful in a previous image frame, and determining a location of a user and a goal position to which a robot apparatus is to move based on the color image and the depth image in the image frame, when user tracking was successful in the previous frame. Accordingly, a current location of the user can be predicted from the depth image, user tracking can be quickly performed, and the user can be re-detected and tracked using user information acquired in user tracking when detection of the user fails due to obstacles or the like.
1. A human-tracking method performed by a robot apparatus, comprising:
receiving an image frame including a color image and a depth image;
determining whether user tracking was successful in a previous image frame;
determining a location of a user and a goal position to which a robot apparatus is to move based on the color image and the depth image in the image frame, when user tracking was successful in the previous frame,
controlling a movement of the robot apparatus; detecting the user in the image frame and determining the location of the user and the goal position to which the robot apparatus is to move, when user tracking failed in the previous frame; and
determining a most recently set goal position as the goal position to which the robot apparatus is to move, when detection fails,
wherein detecting the user includes:
detecting at least one human being from the color image and the depth image and detecting the user based on color information and height information of the user from the detected at least one human being; and
storing tracking information and determining the determined location of the user as the goal position to which the robot apparatus is to move, when detection succeeds.
2. The human-tracking method of claim 1 , wherein the determining of the location of the user and the goal position includes:
performing depth-filtering in a current image frame based on the location of the user successfully tracked in the previous image frame;
performing color-based tracking in an image on which the depth-filtering is performed;
acquiring the location of the user when user tracking is successful in the color-based tracking; and
setting the location of the user as the goal position to which the robot apparatus is to move.
3. The human-tracking method of claim 1 , wherein the color information of the user is color information of the user generated when user tracking succeeds.
4. The human-tracking method of claim 1 , wherein the height information of the user is height information estimated from at least one of a height of a camera, an angle of the camera, a field of view (FOV), and the location and depth of the user within the image frame, when user tracking succeeds.
5. A human-tracking robot apparatus comprising:
an input unit configured to provide a color image and a depth image about a user who is a tracking target;
a control unit configured to track the user based on the provided color image and depth image;
a storage unit configured to store the provided color image and depth image and information generated when tracking the user; and
a movement platform unit configured to control movement of the robot apparatus to a location of the user tracked by the control unit wherein the control unit receives an image frame including the color image and the depth image, determines whether user tracking was successful in a previous image frame, and determines the location of the user and a goal position to which the robot apparatus is to move based on the color image and the depth image in the image frame when user tracking succeeds, and
wherein the control unit detects the user from the image frame and determines the location of the user and the goal position to which the robot apparatus is to move, when user tracking failed in the previous image frame.
6. The human-tracking robot apparatus of claim 5 , wherein the control unit performs depth-filtering in the image frame based on the location of the user successfully tracked in the previous image frame, performs color-based tracking in an image on which the depth-filtering is performed, stores the location of the user and information of the user when user tracking succeeds, and sets the location of the user as the goal position to which the robot apparatus is to move.
7. The human-tracking robot apparatus of claim 5 , wherein the control unit detects at least one human being from the color image and the depth image, detects the user based on color information and height information of the user from the detected at least one human being, stores tracking information when detection succeeds, and determines the detected location of the user as the goal position to which the robot apparatus is to move.