IP Library › Granted Patent US 10,970,935
Granted Patent B2
US 10,970,935 · App. 16/230,278 · Granted Apr 6, 2021

Body pose message system

Inventors: Anthony Mark Jones (Hillsboro, OR); Jessica A. F. Jones (Forest Grove, OR); Bruce A. Young (Le Mars, IA)
G06T19/006G06F3/011G06F3/017G06T2200/04G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,970,935
App. No.
16/230,278
Granted
Apr 6, 2021
Kind
B2
Abstract

A person who is not using a hybrid reality (HR) system communicates with the HR system without using a network communications link using a body pose. Data is received from a sensor and an individual is detected in the sensor data. A first situation of at least one body part of the individual in 3D space is ascertained at a first time and a body pose is determined based on the first situation of the at least one body part. An action is decided on based on the body pose and the action is performed on an HR system worn by a user.

Claims (64)

1. A method comprising:

capturing a first image using a sensor of a hybrid reality (HR) system worn or carried by a user;

detecting, by a computing device, an individual, different than the user, in the first image;

ascertaining, by the computing device, a first situation of at least one body part of the individual in 3D space at a first time using the first image;

recognizing, by the computing device, a body pose of the individual based on the first situation of the at least one body part of the individual;

determining, by the computing device, information for the user to relay to the individual based on the recognized body pose of the individual;

selecting a response body pose that corresponds to the information to relay to the individual; and

prompting the user using the HR system to perform the response body pose to relay the information to the individual.

2. The method of claim 1 , the HR system comprising a head-mounted display and the computing device.

3. The method of claim 1 , wherein the computing device comprises a local processor in the HR system, a remote processor located remotely from the HR system, and a communication interface communicatively coupling the local processor to the remote processor; and

wherein at least one of said detecting the individual in the first image, said ascertaining the first situation of the at least one body part of the individual, said recognizing a body pose of the individual, said determining information for the user to relay to the individual, or said selecting the response body pose for the user, is performed by the remote processor.

4. The method of claim 1 , further comprising selecting, by the computing device, a particular action to be performed by the HR system, in addition to said prompting, from a set of predetermined actions based on the recognized body pose of the individual.

5. The method of claim 1 , said prompting the user to perform the response body pose comprising presenting a virtual image of the response body pose to the user on a display of the HR system.

6. The method of claim 1 , further comprising:

sending a message to a computer remote from the HR system, wherein the message is based on the recognized body pose of the individual; and

receiving a message from the computer remote from the HR system, wherein the message provides the information or the response body pose.

7. The method of claim 1 , further comprising:

capturing a sequence of images of the at least one body part of the individual, including the first image, using the sensor that show the individual making a gesture;

recognizing the gesture, by the computing device, made by the individual using in the sequence of images of the at least one body part of the individual; and

selecting, by the computing device, a particular action to be performed by the HR system, in addition to said prompting, from a set of predetermined actions based on the gesture as the body pose of the individual.

8. The method of claim 1 , said determining further comprising:

accessing data stored in a memory of the computing device to select the response body pose, the data providing a correspondence between a plurality of messages and a plurality of response body poses, wherein the information includes a message of the plurality of messages.

9. The method of claim 1 , further comprising:

distinguishing, by the computing device using the first image, a characteristic of the individual separate from the first situation of the at least one body part of the individual; and

using the characteristic of the individual to determine the information to relay to the information or select the response body pose.

10. The method of claim 9 , wherein the characteristic of the individual comprises a uniform worn by the individual.

11. An article of manufacture comprising a tangible medium, that is not a transitory propagating signal, encoding computer-readable instructions that, when applied to a computer system, instruct the computer system to perform a method comprising:

capturing a first image using a sensor of a hybrid reality (HR) system worn or carried by a user;

detecting an individual, different than the user, in the first image;

ascertaining a first situation of at least one body part of the individual in 3D space at a first time using the first image;

recognizing a body pose of the individual based on the first situation of the at least one body part of the individual;

determining information for the user to relay to the individual based on the recognized body pose of the individual;

selecting a response body pose that corresponds to the information to relay to the individual; and

prompting the user using the HR system to perform the response body pose to relay the information to the individual.

12. The article of manufacture of claim 11 , said prompting the user to perform the response body pose comprising presenting a virtual image of the response body pose to the user on a display of the HR system.

13. The article of manufacture of claim 11 , the method further comprising:

sending a message to a computer remote from the HR system, wherein the message is based on the recognized body pose of the individual; and

receiving a message from the computer remote from the HR system, wherein the message provides the information or the response body pose.

14. The article of manufacture of claim 11 , the method further comprising:

accessing data stored in a memory of the computing device to select the response body pose, the data providing a correspondence between a plurality of messages and a plurality of response body poses, wherein the information includes a message of the plurality of messages.

15. The article of manufacture of claim 11 , the method further comprising:

distinguishing, by the computing device using the first image, a characteristic of the individual separate from the first situation of the at least one body part of the individual; and

using the characteristic of the individual to determine the information to relay to the information or select the response body pose.

16. A head-mounted display (HMD) comprising:

a display;

a structure, coupled to the display and adapted to position the display in a field-of-view (FOV) of a user;

a sensor; and

a processor, coupled to the display and the image sensor, the processor configured to:

receive first image data from the sensor;

detect an individual not wearing the HMD in the first image data;

ascertain a first situation of at least one body part of the individual in 3D space at a first time using the first image data;

recognize a body pose of the individual based on the first situation of the at least one body part of the individual;

determine information for the user to relay to the individual based on the recognized body pose of the individual;

select a response body pose that corresponds to the information to relay to the individual; and

prompt the user to perform the response body pose to relay the information to the individual.

17. The system of claim 16 , the processor further configured to present a virtual image of the response body pose to the user on the display as at least a part of said prompt.

18. The system of claim 16 , the processor further configured to:

send a message to a computer remote from the HMD, wherein the message is based on the recognized body pose of the individual; and

receive a message from the computer remote from the HMD, wherein the message provides the information or the response body pose.

19. The system of claim 16 , the processor further configured to:

access data stored in a memory of the computing device to select the response body pose, the data providing a correspondence between a plurality of messages and a plurality of response body poses, wherein the information includes a message of the plurality of messages.

20. The system of claim 16 , the processor further configured to:

distinguish, using the first image, a characteristic of the individual separate from the first situation of the at least one body part of the individual; and

use the characteristic of the individual to determine the information to relay to the information or select the response body pose.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 26, 2019
From: JONES, ANTHONY MARK; JONES, JESSICA A.F.; YOUNG, BRUCE ARNOLD
To: CURIOUS COMPANY, LLC
Reel/Frame 048706/0291 →
Continuity (1)
Related Publication 20200202625A1 · Jun 25, 2020
Cited By (1)
US 12,515,057