IP Library Granted Patent US 11,586,276
Granted Patent B2
US 11,586,276 · App. 17/276,548 · Granted Feb 21, 2023

Systems and methods for generating complementary data for visual display

Inventors: Hans Winold (Berkeley, CA); Joseph White (Oakland, CA); Ryan Corniel (San Jose, CA); Mark Samuel Gutentag (San Francisco, CA); John Lockhart (San Ramon, CA)
Assignee: PENUMBRA, INC.
G06F3/011G06T19/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,586,276
App. No.
17/276,548
Granted
Feb 21, 2023
Kind
B2
Abstract

A system for generating complementary data for a visual display that includes one or plurality of wearable sensors that collect tracking data for a users position, orientation, and movement. The sensor(s) are in communication with at least one processor that may be configured to receive tracking data, identify missing tracking data, generate complementary data to substitute for missing tracking data, generate a 3D model comprised of tracking data and complementary data, and communicate the 3D model to a display. Complementary tracking data may be generated by comparison to a key pose library, by comparison to past tracking data, or by inverse kinematics.

Claims (30)

1. A system for generating complementary tracking data, the system comprising:

one or a plurality of wearable sensors configured to be disposed on a subject, wherein the wearable sensors are configured to collect and transmit tracking data;

a processor in communication with a memory that includes processor executable instructions, wherein the execution of the instructions cause the system to:

(a) receive tracking data from the one or the plurality of wearable sensors;

(b) map the tracking data onto a 3D model, wherein the 3D model is a virtual representation of an extent and a motion of the subject;

(c) identify an incomplete portion of the 3D model, the incomplete portion comprising a portion of the model not mapped to the tracking data;

(d) generate complementary tracking data to substitute for at least a portion of the incomplete portion; and (e) include the complementary tracking data in the 3D model, and

wherein the memory includes processor executable instructions that cause the system to access one or a series of prior 3D models, wherein the prior 3D models were previously communicated for display.

2. The system of claim 1 , wherein the complementary data is generated by comparing available tracking data to a key pose library.

3. The system of claim 1 , wherein tracking data is compared to blend spaces between two or more key poses in the key pose library.

4. The system of claim 3 , wherein tracking data is compared to key poses of the library of key poses to determine a strongest match.

5. The system of claim 4 , wherein the determination of a match weights similarities of joints and body parts closest to the incomplete portion more heavily than joints and body parts distant from the incomplete portion.

6. The system of claim 1 , wherein the complementary tracking data is generated by identifying a cluster of repetitive tracking data among a series of prior 3D models, matching tracking data to the cluster, and generating complementary data similar to the matched portion of the cluster.

7. The system of claim 1 , wherein the complementary tracking data is generated by identifying a cluster of repetitive tracking data among a series of prior 3D models, determining where available tracking data best fits in the cluster of repetitive tracking data, and generating complementary tracking data that mimics the cluster of repetitive tracking data.

8. The system of claim 1 , wherein the processor is configured to analyze the one or a series of prior 3D models for gesture triggers, wherein the identification of a gesture trigger causes the system to communicate a series of updated 3D models that are at least partially blended with a pre-recorded gesture animation.

9. The system of claim 1 , wherein complementary tracking data is generated with a FABRIK solver when tracking data for an end effector is available and with a cluster function or key pose match when tracking data for an end effector is unavailable.

10. A system for generating complementary tracking data, the system comprising a processor in communication with a memory that includes processor executable instructions, the instructions comprising:

a set of tracking data, wherein the tracking data relates to the position or motion of one or more parts of the user;

a 3D model, wherein the 3D model is a virtual representation of an extent and a motion of a user;

a library, wherein the library comprises a set of poses, gestures, or both;

a set of complementary tracking data, wherein the set of complementary tracking data comprises a least a portion of at least one selected pose or gesture from the library;

a combined model, the combined model comprising the set of tracking data and the set of complementary data, and

wherein the memory includes processor executable instructions that cause the system to access one or a series of prior 3D models, wherein the prior 3D models were previously communicated for display.

11. The system of claim 10 , wherein the complementary data is generated by a comparison of the set of tracking data to a key pose library.

12. The system of claim 11 , wherein the comparison comprises spaces between two or more key poses in the key pose library and the complementary data is generated to blend between those spaces.

13. The system of claim 10 , wherein complementary tracking data is generated with a FABRIK solver when tracking data for an end effector is available and with a cluster function or key pose match when tracking data for an end effector is unavailable.

14. A system for generating complementary data for a visual display, the system comprising:

(a) one or more electromagnetic emitters and one or more electromagnetic sensors configured to be selectively placed on one or more tracked body parts; and

(b) at least one processor in communication with a memory and with the visual display, the one or more electromagnetic emitters, and the one or more electromagnetic sensors and configured to receive tracking data from the one or more electromagnetic emitters and the one or more electromagnetic sensors, and to generate complementary display data comprising projected motion not within the tracking data and based upon a library of potential motions, and

wherein the memory includes processor executable instructions that cause the system to access one or a series of prior 3D models, wherein the prior 3D models were previously communicated for display.

Assignments (4)
RELEASE OF SECURITY INTEREST Recorded Feb 14, 2024
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: PENUMBRA, INC.
Reel/Frame 066596/0706 →
SECURITY INTEREST Recorded Feb 18, 2022
From: PENUMBRA, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 059052/0806 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2021
From: MVI HEALTH, INC.
To: PENUMBRA, INC.
Reel/Frame 057628/0356 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 17, 2021
From: WINOLD, HANS; WHITE, JOSEPH; CORNIEL, RYAN; GUTENTAG, MARK SAMUEL; LOCKHART, JOHN
To: MVI HEALTH INC.
Reel/Frame 055632/0040 →
Continuity (2)
Provisional Application 62734824 · Sep 21, 2018
Related Publication 20220035443A1 · Feb 3, 2022
Cited By (6)
US 12,220,139 US 12,262,903 US 12,343,024 US 12,575,844 US 12,642,542 US 12,642,543