IP Library Granted Patent US 11,836,880
Granted Patent B2
US 11,836,880 · App. 17/572,964 · Granted Dec 5, 2023

Adjusting a digital representation of a head region

Inventors: Eric Sommerlade (Oxford, GB); Alexandros Neophytou (Oxford, GB)
Assignee: RealD Spark, LLC
G06T19/20G06N3/045G06N3/08G06T7/337G06T2207/20081G06T2219/2004G06T2219/2012G06T2219/2021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,836,880
App. No.
17/572,964
Granted
Dec 5, 2023
Kind
B2
Abstract

Methods and devices for generating reference data for adjusting a digital representation of a head region, and methods and devices for adjusting the digital representation of a head region are disclosed. In some arrangements, training data are received. A first machine learning algorithm generates first reference data using the training data. A second machine learning algorithm generates second reference data using the same training data and the first reference data generated by the first machine learning algorithm.

Claims (40)

1. A method of adjusting a digital representation of a head region, the method comprising:

identifying a target patch in the digital representation of the head region, the target patch comprising a target feature of the digital representation of the head region;

deriving a feature vector from plural local descriptors of the target patch;

using the feature vector to select editing instructions from reference data, the reference data comprising editing instructions for a range of possible values of the feature vector;

applying the selected editing instructions to the target patch to adjust the digital representation of the head region, wherein a plurality of the digital representations of the head region are received and adjusted; and

updating a second machine learning algorithm using editing instructions generated by a first machine learning algorithm using one or more of the received and adjusted digital representations.

2. The method of claim 1 , wherein each digital representation is captured from the head region at a different point in time.

3. The method of claim 1 , wherein the updating of the second machine learning algorithm is performed at or near input data frame rate.

4. The method of claim 3 , wherein a subset of the received and adjusted digital representations are used to update the second machine learning algorithm.

5. The method of claim 1 , further comprising receiving image information of the head region from a sensing system.

6. The method of claim 5 , wherein the sensing system comprises a visual camera adapted to receive light in the visible electromagnetic spectrum.

7. The method of claim 6 , wherein the sensing system comprises a depth sensor offset from the visual camera by a predetermined distance, the depth sensor adapted to make depth measurements of the head region.

8. The method of claim 5 , wherein the image information comprises depth information of the head region.

9. The method of claim 1 , wherein the sensing system is a camera system.

10. The method of claim 1 , wherein the editing instructions in the reference data are provided in a compressed representation.

11. The method of claim 10 , wherein the compressed representation comprises one or more of the following: a principle component analysis representation; a wavelet representation; Fourier and/or discrete cosine transform components; and cluster centers.

12. A device for adjusting a digital representation of a head region, the device comprising:

a sensing system arranged to receive a digital representation of a head region; and

a data processing unit communicatively coupled to the sensing system and arranged to process the digital representation of the head region by:

identifying a target patch in the digital representation of the head region, the target patch comprising information about a target feature of the digital representation of the head region;

deriving a feature vector from plural local descriptors of the target patch;

using the feature vector to select editing instructions from reference data, the reference data comprising editing instructions for a range of possible values of the feature vector;

applying the selected editing instructions to the target patch to adjust the digital representation of the head region, wherein a plurality of the digital representations of the head region are received and adjusted; and

updating a second machine learning algorithm using editing instructions generated by a first machine learning algorithm using one or more of the received and adjusted digital representations.

13. The device of claim 12 , wherein each digital representation is captured from the head region at a different point in time.

14. The device of claim 12 , wherein the first machine learning algorithm comprises a first neural network and the second machine learning algorithm comprises a second neural network, wherein the second neural network comprises fewer layers and/or smaller convolution fields than the first neural network.

15. The device of claim 12 , wherein the updating of the second machine learning algorithm is performed at or near input data frame rate.

16. The device of claim 12 , wherein a subset of the received and adjusted digital representations are used to update the second machine learning algorithm.

17. The device of claim 12 , wherein the sensing system comprises a camera system.

18. The device of claim 17 , wherein the camera system comprises first and second cameras facing the head region and offset from each other by a predetermined distance.

19. The device of claim 18 , wherein the first and second cameras have different fields of view.

20. The device of claim 18 , wherein the first and second cameras have different sensing modalities from each other.

21. The device of claim 18 , wherein the first and second cameras have sensing modalities are selected from the group comprising visible light, infrared light, depth, and time-of-flight.

22. The device of claim 12 , further comprising a display, wherein the sensing system is located adjacent to the display.

23. The device of claim 12 , further comprising a display, wherein the sensing system is located offset from the display.

24. The device of claim 12 , further comprising a display, wherein the sensing system is located above the display.

25. The device of claim 12 , wherein the sensing system comprises a visual camera comprising a sensor to receive light in the visible electromagnetic spectrum.

26. The device of claim 25 , wherein the sensing system further comprises a depth sensor offset from the visual camera, the depth sensor adapted to make depth measurements of a head region.

27. The device of claim 12 , wherein the editing instructions in the reference data are provided in a compressed representation.

28. The device of claim 27 , wherein the compressed representation comprises one or more of the following: a principle component analysis representation; a wavelet representation; Fourier and/or discrete cosine transform components; and cluster centers.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 23, 2022
From: SOMMERLADE, ERIC; NEOPHYTOU, ALEXANDROS
To: REALD SPARK, LLC
Reel/Frame 059980/0581 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 28, 2022
From: SOMMERLADE, ERIC; NEOPHYTOU, ALEXANDROS
To: REALD SPARK, LLC
Reel/Frame 059526/0211 →
Continuity (5)
Continuation 16937254 · Jul 23, 2020
Continuation 16057566 · Aug 7, 2018
Provisional Application 62543587 · Aug 10, 2017
Provisional Application 62542661 · Aug 8, 2017
Related Publication 20220207847A1 · Jun 30, 2022