Granted Patent B2

US 12,244,936 · App. 18/148,834 · Granted Mar 4, 2025

Inventors: Andrew Samuel Berkovich (Bellevue, WA); Xinqiao Liu (Medina, WA)

Assignee: Meta Platforms Technologies, LLC

H04N23/84G02B27/017H01L27/14641H01L27/14645H04N25/772G02B2027/0178

Abstract

Claims (53)

1. A sensor apparatus comprising:

an array of pixel cells, each pixel cell of the array of pixel cells including one or more photodiodes configured to generate a charge in response to light, and a charge storage device of one or more charge storage devices to convert the charge to output a voltage of an array of voltages;

one or more analog-to-digital converters (ADC) configured the convert the array of voltages to first pixel data; and

a controller configured to:

receive the first pixel data from the ADC;

input the first pixel data into a machine-learning model to generate output data comprising prediction data associated with one or more features of the first pixel data;

generate, based on the prediction data, second pixel data, the second pixel data associated with one or more transformed features of the first pixel data; and

send, from the sensor apparatus to a separate receiving apparatus, the second pixel data.

2. The apparatus of claim 1 , wherein:

the sensor apparatus is a first sensor apparatus;

the controller is further configured to receive, from a second sensor apparatus, pixel metadata, the pixel metadata associated one or more aspects of third pixel data; and

generating the second pixel data is further based on the pixel metadata.

3. The apparatus of claim 2 , wherein the pixel metadata is input to the machine-learning model to generate the output data.

4. The apparatus of claim 2 , wherein the one or more aspects of the third pixel data comprise one or more features of the third pixel data.

5. The apparatus of claim 1 , wherein the sensor apparatus is a first sensor apparatus and the controller is further configured to:

generate, based on the first pixel data, pixel metadata, the pixel metadata associated with one or more aspects of the first pixel data; and

send, to a second sensor apparatus, the pixel metadata.

6. The apparatus of claim 1 , wherein the sensor apparatus is a first sensor apparatus and the controller is further configured to:

generate, based on the second pixel data, pixel metadata, the pixel metadata associated with one or more aspects of the second pixel data; and

send, to a second sensor apparatus, the pixel metadata.

7. The apparatus of claim 1 , wherein:

the controller is further configured to receive, from an orientation sensor apparatus, orientation metadata, the orientation metadata associated with an orientation of at least the sensor apparatus; and

generating the second pixel data is further based on the orientation metadata.

8. The apparatus of claim 1 , wherein the controller is further configured to input contextual data into the machine-learning model to generate the output data, the contextual data associated with one or more contexts related to the first pixel data.

9. The apparatus of claim 8 , wherein:

the sensor apparatus is a first sensor apparatus; and

the contextual data is sent to the first sensor apparatus from a second sensor apparatus configured to generate the contextual data.

10. The apparatus of claim 8 , wherein the controller is further configured to generate, based on the first pixel data, the contextual data.

11. The apparatus of claim 1 , wherein the first pixel data is first digital pixel data comprising one or more first digital pixel data values representing a first digital image.

12. The apparatus of claim 11 , wherein:

the one or more features of the first pixel data comprise one or more alterable features;

the second pixel data is pixel metadata associated with the first digital pixel data, the pixel metadata comprising at least an indication that the first pixel data comprises the one or more alterable features; and

sending the second pixel data comprises sending the pixel metadata to the separate receiving apparatus instead of the first pixel data.

13. The apparatus of claim 12 , wherein the controller is further configured to, in response to sending the pixel metadata to the separate receiving apparatus, send an indication to the one or more charge storage devices to prevent conversion of the charges to output the array of voltages.

14. The apparatus of claim 13 , wherein the controller is further configured to:

receive, from an orientation sensor apparatus, orientation metadata, the orientation metadata associated with an orientation of at least the sensor apparatus;

determine, based on the orientation metadata, that third pixel data to be converted by the ADC will not include the one or more alterable features; and

send an indication to the one or more charge storage devices to resume conversion of the charges to output the array of voltages.

15. The apparatus of claim 11 , wherein the second pixel data is second digital pixel data comprising one or more second digital pixel data values representing a second digital image, at least a subset of the one or more second digital pixel data values being transformed second digital pixel data values associated with one or more transformed features.

16. The apparatus of claim 15 , wherein generating the second pixel data comprises transforming at least a subset of one or more first digital pixel values into the transformed second digital pixel data values.

17. The apparatus of claim 16 , wherein:

each digital pixel data value of the first digital pixel data values and the second digital pixel data values comprises one or more color values corresponding to a color of a pixel; and

transforming at least the subset of the one or more first digital pixel data values into the transformed second digital pixel data values comprises altering a subset of the one or more color values of the subset of the one or more first digital pixel data values.

18. The apparatus of claim 15 , wherein:

the one or more features of the first pixel data comprise one or more alterable features; and

generating the second pixel data comprises generating the transformed second digital pixel data values associated with the one or more transformed features to replace a subset of one or more first digital pixel data values associated with the one or more alterable features.

19. The apparatus of claim 15 , wherein sending the second pixel data to the separate receiving apparatus comprises sending the one or more second digital pixel data values and not the transformed second digital pixel data values to the separate receiving apparatus.

20. A method comprising:

operating each pixel cell of an array of pixel cells to generate a charge in response to light, and to convert the charge to output a voltage of an array of voltages;

converting an array of voltages to first pixel data;

inputting the first pixel data into a machine-learning model to generate output data comprising prediction data associated with one or more features of the first pixel data;

generating, based on the prediction data, second pixel data, the second pixel data associated with one or more transformed features of the first pixel data; and

sending, the second pixel data to a receiving apparatus.

Assignments (1)

Continuity (2)

References Cited (400)