IP Library Granted Patent US 11,960,638
Granted Patent B2
US 11,960,638 · App. 17/992,648 · Granted Apr 16, 2024

Distributed sensor system

Inventors: Andrew Samuel Berkovich (Bellevue, WA); Xinqiao Liu (Medina, WA); Hans Reyserhove (Seattle, WA)
Assignee: Meta Platforms Technologies, LLC
G06F3/011G06F18/251G06V10/95G06V20/20G06V40/19H04N7/22H04N23/45H04N23/698
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,960,638
App. No.
17/992,648
Granted
Apr 16, 2024
Kind
B2
Abstract

In one example, a mobile device comprises: a physical link; a plurality of image sensors, each image sensor being configured to transmit image data via the physical link; and a controller coupled to the physical link, whereby the physical link, the plurality of image sensors, and the controller form a multi-drop network. The controller is configured to: transmit a control signal to configure image sensing operations at the plurality of image sensors; receive, via the physical link, image data from at least a subset of the plurality of image sensors; combine the image data from the at least a subset of the plurality of image sensors to obtain an extended field of view (FOV); determine information of a surrounding environment of the mobile device captured within the extended FOV; and provide the information to an application to generate content based on the information.

Claims (63)

1. A mobile device comprising:

a physical link;

a plurality of image sensors, each of the plurality of image sensors configured to transmit image data via the physical link; and

a controller coupled to the physical link, whereby the physical link, the plurality of image sensors, and the controller form a multi-drop network;

wherein the controller is configured to:

determine an image capture operation for the plurality of image sensors, wherein the image capture operation configures image data transmitted over the multi-drop network;

transmit a control signal to configure image sensing operations at the plurality of image sensors based on the image capture operation;

receive, via the physical link, the image data from at least a subset of the plurality of image sensors;

determine information about a surrounding environment of the mobile device, wherein the information about the surrounding environment includes a tracking result of an object of interest; and

select the subset of the plurality of image sensors based on determining that the image data generated by the subset of the plurality of image sensors is likely to include the object of interest.

2. The mobile device of claim 1 , wherein based on the image capture operation, the controller is configured to disable the rest of the plurality of image sensors not selected in the subset of the plurality of image sensors.

3. The mobile device of claim 1 , wherein the image data is first image data;

wherein the controller is configured to, based on transmitting the control signal:

control the subset of the plurality of image sensors to transmit the first image data at a first resolution; and

control the rest of the plurality of image sensors to transmit second image data at a second resolution; and

wherein the first resolution is higher than the second resolution.

4. The mobile device of claim 1 , wherein the image data is first image data;

wherein the controller is configured to, based on transmitting the control signal:

control the subset of the plurality of image sensors to transmit the first image data at a first frame rate; and

control the rest of the plurality of image sensors to transmit second image data at a second frame rate; and

wherein the first frame rate is higher than the second frame rate.

5. The mobile device of claim 1 , wherein the controller is configured to determine that the image data generated by the subset of the plurality of image sensors is likely to include the object based on detecting features of the object in prior image data from the subset of the plurality of image sensors.

6. The mobile device of claim 1 , wherein the controller is configured to determine a prior trajectory of relative movement between the object and the mobile device based on prior image data from the plurality of image sensors; and

wherein the controller is configured to:

predict a trajectory of the object based on the prior trajectory; and

determine that the image data generated by the subset of the plurality of image sensors is likely to include the object based on the predicted trajectory.

7. The mobile device of claim 1 , wherein each image sensor of the plurality of image sensors includes an array of pixel cells; and

wherein the controller is configured to, based on the image capturing operation, selectively configure a particular subset of pixel cells of the array of pixel cells in one or more image sensor of the plurality of image sensors to enable.

8. The mobile device of claim 7 , wherein the controller is configured to, based on transmitting the control signal:

enable a first subset of pixel cells of the array of pixel cells of a first image sensor of the plurality of image sensors to transmit first image data via the physical link to the controller; and

enable a second subset of pixel cells of the array of pixel cells of a second image sensor of the plurality of image sensors to transmit second image data via the physical link to the controller; and

wherein the first subset and the second subset are different.

9. The mobile device of claim 7 , wherein the controller is configured to, based on transmitting the control signal:

enable a first subset of pixel cells of the array of pixel cells of a first image sensor of the plurality of image sensors to generate first image data at a first resolution; and

enable a second subset of pixel cells of the array of pixel cells of the first image sensor to generate second image data at a second resolution; and

wherein the first resolution is higher than the second resolution.

10. The mobile device of claim 7 , wherein the controller is configured to, based on transmitting the control signal:

set a first dynamic range of a first subset of pixel cells of the array of pixel cells of a first image sensor of the plurality of image sensors;

set a second dynamic range of a second subset of pixel cells of the array of pixel cells of the first image sensor; and

wherein the first dynamic range is higher than the second dynamic range.

11. The mobile device of claim 7 , wherein the control signal identifies pixel cells of the particular subset of the pixel cells in the array of pixel cells for each image sensor of the one or more image sensors.

12. The mobile device of claim 7 , wherein the control signal includes a guidance signal; and

wherein a first image sensor of plurality of image sensors is configured to determine the particular subset of the pixel cells in the array of pixel cells locally based on the guidance signal.

13. The mobile device of claim 12 , wherein the guidance signal specifies features of an object of interest;

wherein the first image sensor is configured to:

determine a region of interest including the object of interest based on the guidance signal; and

determine the particular subset of the pixel cells in the array of pixel cells based on the region of interest.

14. The mobile device of claim 7 , wherein the particular subset of the pixel cells in the array of pixel cells is determined based on at least one of: a tracking result of an object of interest, or a movement of the mobile device.

15. The mobile device of claim 1 , wherein the image data from at least a first image sensor and a second image sensor of the plurality of image sensors are combined; and

wherein the first image sensor and the second image sensor face different directions.

16. The mobile device of claim 1 , wherein the image data from at least a first image sensor and a second image sensor of the plurality of image sensors are combined;

wherein the first image sensor is configured to capture light of a first frequency range; and

wherein the second image sensor is configured to capture light of a second frequency range different from the first frequency range.

17. The mobile device of claim 1 , wherein the physical link comprises at least one of: a bus based on I3C specification, or an optical link.

18. A method comprising:

determining an image capture operation for a plurality of image sensors of a mobile device, wherein:

the plurality image sensors are part of a multi-drop network; and

the image capture operation reduces a volume of image data transmitted over the multi-drop network by determining a subset of image sensors to capture images of one or more objects of interest;

transmitting a control signal to configure image sensing operations at the plurality of image sensors based on the image capture operation;

receiving, via a physical link, image data from each image sensor of the subset of the plurality of image sensors, wherein the plurality of image sensors and the physical link form the multi-drop network;

determining information about a surrounding environment of the mobile device, wherein the information about the surrounding environment includes a tracking result of an object of interest; and

selecting the subset of the plurality of image sensors based on determining that the image data generated by the subset of the plurality of image sensors is likely to include the object of interest.

19. The method of claim 18 , further comprising: based on transmitting the control signal, disabling the rest of the plurality of image sensors not selected in the subset of the plurality of image sensors.

Continuity (3)
Continuation 17083920 · Oct 29, 2020
Provisional Application 62928233 · Oct 30, 2019
Related Publication 20230080288A1 · Mar 16, 2023
Cited By (2)
US 12,244,936 US 12,282,170