Granted Patent B2

US 11,496,671 · App. 17/179,786 · Granted Nov 8, 2022

Inventors: Shaomin Xiong (Newark, CA); Toshiki Hirano (San Jose, CA); Qian Zhong (Santa Clara, CA); Haoyu Wu (Sunnyvale, CA); David Berman (San Jose, CA)

Assignee: Western Digital Technologies, Inc.

H04N5/23219G06T7/70H04N5/23296H04N5/23299H04N7/181

Abstract

Claims (99)

1. A system, comprising:

a first video camera;

a second video camera, wherein the second video camera is configured to provide an alternative view of an object in a field of view of the first video camera; and

a controller configured to:

receive video data from the first video camera;

receive video data from the second video camera;

determine a first image of an object in the video data from the first video camera;

determine, based on the video data from the first video camera, a primary video stream;

determine, based on a second image of the object from the second video camera, secondary object data;

determine, based on the secondary object data, a set of object feature data for use in object recognition;

embed the set of object feature data in the primary video stream; and

send, after embedding the set of object feature data, the primary video stream to an analytics engine for processing the set of object feature data.

2. The system of claim 1 , wherein the controller is further configured to: determine, for the first image of the object, an object quality metric; and

initiate, responsive to the object quality metric for the first image of the object failing an object quality threshold, the second video camera to capture the video data from the second video camera.

3. The system of claim 2 , wherein the controller is further configured to:

determine a location of the object in a field of view of the second video camera; and

send, responsive to the object quality metric for the first image of the object failing the object quality threshold, a pan-tilt-zoom position control signal to the second video camera to adjust the alternative view of the object for the second image of the object.

4. The system of claim 1 , wherein the controller is further configured to:

determine, in the video data from the second video camera, an object boundary for the object; and

select, based on the object boundary for the object in the video data from the second video camera, the secondary object data to include object image data within the object boundary.

5. The system of claim 1 , wherein:

the controller is further configured to communicate with the analytics engine over a network; and

sending, from the controller to the analytics engine, the primary video stream with the embedded set of object feature data is over the network.

6. The system of claim 1 , wherein:

the object is a human face;

the set of object feature data includes a set of feature vectors from the human face in the video data from the second video camera; and

processing the set of object feature data includes using a facial recognition model and the set of feature vectors to search facial reference data for recognition of the human face.

7. The system of claim 1 , wherein the controller is further configured to:

determine, in the video data from the first video camera, an object boundary for the object;

determine, in the video data from the first video camera, an embed location for the set of object feature data relative to the object boundary; and

encode, in the primary video stream, the set of object feature data in the embed location within the video data from the first video camera.

8. The system of claim 1 , wherein the controller is further configured to:

configure an object data embed location for embedding the set of object feature data;

determine, in the video data from the first video camera and based on the configured object data embed location, an embed location for the set of object feature data; and

selectively replace, in the embed location, video data from the first video camera with the set of object feature data.

9. The system of claim 1 , wherein the controller is further configured to:

process, using a feature abstraction algorithm, the secondary object data to determine the set of object feature data; and

selectively replace a set of pixels in the primary video stream with a series of data values corresponding to the set of object feature data to embed the set of object feature data.

10. The system of claim 1 , further comprising the analytics engine,

wherein:

the controller is embedded in at least one of:

the first video camera; and

the second video camera; and

the analytics engine is configured to:

receive the primary video stream;

determine, in the primary video stream, an embed location for the set of object feature data;

extract the set of object feature data from the embed location;

search, based on the set of object feature data, object reference data for recognition of the object; and

return an object recognition value for the object.

11. A computer-implemented method, comprising:

receiving first video data from a first video camera;

determining a first image of an object in the first video data;

receiving second video data from a second video camera, wherein the second video camera is configured to provide an alternative view of the object in a field of view of the first video camera;

determining, based on the first video data, a primary video stream;

determining, based on a second image of the object from the second video camera, secondary object data;

determining, based on the secondary object data, a set of object feature data for use in object recognition;

embedding the set of object feature data, in the primary video stream; and

sending, after embedding the set of object feature data, the primary video stream to an analytics engine for processing the secondary object data.

12. The computer-implemented method of claim 11 , further comprising:

determining, for the first image of the object, an object quality metric; and

initiating, responsive to the object quality metric for the first image of the object failing an object quality threshold, the second video camera to capture the second video data.

13. The computer-implemented method of claim 12 , further comprising:

determining a location of the object in a field of view of the second video camera; and

sending, responsive to the object quality metric for the first image of the object failing the object quality threshold, a pan-tilt-zoom position control signal to the second video camera to adjust the alternative view of the object for the second image of the object.

14. The computer-implemented method of claim 11 , further comprising:

determining, in the second video data, an object boundary for the object; and

selecting, based on the object boundary for the object in the second video data, the secondary object data to include object image data within the object boundary.

15. The computer-implemented method of claim 11 , wherein sending, to the analytics engine, the primary video stream with the embedded set of object feature data is over a network.

16. The computer-implemented method of claim 11 , wherein:

the object is a human face;

the set of object feature data includes a set of feature vectors from the human face in the second video data; and

processing the set of object feature data includes using a facial recognition model and the set of feature vectors to search facial reference data for recognition of the human face.

17. The computer-implemented method of claim 11 , further comprising:

determining, in the first video data, an object boundary for the object;

determining, in the first video data, an embed location for the set of object feature data relative to the object boundary; and

encoding, in the primary video stream, the set of object feature data in the embed location with the first video data.

18. The computer-implemented method of claim 11 , further comprising:

configuring an object data embed location for embedding the set of object feature data;

determining, in the first video data and based on the configured object data embed location, an embed location for the set of object feature data; and

selectively replacing, in the embed location, first video data with the set of object feature data.

19. The computer-implemented method of claim 11 , further comprising:

receiving the primary video stream;

determining an embed location for the set of object feature data in the primary video stream;

extracting the set of object feature data from the embed location;

searching, based on the set of object feature data, object reference data for recognition of the object; and

returning an object recognition value for the object.

20. A storage system, comprising:

a first video camera;

a second video camera, wherein the second video camera is configured to provide an alternative view of an object in a field of view of the first video camera;

a processor;

a memory;

means for receiving first video data from the first video camera;

means for determining a first image of an object in the first video data;

means for receiving second video data from the second video camera;

means for determining, based on the first video data, a primary video stream;

means for determining, based on a second image of the object from the second video camera, secondary object data;

means for determining, based on the secondary object data, a set of object feature data for use in object recognition;

means for embedding the set of object feature data in the primary video stream; and

means for sending, after embedding the set of object feature data, the primary video stream to an analytics engine for processing the secondary object data.

Assignments (10)

Continuity (1)

Cited By (1)

US 12,335,483