IP Library Granted Patent US 10,372,995
Granted Patent B2
US 10,372,995 · App. 15/092,557 · Granted Aug 6, 2019

System and method for previewing video

Inventors: Junfeng Zhang (Shanghai, CN); Lili Zhao (Shanghai, CN); Feng Li (Shanghai, CN)
Assignee: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
G06K9/00751G06F16/739G11B27/031H04N7/188G06K2009/00738
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,372,995
App. No.
15/092,557
Granted
Aug 6, 2019
Kind
B2
Abstract

A device for previewing a video includes a memory device configured to store instructions, and one or more processors configured to execute the instructions to receive a video captured by a camera and analyze the video. The one or more processors are also configured to detect a special event from the analysis of the video. The special event includes at least one of an object, a moving object, or a sound detected in the video. The one or more processors are further configured to extract, from the video, at least one of a video preview frame or audio segment corresponding to the special event, and generate a preview video including the at least one of the video preview frame or audio segment corresponding to the special event.

Claims (112)

1. A device for previewing a video, the device comprising:

a memory device configured to store instructions; and

one or more processors configured to execute the instructions to:

receive the video captured by a camera, wherein the camera includes an image sensor, a sound sensor, and a motion sensor including an ultrasonic sensor;

obtain information relating to a potential special event detected by the camera, wherein the information relating to the potential special event is obtained by at least one of the motion sensor or the sound sensor, the ultrasonic sensor configured to emit ultrasonic signals to detect an object within a vicinity of the camera based on reflected ultrasonic signals and the motion sensor configured to generate a signal indicating that the object is present, the sound sensor configured to generate a signal if the sounds sensor detects an ambient sound level exceeding a threshold, wherein a processor of the camera is responsive to the signal generated by the motion sensor or the sound sensor to instruct the image sensor to start capturing the video;

analyze the captured video;

detect a special event from the analysis of the video by comparing an identified feature with one or more object, motion, or sound models previously constructed, the special event including at least one of an object, a moving object, or a sound detected in the video;

extract, from the video, at least a first video preview frame or audio segment corresponding to the special event based on information relating to the potential special event, wherein the one or more processors are further configured to skip frames before extracting a second video preview frame or audio segment, depending on whether the special event is detected in the first video preview frame or audio segment, wherein the skipped frames correspond to a time period selected by the one or more processors for skipping frames, a length of the time period being selected based on whether the special event is detected in a previous time period; and

generate a preview video including the at least first video preview frame or audio segment corresponding to the special event.

2. The device of claim 1 , wherein the one or more processors are further configured to execute the instructions to transmit the preview video to a user.

3. The device of claim 1 , wherein the special event is a first special event, the one or more processors being further configured to execute the instructions to:

detect a second special event from the analysis of the video, the second special event including at least one of an object, a moving object, or a sound detected in the video; and

extract, from the video, at least one of a video preview frame or audio segment corresponding to the second special event, wherein the preview video further includes the at least one of the video preview frame or audio segment corresponding to the second special event.

4. The device of claim 1 , wherein the video is a first video and the special event is a first special event, the one or more processors being further configured to execute the instructions to:

receive a second video captured by the camera;

analyze the second video;

detect a second special event from the analysis of the second video, the second special event including at least one of an object, a moving object, or a sound detected in the second video; and

extract, from the second video, at least one of a video preview frame or audio segment corresponding to the second special event, wherein the preview video further includes the at least one of the video preview frame or audio segment corresponding to the second special event.

5. The device of claim 3 , wherein the one or more processors are further configured to execute the instructions to:

receive another video captured by the camera;

analyze the another video;

detect a third special event from the analysis of the another video, the third special event including at least one of an object, a moving object, or a sound detected in the another video; and

extract, from the another video, at least one of a video preview frame or audio segment corresponding to the third special event, wherein the preview video further includes the at least one of the video preview frame or audio segment corresponding to the third special event.

6. A system for previewing a video, the system comprising:

a camera including an image sensor, a sound sensor, a motion sensor including an ultrasonic sensor, and a first processor, the camera configured to:

emit ultrasonic signals, by the ultrasonic sensor, to detect an object within a vicinity of the camera;

detect at least one of an object, by the motion sensor, based on reflected ultrasonic signals or an ambient sound, by the sound sensor, that exceeds a threshold;

generate a signal by the motion sensor indicating that the object is present or a signal by the sound sensor indicating the ambient sound exceeds the threshold;

transmit the signal generated by the motion sensor or the sound sensor to the first processor;

receive an instruction, from the first processor responsive to the signal generated by the motion sensor or the sound sensor, to start capturing a video;

capture, by the image and sound sensors, the video,

transmit a request for uploading the video,

receive a response including an address for uploading the video, and upload the video; and

a processing server including a second processor, the processing server configured to:

receive the video,

analyze the video,

detect a special event from the analysis of the video by comparing an identified feature with one or more object, motion, or sound models previously constructed, the special event including at least one of an object, a moving object, or a sound detected in the video,

extract, from the video, at least a first video preview frame or audio segment corresponding to the special event, wherein the processing server is further configured to skip frames before extracting a second video preview frame or audio segment, depending on whether the special event is detected in the first video preview frame or audio segment, wherein the skipped frames correspond to a time period selected by the processing server for skipping frames, a length of the time period being selected based on whether the special event is detected in a previous time period, and

generate a preview video including the at least first video preview frame or audio segment corresponding to the special event.

7. The system of claim 6 , further comprising a distributing server, including a third processor, the distributing server configured to:

receive, from the camera, the request for uploading the first video, and

generate, in response to the request, the response including the address for uploading the video, wherein

the camera is further configured to upload the video to the address included in the response, and

the processing server is further configured to receive the video from the address to which the first video is uploaded.

8. The system of claim 6 , wherein the camera is further configured to:

receive a signal indicating the potential special event including at least one of a potential object, a potential moving object, or a potential sound; and

capture the video including the potential special event.

9. The system of claim 8 , wherein:

the camera is further configured to:

obtain information relating to the potential special event, and

transmit, to the processing server, the information relating to the potential special event; and

the processing server is further configured to:

receive the information relating to the potential special event, and

detect the special event based, at least in part, on the information relating to the potential special event.

10. The system of claim 6 , wherein the special event is a first special event, and the processing server is further configured:

detect a second special event from the analysis of the video, the second special event including at least one of an object, a moving object, or a sound detected in the video; and

extract, from the video, at least one of a video preview frame or audio segment corresponding to the second special event, wherein the preview video further includes the at least one of the video preview frame or audio segment corresponding to the second special event.

11. A method for previewing a video, the method comprising:

receiving the video captured by a camera, wherein the camera includes an image sensor, a sound sensor, and a motion sensor including an ultrasonic sensor;

obtaining information relating to a potential special event detected by the camera, wherein the information relating to the potential special event is obtained by at least one of the motion sensor or the sound sensor, the ultrasonic sensor configured to emit ultrasonic signals to detect an object within a vicinity of the camera based on reflected ultrasonic signals and the motion sensor configured to generate a signal indicating that the object is present, the sound sensor configured to generate a signal if the sounds sensor detects an ambient sound level exceeding a threshold, wherein a processor of the camera is responsive to the signal generated by the motion sensor or the sound sensor to instruct the image sensor to start capturing the video;

analyzing the captured video;

detecting a special event from the analysis of the video by comparing an identified feature with one or more object, motion, or sound models previously constructed, the special event including at least one of an object, a moving object, or a sound detected in the video;

extracting, from the video, at least a first video preview frame or audio segment corresponding to the special event based on information relating to the potential special event, and skipping frames before extracting a second video preview frame or audio segment, depending on whether the special event is detected in the first video preview frame or audio segment, wherein the skipped frames correspond to a time period selected for skipping frames, a length of the time period being selected based on whether the special event is detected in a previous time period; and

generating a preview video including the at least first video preview frame or audio segment corresponding to the special event.

12. The method of claim 11 , further comprising transmitting the preview video to a user.

13. The method of claim 11 , wherein the special event is a first special event, the method further comprising:

detecting a second special event from the analysis of the video, the second special event including at least one of an object, a moving object, or a sound detected in the video; and

extracting, from the video, at least one of a video preview frame or audio segment corresponding to the second special event, wherein the preview video further includes the at least one of the video preview frame or audio segment corresponding to the second special event.

14. The method of claim 11 , wherein the video is a first video and the special event is a first special event, the method further comprising:

receiving a second video captured by the camera;

analyzing the second video;

detecting a second special event from the analysis of the second video, the second special event including at least one of an object, a moving object, or a sound detected in the second video; and

extracting, from the second video, at least one of a video preview frame or audio segment corresponding to the second special event, wherein the preview video further includes the at least one of the video preview frame or audio segment corresponding to the second special event.

15. The method of claim 13 , further comprising:

receiving another video captured by the camera;

analyzing the another video;

detecting a third special event from the analysis of the another video, the third special event including at least one of an object, a moving object, or a sound detected in the another video; and

extracting, from the another video, at least one of a video preview frame or audio segment corresponding to the third special event, wherein the preview video further includes the at least one of the video preview frame or audio segment corresponding to the third special event.

16. A method for previewing a video, the method comprising:

emitting ultrasonic signals, by an ultrasonic sensor, to detect an object within a vicinity of a camera, wherein the camera includes an image sensor, a sound sensor, a motion sensor including the ultrasonic sensor, and a camera processor;

at least one of detecting an object based on reflected ultrasonic signals or detecting by the sound sensor if an ambient sound level exceeds a threshold;

at least one of generating a signal, by the motion sensor, indicating that the object is present or generating a signal, by the sound sensor, indicating the ambient sound level exceeds the threshold;

receiving an instruction, from the camera processor responsive to the signal generated by the motion sensor or the sound sensor, to start capturing the video by the image sensor and the sound sensor;

capturing, by the camera, the video;

transmitting, by the camera, a request for uploading the video;

receiving, by the camera, a response including an address for uploading the video;

transmitting, by the camera, the video to the address included in the response;

receiving, by a processing server, the video;

analyzing, by the processing server, the video;

detecting, by the processing server, a special event from the analysis of the video by comparing an identified feature with one or more object, motion, or sound models previously constructed, the special event including at least one of an object, a moving object, or a sound detected in the video;

extracting, by the processing server, from the video at least a first video preview frame or audio segment corresponding to the special event, wherein the processing server is further configured to skip frames before extracting a second video preview frame or audio segment, depending on whether the special event is detected in the first video preview frame or audio segment, wherein the skipped frames correspond to a time period selected by the processing server for skipping frames, a length of the time period being selected based on whether the special event is detected in a previous time period; and

generating, by the processing server, a preview video including the at least first video preview frame or audio segment corresponding to the special event.

17. The method of claim 16 , further comprising:

receiving, by a distributing server from the camera, the request for uploading the video, and

generating, by the distributing server, the response including the address for uploading the video, wherein

the camera uploads the video to the address included in the response, and

the processing server receives the video from the address to which the video is uploaded.

18. The method of claim 16 , further comprising:

receiving, by the camera, a signal indicating the potential special event including at least one of a potential object, a potential moving object, or a potential sound; and

capturing, by the camera, the video including the potential special event.

19. The method of claim 18 , further comprising:

obtaining, by the camera, information relating to the potential special event;

transmitting, from the camera to the processing server, the information relating to the potential special event;

receiving, by the processing server, the information relating to the potential special event; and

detecting, by the processing server, the special event based, at least in part, on the information relating to the potential special event.

20. A non-transitory computer readable medium embodying a computer program product, the computer program product comprising instructions configured to cause a computing device to:

receive the video captured by a camera, wherein the camera includes an image sensor, a sound sensor, and a motion sensor including an ultrasonic sensor;

obtain information relating to a potential special event detected by the camera, wherein the information relating to the potential special event is obtained by at least one of the motion sensor or the sound sensor, the ultrasonic sensor configured to emit ultrasonic signals to detect an object within a vicinity of the camera based on reflected ultrasonic signals and the motion sensor configured to generate a signal indicating that the object is present, the sound sensor configured to generate a signal if the sounds sensor detects an ambient sound level exceeding a threshold, wherein a processor of the camera is responsive to the signal generated by the motion sensor or the sound sensor to instruct the image sensor to start capturing the video;

analyze the captured video;

detect a special event from the analysis of the video by comparing an identified feature with one or more object, motion, or sound models previously constructed, the special event including at least one of an object, a moving object, or a sound detected in the video;

extract, from the video, at least a first video preview frame or audio segment corresponding to the special event based on information relating to the potential special event, wherein the instructions further cause the computing device to skip frames before extracting a second video preview frame or audio segment, depending on whether the special event is detected in the first video preview frame or audio segment, wherein the skipped frames correspond to a time period selected for skipping frames, a length of the time period being selected based on whether the special event is detected in a previous time period; and

generate a preview video including the at least first video preview frame or audio segment corresponding to the special event.

Assignments (5)
RELEASE OF SECURITY INTEREST Recorded Nov 5, 2024
From: EAST WEST BANK
To: KAMI VISION INCORPORATED
Reel/Frame 070792/0551 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Mar 28, 2022
From: KAMI VISION INCORPORATED
To: EAST WEST BANK
Reel/Frame 059512/0101 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2022
From: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
To: KAMI VISION INC.
Reel/Frame 059275/0652 →
CORRECTIVE ASSIGNMENT TO CORRECT THE CORRECTING RECEIVING PARTY'S NAME TO "SHANGHAI XIAOYI TECHNOLOGY CO., LTD." PREVIOUSLY RECORDED ON REEL 038211 FRAME 0043. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 30, 2018
From: ZHANG, JUNFENG; ZHAO, LILI; LI, FENG
To: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 045195/0546 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2016
From: ZHANG, JUNFENG; ZHAO, LILI; LI, FENG
To: XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 038211/0043 →
Priority Claims (1)
CN 2016 1 0018961 · Jan 12, 2016 · national
Continuity (1)
Related Publication 20170200050A1 · Jul 13, 2017