Combining video streams having different information-bearing levels
A method includes recording a first video stream characterized by a first value of a first quality characteristic. The method includes determining that the first video stream satisfies a trigger criterion. The trigger criterion characterizes a threshold amount of video content change information. The method includes, in response to determining that the first video stream satisfies the trigger criterion, obtaining a second video stream characterized by a second value of a second quality characteristic. The second video stream includes scene information also included in the first video stream. The second value of the second quality characteristic is indicative of a higher quality video stream than the first value of the first quality characteristic. The method includes generating a third video stream by adding information from the second video stream to the first video stream. The third video stream corresponds to a higher quality version of the first video stream.
1 . A method comprising:
at an electronic device with one or more processors, a non-transitory memory, one or more input devices, and a camera system:
recording, using the camera system, a first video stream characterized by a first value of a first quality characteristic; and
in response to detecting, via the one or more input devices, a capture input:
obtaining a first image frame characterized by a second value of a second quality characteristic, wherein the second value of the second quality characteristic is indicative of a higher quality than the first value of the first quality characteristic, and wherein the first image frame includes scene information also included in the first video stream;
storing the first video stream in a buffer; and
generating a second image frame by adding information from the first image frame to the first video stream, wherein the second image frame corresponds to a higher quality version of the first video stream, and wherein generating the second image frame by adding the information from the first image frame to the first video stream includes retrieving the first video stream from the buffer.
2 . The method of claim 1 , further comprising detecting the capture input at a first point in time, wherein the information from the first image frame is added to a portion of the first video stream that is associated with a second point in time, and wherein the first point in time and the second point in time collectively satisfy a temporal proximity threshold.
3 . The method of claim 1 , wherein the electronic device includes a touch-sensitive surface, the method further comprising detecting the capture input on the touch-sensitive surface.
4 . The method of claim 1 , wherein the electronic device corresponds to a head-mountable device (HMD), and wherein the capture input is directed to the HMD.
5 . The method of claim 1 , wherein the camera system records the first video stream in a first mode of operation, and wherein obtaining the first image frame includes capturing the first image frame in a second mode of operation that is different from the first mode of operation.
6 . The method of claim 5 , wherein the first mode of operation is associated with a first power profile, and wherein the second mode of operation is associated with a second power profile that is larger than the first power profile.
7 . The method of claim 5 , wherein the camera system includes a first image sensor configured to operate in the first and second modes of operation.
8 . The method of claim 1 , wherein the camera system includes a first image sensor that records the first video stream, the method further comprising, in response to detecting the capture input, capturing the first image frame via a second image sensor included in the camera system, wherein the second image sensor is different from the first image sensor.
9 . The method of claim 8 , wherein the first image sensor includes a combination of a black-and-white (BW) camera and an infrared (IR) camera.
10 . The method of claim 8 , wherein the second image sensor includes a red-green-blue (RGB) camera.
11 . The method of claim 1 , wherein the electronic device includes a depth sensor that detects depth information, and wherein generating the second image frame includes adding the depth information to the first video stream.
12 . The method of claim 1 , wherein adding the information from the first image frame to the first video stream is performed by a neural network.
13 . The method of claim 1 , wherein the first value of the first quality characteristic corresponds to a first resolution value, and wherein the second value of the second quality characteristic corresponds to a second resolution value that is greater than the first resolution value.
14 . An electronic device comprising:
one or more processors;
a non-transitory memory;
a camera system;
one or more input devices; and
one or more programs, wherein the one or more programs are stored in the non-transitory memory and are configured to be executed by the one or more processors, the one or more programs including instructions for performing or causing performance of a method comprising:
recording, using the camera system, a first video stream characterized by a first value of a first quality characteristic; and
in response to detecting, via the one or more input devices, a capture input:
obtaining a first image frame characterized by a second value of a second quality characteristic, wherein the second value of the second quality characteristic is indicative of a higher quality than the first value of the first quality characteristic, and wherein the first image frame includes scene information also included in the first video stream;
storing the first video stream in a buffer; and
generating a second image frame by adding information from the first image frame to the first video stream, wherein the second image frame corresponds to a higher quality version of the first video stream, and wherein generating the second image frame by adding the information from the first image frame to the first video stream includes retrieving the first video stream from the buffer.
15 . The electronic device of claim 14 , wherein the one or more programs include instructions for performing or causing performance of detecting the capture input at a first point in time, wherein the information from the first image frame is added to a portion of the first video stream that is associated with a second point in time, and wherein the first point in time and the second point in time collectively satisfy a temporal proximity threshold.
16 . The electronic device of claim 14 , wherein the camera system records the first video stream in a first mode of operation that is associated with a first power profile, and wherein obtaining the first image frame includes capturing the first image frame in a second mode of operation that is different from the first mode of operation and that is associated with a second power profile that is larger than the first power profile.
17 . The electronic device of claim 14 , wherein the first value of the first quality characteristic corresponds to a first resolution value, and wherein the second value of the second quality characteristic corresponds to a second resolution value that is greater than the first resolution value.
18 . The electronic device of claim 14 , wherein the electronic device corresponds to a head-mountable device (HMD), and wherein the capture input is directed to the HMD.
19 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by an electronic device comprising a camera system and one or more input devices, cause the electronic device to perform or cause performance of a method comprising:
recording, using the camera system, a first video stream characterized by a first value of a first quality characteristic; and
in response to detecting, via the one or more input devices, a capture input:
obtaining a first image frame characterized by a second value of a second quality characteristic, wherein the second value of the second quality characteristic is indicative of a higher quality than the first value of the first quality characteristic, and wherein the first image frame includes scene information also included in the first video stream;
storing the first video stream in a buffer; and
generating a second image frame by adding information from the first image frame to the first video stream, wherein the second image frame corresponds to a higher quality version of the first video stream, and wherein generating the second image frame by adding the information from the first image frame to the first video stream includes retrieving the first video stream from the buffer.
20 . The non-transitory computer readable storage medium of claim 19 , wherein the one or more programs include instructions for performing or causing performance of detecting the capture input at a first point in time, wherein the information from the first image frame is added to a portion of the first video stream that is associated with a second point in time, and wherein the first point in time and the second point in time collectively satisfy a temporal proximity threshold.