System and method for generating astro-lapse video on user device
A method ( 1300 ) and system ( 201 ) for generating an astro-lapse video on a user device is disclosed. Initially, at least one current frame comprising one or more stars, is captured via the user device. Further, region(s) with a degree of exposure and focus greater or less than a predefined threshold are determined and a set of subsequent frames including the stars are captured. The exposure and focus of region(s) in the subsequent frames is corrected during the capturing based on degree of exposure and focus of the region(s) within the current frame. Further, fusion weights of the current and subsequent frames are determined based on one or more cloud categories and associated cloud density scores. Finally, an astro-lapse video is generated by fusing the at least one current frame and the set of subsequent frames based on the determined fusion weights.
1 . A method for generating an astro-lapse video on a user device, the method comprising:
obtaining at least one current frame by capturing one or more stars via a camera of the user device, wherein the at least one current frame comprises the one or more stars;
determining one or more regions within the at least one current frame, based on a degree of exposure and focus corresponding to the one or more regions;
obtaining a set of subsequent frames comprising the one or more stars by capturing the one or more stars via the camera, wherein the exposure and focus of corresponding to one or more regions within the set of subsequent frames is corrected during the capturing based on the degree of exposure and focus corresponding to the one or more regions within the at least one current frame;
determining fusion weights of the at least one current frame and the set of subsequent frames based on one or more cloud categories and a cloud density score associated with each of the one or more cloud categories; and
generating an astro-lapse video by fusing the at least one current frame and the set of subsequent frames based on the determined fusion weights.
2 . The method of claim 1 , wherein the degree of exposure and focus is determined based on frame segmentation and a true north proximity factor associated with the one or more stars in the at least one current frame.
3 . The method of claim 2 , further comprising:
performing the frame segmentation for at least one of the at least one current frame or the set of subsequent frames by segmenting the corresponding frame into one or more areas and classifying each area into one or more objects.
4 . The method of claim 3 , wherein determining the one or more regions comprises:
identifying a preferred type of area and one or more non-preferred types of areas from the one or more segmented areas;
dividing the preferred type of area into a plurality of blocks;
comparing a brightness of each block with an average brightness of the plurality of blocks to obtain the degree of exposure and focus; and
determining, based on the comparison, the one or more regions corresponding to one or more blocks with the degree of exposure and focus being one of greater or less than a predefined threshold.
5 . The method of claim 4 , comprising:
classifying each block of the corresponding frame into one or more cloud categories;
determining, for each cloud category, the cloud density score; and
determining the fusion weights for the corresponding frames, based on the cloud classification and the cloud density score.
6 . The method of claim 1 , wherein generating the astro-lapse video comprises:
correcting a star shape associated with the one or more stars;
estimating a star trajectory associated with the one or more stars; and
generating the astro-lapse video by fusing the at least one current frame and the set of subsequent frames based on the determined fusion weights, the corrected star shape, and the estimated star trajectory.
7 . The method of claim 6 , wherein correcting the star shape comprises:
correcting a shape of at least one star of the one or more stars in the at least one current frame and the set of subsequent frames to obtain a plurality of star shape corrected frames;
assigning trail length weight to each star in a corresponding frame based on an angular separation;
estimating, for each star in the corresponding frames, a star trajectory based on the celestial positioning information of the one or more stars; and
obtaining a pixel location of each star in the corresponding frames based on the determined celestial positioning information of the respective star and a transformation map associated with celestial positioning information of the one or more stars.
8 . The method of claim 7 , comprising:
correcting a star trail of at least one star of the one or more stars, based on a corresponding star trajectory of the one or more star trajectories, to obtain a plurality of star trail corrected frames.
9 . The method of claim 6 , comprising:
receiving an input to generate colored astro-lapse videos; and
generating a colored astro-lapse by fusing the at least one current frame and the set of subsequent frames based on the determined fusion weights, the corrected star shape, and the estimated star trajectory along with a predetermined color map, wherein the predetermined color map is applied to the corresponding pixel location of the one or more stars in the plurality of frames.
10 . The method of claim 7 , wherein the fusing comprises:
fusing the at least one current frame and the set of subsequent frames, using a custom alpha blending technique, based on the plurality of star shape corrected frames, corresponding trail length weight, the plurality of star trail corrected frames, and the determined fusion weights.
11 . The method of claim 1 , comprising:
providing, to a camera hardware abstraction layer (HAL) of the user device, information associated with the one or more regions having the exposure and focus being one of greater or less than the threshold degree of exposure and focus; and
correcting the degree of exposure and focus of the determined one or more regions in a set of subsequent frames captured by the camera.
12 . A system for generating an astro-lapse video on a user device, the system comprises:
memory storing instructions;
at least one processor;
wherein the instructions, when executed by the at least one processor, causes the system to:
obtain at least one current frame by capturing one or more stars via a camera of the user device, wherein the at least one current frame comprises the one or more stars;
determine one or more regions within the at least one current frame, based on a degree of exposure and focus;
obtain a set of subsequent frames comprising the one or more stars by capturing the one or more stars via the camera, wherein the exposure and focus of corresponding one or more regions within the set of subsequent frames is corrected during the capturing based on the degree of exposure and focus corresponding to the one or more regions within the at least one current frame;
determine fusion weights of the at least one current frame and the set of subsequent frames based on one or more cloud categories and a cloud density score associated with each of the one or more cloud categories; and
generate an astro-lapse video by fusing the at least one current frame and the set of subsequent frames based on the determined fusion weights.
13 . The system of claim 12 , wherein the degree of exposure and focus is determined based on frame segmentation and a true north proximity factor associated with the one or more stars in the at least one current frame.
14 . The system of claim 13 , wherein the instructions, when executed by the at least one processor, causes the system to:
perform the frame segmentation for at least one of the at least one current frame or the set of subsequent frames by segmenting the corresponding frame into one or more areas and classifying each area into one or more objects.
15 . The system of claim 14 , wherein to determine the one or more regions, the instructions, when executed by the at least one processor, causes the system to:
identify a preferred type of area and one or more non-preferred types of areas from the one or more segmented areas;
divide the preferred type of area into a plurality of blocks;
compare a brightness of each block with an average brightness of the plurality of blocks to obtain the degree of exposure and focus; and
determine, based on the comparison, the one or more regions corresponding to one or more blocks with the degree of exposure and focus being one of greater or less than a predefined threshold.
16 . The system of claim 15 , wherein the instructions, when executed by the at least one processor, causes the system to:
classify each block of the corresponding frame into one or more cloud categories;
determine, for each cloud category, the cloud density score; and
determine the fusion weights for the corresponding frames, based on the cloud classification and the cloud density score.
17 . The system of claim 12 , wherein to generate the astro-lapse video, the instructions, when executed by the at least one processor, causes the system to:
correct a star shape associated with the one or more stars;
estimate a star trajectory associated with the one or more stars; and
generate the astro-lapse video by fusing the at least one current frame and the set of subsequent frames based on the determined fusion weights, the corrected star shape, and the estimated star trajectory.
18 . The system of claim 17 , wherein to correct the star shape, the instructions, when executed by the at least one processor, causes the system to:
correct a shape of at least one star of the one or more stars in the at least one current frame and the set of subsequent frames to obtain a plurality of star shape corrected frames;
assign trail length weight to each star in a corresponding frame based on an angular separation;
estimate, for each star in the corresponding frames, a star trajectory based on the celestial positioning information of the one or more stars; and
obtain a pixel location of each star in the corresponding frames based on the determined celestial positioning information of the respective star and a transformation map associated with celestial positioning information of the one or more stars.
19 . The system of claim 18 , wherein the instructions, when executed by the at least one processor, causes the system to:
correct a star trail of at least one star of the one or more stars, based on a corresponding star trajectory of the one or more star trajectories, to obtain a plurality of star trail corrected frames.
20 . The system of claim 17 , wherein the instructions, when executed by the at least one processor, causes the system to:
receive an input to generate colored astro-lapse videos; and
generate a colored astro-lapse by fusing the at least one current frame and the set of subsequent frames based on the determined fusion weights, the star shape correction associated with the one or more stars, and a star trajectory estimation associated with the one or more stars along with a predetermined color map, wherein the predetermined color map is applied to the corresponding pixel location of the one or more stars in the plurality of frame.