Granted Patent B2

US 12,597,162 · App. 17/721,915 · Granted Apr 7, 2026

Inventors: Katerina Patrice Schulz (San Francisco, CA); Vishal Mandal (Ypsilanti, MI); Abhishek Baral (Farmington Hills, MI); Cyrus Blankinship (Brooklyn, NY); David Hahn Clifford (Royal Oak, MI)

Assignee: Cavnue Technology, LLC

G06T7/80G01S19/485G06T11/40G06T2207/30184G06T2207/30236

Abstract

Claims (90)

1 . A computer-implemented method comprising:

obtaining, by one or more processors, a plurality of first image frames from a camera that is monitoring a specific geographical area from an elevated viewpoint;

for each first image frame of the plurality of first image frames from the camera that is monitoring the specific geographical area from the elevated viewpoint:

generating, by the one or more processors, a second image frame from the first image frame from the camera that is monitoring the specific geographical area from the elevated viewpoint by generating vertical projection of the first image frame to a birds-eye view;

generating, by the one or more processors, a third image frame from the second image frame that is a vertical projection of the first image frame by converting pixel values of the second image frame to produce rasterized pixel data in the third image frame; and

identifying, by the one or more processors, photonic content from the rasterized pixel data in the third image frame;

determining, by the one or more processors, one or more invariant components classified as stationary in a plurality of third image frames based on the photonic content identified in subsequent third image frames;

determining, by the one or more processors, one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames;

comparing, by the one or more processors, the one or more position changes of the camera to a threshold value; and

in response to determining the one or more position changes of the camera satisfies the threshold value, adjusting, by the one or more processors, a position of the camera.

2 . The computer-implemented method of claim 1 , wherein obtaining the plurality of first image frames from the camera monitoring the specific geographic area comprises:

determining, by the one or more processors, one or more characteristics of the camera based on content of the plurality of first image frames; and

adjusting, by the one or more processors, the one or more characteristics of the camera for monitoring the specific geographical area.

3 . The computer-implemented method of claim 1 , wherein

obtaining, by the one or more processors, a plurality of fourth image frames of the specific geographic area, wherein each fourth image frame of the plurality of fourth image frames include content of the specific geographical area and imagery of the camera;

identifying, by the one or more processors, one or more geographical components in each first image frame of the plurality of first image frames using geographical content from the plurality of fourth image frames; and

determining, by the one or more processors, a geographical field of view of the camera based on the one or more geographical components in each first image frame of the plurality of first image frames.

4 . The computer-implemented method of claim 3 , wherein the plurality of fourth image frames are obtained from at least one of a satellite, an unmanned aerial vehicle, and an airplane.

5 . The computer-implemented method of claim 1 , wherein generating the second image frame from the first image frame by adjusting the elevated viewpoint of the first image frame comprises:

transposing, by the one or more processors, the first image frame by an orthogonal projection to the second image frame, wherein the second image frame is a 90 degree rotated viewpoint of the first image frame; and

assigning, by the one or more processors, data indicative of cardinality to the second image frame.

6 . The computer-implemented method of claim 5 , wherein identifying the photonic content from the rasterized pixel data in the third image frame comprises:

identifying, by the one or more processors, one or more edges in the third image frame;

identifying, by the one or more processors, one or more contours in the third image frame; and

identifying, by the one or more processors, a specific color in the third image frame.

7 . The computer-implemented method of claim 1 , wherein determining the one or more invariant components classified as stationary in the plurality of third image frames based on the photonic content identified in subsequent third image frames comprises:

obtaining, by the one or more processors, the plurality of third image frames by aggregating each subsequent third image frame; and

detecting, by the one or more processors, one or more static features in each of the plurality of third image frames by (i) comparing features between subsequent third image frames, (ii) identifying the one or more static features between the subsequent third image frames of the features that do not change location, and (iii) identifying one or more non static features between the subsequent third image frames of the features that do change location.

8 . The computer-implemented method of claim 7 , wherein determining the one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames further comprises:

determining, by the one or more processors, a location for each of the one or more static features in the plurality of third image frames;

for each third image frame of the plurality of third image frames:

determining, by the one or more processors, a difference amount between the location of each of the one or more static features in between an initial third image frame and a subsequent third image frame;

comparing, by the one or more processors, the difference amount to a second threshold value; and

in response to determining the difference amount satisfies the second threshold value, identifying, by the one or more processors, a position change of the camera.

9 . The computer-implemented method of claim 7 , wherein determining the one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames further comprises:

determining, by the one or more processors, a location for each of the one or more static features in the plurality of third image frames;

for each third image frame of the plurality of third image frames:

determining, by the one or more processors, a difference amount between the location of each of the one or more static features in between the third image frame and an initial raster frame;

comparing, by the one or more processors, the difference amount to a threshold value; and

in response to determining the difference amount satisfies the threshold value, identifying, by the one or more processors, a position change of the camera from an initial calibrated position.

10 . The computer-implemented method of claim 1 , wherein adjusting the position of the camera further comprises at least one of:

recalibrating, by the one or more processors, optical characteristics of the camera to match to an initial calibrated position of the camera; or

adjusting, by the one or more processors, the position of the camera to the initial calibrated position of the camera.

11 . The computer-implemented method of claim 1 , wherein the one or more invariant components represent geographic components comprising at least one of a pole, a lane marker, a hill, a median, crosswalks, mountain, and a tree.

12 . A system comprising:

one or more processors;

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

obtaining, by the one or more processors, a plurality of first image frames from a camera that is monitoring a specific geographical area from an elevated viewpoint;

for each first image frame of the plurality of first image frames from the camera that is monitoring the specific geographical area from the elevated viewpoint:

generating, by the one or more processors, a second image frame from the first image frame from the camera that is monitoring the specific geographical area from the elevated viewpoint by generating vertical projection of the first image frame to a birds-eye view;

generating, by the one or more processors, a third image frame from the second image frame that is a vertical projection of the first image frame by converting pixel values of the second image frame to produce rasterized pixel data in the third image frame; and

identifying, by the one or more processors, photonic content from the rasterized pixel data in the third image frame;

determining, by the one or more processors, one or more invariant components classified as stationary in a plurality of third image frames based on the photonic content identified in subsequent third image frames;

determining, by the one or more processors, one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames;

comparing, by the one or more processors, the one or more position changes of the camera to a threshold value; and

in response to determining the one or more position changes of the camera satisfies the threshold value, adjusting, by the one or more processors, a position of the camera.

13 . The system of claim 12 , wherein obtaining the plurality of first image frames from the camera monitoring the specific geographic area further comprises:

determining, by the one or more processors, one or more characteristics of the camera based on content of the plurality of first image frames; and

adjusting, by the one or more processors, the one or more characteristics of the camera for monitoring the specific geographical area.

14 . The system of claim 12 , wherein

obtaining, by the one or more processors, a plurality of fourth image frames of the specific geographic area, wherein each fourth image frame of the plurality of fourth image frames include content of the specific geographical area and imagery of the camera;

identifying, by the one or more processors, one or more geographical components in each first image frame of the plurality of first image frames using geographical content from the plurality of fourth image frames; and

determining, by the one or more processors, a geographical field of view of the camera based on the one or more geographical components in each first image frame of the plurality of first image frames.

15 . The system of claim 14 , wherein the plurality of fourth image frames are obtained from at least one of a satellite, an unmanned aerial vehicle, and an airplane.

16 . The system of claim 12 , wherein generating the second image frame from the first image frame by adjusting the elevated viewpoint of the first image frame comprises:

transposing, by the one or more processors, the first image frame by an orthogonal projection to the second image frame, wherein the second image frame is a 90 degree rotated viewpoint of the first image frame; and

assigning, by the one or more processors, data indicative of cardinality to the second image frame.

17 . The system of claim 16 , wherein identifying the photonic content from the rasterized pixel data in the third image frame comprises:

identifying, by the one or more processors, one or more edges in the third image frame;

identifying, by the one or more processors, one or more contours in the third image frame; and

identifying, by the one or more processors, a specific color in the third image frame.

18 . The system of claim 12 , wherein determining the one or more invariant components classified as stationary in the plurality of third image frames based on the photonic content identified in subsequent third image frames further comprises:

obtaining, by the one or more processors, the plurality of third image frames by aggregating each subsequent third image frame; and

detecting, by the one or more processors, one or more static features in each of the plurality of third image frames by (i) comparing features between subsequent third image frames, (ii) identifying the one or more static features between the subsequent third image frames of the features that do not change location, and (iii) identifying one or more non static features between the subsequent third image frames of the features that do change location.

19 . The system of claim 18 , wherein determining the one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames comprises:

determining, by the one or more processors, a location for each of the one or more static features in the plurality of third image frames;

for each third image frame of the plurality of third image frames:

determining, by the one or more processors, a difference amount between the location of each of the one or more static features in between an initial third image frame and a subsequent third image frame;

comparing, by the one or more processors, the difference amount to a threshold value; and

in response to determining the difference amount satisfies the threshold value, identifying, by the one or more processors, a position change of the camera.

20 . A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

obtaining, by one or more processors, a plurality of first image frames from a camera that is monitoring a specific geographical area from an elevated viewpoint;

for each first image frame of the plurality of first image frames from the camera that is monitoring the specific geographical area from the elevated viewpoint:

generating, by the one or more processors, a second image frame from the first image frame from the camera that is monitoring the specific geographical area from the elevated viewpoint by generating vertical projection of the first image frame to a birds-eye view;

generating, by the one or more processors, a third image frame from the second image frame that is a vertical projection of the first image frame by converting pixel values of the second image frame to produce rasterized pixel data in the third image frame; and

identifying, by the one or more processors, photonic content from the rasterized pixel data in the third image frame;

determining, by the one or more processors, one or more invariant components classified as stationary in a plurality of third image frames based on the photonic content identified in subsequent third image frames;

determining, by the one or more processors, one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames;

comparing, by the one or more processors, the one or more position changes of the camera to a threshold value; and

in response to determining the one or more position changes of the camera satisfies the threshold value, adjusting, by the one or more processors, a position of the camera.

Assignments (5)

Continuity (1)

References Cited (16)