Granted Patent B2

US 12,646,314 · App. 17/783,998 · Granted Jun 2, 2026

Inventors: Weronika Nowicka (Fareham, GB); Mark Johnson (Vannes Cedex, FR)

Assignee: Raymarine UK Limited

G06V20/00G05D1/622G06T3/40G06V10/454G06V10/82G06V20/05G06V20/41

Abstract

Claims (60)

1 . An apparatus comprising:

an imaging system mounted to a mobile structure, the imaging system comprising image capture components configured to rotate and generate a stream of 360-degree images over time, each image corresponding to a field of view from the mobile structure of a water surface and a horizon; and

a logic device configured to:

analyze a selected image from the stream of 360-degree images to detect the horizon;

define a plurality of image tiles in the selected image along the detected horizon to detect far objects on the water surface along the horizon, wherein at least one image tile in the plurality of image tiles overlaps with at least one other image tile in the plurality of image tiles, wherein the plurality of image tiles span the detected horizon, and wherein each tile in the plurality of image tiles includes a first portion below the detected horizon including the water surface and a second portion above the detected horizon;

analyze each image tile to detect and classify an object therein;

generate a first set of object data for the selected image representing far objects detected in the plurality of image tiles;

determine a route for the mobile structure based on the first set of object data; and

navigate the mobile structure along the determined route.

2 . The apparatus of claim 1 , wherein the logic device is further configured to analyze each image tile to detect and classify the object therein by inputting each image tile to a convolutional neural network trained to detect and classify the far objects therein.

3 . The apparatus of claim 1 , wherein the logic device is further configured to execute a near object detection process configured to:

downsample the selected image to a low-resolution image for input to a convolutional neural network trained to detect and classify near objects therein;

detect and classify one or more near objects in the downsampled selected image;

generate a second set of object data for the image; and

fuse the first set of object data representing the far objects detected on the surface of the water and the second set of object data representing the near objects detected on the surface of the water.

4 . The apparatus of claim 1 , wherein the logic device is further configured to determine a location of a detected object using the first set of object data; and

wherein to determine the route the logic device is further configured to determine the route for the mobile structure based, at least in part, on the location of the detected object.

5 . The apparatus of claim 1 , further comprising a user interface comprising a display wherein the logic device is configured to display the stream of 360-degree images, an object identifier, and other object detection data.

6 . The apparatus of claim 1 , wherein the logic device is further configured to:

receive navigation data;

generate and/or transmit maneuvering and/or monitoring instructions for the mobile structure based on the received navigation data; and

monitor the navigation data as the mobile structure traverses an associated navigation path for a detected object.

7 . A method comprising:

capturing a stream of images over time corresponding to a field of view from a mobile structure of a water surface and a horizon;

analyzing a selected image from the stream of images to detect the horizon;

processing the selected image using a first path, wherein the first path comprises:

downsampling the selected image to a low-resolution image;

generating, using a convolutional neural network, first metadata corresponding to one or more first objects identified in the downsampled selected image that are on the water surface and are near to the mobile structure;

processing the selected image using a second path, wherein the second path comprises:

defining a plurality of image tiles in the selected image along the detected horizon to detect far objects on the water surface along the horizon, wherein each tile in the plurality of image tiles includes a first portion below the detected horizon including the water surface and a second portion above the detected horizon;

generating, using the convolutional neural network, second metadata from the plurality of image tiles, the second metadata corresponding to one or more second objects identified in the plurality of image tiles that are on the water surface and are the far objects along the horizon;

combining the first metadata and the second metadata, wherein the combined metadata corresponds to the one or more first and second objects in the selected image that are on the water surface;

determine a route for the mobile structure based on the one or more first and second objects in the selected image; and

navigate the mobile structure along the determined route.

8 . The method of claim 7 , wherein capturing the stream of images further comprises capturing a stream of 360-degree images using at least one imaging device mounted to the mobile structure, wherein the 360-degree images comprise image data corresponding to the water surface and the horizon surrounding the mobile structure.

9 . The method of claim 7 , wherein generating the second metadata further comprises inputting each image tile to the convolutional neural network trained to detect and classify objects therein.

10 . The method of claim 7 , wherein the plurality of image tiles comprises overlapping subregions of the selected image spanning the detected horizon and including the first portion below the detected horizon including the water surface and the second portion above the detected horizon.

11 . The method of claim 7 , wherein the combined metadata further comprises a location and classification of detected objects.

12 . The method of claim 7 , further comprising:

determining a location of a detected object using the one or more first and second objects; and

wherein to determine the route further comprises determining the route for the mobile structure based, at least in part, on the location of the detected object.

13 . An apparatus comprising:

an imaging system mounted to a mobile structure, the imaging system comprising image capture components configured to rotate and generate a stream of 360-degree images over time corresponding to a field of view from the mobile structure of a water surface and a horizon surrounding the mobile structure; and

a logic device configured to:

analyze the stream of 360-degree images to detect the horizon and the water surface below the horizon;

define a plurality of image tiles in the stream of 360-degree images along the detected horizon to detect far objects on the water surface along the horizon, wherein each image tile in the plurality of image tiles overlaps with at least one other image tile in the plurality of image tiles and wherein each image tile includes a first portion below the detected horizon including the water surface and a second portion above the detected horizon;

detect one or more far objects on the water surface along the horizon in the plurality of image tiles using a marine video analytics engine;

classify each detected object using a convolutional neural network;

generate co-ordinates of a location of each classified object for the stream of 360-degree images; and

determine a route for the mobile structure based, at least in part, on at least one of the co-ordinates of the location of each classified object; and

navigate the mobile structure along the determined route.

14 . The apparatus of claim 13 , wherein the image capture components are mounted to the mobile structure and configured to capture the stream of 360-degree images comprising image data corresponding to the water surface and the horizon surrounding the mobile structure.

15 . The apparatus of claim 13 , wherein the logic device is further configured to:

select a location of a classified object; and

wherein to determine the route the logic device is further configured to determine the route for the mobile structure based, at least in part, on the location of the selected classified object.

16 . The apparatus of claim 13 , further comprising a user interface comprising a display wherein the logic device is configured to display the stream of images, an object identifier, and other object detection data.

17 . The apparatus of claim 13 , wherein the logic device is further configured to:

receive navigation data;

generate and/or transmit maneuvering and/or monitoring instructions for the mobile structure based on the received navigation data; and

monitor the navigation data as the mobile structure traverses an associated navigation path for a detected far object in the one or more objects.

Assignments (2)

Continuity (2)

References Cited (17)