IP Library Granted Patent US 12,374,224
Granted Patent B2
US 12,374,224 · App. 17/755,194 · Granted Jul 29, 2025

Vehicle control in geographical control zones

Inventors: Ranjeet Ranjan Raj (Bangalore, IN); Madhvesh Sulibhavi (Bangalore, IN)
Assignee: SONY GROUP CORPORATION
G08G1/164G01C21/3415G01C21/3461G08G1/052G08G1/056G08G1/096716G08G1/096725G08G1/096775G08G1/096783G08G1/096791G08G1/166B64U2101/20B64U2101/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,374,224
App. No.
17/755,194
Granted
Jul 29, 2025
Kind
B2
Abstract

A control system and a method for vehicle control in geographical control zones is provided. The control system receives traffic information, including a plurality of image frames of a group of moving objects in a geographical control zone and generates a set of images frames of a first moving object of the group of moving objects based on application of a trained Neural Network (NN) model on the received traffic information. The generated set of image frames corresponds to a set of likely positions of the first moving object at a future time instant. The control system predicts the unsafe behavior of the first moving object based on the generated set of image frames and generates first control information, including an alternate route for a first vehicle in the geographical control zone based on the predicted unsafe behavior. The first vehicle is controlled based on the generated first control information.

Claims (82)

1. A control system, comprising:

a first electronic device that comprises circuitry configured to:

receive traffic information that comprises a plurality of image frames, wherein the plurality of image frames is associated with a group of moving objects in a geographical control zone;

generate a set of image frames of the plurality of image frames based on application of a trained neural network (NN) model on the received traffic information, wherein

the generated set of image frames is associated with a moving object of the group of moving objects, and

the generated set of image frames corresponds to a set of positions associated with the moving object at a specific time instant;

determine an unsafe behavior of the moving object based on the generated set of image frames, wherein the determined unsafe behavior is one of an avoidable collision or an unavoidable collision;

generate first control information based on the determined unsafe behaviour is the avoidable collision, wherein the first control information comprises an alternate route for a specific vehicle in the geographical control zone;

control the specific vehicle based on the generated first control information; and

control, based on the determined unsafe behavior is the unavoidable collision, a safety hardware associated with the specific vehicle.

2. The control system according to claim 1 , wherein the moving object is a moving vehicle.

3. The control system according to claim 1 , wherein the moving object is one of a pedestrian, an animal, an aerial vehicle, a flying debris, or a human.

4. The control system according to claim 1 , wherein the circuitry is further configured to:

detect the specific vehicle based on the specific vehicle is in a communication range of the control system;

receive, from the detected specific vehicle, an input that corresponds to an acceptance associated with the geographical control zone, wherein the geographical control zone spans up to a defined distance; and

establish, based on the received input, the geographical control zone in a specific vicinity of the detected specific vehicle.

5. The control system according to claim 1 , further comprising a set of electronic devices configured to communicatively couple with the first electronic device, wherein

the circuitry is further configured to receive the traffic information from at least one of the set of electronic devices, and

the set of electronic devices comprises a vehicle, an in-vehicle Electronic Control Unit (ECU), an in-vehicle camera, an external vehicle camera, a road side unit (RSU), a Vehicle-to-Infrastructure (V2I) communication device, and a camera drone.

6. The control system according to claim 1 , wherein

the traffic information further comprises Global Navigation Satellite System (GNSS) information, and

the GNSS information is associated with a plurality of locations in the geographical control zone.

7. The control system according to claim 1 , wherein the circuitry is further configured to:

determine motion information of the group of moving objects based on the received traffic information; and

determine the unsafe behavior of the moving object based on the determined motion information.

8. The control system according to claim 7 , wherein the motion information comprises at least one of a moving direction, a moving speed, a driving pattern, a driving lane, or a driving side of a road.

9. The control system according to claim 1 , wherein the determined unsafe behavior corresponds to a violation of a traffic rule by the moving object.

10. The control system according to claim 1 , wherein

the circuitry is further configured to register a set of vehicles that comprises the specific vehicle on a control network, and

the registered set of vehicles are in the geographical control zone.

11. The control system according to claim 10 , wherein the circuitry is further configured to:

generate second control information for each vehicle of the registered set of vehicles, wherein the second control information is based on the determined unsafe behavior; and

communicate the generated second control information to each vehicle of the registered set of vehicles.

12. The control system according to claim 10 , wherein the circuitry is further configured to:

generate, based on the moving object is a vehicle registered on the control network, second control information that comprises a set of safety-measures for the moving object; and

communicate the generated second control information to the moving object.

13. The control system according to claim 1 , further comprising a set of electronic devices configured to communicatively couple with the first electronic device, wherein

the circuitry is further configured to select, from the set of electronic devices, a second electronic device,

the second electronic device corresponds to a first zone master of the geographical control zone, and

the first zone master is configured to:

collect the traffic information of the group of moving objects in the geographical control zone;

receive a retrieval request from at least one of the set of electronic devices, wherein the retrieval request is associated with the collected traffic information; and

transmit the collected traffic information to at least one of the set of electronic devices based on the received retrieval request.

14. The control system according to claim 1 , wherein the circuitry is further configured to determine the unsafe behavior of the moving object based on an absence of the moving object from a field-of-view (FOV) of the specific vehicle.

15. The control system according to claim 1 , wherein the circuitry is further configured to determine the unsafe behavior of the moving object based on the moving object is in a field-of-view (FOV) of the specific vehicle.

16. The control system according to claim 1 , wherein the circuitry is further configured to:

determine a likelihood of one of the avoidable collision or the unavoidable collision based on the generated set of image frames, wherein each of the avoidable collision and the unavoidable collision is between the specific vehicle and the moving object; and

determine the unsafe behavior of the moving object based on the determined likelihood.

17. The control system according to claim 1 , wherein the circuitry is further configured to:

determine a first travel route of the moving object based on the determined unsafe behavior, wherein the determined unsafe behavior corresponds to a violation of at least one traffic rule by the moving object;

determine a current travel route of the specific vehicle;

determine a likelihood of one of the avoidable collision or the unavoidable collision based on the determined first travel route and the determined current travel route, wherein each of the avoidable collision and the unavoidable collision is between the specific vehicle and the moving object; and

generate the first control information based on the determined likelihood, wherein the generated first control information comprises the alternate route for the specific vehicle.

18. The control system according to claim 1 , wherein the circuitry is further configured to:

receive safety information associated with a subsequent geographical control zone in a current travel route of the specific vehicle, wherein the safety information corresponds to an event associated with the subsequent geographical control zone;

determine a safe route for the specific vehicle based on the received safety information;

generate second control information that comprises the determined safe route; and

control the specific vehicle based on the generated second control information.

19. The control system according to claim 1 , wherein the circuitry is further configured to:

determine at least one safe driving zone for the specific vehicle;

determine a safe route for the specific vehicle based on the at least one determined safe driving zone;

generate second control information that comprises the determined safe route; and

control the specific vehicle based on

the generated second control information, and

a current travel route overridden on the specific vehicle.

20. The control system according to claim 1 , further comprising a drone on the specific vehicle, wherein the circuitry is further configured to:

dispatch the drone to a location of the moving object, wherein

the moving object is absent from a field-of-view (FOV) of the specific vehicle, and

the drone is configured to capture the traffic information of the moving object; and

receive the captured traffic information from the drone.

21. The control system according to claim 20 , wherein the circuitry is further configured to:

detect a user discomfort level for the specific vehicle; and

dispatch the drone based on the detected user discomfort level.

22. A method, comprising:

receiving traffic information that comprises a plurality of image frames, wherein the plurality of image frames is associated with a group of moving objects in a geographical control zone;

generating a set of image frames of the plurality of image frames based on application of a trained neural network (NN) model on the received traffic information, wherein

the generated set of image frames is associated with a moving object of the group of moving objects, and

the generated set of image frames corresponds to a set of positions associated with the moving object at a specific time instant;

determining an unsafe behavior of the moving object based on the generated set of image frames, wherein the determined unsafe behavior is one of an avoidable collision or an unavoidable collision;

generating first control information based on the determined unsafe behavior is the avoidable collision, wherein the first control information comprises an alternate route for a specific vehicle in the geographical control zone;

controlling the specific vehicle based on the generated first control information; and

controlling, based on the determined unsafe behavior is the unavoidable collision, a safety hardware associated with the specific vehicle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 26, 2022
From: RAJ, RANJEET RANJAN; SULIBHAVI, MADHVESH
To: SONY GROUP CORPORATION
Reel/Frame 061213/0639 →
Priority Claims (1)
IN 201911043742 · Oct 29, 2019 · national
Continuity (1)
Related Publication 20220383748A1 · Dec 1, 2022
References Cited (85)
US 10235882B1 · Aoude et al. · 2019 [cited by applicant]
US 10479274B2 · Park · 2019 [cited by examiner]
US 10755115B2 · Avidan · 2020 [cited by examiner]
US 10871783B2 · Julian · 2020 [cited by examiner]
US 11200795B2 · Awai · 2021 [cited by examiner]
US 11475677B2 · Avidan · 2022 [cited by examiner]
US 11508238B2 · Sui · 2022 [cited by examiner]
US 11521493B2 · Vaughn · 2022 [cited by examiner]
US 11531109B2 · Conde · 2022 [cited by examiner]
US 11630998B2 · Khan · 2023 [cited by examiner]
US 11636307B2 · Urtasun · 2023 [cited by examiner]
US 11741696B2 · Julian · 2023 [cited by examiner]
US 11807234B2 · Hotta · 2023 [cited by examiner]
US 12008454B2 · Urtasun · 2024 [cited by examiner]
US 20170036673A1 · Lee · 2017 [cited by examiner]
US 20170267256A1 · Minster · 2017 [cited by examiner]
US 20170327035A1 · Keiser · 2017 [cited by applicant]
US 20170372151A1 · Divekar · 2017 [cited by applicant]
US 20180068206A1 · Pollach · 2018 [cited by examiner]
US 20180173240A1 · Fang et al. · 2018 [cited by applicant]
US 20180335785A1 · Miller · 2018 [cited by examiner]
US 20180370502A1 · Wang et al. · 2018 [cited by applicant]
US 20190025853A1 · Julian · 2019 [cited by examiner]
US 20190065951A1 · Luo · 2019 [cited by examiner]
US 20190073545A1 · Dolgov · 2019 [cited by examiner]
US 20190088148A1 · Jacobus · 2019 [cited by examiner]
US 20190094882A1 · Kim · 2019 [cited by examiner]
US 20190129436A1 · Sun · 2019 [cited by examiner]
US 20190147260A1 · May · 2019 [cited by examiner]
US 20190171208A1 · Magalhães de Matos · 2019 [cited by examiner]
US 20190179328A1 · Movert et al. · 2019 [cited by applicant]
US 20190204843A1 · Fang et al. · 2019 [cited by applicant]
US 20190205115A1 · Gomes · 2019 [cited by examiner]
US 20190205667A1 · Avidan · 2019 [cited by examiner]
US 20190287394A1 · Aoude et al. · 2019 [cited by applicant]
US 20190287395A1 · Aoude et al. · 2019 [cited by applicant]
US 20190287401A1 · Aoude et al. · 2019 [cited by applicant]
US 20190287402A1 · Aoude et al. · 2019 [cited by applicant]
US 20190287403A1 · Aoude et al. · 2019 [cited by applicant]
US 20190294966A1 · Khan · 2019 [cited by examiner]
US 20190325754A1 · Aoude et al. · 2019 [cited by applicant]
US 20190392712A1 · Ran · 2019 [cited by examiner]
US 20200004242A1 · Kim · 2020 [cited by examiner]
US 20200249683A1 · Rosales · 2020 [cited by examiner]
US 20200262423A1 · Oboril · 2020 [cited by examiner]
US 20200290638A1 · Damnjanovic · 2020 [cited by examiner]
US 20200294394A1 · Guo · 2020 [cited by examiner]
US 20200326203A1 · Lund · 2020 [cited by examiner]
US 20200342242A1 · Avidan · 2020 [cited by examiner]
US 20200388156A1 · Aoude et al. · 2020 [cited by applicant]
US 20200409389A1 · Julian · 2020 [cited by examiner]
US 20210125076A1 · Zhang · 2021 [cited by examiner]
US 20210331701A1 · Hur · 2021 [cited by examiner]
US 20220105926A1 · Zhang · 2022 [cited by examiner]
US 20220126864A1 · Moustafa · 2022 [cited by examiner]
US 20220198930A1 · Ookuma · 2022 [cited by examiner]
US 20230394810A1 · Julian · 2023 [cited by examiner]
CA 3094275A1 · 2019 [cited by applicant]
CN 1782665A · 2006 [cited by applicant]
CN 104574953A · 2015 [cited by applicant]
CN 106873580A · 2017 [cited by applicant]
CN 107358816A · 2017 [cited by applicant]
CN 107539313A · 2018 [cited by applicant]
CN 108447291A · 2018 [cited by applicant]
CN 108475057A · 2018 [cited by applicant]
CN 109117709A · 2019 [cited by applicant]
CN 109804223A · 2019 [cited by applicant]
CN 109841088A · 2019 [cited by applicant]
CN 110027563A · 2019 [cited by applicant]
CN 110072749A · 2019 [cited by applicant]
CN 110171414A · 2019 [cited by applicant]
CN 110546696A · 2019 [cited by applicant]
CN 112154492A · 2020 [cited by applicant]
DE 102017109513A1 · 2017 [cited by applicant]
EP 3356900A1 · 2018 [cited by applicant]
EP 3495220A1 · 2019 [cited by applicant]
EP 3753001A1 · 2020 [cited by applicant]
GB 2552241A · 2018 [cited by applicant]
JP 6684346B2 · 2020 [cited by applicant]
KR 20090130977A · 2009 [cited by applicant]
KR 1020180088789A · 2018 [cited by applicant]
RU 2017115671A · 2018 [cited by applicant]
WO 2018118112A1 · 2018 [cited by applicant]
WO 2019180551A1 · 2019 [cited by applicant]
International Search Report and Written Opinion of PCT Application No. PCT/IB2020/060065, issued on Mar. 19, 2021, 16 pages of ISRWO. [cited by applicant]