IP Library Granted Patent US 11,698,642
Granted Patent B2
US 11,698,642 · App. 16/768,875 · Granted Jul 11, 2023

Information processing apparatus, mobile object, control system, and information processing method

Inventor: Masashi Eshima (Chiba, JP)
Assignee: SONY CORPORATION
G05D1/0246G01C21/1656G05D1/0274G06T7/73G05D2201/0213G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,698,642
App. No.
16/768,875
Granted
Jul 11, 2023
Kind
B2
Abstract

There is provided an information processing apparatus that includes an information acquisition unit and a control prediction unit. The information acquisition unit acquires information regarding a location and a posture of a first mobile object that includes a sensing device. The control prediction unit predicts control performed with respect to the sensing device, on the basis of the information regarding the location and the posture and map information, the information regarding the location and the posture being acquired by the information acquisition unit.

Claims (94)

1. An information processing apparatus, comprising:

a central processing unit (CPU) configured to:

acquire first information regarding a first location and a first posture of a first mobile object, wherein

the first location and the first posture correspond to a first time instant,

the first mobile object includes a first sensing device, and

the first sensing device includes an image-capturing device;

predict a second location and a second posture of the first mobile object based on the first information, wherein

the second location and the second posture correspond to a second time instant, and

the second time instant is subsequent to the first time instant;

predict a position and a location of a disturbance factor in an image based on the predicted second location of the first mobile object at the second time instant, the predicted second posture of the first mobile object at the second time instant, and map information, wherein the image is to be captured by the image-capturing device at the second time instant;

generate, based on the predicted position of the disturbance factor and the predicted location of the disturbance factor in the image, first control prediction information for control of the first sensing device; and

transmit the first control prediction information,

wherein the first sensing device is controlled at the second time instant based on the first control prediction information.

2. The information processing apparatus according to claim 1 , wherein the CPU is further configured to predict an exposure control of the image-capturing device.

3. The information processing apparatus according to claim 1 , wherein the CPU is further configured to predict photometry-region control of the image-capturing device.

4. The information processing apparatus according to claim 1 , wherein

the first mobile object further includes a self-location estimation system that estimates the first location and the first posture of the first mobile object using a characteristic point that is extracted from image information from the image-capturing device,

the CPU is further configured to generate, based on the predicted position of the disturbance factor and the predicted location of the disturbance factor in the image, the first control prediction information to control the self-location estimation system, and

the self-location estimation system is controlled at the second time instant based on the first control prediction information.

5. The information processing apparatus according to claim 4 , wherein the CPU is further configured to predict, based on the predicted position of the disturbance factor, a region in which the extraction of the characteristic point in the image is not performed in the self-location estimation system.

6. The information processing apparatus according to claim 4 , wherein

the first sensing device further includes a mobile-object-state detection sensor that detects a state of the first mobile object,

the self-location estimation system further estimates the first location and the first posture of the first mobile object based on at least one of the image information, or mobile-object-state information from the mobile-object-state detection sensor, and

the CPU is further configured to assign, based on the predicted position of the disturbance factor, a weight to each of the image information and the mobile-object-state information.

7. The information processing apparatus according to claim 4 , wherein

the first sensing device further includes a plurality of image-capturing devices,

the plurality of image-capturing devices include the image-capturing device,

the CPU is further configured to assign, based on the predicted position of the disturbance factor, a weight to image information from each image-capturing device of the plurality of image-capturing devices, and

the first location and the first posture of the first mobile object are estimated in the self-location estimation system based on the image information.

8. The information processing apparatus according to claim 4 , wherein the CPU is further configured to generate, based on the predicted position of the disturbance factor, the first control prediction information for control of the image-capturing device.

9. The information processing apparatus according to claim 1 , wherein

the disturbance factor is the sun, and

the CPU is further configured to predict, based on the map information, the predicted second location of the first mobile object, the predicted second posture of the first mobile object, and sun position information, a position of the sun in the image to be captured by the image-capturing device.

10. The information processing apparatus according to claim 1 , wherein

the disturbance factor is a tunnel, and

the CPU is further configured to predict, based on the map information, the predicted second location of the first mobile object, and the predicted second posture of the first mobile object, a position of the tunnel in the image to be captured by the image-capturing device.

11. The information processing apparatus according to claim 1 , wherein

the disturbance factor is a shadow of a structure, and

the CPU is further configured to predict, based on the map information, the predicted second location of the first mobile object, the predicted second posture of the first mobile object, and sun position information, a position of the shadow created due to the structure in the image to be captured by the image-capturing device.

12. The information processing apparatus according to claim 1 , wherein the CPU is further configured to:

acquire a third location and a third posture of a second mobile object that includes a second sensing device,

wherein the second mobile object is different from the first mobile object, and

set, as second control prediction information for control of the second sensing device of the second mobile object, the first control prediction information for the control of the first sensing device of the first mobile object.

13. A mobile object, comprising:

a sensing device; and

circuitry configured to:

acquire information regarding a self-location and a posture of the mobile object, wherein the self-location and the posture of the mobile object correspond to a first time instant;

acquire control prediction information; and

control the sensing device at a second time instant based on the control prediction information,

wherein

the second time instant is subsequent to the first time instant,

the control prediction information is based on the information regarding the self-location and the posture, a location of a disturbance factor in an image, and map information, and

the sensing device is configured to capture the image at the second time instant based on the control prediction information.

14. A control system, comprising:

a mobile object that includes a sensing device and circuitry; and

an information processing device comprising a central processing unit (CPU) configured to:

acquire first information regarding a first location and a first posture of the mobile object,

wherein the first location and the first posture correspond to a first time instant;

predict a second location and a second posture of the mobile object based on the first information, wherein

the second location and the second posture correspond to a second time instant, and

the second time instant is subsequent to the first time instant;

predict a position and a location of a disturbance factor in an image based on the predicted second location of the mobile object at the second time instant, the predicted second posture of the mobile object at the second time instant, and map information,

wherein the image is to be captured by the sensing device at the second time instant;

generate, based on the predicted position of the disturbance factor and the predicted location of the disturbance factor in the image, control prediction information for control of the sensing device; and

transmit the control prediction information,

wherein the circuitry is configured to control the sensing device based on the control prediction information.

15. An information processing method, comprising:

acquiring first information regarding a first location and a first posture of a mobile object,

wherein

the first location and the first posture correspond to a first time instant,

the mobile object includes a sensing device, and

the sensing device includes an image-capturing device;

predicting a second location and a second posture of the mobile object based on the first information, wherein

the second location and the second posture correspond to a second time instant, and

the second time instant is subsequent to the first time instant;

predicting a position and a location of a disturbance factor in an image based on the predicted second location of the mobile object at the second time instant, the predicted second posture of the mobile object at the second time instant, and map information,

wherein the image is to be captured by the image-capturing device at the second time instant;

generating, based on the predicted position of the disturbance factor and the predicted location of the disturbance factor in the image, first control prediction information for control of the sensing device; and

transmitting the first control prediction information,

wherein the sensing device is controlled at the second time instant based on the first control prediction information.

16. A non-transitory computer-readable medium having stored thereon, computer-executable instructions which, when executed by a processor, cause the processor to execute operations, the operations comprising:

acquiring first information regarding a first location and a first posture of a mobile object,

wherein

the first location and the first posture correspond to a first time instant,

the mobile object includes a sensing device, and

the sensing device includes an image-capturing device;

predicting a second location and a second posture of the mobile object based on the first information, wherein

the second location and the second posture correspond to a second time instant, and

the second time instant is subsequent to the first time instant;

predicting a position and a location of a disturbance factor in an image based on the predicted second location of the mobile object at the second time instant, the predicted second posture of the mobile object at the second time instant, and map information,

wherein the image is to be captured by the image-capturing device at the second time instant;

generating, based on the predicted position of the disturbance factor and the predicted location of the disturbance factor in the image, first control prediction information for control of the sensing device; and

transmitting the first control prediction information,

wherein the sensing device is controlled at the second time instant based on the first control prediction information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 1, 2020
From: ESHIMA, MASASHI
To: SONY CORPORATION
Reel/Frame 053654/0899 →
Priority Claims (1)
JP 2017-237534 · Dec 12, 2017 · national
Continuity (1)
Related Publication 20210141386A1 · May 13, 2021