IP Library › Granted Patent US 12,243,331
Granted Patent B2
US 12,243,331 · App. 17/924,357 · Granted Mar 4, 2025

Control device, projection system, control method, and program

Inventors: Takashi Takamatsu (Tokyo, JP); Hiroshi Imamura (Tokyo, JP); Seishi Tomonaga (Tokyo, JP); Koji Nagata (Tokyo, JP); Toru Nagara (Tokyo, JP)
Assignee: SONY GROUP CORPORATION
G06V20/593G03B21/2053G03B21/53G03B21/60G06T7/521
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,243,331
App. No.
17/924,357
Granted
Mar 4, 2025
Kind
B2
Abstract

The control device 50 includes a control unit 52 that controls a state of video projected onto a predetermined projection plane by a projector 20 provided in a space in the moving body 10 on the basis of space situation information input from a sensor that detects a space situation in the moving body 10.

Claims (72)

1. A control device comprising:

circuitry configured to

control a state of video projected onto a predetermined projection plane by a projection device provided in a space in a moving body based on space situation information input from a sensor that detects a space situation in the moving body,

recognize a number of passengers and a seating position of each of the passengers in the moving body based on the space situation information, and

generate a control signal for controlling a focus state of the video projected by the projection device based on the number of passengers and the seating position of each of the passengers.

2. The control device according to claim 1 , wherein

the circuitry is further configured to

calculate, for each of the passengers, a distance between the seating position each of the passengers and the predetermined projection plane of onto which the video is projected by the projection device, and

generate the control signal based on the number of passengers, the seating position of each of the passengers, and the distance for each of the passengers.

3. The control device according to claim 2 , wherein

the circuitry is further configured to

recognize at least one of a head position and an eye position of each of the passengers based on the space situation information, and

calculate the distance for each of the passengers based on at least one of the head position and the eye position of each of the passengers, and the seating position of each of the passengers.

4. The control device according to claim 1 , wherein

the circuitry is further configured to

generate the control signal based on the number of passengers, the seating position of each of the passengers, and acceleration information acquired by an acceleration sensor that detects shake generated in the moving body.

5. The control device according to claim 1 , wherein the sensor is any one or more of

a camera that generates color image data by imaging an inside of the moving body,

a stereo camera that generates parallax image data having parallax by imaging the inside of the moving body, and

a TOF camera that generates distance image data capable of detecting distance information for each pixel by imaging the inside of the moving body, and

the space situation information is any one or more of

the color image data, the parallax image data, and the distance image data.

6. The control device according to claim 5 , wherein

the circuitry is further configured to

recognize the number of passengers and the seating position of each of the passengers based on any one or more of the color image data, the parallax image data, and the distance image data.

7. The control device according to claim 1 , wherein

the circuitry is further configured to

generate the control signal based on content data as the video projected by the projection device.

8. The control device according to claim 7 , wherein

the circuitry is further configured to

specify an emphasis position to be emphasized in the video based on the content data, and

generate the control signal based on the emphasis position.

9. The control device according to claim 1 , wherein

the sensor is an interaction device that detects motions of the passengers in the moving body,

the space situation informatation is the motions of the passengers, and

the circuitry is further configured to generate the control signal based on the motions of the passengers.

10. The control device according to claim 1 , wherein

the control signal is a signal for controlling a focus position of the projection device.

11. The control device according to claim 1 , wherein

the control signal is a signal for adjusting brightness of the video projected by the projection device.

12. The control device according to claim 1 , wherein

the predetermined projection plane is a free-form curved surface.

13. A projection system comprising:

a projection device that projects video onto a predetermined projection plane in a moving body;

a sensor that detects a space situation in the moving body; and

circuitry configured to

control a state of the video based on space situation information regarding a space situation detected by the sensor,

recognize a number of passengers and a seating position of each of the passengers in the moving body based on the space situation information, and

generate a control signal for controlling a focus state of the video projected by the projection device based on the number of passengers and the seating position of each of the passengers.

14. The projection system according to claim 13 , wherein

the projection device includes

a projection lens that projects the video onto the predetermined projection plane, and

a drive actuator that adjusts the focus state of the video by moving the projection lens along an optical path.

15. The projection system according to claim 13 , wherein

the circuitry is further configured to

calculate, for each of the passengers, a distance between the seating position of each of the passengers and the predetermined projection plane onto which the video is projected by the projection device, and

generate the control signal based on the number of passengers, the seating position of each of the passengers, and the distance for each of the passengers.

16. The projection system according to claim 15 , wherein

the circuitry is further configured to

recognize at least one of a head position and an eye position of each of the passengers based on the space situation information, and

calculate the distance for each of the passengers based on at least one of the head position and the eye position of each of the passengers, and the seating position of each of the passengers.

17. The projection system according to claim 14 , wherein

the circuitry is further configured to

generate the control signal based on the number of passengers, the seating position of each of the passengers, and acceleration information acquired by an acceleration sensor that detects shake generated in the moving body.

18. A control method of controlling a projection device provided in a space in a moving body, the control method comprising:

controlling a state of video projected onto a predetermined projection plane by a projection device in a moving body based on space situation information input from a sensor that detects a space situation in the moving body;

recognizing a number of passengers and a seating position of each of the passengers in the moving body based on the space situation information; and

generating a control signal for controlling a focus state of the video projected by the projection device based on the number of passengers and the seating position of each of the passengers.

19. A non-transitory computer-readable storage medium storing a program for causing circuitry that controls a projection device provided in a space in a moving body to execute:

controlling a state of video projected onto a predetermined projection plane by the projection device based on space situation information input from a sensor that detects a space situation in the moving body;

recognizing a number of passengers and a seating position of each of the passengers in the moving body based on the space situation information; and

generating a control signal for controlling a focus state of the video projected by the projection device based on the number of passengers and the seating position of each of the passengers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 10, 2022
From: TAKAMATSU, TAKASHI; IMAMURA, HIROSHI; TOMONAGA, SEISHI; NAGATA, KOJI; NAGARA, TORU
To: SONY GROUP CORPORATION
Reel/Frame 061715/0255 →
Priority Claims (1)
JP 2020-105856 · Jun 19, 2020 · national
Continuity (1)
Related Publication 20230186651A1 · Jun 15, 2023
References Cited (15)
US 10242457B1 · Sibley · 2019 [cited by examiner]
US 20010054989A1 · Zavracky · 2001 [cited by examiner]
US 20080106702A1 · Huonker · 2008 [cited by examiner]
US 20090288123A1 · Havlovick · 2009 [cited by examiner]
US 20140267687A1 · Loghmani · 2014 [cited by examiner]
US 20160041386A1 · Rodriguez Moreno · 2016 [cited by examiner]
US 20200184219A1 · Mugura · 2020 [cited by applicant]
US 20210155122A1 · Lee · 2021 [cited by examiner]
JP 200588864A · 2005 [cited by applicant]
JP 4955309B2 · 2012 [cited by applicant]
JP 201515677A · 2015 [cited by applicant]
WO 2019044536A1 · 2019 [cited by applicant]
WO 2019124158A1 · 2019 [cited by applicant]
WO 2019181233A1 · 2019 [cited by applicant]
International Search Report and Written Opinion mailed on Sep. 7, 2021, received for PCT Application PCT/JP2021/022802, filed on Jun. 16, 2021, 9 pages including English Translation. [cited by applicant]