IP Library Granted Patent US 12,137,285
Granted Patent B2
US 12,137,285 · App. 17/207,787 · Granted Nov 5, 2024

Camera peek into turn

Inventor: Nirmal Patel (Sunnyvale, CA)
Assignee: WAYMO LLC
H04N23/64B60R1/27B60R1/28G05D1/0094G05D1/0246G06T15/20H04N23/63H04N23/69B60R2300/302B60R2300/602B60R2300/605B60R2300/8086
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,137,285
App. No.
17/207,787
Granted
Nov 5, 2024
Kind
B2
Abstract

Aspects of the disclosure relate to adjusting a virtual camera's orientation when a vehicle is making a turn. One or more computing devices may receive the vehicle's original heading prior to making the turn and the vehicle's current heading. Based on the vehicle's original heading and the vehicle's current heading, the one or more computing devices may determine an angle of a turn the vehicle is performing and The one or more computing devices may determine a camera rotation angle and adjust the virtual camera's orientation relative to the vehicle to an updated orientation by rotating the virtual camera by the camera rotation angle and generate a video corresponding to the virtual camera's updated orientation. The video may be displayed on the display by the one or more computing devices.

Claims (37)

1. A computer implemented method, comprising:

determining, by one or more computing devices of a vehicle operating in an autonomous mode, a projected trajectory of the vehicle;

determining, by the one or more computing devices based on the vehicle's projected trajectory and positioning of the vehicle, an anticipated angle of an upcoming turn the vehicle is planning to perform;

comparing, by the one or more computing devices, how far away the upcoming turn is to a predetermined distance that is based on a current speed of the vehicle and a current distance the vehicle is from the upcoming turn;

adjusting, by the one or more computing devices, an orientation of a virtual camera relative to the vehicle from an initial orientation to an updated orientation by rotating the virtual camera by a camera rotation angle corresponding to the determined anticipated angle of the upcoming turn; and

generating for display, by the one or more computing devices, a video corresponding to the virtual camera's updated orientation, the video being configured for presentation to passengers within the vehicle as the vehicle is operating in the autonomous mode;

wherein upon the one or more computing devices determining that the upcoming turn is located at or less than the predetermined distance, beginning to rotate the virtual camera through a series camera rotation angles corresponding to a set of anticipated angles of the upcoming turn.

2. The computer implemented method of claim 1 , wherein when the vehicle is traveling at a higher rate of speed the predetermined distance is greater than when the vehicle is traveling at a slower rate of speed.

3. The computer implemented method of claim 1 , further comprising mapping one or more angles of the upcoming turn to the camera rotation angle.

4. The computer implemented method of claim 3 , wherein the mapping is not a 1:1 correspondence.

5. The computer implemented method of claim 4 , wherein when a given angle of the upcoming turn is 90° the given angle is mapped to a 45° camera rotation angle.

6. The computer implemented method of claim 1 , wherein as the vehicle progresses through the upcoming turn, the camera rotation angle is continually updated.

7. A computer implemented method, comprising:

determining, by one or more computing devices of a vehicle operating in an autonomous mode, a projected trajectory of the vehicle;

determining, by the one or more computing devices based on the vehicle's projected trajectory and positioning of the vehicle, an anticipated angle of an upcoming turn the vehicle is planning to perform;

adjusting, by the one or more computing devices, an orientation of a virtual camera relative to the vehicle from an initial orientation to an updated orientation by rotating the virtual camera by a camera rotation angle corresponding to the determined anticipated angle of the upcoming turn, wherein the rotating of the virtual camera temporarily ceases when the vehicle is stopped or projected to stop; and

generating for display, by the one or more computing devices, a video corresponding to the virtual camera's updated orientation, the video being configured for presentation to passengers within the vehicle as the vehicle is operating in the autonomous mode.

8. The computer implemented method of claim 1 , further comprising returning the virtual camera to a default position upon completion of the upcoming turn.

9. The computer implemented method of claim 1 , wherein a height and a pitch of the virtual camera is adjusted to capture more or less of the vehicle's surroundings during a driving operation in the autonomous mode.

10. A vehicle configured to operate in an autonomous driving mode, the vehicle comprising:

a steering system;

an acceleration system;

a deceleration system;

a perception system including one or more sensors configured to detect objects in an environment external to the vehicle;

an autonomous driving system including one or more processors, the autonomous driving system operatively coupled to the steering system, the acceleration system, the deceleration system and the perception system to control driving of the vehicle in the autonomous driving mode; and

a virtual camera system having one or more processors, the virtual camera system being configured to determine a set of anticipated angles of an unprotected turn, and rotate a virtual camera to generate imagery corresponding to oncoming traffic at the unprotected turn according to the determined set of anticipated angles, wherein a camera rotation angle of the virtual camera is predetermined for the unprotected turn.

11. The vehicle of claim 10 , wherein the camera rotation angle is 45° or less.

12. The vehicle of claim 10 , wherein the unprotected turn is either where the vehicle is turning onto a road where traffic is moving, or where the road the vehicle is turning onto does not require other vehicles to stop at an intersection.

13. The vehicle of claim 10 , wherein:

the camera rotation angle is a series of camera rotation angles corresponding to the set of anticipated angles of the unprotected turn; and

the virtual camera system is further configured to:

compare how far away the unprotected turn is to a predetermined distance; and

upon a determination that the unprotected turn is located at or less than the predetermined distance, begin to rotate the virtual camera through the series of camera rotation angles.

14. The vehicle of claim 10 , wherein the virtual camera system is further configured to map one or more angles of the unprotected turn to the camera rotation angle.

15. The vehicle of claim 14 , wherein the mapping is not a 1:1 correspondence.

16. The vehicle of claim 10 , wherein the virtual camera system is further configured to temporarily cease rotation of the virtual camera when the vehicle is stopped or projected to stop.

17. The vehicle of claim 10 , wherein the virtual camera system is further configured to return the virtual camera to a default position upon completion of the unprotected turn.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2021
From: PATEL, NIRMAL
To: GOOGLE INC.
Reel/Frame 055665/0888 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2021
From: GOOGLE INC.
To: WAYMO HOLDING INC.
Reel/Frame 055735/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2021
From: WAYMO HOLDING INC.
To: WAYMO LLC
Reel/Frame 055735/0073 →
Continuity (2)
Continuation 15364914 · Nov 30, 2016
Related Publication 20210211576A1 · Jul 8, 2021