IP Library › Granted Patent US 11,580,628
Granted Patent B2
US 11,580,628 · App. 16/880,538 · Granted Feb 14, 2023

Apparatus and methods for augmented reality vehicle condition inspection

Inventors: Stephen Gilbert (Ames, IA); Eliot Winer (Ames, IA); Jack Miller (Ames, IA); Alex Renner (Ames, IA); Nathan Sepich (Ames, IA); Vinodh Sankaranthi (Urbandale, IA); Chris Gallant (Des Moines, IA); David Wehr (Redmond, WA); Rafael Radkowski (Ames, IA); Cheng Song (Ames, IA)
Assignee: Deere & Company
G06T7/001G06T7/74G06V20/20G07C5/0825G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,580,628
App. No.
16/880,538
Granted
Feb 14, 2023
Kind
B2
Abstract

Methods, apparatus, systems and articles of manufacture are disclosed for augmented reality vehicle condition inspection. An example apparatus disclosed herein includes a location analyzer to determine whether a camera is at an inspection location and directed towards a first vehicle in an inspection profile, the inspection location corresponding to a location of the camera relative to the first vehicle, an interface generator to generate an indication on a display that the camera is at the inspection location, the indication associated with an inspection image being captured, and an image analyzer to compare the inspection image captured at the inspection location with a reference image taken of a reference vehicle of a same type as the first vehicle, and determine a vehicle part condition or a vehicle condition based on the comparison of the inspection image and the reference image.

Claims (42)

1. An apparatus comprising:

device-specific location translator circuitry to determine a distance from a point of interest for a camera to capture a dimension of a horizontal area around the point of interest based on a focal length of the camera and a width of the horizontal area;

location analyzer circuitry to determine whether the camera is at an inspection location and directed towards a first vehicle in an inspection profile, the inspection location determined based on the distance and corresponding to a location of the camera relative to the first vehicle, the inspection profile corresponding to the point of interest;

interface generator circuitry to:

display, utilizing augmented reality, navigation instructions responsive to determining that the camera is not at the inspection location and directed towards the first vehicle, the navigation instructions including a beacon overlay indicating a location on a ground to which the camera should be moved to facilitate an adjustment of a position of the camera to the inspection location; and

generate, responsive to facilitation of the adjustment of the position of the camera to the inspection location, an indication on a display that the camera is at the inspection location, the indication associated with an inspection image being captured; and

image analyzer circuitry to:

compare the inspection image captured at the inspection location with a reference image taken of a reference vehicle of a same type as the first vehicle; and

determine a vehicle part condition or a vehicle condition based on the comparison of the inspection image and the reference image.

2. The apparatus of claim 1 , wherein the interface generator circuitry is to generate a maintenance alert in response to the vehicle condition or vehicle part condition.

3. The apparatus of claim 1 , wherein the interface generator circuitry is to display part ordering information in response to the vehicle condition or vehicle part condition.

4. The apparatus of claim 1 , further including inspection data transmitter circuitry to transmit the inspection image to a third party in response to the vehicle part condition or the vehicle condition.

5. The apparatus of claim 4 , wherein the third party is at least one of a vehicle owner, an insurer, a leasing or rental entity, or a maintenance entity.

6. The apparatus of claim 1 , wherein the vehicle part condition indicates a vehicle part is missing from the first vehicle.

7. The apparatus of claim 6 , further including part information manager circuitry to access information corresponding to the vehicle part missing from the first vehicle, the interface generator circuitry to display part ordering information in response to the vehicle part missing from the first vehicle.

8. A non-transitory computer readable storage medium comprising instructions that, when executed, cause a processor to at least:

determine a distance from a point of interest for a camera to capture a dimension of a horizontal area around the point of interest based on a focal length of the camera and a width of the horizontal area;

determine whether the camera is at an inspection location, based on the distance, and directed towards a first vehicle in an inspection profile, the inspection location corresponding to a location of the camera relative to the first vehicle, the inspection profile corresponding to the point of interest;

generate navigation instructions responsive to determining that the camera is not at the inspection location and directed towards the first vehicle, the navigation instructions including a beacon overlay indicating a location on a ground to which the camera should be moved to;

display, utilizing augmented reality, the navigation instructions to facilitate an adjustment of a position of the camera to the inspection location;

generate an indication on a display that the camera is at the inspection location, the indication associated with an inspection image being captured;

compare the inspection image captured at the inspection location with a reference image taken of a reference vehicle of a same type as the first vehicle; and

determine a vehicle part condition or a vehicle condition based on the comparison of the inspection image and the reference image.

9. The non-transitory computer readable storage medium of claim 8 , wherein the instructions, when executed, cause the processor to generate a maintenance alert in response to the vehicle condition or vehicle part condition.

10. The non-transitory computer readable storage medium of claim 8 , wherein the instructions, when executed, cause the processor to display part ordering information in response to the vehicle condition or vehicle part condition.

11. The non-transitory computer readable storage medium of claim 8 , wherein the instructions, when executed, cause the processor to transmit the inspection image to a third party in response to the vehicle part condition or the vehicle condition.

12. The non-transitory computer readable storage medium of claim 11 , wherein the third party is at least one of a vehicle owner, an insurer, a leasing or rental entity, or a maintenance entity.

13. The non-transitory computer readable storage medium of claim 8 , wherein the vehicle part condition indicates a vehicle part is missing from the first vehicle.

14. The non-transitory computer readable storage medium of claim 13 , wherein the instructions, when executed, further cause the processor to access information corresponding to the vehicle part missing from the first vehicle and display part ordering information in response to the vehicle part missing from the first vehicle.

15. A method comprising

determining a distance from a point of interest for a camera to capture a dimension of a horizontal area around the point of interest based on a focal length of the camera and a width of the horizontal area;

determining whether the camera is at an inspection location, based on the distance, and directed towards a first vehicle in an inspection profile, the inspection location corresponding to a location of the camera relative to the first vehicle, the inspection profile corresponding to the point of interest;

generating navigation instructions responsive to determining that the camera is not at the inspection location and directed towards the first vehicle, the navigation instructions including a beacon overlay indicating a location on a ground to which the camera should be moved to;

displaying, utilizing augmented reality, the navigation instructions to facilitate an adjustment of a position of the camera to the inspection location;

generating an indication on a display that the camera is at the inspection location, the indication associated with an inspection image being captured;

comparing the inspection image captured at the inspection location with a reference image taken of a reference vehicle of a same type as the first vehicle; and

determining a vehicle part condition or a vehicle condition based on the comparison of the inspection image and the reference image.

16. The method of claim 15 , further including generating a maintenance alert in response to the vehicle condition or vehicle part condition.

17. The method of claim 15 , further including displaying part ordering information in response to the vehicle condition or vehicle part condition.

18. The method of claim 15 , further including transmitting the inspection image to a third party in response to the vehicle part condition or the vehicle condition.

19. The method of claim 15 , wherein the vehicle part condition indicates a vehicle part is missing from the first vehicle.

20. The method of claim 19 , further including accessing information corresponding to the vehicle part missing from the first vehicle and display part ordering information in response to the vehicle part missing from the first vehicle.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE ATTORNEY DOCKET NUMBER PREVIOUSLY RECORDED ON REEL 055265 FRAME 0702. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Mar 16, 2021
From: SANKARANTHI, VINODH; GALLANT, CHRIS
To: DEERE & COMPANY
Reel/Frame 055604/0544 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2021
From: GILBERT, STEPHEN; WINER, ELIOT; MILLER, JACK; RENNER, ALEX; SEPICH, NATHAN; WEHR, DAVID; RADKOWSKI, RAFAEL; SONG, CHENG
To: IOWA STATE UNIVERSITY RESEARCH FOUNDATION, INC.
Reel/Frame 055201/0816 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2021
From: SANKARANTHI, VINODH; GALLANT, CHRIS
To: DEERE & COMPANY
Reel/Frame 055265/0702 →
Continuity (2)
Provisional Application 62863836 · Jun 19, 2019
Related Publication 20200402219A1 · Dec 24, 2020