IP Library Granted Patent US 11,313,765
Granted Patent B2
US 11,313,765 · App. 16/255,406 · Granted Apr 26, 2022

Systems and methods for remote visual inspection and emissions testing of vehicles

Inventors: Jeremy J. Onekea (Plano, TX); Darren K. Masumori (Richardson, TX); Randall M. Harris (Dallas, TX)
Assignee: Toyota Connected North America, Inc.
G01M17/007G01M15/102G06K9/00671G06K9/6215G06T7/0002H04N5/44504G06K2209/23
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,313,765
App. No.
16/255,406
Granted
Apr 26, 2022
Kind
B2
Abstract

A vehicle certification system includes an emissions sensor configured to capture emissions data of a subject vehicle, a local computing system that includes a processor and a non-transitory computer readable memory, an imaging device communicatively coupled to the processor, a display communicatively coupled to the processor, and a machine-readable instruction set stored in the non-transitory computer readable memory that, when executed by the processor, causes the local computing system to capture image data of the subject vehicle, and a remote server that is configured to receive the emissions data captured by the emissions sensor, determine compliance with at least one emissions criterion based on the emissions data, receive the image data of the subject vehicle captured by the local computing system, and determine compliance with at least one visually verifiable inspection criterion based on the image data.

Claims (54)

1. A vehicle certification system configured to:

receive the emissions data of a subject vehicle captured by an emissions sensor;

determine compliance with at least one emissions criterion based on the emissions data;

display an augmented reality portion on a display, wherein the augmented reality portion comprises a dynamic border around an outline of a dynamic inspection feature of the subject vehicle to indicate to a user where to place a computing system to best image the dynamic inspection feature, wherein the dynamic border includes dynamic markers that change to indicate a necessary change in the dynamic inspection feature;

capture image data of the dynamic inspection feature when the dynamic border on the augmented reality portion is aligned with the dynamic inspection feature;

receive the image data of the dynamic inspection feature; and

determine whether one or more physical parameters of the dynamic inspection feature satisfy a requirement for passing a certification test based on the image data.

2. The vehicle certification system of claim 1 , further configured to:

compare the image data of the dynamic inspection feature to baseline image data of a satisfactory vehicle condition, and

determine whether the one or more physical parameters of the dynamic inspection feature satisfy the requirement for passing the certification test based on the comparison between the image data and the baseline image data.

3. The vehicle certification system of claim 2 , further comprising:

an emissions sensor configured to capture the emissions data of the subject vehicle; and

a local computing system comprising:

a processor and a non-transitory computer readable memory;

an imaging device communicatively coupled to the processor;

a display communicatively coupled to the processor; and

a machine-readable instruction set stored in the non-transitory computer readable memory that, when executed by the processor, causes the local computing system to capture the image data of the dynamic inspection feature,

wherein a real time image is displayed on the display that is based on the image data captured by the imaging device.

4. The vehicle certification system of claim 3 , wherein

the augmented reality portion is a display overlay that overlays simultaneously with the real time image on the display and is configured to identify one or more inspection features on the subject vehicle.

5. The vehicle certification system of claim 4 , wherein the augmented reality portion is a semi-transparent display overlay that can align with the one or more inspection features as displayed on the display.

6. The vehicle certification system of claim 5 , wherein the augmented reality portion is a dynamic graphical representation that indicates an inspection feature.

7. The vehicle certification system of claim 5 , wherein the vehicle certification system is further configured to:

identify one or more of a make, a model, or a trim package of the subject vehicle;

select a sub-group of baseline image data based on one or more of the make, the model, or the trim package of the subject vehicle;

compare the image data of the inspection feature to the sub-group of baseline image data; and

determine whether the one or more physical parameters of the inspection feature satisfy the requirement for passing the certification test based on the comparison between the image data of the inspection feature and the sub-group of baseline image data.

8. The vehicle certification system of claim 7 , wherein a shape of the augmented reality portion is based on the baseline image data or the sub-group of baseline image data.

9. The vehicle certification system of claim 7 , wherein identifying at least one of the make, the model, or the trim package of the subject vehicle includes receiving a VIN of the subject vehicle.

10. The vehicle certification system of claim 3 , wherein the machine-readable instruction set, when executed, further causes a satisfactory inspection indication to be displayed on the display based on a determination that the one or more physical parameters of the dynamic inspection feature have satisfied the requirement for passing the certification test.

11. An augmented reality system configured to:

capture an image of a subject vehicle with an imaging device;

present on a display the image captured by the imaging device;

present on the display an augmented reality portion, wherein the augmented reality portion is a display overlay that overlays simultaneously with the image on the display and comprises a dynamic border around an outline of a dynamic inspection feature on the subject vehicle to be inspected as part of a certification test of the subject vehicle, wherein the dynamic border includes dynamic markers that change to indicate a necessary change in the dynamic inspection feature; and

capture image data of the dynamic inspection feature when the dynamic border on the augmented reality portion is aligned with the dynamic inspection feature.

12. The augmented reality system of claim 11 , further configured to:

compare the image of the subject vehicle to a library of images of satisfactory vehicle conditions; and

determine a similarity value for the image of the subject vehicle, wherein the similarity value defines a similarity between the image of the subject vehicle and one or more images in the library of images of satisfactory vehicle conditions.

13. The augmented reality system of claim 12 , further configured to:

identify one or more of a make, a model, or a trim package of the subject vehicle; and

select a sub-group of images from the library of images of satisfactory vehicle conditions based on one or more of the make, the model, or the trim package of the subject vehicle, wherein

the image of the subject vehicle is compared to the sub-group of images from the library of images of satisfactory vehicle conditions.

14. The augmented reality system of claim 13 , wherein a shape of the augmented reality portion is based on the sub-group of images from the library of images of satisfactory vehicle conditions.

15. The augmented reality system of claim 13 , wherein identifying one or more of the make, the model, or the trim package of the subject vehicle includes receiving a VIN of the subject vehicle.

16. The augmented reality system of claim 11 , wherein one or more of a processor, the imaging device, the display, and a non-transitory computer readable memory are contained in a local computing system.

17. A method comprising:

displaying an augmented reality portion on a display, wherein the augmented reality portion comprises a dynamic border around an outline of a dynamic inspection feature of a subject vehicle to indicate to a user where to place a computing system to best image the dynamic inspection feature, wherein the dynamic border includes dynamic markers that change to indicate a necessary change in the dynamic inspection feature;

capturing image data of the dynamic inspection feature when the dynamic border on the augmented reality portion is aligned with the dynamic inspection feature; and

transmitting emissions data associated with the subject vehicle and the image data to a remote server.

18. The method of claim 17 further comprising

determining whether the subject vehicle has satisfied at least one emissions criterion based on the emissions data;

comparing the image data of the dynamic inspection feature to baseline image data of a satisfactory vehicle condition, and

determining whether one or more physical parameters of the dynamic inspection feature satisfy a requirement for passing a certification test based on a comparison between the image data and the baseline image data.

19. The method of claim 18 , wherein the augmented reality portion is a display overlay that overlays simultaneously with an image on the display and is configured to identify one or more inspection features on the subject vehicle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 23, 2019
From: ONEKEA, JEREMY J.; MASUMORI, DARREN K.; HARRIS, RANDALL M.
To: TOYOTA CONNECTED NORTH AMERICA, INC.
Reel/Frame 048112/0157 →
Continuity (1)
Related Publication 20200232884A1 · Jul 23, 2020
Cited By (1)
US 12,644,795