IP Library Granted Patent US 12,260,621
Granted Patent B2
US 12,260,621 · App. 18/008,548 · Granted Mar 25, 2025

Method for generating images of a vehicle-interior camera

Inventors: Martin Petrov (Toulouse, FR); Sonia Yousfi (Toulouse, FR); Paul Mendez (Toulouse, FR); Holmer-Geert Grundmann (Toulouse, FR)
Assignee: Continental Automotive Technologies GmbH
G06V10/774G06T13/40G06T17/00G06V10/776G06V20/59
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,260,621
App. No.
18/008,548
Granted
Mar 25, 2025
Kind
B2
Abstract

A method for generating synthetic images, each image simulating an image of an individual acquired by a vehicle interior camera, including: generating a plurality of models of individuals, each model including a three-dimensional representation of an individual's head; receiving a set of variable parameters and a probability distribution associated with each parameter; generating a set of configurations, each configuration corresponding to a combination of values or states taken by each parameter, such that the set of configurations is representative of the probability distribution of each parameter; generating, for each model of an individual, a set of images simulating images of the model of an individual acquired by a vehicle interior camera, where each image corresponds to a configuration generated for a variable parameter, and where each image further includes the three-dimensional positions of a set of points characterizing the individual's head; and storing all the images in a memory.

Claims (39)

1. A method for generating synthetic images, each image simulating an image of an individual acquired by a vehicle interior camera, the method being implemented by a computer and comprising:

generating a plurality of models of individuals, each model of the plurality of models of individuals comprising a three-dimensional representation of an individual's head;

receiving a set of variable parameters and a probability distribution associated with each parameter of the set of variable parameters, the set of variable parameters relating to at least one of an environment of a corresponding model of an individual and a posture or a position of the corresponding model of the individual in relation to the environment, wherein at least one parameter of the set of variable parameters relates to the posture or the position of the individual in an interior of the vehicle, characteristics of the vehicle interior camera, or an image acquired by the vehicle interior camera;

receiving at least one parameter of the set of variable parameters, the at least one parameter relating to characteristics of the vehicle interior camera, including at least one of:

a position and/or an orientation of the vehicle interior camera,

a contrast level,

a blur level,

a camera exposure, or

a noise level;

generating a set of configurations, each configuration corresponding to a combination of values or states taken by each parameter of the set of variable parameters, such that the set of configurations is representative of the probability distribution associated with each parameter of the set of variable parameters;

generating, for each corresponding model of the individual, a set of images simulating images of the corresponding model of the individual acquired by the vehicle interior camera, where each image of the set of images corresponds to a configuration generated for a corresponding parameter of the set of variable parameters, and where each image of the set of images further comprises three-dimensional positions of a set of points characterizing the individual's head; and

storing all of the images in a memory.

2. The method as claimed in claim 1 , further comprising generating a plurality of models of vehicles, and the image-generating step is implemented for each model of the plurality of models of individuals in each model of a vehicle.

3. The method as claimed in claim 1 , comprising generating, for each model of the plurality of models of individuals, a set of images comprising an image for each of the configurations generated for each parameter of the received set of variable parameters.

4. The method as claimed in claim 1 , further comprising receiving a number N of configurations per the corresponding model of the individual, and generating, for each corresponding model of the individual, a set of configurations comprising a total of N configurations for the set of variable parameters.

5. The method as claimed in claim 1 , further comprising:

receiving, for each model of the plurality of models of individuals, at least one movement performed by the corresponding model of the individual;

generating, for at least one configuration of the set of variable parameters, a video representing the corresponding model of the individual performing the movement; and

storing said video and each image making up said video in the memory.

6. The use of a database of images generated by implementing the method as claimed in claim 1 for learning or validating an algorithm based on processing of the image acquired by the vehicle interior camera.

7. A non-transitory computer program product comprising code instructions for implementing the method as claimed in claim 1 when the code instructions are executed by a computer.

8. A device for generating synthetic images simulating images of individuals acquired by a vehicle interior camera, comprising a computer and a memory, in which the computer is configured to implement the method as claimed in claim 1 .

9. A method for generating synthetic images, each image simulating an image of an individual acquired by a vehicle interior camera, the method being implemented by a computer and comprising:

generating a plurality of models of individuals, each model of the plurality of models of individuals comprising a three-dimensional representation of an individual's head;

receiving a set of variable parameters and a probability distribution associated with each parameter of the set of variable parameters, the set of variable parameters relating to at least one of an environment of a corresponding model of an individual and a posture or a position of the corresponding model of the individual in relation to the environment;

receiving at least one parameter of the set of variable parameters, the at least one parameter relating to a position of the individual relative to a seat of the vehicle and/or at least one parameter of the set of variable parameters relating to the environment of the corresponding model of the individual, including at least one of:

a brightness and/or a light direction, and

an environment outside the vehicle;

generating a set of configurations, each configuration corresponding to a combination of values or states taken by each parameter of the set of variable parameters, such that the set of configurations is representative of the probability distribution associated with each parameter of the set of variable parameters;

generating, for each corresponding model of the individual, a set of images simulating images of the corresponding model of the individual acquired by the vehicle interior camera, where each image of the set of images corresponds to a configuration generated for a corresponding parameter of the set of variable parameters, and where each image of the set of images further comprises three-dimensional positions of a set of points characterizing the individual's head; and

storing all of the images in a memory.

10. A method for learning or validating an algorithm based on processing an image acquired by a vehicle interior camera, the method being implemented by a computer and comprising:

generating a plurality of models of individuals, each model of the plurality of models of individuals comprising a three-dimensional representation of an individual's head;

receiving a set of variable parameters and a probability distribution associated with each parameter of the set of variable parameters, the set of variable parameters relating to at least one of an environment of a corresponding model of an individual and a posture or a position of the corresponding model of the individual in relation to the environment, wherein at least one parameter of the set of variable parameters relates to the posture or the position of the individual in an interior of the vehicle, characteristics of the vehicle interior camera, or an image acquired by the vehicle interior camera;

generating a set of configurations, each configuration corresponding to a combination of values or states taken by each parameter of the set of variable parameters, such that the set of configurations is representative of the probability distribution associated with each parameter of the set of variable parameters:

generating, for each corresponding model of the individual, a set of images simulating images of the corresponding model of the individual acquired by the vehicle interior camera, where each image of the set of images corresponds to a configuration generated for a corresponding parameter of the set of variable parameters, and where each image of the set of images further comprises three-dimensional positions of a set of points characterizing the individual's head;

storing all of the images in a memory;

creating an image database where each synthetic image or each image simulates an image of the individual acquired by the vehicle interior camera and further comprises three-dimensional positions of points characterizing an individual's face; and

learning or validating the algorithm using the created image database.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 6, 2023
From: PETROV, MARTIN; YOUSFI, SONIA; MENDEZ, PAUL; GRUNDMANN, HOLMER-GEERT
To: CONTINENTAL AUTOMOTIVE TECHNOLOGIES GMBH
Reel/Frame 062601/0946 →
Priority Claims (1)
FR 2006233 · Jun 16, 2020 · national
Continuity (1)
Related Publication 20230230359A1 · Jul 20, 2023
References Cited (29)
US 5161204A · Hutcheson · 1992 [cited by examiner]
US 9665800B1 · Kuffner, Jr. · 2017 [cited by examiner]
US 9760827B1 · Lin · 2017 [cited by examiner]
US 10068385B2 · Bleiweiss · 2018 [cited by examiner]
US 10235601B1 · Wrenninge · 2019 [cited by examiner]
US 20110002531A1 · Heisele · 2011 [cited by examiner]
US 20130329951A1 · Kannan · 2013 [cited by examiner]
US 20140176551A1 · Heisele · 2014 [cited by examiner]
US 20170069124A1 · Tong et al. · 2017 [cited by applicant]
US 20170169620A1 · Bleiweiss · 2017 [cited by examiner]
US 20180181802A1 · Chen · 2018 [cited by examiner]
US 20180268255A1 · Surazhsky · 2018 [cited by examiner]
US 20190130603A1 · Sun · 2019 [cited by examiner]
US 20190251397A1 · Tremblay · 2019 [cited by examiner]
US 20190303759A1 · Farabet · 2019 [cited by examiner]
US 20190370580A1 · Aoi · 2019 [cited by examiner]
US 20200242379A1 · Mabuchi · 2020 [cited by examiner]
US 20210182609A1 · Arar · 2021 [cited by examiner]
US 20210390767A1 · Johnson · 2021 [cited by examiner]
US 20220058407A1 · Yang · 2022 [cited by examiner]
Chen et al., “Synthesizing Training Images for Boosting 3D Pose Estimation”, 2016 Fourth International Conference on 3D Vision, 2016, pp. 479-488. [cited by applicant]
Dahmen et al., “Digital Reality: A Model-Based Approach to Supervised Learning from Synthetic Data”, A1 Prospectives, 2019, vol. 1, No. 1, 12 pages. [cited by applicant]
Fridman et al., “MIT Advanced Vehicle Study: Large-Scale Naturalistic Driving Study of Driver Behavior and Interaction with Automation”, 2019, 16 pages. [cited by applicant]
Gu et al., “Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Network”, 2017, pp. 1548-1557. [cited by applicant]
Liu et al., “3D Head Pose Estimation With Convolutional Neural Network Trained on Synthetic Images”, 2016 IEEE International Conference on Image Processing, 2016, pp. 1289-1293. [cited by applicant]
Sancheti et al., “Camera Based Driver Monitoring System Using Deep Learning”, downloaded at www.visteon.com/wp-content/uploads/2019/04/camera-based-driver-monitoring-system-using-deep-learning.pdf, 2019, 5 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/EP2021/065160, dated Aug. 20, 2021, 11 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/EP2021/065160, dated Aug. 20, 2021, 16 pages (French). [cited by applicant]
French Search Report for French Application No. 2006233, dated Nov. 24, 2020 with translation, 13 pages. [cited by applicant]