Driver recognition system and method of use thereof

1 . A driver recognition system, comprising:

a driver input unit configured to receive image data of a driver seated in a driver's seat in a vehicle in which an automatic driver recognition function of the driver recognition system is set;

a driver area extraction unit configured to extract driver's head area data by analyzing the image data of the driver using a stored machine learning-based object detection model;

a driver determination unit configured to analyze the extracted driver's head area data and registered driver's head area data using a stored machine learning-based driver matching model, and to determine whether there is matched data between the extracted driver's head area data and the registered driver's head area data;

a driver recognition unit configured to extract unique information matched to corresponding driver's head area data when it is determined that there is matched data according to a determination result of the driver determination unit, and to perform an automatic driver recognition function using the extracted unique information; and

a first image collection unit configured to acquire a first image data set, which is first image data of a driver's seat, in a driverless situation, for each of a plurality of illumination conditions around the driver's seat controlled according to a predetermined condition, for each vehicle model for which the automatic driver recognition function is to be set in advance,

wherein first image data set is used in generating the machine learning-based object detection model.

2 . The driver recognition system of claim 1 , wherein the driver recognition unit is configured to perform an automatic driver recognition function for controlling a state of the vehicle according to vehicle setting information previously received from the driver matching the unique information using the extracted unique information.

3 . The driver recognition system of claim 2 , further comprising:

a registered driver collection unit configured to acquire registered driver image data of a registered driver when the registered driver, who is a driver who wants to use the vehicle in which an automatic driver recognition function is set, sits in the driver's seat of the vehicle;

a registered driver area extraction unit configured to extract driver's head area data of the registered driver by analyzing the registered driver image data of the registered driver using a stored machine learning-based object detection model;

a vehicle setting input unit configured to receive desired vehicle setting information from the registered driver; and

a registered driver management unit configured to generate unique information for each of a plurality of registered drivers, and to match the unique information generated for each driver who sits in the driver's seat of the vehicle, the head area data extracted by the registered driver area extraction unit, and the setting information received by the vehicle setting input unit to store and manage the matched information,

wherein the driver determination unit receives head area data stored and managed by the registered driver management unit as head area data of the registered driver.

4 . The driver recognition system of claim 3 , wherein the registered driver collection unit is configured to repeatedly performs a registered driver collection operation for a predetermined number of times, to request the driver to change a head angle for each repetition of the registered driver collection operation, and to acquire the registered driver image data of the registered driver.

5 . The driver recognition system of claim 3 , further comprising:

a second image collection unit configured to acquire a second image data set of second image data when a driver to which unique information is assigned sits in the driver's seat, for each vehicle match request to the driver to change a head angle a predetermined number of times;

a custom data generation unit configured to analyze the second image data set collected by the second image collection unit to perform labeling on the head area, and to generate a custom data set including corresponding custom image data and labeling coordinate information based on each piece of unique information;

a region of interest setting unit configured to overlap all labeling areas comprising the custom data set and to setting an entire overlapping area as a region of interest (ROI); and

an anchor box setting unit configured to set the region of interest to a predetermined number of anchor box areas based on an average size value and an average ratio value of all the labeling areas comprising the custom data set.

6 . The driver recognition system of claim 5 , further comprising:

an input data selection unit configured to select at least one data set from among an open data set pre-collected based on a probability and a stored custom data set;

a data pre-processing unit configured to extract a predetermined number of anchor box areas set by the anchor box setting unit for each data comprising the data set selected by the input data selection unit; and

a learning processing unit configured to input each anchor box area-based data extracted by the data pre-processing unit to a pre-stored object detection network and to perform learning processing to generate, store, and transmit an object detection model,

wherein the data pre-processing unit includes a first pre-processing unit configured to extract the predetermined number of anchor box areas set by the anchor box setting unit for each data comprising the custom data set when a custom data set is selected by the input data selection unit.

7 . The driver recognition system of claim 6 , wherein the data pre-processing unit further includes:

when the open data set is selected by the input data selection unit, performing:

a 2-1-th pre-processing unit configured to detect head area data for each data comprising the open data set by using the pre-stored object detection model;

a 2-2-th pre-processing unit configured to randomly adjust at least one of hue, saturation, brightness, and transparency of the head area data detected by the 2-1-th pre-processing unit;

a 2-3-th pre-processing unit configured to randomly receive at least one image data of the first image data by the first image collection unit, and to position the head area data by the 2-2-th pre-processing unit in the corresponding image data to generate a synthetic data set, and to randomly position the head area data in region of interest coordinates set by the region of interest setting unit; and

a 2-4-th pre-processing unit configured to extract the predetermined number of anchor box areas set by the anchor box setting unit for each data comprising the synthetic data set.

8 . The driver recognition system of claim 7 , further comprising:

a feature pre-processing unit configured to analyze, in advance, a custom data set generated by the custom data generation unit and a synthetic data set generated by the 2-3-th pre-processing unit by using the object detection model generated by the learning processing unit to extract face area data from each image data, and to assign the unique information to each extracted face area data;

a first learning processing unit configured to input the face area data to which the unique information is assigned by the feature pre-processing unit to a pre-stored classification network to perform learning processing;

a feature extraction unit configured to use at least two feature extraction networks including a base network in which a learning result of the first learning processing unit is reflected to input the face area data to which the unique information is assigned by the feature pre-processing unit to any one selected feature extraction network and input the face area data corresponding to the custom data set among the face area data to which the unique information is assigned by the feature pre-processing unit to another feature extraction network, and to extract features of the face area data input from each network; and

a second learning processing unit configured to include a feature comparator layer connected to rear ends of the two feature extraction networks to compare the features of each face area data extracted by the feature extraction unit and to perform the learning processing of determining whether the features are identical,

wherein a final layer comprising the base network of the classification network of the first learning processing unit includes a global average pooling (GAP) layer,

the final layer comprising each base network of the two feature extraction networks of the second learning processing unit includes an adaptive average pooling (AAP) layer, and

the driver matching model is generated as a learning processing result of the second learning processing unit.

9 . The driver recognition system of claim 1 , wherein generating and using at least one of the learning-based object detection model and the learning-based driver matching model is based on an open-data set, pre-collected by the driver recognition system based on a probability, including a plurality of sample face images and labels for each of the sample face images, and on a custom data set for the vehicle acquired and stored by the driver recognition system.

10 . The driver recognition system of claim 1 , further comprising:

an input data selection unit configured to select at least one data set from among an open data set and stored custom data set; and

a first learning processing unit configured to perform learning processing to generate, store, and transmit the object detection model, based on the at least one data set, to the driver area extraction unit and to a second learning processing unit,

wherein the second learning processing unit is configured to use the object detection model generated by the first learning processing unit to generate the learning-based driver matching model to provide to the driver determination unit to determine whether there is matched data between the extracted driver's head area data and the registered driver's head area data.

11 . A driver recognition method using a driver recognition system in which each step is performed by an arithmetic processing means, the driver recognition method comprising:

a first image collection step of acquiring a first image data set, which is first image data of a driver's seat, in a driverless situation for each of a plurality of illumination conditions controlled according to a predetermined condition, for each vehicle model for which an automatic driver recognition function is to be set for the driver recognition system;

a second image collection step of seating a driver, to whom unique information is assigned, in the driver's seat, for each vehicle model for which the automatic driver recognition function is to be set and acquiring a second image data set which is second image data of the driver showing a plurality of different head angles of the driver in a head area of the driver sitting in the driver's seat;

a custom data generation step of analyzing the second image data set obtained by the second image collection step to perform labeling on the head area, and generating a custom data set including corresponding custom image data and labeling coordinate information based on each piece of unique information;

a region of interest setting step of overlapping all labeling areas comprising the custom data set by the custom data generation step and setting an entire overlapping area as a region of interest (ROI);

an anchor box setting step of setting the region of interest to a predetermined number of anchor box areas based on an average size value and an average ratio value of all the labeling areas comprising the custom data set by the custom data generation step;

a data pre-processing step of selecting at least one data set from among an open data set pre-collected based on a probability and a stored custom data set, and extracting a predetermined number of anchor box areas set by the anchor box setting step for each data comprising the selected data set;

a first learning processing step of inputting each anchor box area-based data extracted by the data pre-processing step to a pre-stored object detection network and performing learning processing to generate an object detection model;

a driver input step of receiving image data of the driver seated in a driver's seat in a vehicle in which an automatic driver recognition function is set;

a driver area extraction step of extracting the driver's head area data by analyzing the image data of the driver by the driver input step using the object detection model generated by the first learning processing step;

a driver determination step of analyzing the extracted head area data and pre-registered driver's head area data by the driver area extraction step using a stored machine learning-based driver matching model, and determining whether there is matched data; and

a driver recognition step of extracting unique information matched to the corresponding head area data when it is determined that there is the matched data according to a determination result of the driver determination step, and performing an automatic driver recognition function using the extracted unique information,

wherein the driver recognition step uses the extracted unique information to control a state of the vehicle according to the vehicle setting information previously received from the driver matching the unique information.

12 . The driver recognition method of claim 11 , wherein the data pre-processing step includes a first pre-processing step of extracting a predetermined number of anchor box areas set by the anchor box setting step for each data comprising the custom data set when a custom data set is selected.

13 . The driver recognition method of claim 12 , wherein the data pre-processing step further includes:

when the open data set is selected, performing:

a 2-1-th pre-processing step of detecting head area data for each data comprising the open data set by using the pre-stored object detection model;

a 2-2-th pre-processing step of randomly adjusting at least one of hue, saturation, brightness, and transparency of the head area data detected by the 2-1-th pre-processing step;

a 2-3-th pre-processing step of randomly receiving at least one image data of the first image data by the first image collection step, and positioning the head area data by the 2-2-th pre-processing step in the corresponding image data to generate a synthetic data set, and randomly positioning the head area data in region of interest coordinates set by the region of interest setting step; and

a 2-4-th pre-processing step of extracting the predetermined number of anchor box areas set by the anchor box setting step for each data comprising the synthetic data set.

14 . The driver recognition method of claim 13 , further comprising:

prior to performing the driver determination step, in order to store a machine learning-based driver matching model, performing:

a feature pre-processing step of analyzing, in advance, a custom data set generated by the custom data generation step and a synthetic data set generated by the 2-3-th pre-processing step by using the object detection model generated by the first learning processing step to extract face area data from each image data, and assigning the unique information to each extracted face area data;

a second learning processing step of inputting the face area data to which the unique information is assigned by the feature pre-processing step to a pre-stored classification network to perform learning processing;

a feature extraction step of using at least two feature extraction networks including a base network in which a learning result of the second learning processing step is reflected to input the face area data to which the unique information is assigned by the feature pre-processing step to any one selected feature extraction network and to input the face area data corresponding to the custom data set among the face area data to which the unique information is assigned by the feature pre-processing step to another feature extraction network, and extracting features from each network; and

a third learning processing step of comparing the features of each face area data extracted by the feature extraction step by including a feature comparator layer connected to the two feature extraction networks to perform the learning processing of determining whether the features are identical,

wherein a final layer comprising the base network of the classification network includes a global average pooling (GAP) layer,

the final layer comprising the base network of the feature extraction network includes an adaptive average pooling (AAP) layer, and

the driver matching model is generated as a learning processing result of the third learning process step.

15 . The driver recognition method of claim 11 , further comprising:

prior to performing, in advance, the driver determination step in order to register the driver's head area data, performing:

a registered driver collection step of acquiring registered driver image data of a registered driver, wherein the registered driver is a driver who wants to use a vehicle in which an automatic driver recognition function is set when the registered driver sits in the driver's seat of the vehicle;

a registered driver area extraction step of extracting the driver's head area data by analyzing the registered image data of the driver by the registered driver collection step using the object detection model generated by the first learning processing step;

a vehicle setting input unit of receiving desired vehicle setting information from the registered driver; and

a registered driver management step of generating unique information for each driver who sits in the driver's seat of the vehicle, and matching the unique information generated for each driver who sits in the driver's seat of the vehicle, the head area data extracted by the registered driver area extraction step and the setting information received by the vehicle setting input step to store and manage the matched information,

wherein, the registered driver collection step requests the registered driver to change a head angle for multiple repetitions of the registered driver collection step and acquires the registered driver image data of the registered driver while repeatedly performing the repetitions for a predetermined number of times.

16 . A driver recognition method using a driver recognition system, comprising:

receiving, via a driver input unit, image data of a driver seated in a driver's seat in a vehicle in which an automatic driver recognition function of the driver recognition system is set;

extracting, via a driver area extraction unit, driver's head area data by analyzing the image data of the driver using a stored machine learning-based object detection model;

analyzing, via a driver determination unit, the extracted driver's head area data and registered driver's head area data using a stored machine learning-based driver matching model,

determining whether there is matched data between the extracted driver's head area data and the register driver's head area data; and

extracting, via a driver recognition unit, unique information matched to corresponding head area data when it is determined that there is matched data according to a determination result of the driver determination unit, and to perform an automatic driver recognition function using the extracted unique information,

wherein the method further includes:

receiving a first image data set received from a first image collection unit configured to acquire the first image data set, which is first image data of a driver's seat, in a driverless situation, for each of a plurality of illumination conditions around the driver's seat controlled according to a predetermined condition, for each vehicle model for which the automatic driver recognition function is to be set in advance; and

using the first image data set for generating the machine learning-based object detection model.

17 . The driver recognition method of claim 16 , further comprising performing an automatic driver recognition function, via the driver recognition unit, for controlling a state of the vehicle according to vehicle setting information previously received from the driver matching the unique information using the extracted unique information.

18 . The driver recognition method of claim 16 , further comprising:

selecting, via an input data selection unit, at least one data set from among an open data set and a stored custom data set;

performing a learning processing, via a first learning processing unit, to generate, store, and transmit the object detection model, based on the at least one data set, to the driver area extraction unit and to a second learning processing unit; and

generating the learning-based driver matching model, via the second learning processing unit using the object detection model generated by the first learning processing unit, to provide to the driver determination unit to determine whether there is matched data between the extracted driver's head area data and the registered driver's head area data.

19 . The driver recognition method of claim 16 , wherein a learning data transfer for the driver recognition method, which includes generating and using the learning-based object detection model and the learning-based driver matching model, is based on an open-data set, pre-collected for the driver recognition method based on a probability, including a plurality of sample face images and labels for each of the sample face images, and on a custom data set for the vehicle acquired and stored by the driver recognition method.