Methods for automatically generating a training dataset for training an optical recognition model for reading street signs
Various embodiments include methods for generating image datasets for training an artificial intelligence machine learning (AI/ML) optical character recognition (OCR) model. Image processing may be performed on a plurality of roadway images to identify street signs within the images and generate a dataset of sign images categorized into sign variants of the same shape, color, pictogram, and characters. An OCR model may process sign images to obtain OCR results for images of each sign variant. An aggregation process may be performed on the OCR results for all sign images within each sign variant to identify a ground truth OCR result for each sign variant. The ground truth OCR result may be used to automatically label all sign images of each sign variant to produce an OCR model training dataset. The produced training dataset may then be used to retrain the initial AI/ML OCR model and/or train other AI/ML OCR models.
1 . A method of turning a dataset of street sign images into a training dataset for training an artificial intelligence machine learning (AI/ML) optical character recognition (OCR) model, comprising:
performing image processing on a plurality of roadway images to identify street signs within the images and generate a dataset of street sign images categorized into a plurality of sign variants, in which each sign variant exhibits same or similar shape, colors, pictograms, and characters;
processing the dataset of street sign images using an AI/ML OCR model to obtain OCR results for each sign image of each sign variant;
performing an aggregation process on the OCR results for all sign images within each sign variant to identify a single correct OCR result for each sign variant, wherein each single correct OCR result comprises text;
automatically labeling all sign images of each sign variant with the identified single correct OCR result to produce an OCR model training dataset; and
training the AI/ML OCR model using the produced OCR model training dataset.
2 . The method of claim 1 , wherein:
performing image processing on a plurality of roadway images to identify street signs within the images includes applying a bounding box that encompasses each sign image; and
processing the dataset of street sign images using the AI/ML OCR model to obtain OCR results for each sign image of each sign variant includes applying the AI/ML OCR model to characters that appear within each sign image bounding box.
3 . The method of claim 1 , wherein the AI/ML OCR model is partially trained.
4 . The method of claim 1 , wherein performing the aggregation process on the OCR results for all sign images within each sign variant to identify a single correct OCR result for each sign variant includes identifying as the single correct OCR result the OCR result produced for a majority of the sign images in each sign variant.
5 . The method of claim 1 , wherein performing the aggregation process on the OCR results for all sign images within each sign variant to identify a single correct OCR result for each sign variant includes using a probability associated with each OCR result to identify the single correct OCR result for each sign variant.
6 . The method of claim 1 , further comprising filtering the OCR results obtained for each sign image of each sign variant to remove from further evaluation OCR results that are incorrect.
7 . The method of claim 1 , wherein the AI/ML OCR model is a fully trained AI/ML OCR model.
8 . The method of claim 1 , wherein processing the dataset of street sign images using the AI/ML OCR model to obtain OCR results for each sign images of each sign variant includes using a trained neural network to process visual embeddings for the images of text lines on sign images, using principal component analysis (PCA) to reduce dimensionality of the embeddings, and clustering the embeddings into clusters of text lines with the same text.
9 . The method of claim 1 , wherein performing the aggregation process on the OCR results for all sign images within each sign variant to identify a single correct OCR result for each sign variant includes comparing OCR results to a dictionary of words that appear on street signs.
10 . A computing system, comprising:
a memory; and
a processing system coupled to the memory and including one or more processors configured to:
perform image processing on a plurality of roadway images to identify street signs within the images and generate a dataset of street sign images categorized into a plurality of sign variants, in which each sign variant exhibits same or similar shape, colors, pictograms, and characters;
process the dataset of street sign images using an artificial intelligence machine learning (AI/ML) optical character recognition (OCR) model to obtain OCR results for each sign image of each sign variant;
perform an aggregation process on the OCR results for all sign images within each sign variant to identify a single correct OCR result for each sign variant, wherein each single correct OCR result comprises text;
automatically label all sign images of each sign variant with the identified single correct OCR result to produce an OCR model training dataset; and
train the AI/ML OCR model using the produced OCR model training dataset.
11 . The computing system of claim 10 , wherein the one or more processors are further configured to:
apply a bounding box that encompasses each sign image; and
apply the AI/ML OCR model to characters that appear within each sign image bounding box.
12 . The computing system of claim 10 , wherein the AI/ML OCR model a partially trained AI/ML OCR model.
13 . The computing system of claim 10 , wherein the one or more processors are further configured to identify as the single correct OCR result the OCR result produced for a majority of the sign images in each sign variant.
14 . The computing system of claim 10 , wherein the one or more processors are further configured to use a probability associated with each OCR result to identify the single correct OCR result for each sign variant.
15 . The computing system of claim 10 , wherein the one or more processors are further configured to filter the OCR results obtained for each sign image of each sign variant to remove from further evaluation OCR results that are incorrect.
16 . The computing system of claim 10 , wherein the AI/ML OCR model is a fully trained AI/ML OCR model; and train a different AI/ML OCR model using the produced OCR model training dataset.
17 . The computing system of claim 10 , wherein the one or more processors are further configured to process the dataset of street sign images using the AI/ML OCR model to obtain OCR results for each sign image of each sign variant by using a trained neural network to process visual embeddings for the images of text lines on sign images, using principal component analysis (PCA) to reduce dimensionality of the embeddings, and clustering the embeddings into clusters of text lines with the same text.
18 . The computing system of claim 10 , wherein the one or more processors are further configured to compare OCR results to a dictionary of words that appear on street signs to identify the single correct OCR result for each sign variant.
19 . A non-transitory processor-readable medium having stored thereon processor-executable instructions configured to cause one or more processors of a computing system to perform operations comprising:
performing image processing on a plurality of roadway images to identify street signs within the images and generate a dataset of street sign images categorized into a plurality of sign variants, in which each sign variant exhibits same or similar shape, colors, pictograms, and characters;
processing the dataset of street sign images using an artificial intelligence machine learning (AI/ML) optical character recognition (OCR) model to obtain OCR results for each sign image of each sign variant;
performing an aggregation process on the OCR results for all sign images within each sign variant to identify a single correct OCR result for each sign variant, wherein each single correct OCR result comprises text;
automatically labeling all sign images of each sign variant with the identified single correct OCR result to produce an OCR model training dataset; and
training the AI/ML OCR model using the produced OCR model training dataset.
20 . The non-transitory processor-readable medium of claim 19 , wherein AI/ML OCR model is a partially trained AI/ML OCR model.