Method and system for data generation

1 . A system for generating a dataset of damaged signs, the system comprising at least one computer configured to:

receive a first set of image data indicating a plurality of damaged signs from a first set of classes, and a plurality of undamaged signs from a second set of classes, wherein the first set of classes and the second set of classes are mutually exclusive with respect to class type such that the first set of classes lacks class types from the second set of classes and the second set of classes lacks class types from the first set of classes;

receive class information indicating a class type for each sign in the plurality of damaged signs and the plurality of undamaged signs;

receive damage information indicating a damage type for each sign in the plurality of damaged signs;

process the first set of image data, the class information, and the damage information together with at least one machine learning algorithm to produce output features; and

generate output image data indicating an output sign from the second set of classes using a generative diffusion model that processes the output features, wherein the output image data indicates a damage type matching the damage type of at least one of the plurality of damaged signs,

wherein the generative diffusion model generates the output sign from the second set of classes by applying the damage type learned from the first set of classes to a sign type of the second set of classes that is different from sign types of the first set of classes.

2 . The system of claim 1 , further comprising a camera operatively connected with the at least one computer, wherein the at least one computer is configured to:

generate the output image data indicating a plurality of output signs from a same class type in the second set of classes, wherein each of the output signs indicate the damage type matching the damage type of at least one of the plurality of damaged signs in the first set of classes;

train an identification model for processing image data using the output image data;

receive a second set of image data from the camera; and

generate sign information identifying a sign in the second set of image data using the identification model, wherein the sign information indicates the class type and the damage type of the sign identified in the second set of image data.

3 . The system of claim 2 , wherein the at least one computer is a portable device operatively connected to an external server via a network, and configured to transmit a notification including the sign information to the external server.

4 . The system of claim 2 , in combination with a vehicle wherein the camera and the at least one computer are included in the vehicle to generate traffic sign information identifying a traffic sign in the second set of image data, including the class type and the damage type of the traffic sign identified in the second set of image data.

5 . The combination of claim 4 , wherein the vehicle is an autonomous vehicle configured to navigate a path based on the second set of image data, and the at least one computer includes an electronic control unit (ECU) configured to actuate autonomous travel by the vehicle, wherein the ECU causes the vehicle to follow an instruction associated with the traffic sign identified in the second set of image data.

6 . The system of claim 1 , wherein the at least one computer is configured to generate output image data indicating output signs from both the first set of classes and the second set of classes, wherein the output image data indicates the damage type for each of the output signs, and the damage type for each of the output signs matches the damage type of a damaged sign of the plurality of damaged signs in the first set of classes.

7 . The system of claim 1 , wherein the at least one machine learning algorithm includes a first machine learning algorithm configured to extract image features from the first set of image data, and the at least one computer is configured to concatenate the image features with the class information and the damage information to produce the output features.

8 . The system of claim 7 , wherein the at least one machine learning algorithm includes a second machine learning algorithm configured to process the concatenated image features from the first machine learning algorithm with added conditions regarding the class type and the damage type to produce the output features.

9 . The system of claim 1 , wherein the at least one machine learning algorithm includes a neural network structure configured to control the generative diffusion model by adding conditions regarding the class type and the damage type to produce the output features.

10 . The system of claim 1 , wherein the generative diffusion model processes the output features based on additional instruction that is a text embedding indicating the class type and the damage type of the output sign to be generated.

11 . A method for generating a dataset of damaged signs, the method comprising:

receiving a first set of image data indicating a plurality of damaged signs from a first set of classes, and a plurality of undamaged signs from a second set of classes, wherein the first set of classes and the second set of classes are mutually exclusive with respect to class type such that the first set of classes lacks class types from the second set of classes and the second set of classes lacks class types from the first set of classes;

receiving class information indicating a class type for signs in the plurality of damaged signs and the plurality of undamaged signs;

receiving damage information indicating a damage type corresponding to each sign in the plurality of damaged signs;

processing the first set of image data, the class information, and the damage information together with at least one machine learning algorithm to produce output features; and

generating output image data indicating an output sign from the second set of classes using a generative diffusion model that processes the output features, wherein the output image data indicates a damage type matching the damage type of at least one of the plurality of damaged signs,

wherein the generative diffusion model generates the output sign from the second set of classes by applying the damage type learned from the first set of classes to a sign type of the second set of classes that is different from sign types of the first set of classes.

12 . The method of claim 11 , further comprising:

generating the output image data indicating a plurality of output signs from a same class type in the second set of classes, wherein each of the output signs indicate the damage type matching the damage type of at least one of the plurality of damaged signs in the first set of classes;

training an identification model for processing image data using the output image data;

receiving a second set of image data from a camera; and

generating sign information identifying a sign in the second set of image data using the identification model, wherein the sign information indicates the class type and the damage type of the sign identified in the second set of image data.

13 . The method of claim 12 , further comprising transmitting a notification including the sign information to an external server via a network.

14 . The method of claim 12 , further comprising causing a vehicle to navigate a path based on the second set of image data, wherein the vehicle follows an instruction associated with a traffic sign identified in the second set of image data.

15 . The method of claim 11 , further comprising determining a target size for a dataset including signs having class types from the first set of classes and the second set of classes, including determining target sizes for subsets of the dataset, wherein each subset is defined by the class type and the damage type; and

generating the output image data for each subset of the dataset.

16 . The method of claim 11 , further comprising generating the output image data indicating output signs from both the first set of classes and the second set of classes, wherein the output image data indicates the damage type for each of the output signs, and the damage type for each of the output signs matches the damage type of a damaged sign of the plurality of damaged signs in the first set of classes.

17 . The method of claim 11 , wherein processing the first set of image data, the class information, and the damage information together with the at least one machine learning algorithm includes:

extracting image features from the first set of image data with a first machine learning algorithm;

concatenating the extracted image features with the class information and the damage information; and

processing the concatenated image features with a second machine learning algorithm, with added conditions regarding the class type and the damage type of the output sign to be generated, to produce the output features.

18 . A non-transitory computer readable storage medium storing instructions that, when executed by a computer having a processor, causes the processor to perform a method, the method comprising:

receiving a first set of image data indicating a plurality of damaged signs from a first set of classes, and a plurality of undamaged signs from a second set of classes, wherein the first set of classes and the second set of classes are mutually exclusive with respect to class type such that the first set of classes lacks class types from the second set of classes and the second set of classes lacks class types from the first set of classes;

receiving class information indicating a class type for signs in the plurality of damaged signs and the plurality of undamaged signs;

receiving damage information indicating a damage type corresponding to each sign in the plurality of damaged signs;

processing the first set of image data, the class information, and the damage information together with at least one machine learning algorithm to produce output features; and

generating output image data indicating an output sign from the second set of classes using a generative diffusion model that processes the output features, wherein the output image data indicates a damage type matching the damage type of at least one of the plurality of damaged signs,

wherein the generative diffusion model generates the output sign from the second set of classes by applying the damage type learned from the first set of classes to a sign type of the second set of classes that is different from sign types of the first set of classes.