Granted Patent B2

US 12,387,581 · App. 17/613,674 · Granted Aug 12, 2025

Inventor: Andreas Wellhausen (Hannover, DE)

Assignee: Robert Bosch GmbH

G08B17/125G06T7/0002G06T7/73G06T2207/20081G06T2207/20084G06T2207/30232

Abstract

Claims (45)

1. A machine learning system comprising:

a convolutional recurrent neural network, wherein the machine learning system is parameterized, on the basis of successive images of a sequence of images, to detect smoke in the images, and wherein the convolutional recurrent neural network is configured as a plurality of concatenated convolutional LongShortTermMemory (convolutional LSTM) modules connected to each other in a predetermined sequence;

wherein a first convolutional LSTM module in the predetermined sequence is configured to receive an input variable from the sequence of images,

wherein a number of filters that each respective convolutional LSTM module of the plurality of concatenated convolutional LSTM modules comprises increases with increasing depth of a position of each respective convolutional LSTM module,

wherein the depth of the position is a position within a sequence of each respective convolutional LSTM module of the machine learning system relative to the first convolutional LSTM module of the plurality of concatenated convolutional LSTM modules,

wherein a pooling layer is positioned between each of the convolutional LSTM modules, and

wherein a second convolutional LSTM module of the plurality of the concatenated convolutional LSTM modules is connected to a fully connected neural network, which is parameterized to output an output variable of the machine learning system.

2. The machine learning system as claimed in claim 1 , which is further parameterized to output an output variable that characterizes whether smoke is represented within the images.

3. The machine learning system as claimed in claim 1 , which is parameterized, on the basis of each individual image of the images, to output a matrix as an output variable,

wherein elements of the matrix are each assigned to a section of a predetermined plurality of sections of a respective image of the images, and

wherein the elements of the matrix characterize whether smoke is represented within this assigned section of the respective image.

4. The machine learning system as claimed in claim 1 , further comprising an input, wherein the input is configured to provide the sequence of images captured by a camera to the machine learning system.

5. The machine learning system as claimed in claim 1 , wherein the machine learning system is further parameterized to determine an amount of the detected smoke based on the successive images of the sequence of images, wherein the amount of the detected smoke is at least one selected from a group consisting of: an intensity and a density.

6. The machine learning system as claimed in claim 1 , wherein the convolutional LSTM module is configured to extract global information and temporal information of the successive images of the sequence of images together.

7. The machine learning system as claimed in claim 1 , wherein the convolutional LSTM module is configured to process global information and temporal information of the successive images of the sequence of images together.

8. The machine learning system as claimed in claim 1 ,

wherein the images are processed successively by the plurality of layers of the machine learning system,

wherein a resolution of the processed image decreases with increasing depth of a position of each respective convolutional LSTM module, wherein the depth of the position is a position within a sequence of each respective convolutional LSTM module of the machine learning system relative to the first convolutional LSTM module of the plurality of concatenated convolutional LSTM modules.

9. The machine learning system as claimed in claim 1 , wherein each of the plurality of convolutional LSTM modules are configured to process temporal and global information simultaneously.

10. A method for detecting smoke within an image by means of a machine learning system, which is parameterized, on the basis of a plurality of images captured in direct succession to detect smoke in the images,

wherein the machine learning system comprises a plurality of layers connected in a specified sequence and at least one of the plurality of layers comprises a convolutional recurrent neural network, wherein the convolutional recurrent neural network is a plurality of concatenated convolutional LongShortTermMemory (convolutional LSTM) modules,

wherein a number of filters that each respective convolutional LSTM module of the plurality of concatenated convolutional LSTM modules comprises increases with increasing depth of a position of each respective convolutional LSTM module,

wherein the depth of the position is a position within a sequence of each respective convolutional LSTM module of the machine learning system relative to a first convolutional LSTM module of the plurality of concatenated convolutional LSTM modules,

wherein a pooling layer is positioned between each of the convolutional LSTM modules, and

wherein a second convolutional LSTM module of the plurality of the concatenated convolutional LSTM modules is connected to a fully connected neural network, which is parameterized to output an output variable of the machine learning system, said method comprising the following steps:

obtaining a plurality of images captured in direct succession;

receiving an input variable at a first layer in the specified sequence, the first layer comprising the first convolutional LSTM module;

propagating the plurality of the captured images through the machine learning system in succession,

wherein during the propagation, the images are processed successively by the plurality of layers of the machine learning system and a final layer of the sequence of layers outputs an output variable.

11. The method as claimed in claim 10 , wherein the machine learning system is trained on the basis of a plurality of training data, comprising training images (x) and respectively assigned training output variables (y s ),

wherein the training images (x) are propagated through the machine learning system during the training and, on the basis of the determined output variables of the machine learning system and the respectively assigned training output variables of the training images, a parameterization of the machine learning system is adjusted in such a way that a deviation between the determined output variables and the training output variables becomes a minimum.

12. The method as claimed in claim 10 , wherein a smoke detector is activated on the basis of the determined output variable of the machine learning system.

13. The method of claim 10 , wherein a resolution of the processed image decreases with increasing depth of a position of each respective layer, wherein the depth of the position is a position within a sequence of each respective layer of the machine learning system relative to the first layer.

14. The method of claim 10 , further comprising processing temporal and global information simultaneously at each layer.

15. A non-transitory, computer-readable medium that contains instructions that when executed on a computer cause said computer to detect smoke within an image by obtaining a plurality of images captured in direct succession;

propagating the plurality of the captured images through a machine learning system in succession;

wherein the machine learning system comprises a plurality of layers connected in a specified sequence and at least one of the layers comprises a convolutional recurrent neural network,

wherein the convolutional recurrent neural network is a plurality of concatenated convolutional LongShortTermMemory (convolutional LSTM) modules,

wherein a number of filters that each respective convolutional LSTM module of the plurality of concatenated convolutional LSTM modules comprises increases with increasing depth of a position of each respective convolutional LSTM module,

wherein the depth of the position is a position within a sequence of each respective convolutional LSTM module of the machine learning system relative to a first convolutional LSTM module of the plurality of concatenated convolutional LSTM modules,

wherein a pooling layer is positioned between each of the convolutional LSTM modules,

wherein a second convolutional LSTM module of the plurality of the concatenated convolutional LSTM modules is connected to a fully connected neural network, which is parameterized to output an output variable of the machine learning system, and

wherein during the propagation, the images are processed successively by the layers of the machine learning system, wherein a first layer of the plurality of layers is the first convolutional LSTM module configured to receive an input variable from the plurality of images and a final layer of the sequence of layers outputs an output variable.

16. The non-transitory, computer-readable medium of claim 15 , wherein each of the plurality of convolutional LSTM modules are configured to process temporal and global information simultaneously.

17. The non-transitory, computer-readable medium of claim 15 , wherein a resolution of the processed image decreases with increasing depth of a position of each respective layer, wherein the depth of the position is a position within a sequence of each respective layer of the machine learning system relative to the first layer.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (27)