IP Library Patent Application 15949246
Patent Application
App. No. 15/949,246

METHOD FOR THE SEMANTIC SEGMENTATION OF AN IMAGE

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
15/949,246
Abstract

A method for the semantic segmentation of an image having a two-dimensional arrangement of pixels comprises the steps of segmenting at least a part of the image into superpixels, determining image descriptors for the superpixels, wherein each image descriptor comprises a plurality of image features, feeding the image descriptors of the superpixels to a convolutional network and labeling the pixels of the image according to semantic categories by means of the convolutional network, wherein the superpixels are assigned to corresponding positions of a regular grid structure extending across the image and the image descriptors are fed to the convolutional network based on the assignment.

Claims (43)

1 . A method for the semantic segmentation of an image ( 20 ) having a two-dimensional arrangement of pixels, comprising the steps:

segmenting at least a part of the image into superpixels ( 30 ), wherein the superpixels ( 30 ) are coherent image regions comprising a plurality of pixels having similar image features,

determining image descriptors for the superpixels, wherein each image descriptor comprises a plurality of image features,

feeding the image descriptors of the superpixels to a convolutional network ( 40 ) and

labeling the pixels of the image ( 20 ) according to semantic categories by means of the convolutional network ( 40 ), wherein

the superpixels ( 30 ) are assigned to corresponding positions of a grid structure ( 37 ) extending across the image ( 20 ) and the image descriptors are fed to the convolutional network ( 40 ) based on the assignment,

characterized in that

the grid structure ( 37 ) is a regular grid structure, wherein the assigning of the superpixels ( 30 ) to corresponding positions of the regular grid structure ( 37 ) is carried out by means of a grid projection process.

2 . The method in accordance with claim 1 ,

characterized in that

the image descriptors are fed to a convolutional neural network (CNN).

3 . The method in accordance with claim 1 ,

characterized in that

the segmentation of at least a part of the image ( 20 ) into superpixels ( 30 ) is carried out by means of an iterative clustering algorithm, in particular by means of a simple linear iterative clustering algorithm (SLIC).

4 . The method in accordance with claim 3 ,

characterized in that

the iterative clustering algorithm comprises a plurality of iteration steps, in particular at least five iteration steps, wherein the regular grid structure ( 37 ) is extracted from the first iteration step.

5 . The method in accordance with claim 4 ,

characterized in that

the superpixels ( 30 ) generated by the last iteration step are matched to the regular grid structure ( 37 ) extracted from the first iteration step.

6 . The method in accordance with claim 4 ,

characterized in that

the regular grid structure ( 37 ) is generated based on the positions of the centers of those superpixels ( 30 ) which are generated by the first iteration step.

7 . The method in accordance with claim 1 ,

characterized in that

the convolutional network ( 40 ) includes 10 or less layers, preferably 5 or less layers.

8 . The method in accordance with claim 7 ,

characterized in that

the convolutional network ( 40 ) is composed of two convolutional layers and two fully connected layers.

9 . The method in accordance with claim 1 ,

characterized in that

each of the image descriptors comprises at least thirty image features.

10 . The method in accordance with claim 1 ,

characterized in that

each of the image descriptors comprises a plurality of “histogram of oriented gradients”-features (HOG-features) and/or a plurality of “local binary pattern”-features (LBP-features).

11 . A method for the recognition of objects ( 10 , 11 , 13 ) in an image ( 20 ) of a vehicle environment, comprising a semantic segmentation method in accordance with any one of the preceding claims.

12 . The system for the recognition of objects ( 10 , 11 , 13 ) from a motor vehicle, wherein the system includes a camera to be arranged at the motor vehicle and an image processing device for processing images ( 20 ) captured by the camera,

characterized in that

the image processing device is configured for carrying out a method in accordance with any one of claims 1 to 11 .

13 . The system in accordance with claim 12 ,

characterized in that

the camera is configured for repeatedly or continuously capturing images ( 20 ) and the image processing device is configured for a real-time processing of the captured images ( 20 ).

14 . A computer program product including executable program code which, when executed, carries out a method in accordance with claim 1 .

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 27, 2020
From: DELPHI TECHNOLOGIES LLC
To: APTIV TECHNOLOGIES LIMITED
Reel/Frame 052044/0428 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 16, 2018
From: ZOHOURIAN, FARNOUSH; ANTIC, BORISLAV; SIEGEMUND, JAN; MEUTER, MIRKO
To: DELPHI TECHNOLOGIES, LLC
Reel/Frame 045549/0228 →