Semantic segmentation based on global optimization
Methods and systems for data segmentation include determining initial segmentation scores for each unit of an input data set using a neural network, with each unit being assigned an initial segmentation score for each of multiple segmentation classes. Final segmentation scores are determined for each unit of the input data set by enforcing a smoothness criterion. The input data set is segmented in accordance with the final segmentation scores.
1. A method for data segmentation, comprising:
determining initial segmentation scores for each unit of an input data set using a neural network, with each unit being assigned an initial segmentation score for each of a plurality of segmentation classes;
determining final segmentation scores for each unit of the input data set by enforcing a smoothness criterion;
segmenting the input data set in accordance with the final segmentation scores,
wherein the determining final segmentation scores comprises applying variational reaction diffusion (VRD),
wherein the applying VRD comprises solving an optimization problem based on the initial segmentation scores and on positive-definite parameter matrices, and
wherein the optimization problem is expressed as:
arg
min
s
o
∫
I
s
T
Qs
+
∑
k
=
1
2
∂
s
T
∂
x
k
B
∂
s
∂
x
k
dx
where s o is a vector of the final segmentation scores for a unit of data, x is a unit of data in data set I, d is a dimension of the data set I, and s is a vector combining s o and a vector of the initial segmentation scores for the unit of data; and
backpropagating information for efficiently training the neural network using the final segmentation scores and an error signal, wherein computational complexity for the training is reduced by reducing computations for the training to the same steps for generating the final segmentation scores.
2. The method of claim 1 , wherein solving the optimization problem comprises solving a system of partial differential equations.
3. The method of claim 2 , wherein applying VRD comprises converting the optimization problem to the system of partial differential equations using Schur decomposition on the positive-definite parameter matrices.
4. The method of claim 2 , wherein the system of partial differential equations is expressed as:
Δ
z
k
-
U
kk
z
k
=
(
V
T
(
B
o
)
-
1
s
p
)
k
+
∑
j
=
k
+
1
N
o
U
kj
z
j
where s P is defined as the expression Q i s i −B i Δs i , Q i , Q o , B i , and B o are sub-matrices of the positive-definite parameter matrices, Δ is the vector Laplacian operator, s i is the initial segmentation scores for a unit of data, each z k represents a transformed score on a k th segmentation class, N o is a number of segmentation classes, and V and U are transformed versions of the positive-definite parameter matrices after Schur decomposition.
5. The method of claim 1 , wherein backpropagating information comprises determining derivatives of the error signal with respect to the positive-definite parameter matrices.
6. The method of claim 1 , wherein the input data set is an image and each unit of the input data set is a pixel in the image.
7. A system for data segmentation, comprising:
a neural network configured to determine initial segmentation scores for each unit of an input data set, with each unit being assigned an initial segmentation score for each of a plurality of segmentation classes;
a segmentation module configured to determine final segmentation scores for each unit of the input data set by enforcing a smoothness criterion and to segment the input data set in accordance with the final segmentation scores,
wherein the segmentation module is further configured to apply variational reaction diffusion (VRD) to determine the final segmentation scores.,
wherein the segmentation module is further configured to solve a VRD optimization problem based on the initial segmentation scores and on positive-definite parameter matrices,
wherein the optimization problem is expressed as:
arg
min
s
o
∫
I
s
T
Qs
+
∑
k
=
1
2
∂
s
T
∂
x
k
B
∂
s
∂
x
k
dx
where s o is a vector of the final segmentation scores for a unit of data, x is a unit of data in data set I, d is a dimension of the data set I, and s is a vector combining s o and a vector of the initial segmentation scores for the unit of data; and
wherein the neural network is further configured to backpropagate information for training the neural network using the final segmentation scores and an error signal, wherein computational complexity for the training is reduced by reducing computations for the training to the same steps for generating the final segmentation scores.
8. The system of claim 7 , wherein the segmentation module is further configured to solve a system of partial differential equations to solve the optimization problem.
9. The system of claim 8 , wherein the segmentation module is further configured to convert the optimization problem to the system of partial differential equations using Schur decomposition on the positive-definite parameter matrices.
10. The system of claim 8 , wherein the system of partial differential equations is expressed as:
Δ
z
k
=
U
kk
z
k
=
(
V
T
(
B
o
)
-
1
s
p
)
k
+
∑
j
=
k
+
1
N
o
U
kj
z
j
where s p is defined as the expression Q i s i −B i Δs i , Q i , Q o , B i , and B o are sub-matrices of the positive-definite parameter matrices, Δ is the vector Laplacian operator, s i is the initial segmentation scores for a unit of data, each z k represents a transformed score on a k th segmentation class, N o , is a number of segmentation classes, and V and U are transformed versions of the positive-definite parameter matrices after Schur decomposition.
11. The system of claim 7 , wherein the neural network is further configured to determine derivatives of the error signal with respect to the positive-definite parameter matrices.
12. The system of claim 7 , wherein the input data set is an image and each unit of the input data set is a pixel in the image.