Annealed sparsity via adaptive and dynamic shrinking

1. A method for acquiring data from an input signal using multitask regression, comprising:

receiving, into a memory, the input signal, the input signal including data that includes a plurality of features;

determining at least two computational tasks to analyze within the input signal;

regularizing all of the at least two tasks using shared adaptive weights;

performing, using a processor, a multitask regression on the input signal to create a solution path for all of the at least two tasks, wherein the multitask regression includes:

concurrently updating a model coefficient and a regularization weight together under an equality norm constraint until convergence is reached to produce a sparse model in the solution path;

concurrently updating the model coefficient and regularization weight together under an updated equality norm constraint that has a greater l 1 -penalty than the previous equality norm constraint until convergence is reached to produce another sparse model in the solution path; and

gradually strengthening a global magnitude of the l 1 -penalty by adjusting the equality norm constraint on the regularization weight by annealing to improve the convergence and reduce a sensitivity on initial conditions,

wherein the shared adaptive weights w=[w 1 , w 2 , . . . , w D ] T are used for the regularizing over all K tasks, as

min

w

,

B

⁢

∑

K

k

=

1

⁢

(



X

k

⁢

β

k

-

y

k



2

2

+



w

-

γ

⊙

β

k



)

,

s.t. Σ d w d =ω, w d ≥0, wherein β k ∈ D×1 represents the model coefficient for a l th task for k=1, 2, . . . K, and B=[β 1 , β 2 , . . . , β k ];

selecting a sparse model from the solution path;

constructing an image using the sparse model; and

displaying the image on a display.

2. The method as recited in claim 1 , wherein the updating the model coefficient and the regularization weight further comprises:

updating the model coefficient by training a Least Absolute Shrinkage and Selection Operator problem with a fixed regularization weight and solving the Least Absolute Shrinkage and Selection Operator problem for the model coefficient; and

updating the regularization weight by training a constrained optimization problem with a fixed model coefficient and solving the constrained optimization problem for the regularization weight.

3. The method as recited in claim 1 , further comprising recording the input signal, using a signal recorder, prior to receiving the input signal.

4. The method as recited in claim 1 , wherein the performing the multitask regression increases a global regularization strength across the at least two tasks.

5. The method as recited in claim 1 , wherein updating the equality norm constraint decreases a number of input signal features in the produced model.

6. The method as recited in claim 5 , wherein the updating the model coefficient and regularization weight together under an updated equality norm constraint is repeated until a model is produced that is without features in the input signal.

7. The method as recited in claim 1 , wherein the selecting a sparse model from the solution path further includes performing a cross-validation of all of the sparse models in the solution path.

8. A system for acquiring data from an input signal using multitask regression, comprising:

a memory to receive the input signal, the input signal including data that includes a plurality of features;

a processor configured to:

determine at least two computational tasks to analyze within the input signal;

regularize all of the at least two tasks using shared adaptive weights;

perform a multitask regression on the input signal to create a solution path for all of the at least two tasks, wherein the processor performs the multitask regression by:

concurrently updating a model coefficient and a regularization weight together under an equality norm constraint until convergence is reached to produce a sparse model in the solution path;

concurrently updating the model coefficient and regularization weight together under an updated equality norm constraint that has a greater l 1 -penalty than the previous equality norm constraint until convergence is reached to produce another sparse model in the solution path; and

gradually strengthening a global magnitude of the l 1 -penalty by adjusting the equality norm constraint on the regularization weight by annealing to improve the convergence and reduce a sensitivity on initial conditions,

wherein the shared adaptive weights w=[w 1 , w 2 , . . . , w D ] T are used for the regularizing over all K tasks, as

min

w

,

B

⁢

∑

K

k

=

1

⁢

(



X

k

⁢

β

k

-

y

k



2

2

+



w

-

γ

⊙

β

k



)

,

s.t. Σ d w d =ω, w d ≥0, wherein β k ∈ D×1 represents the model coefficient for a k th task for k=1, 2, . . . , K, and B=[β 1 , β 2 , . . . , β k ];

select a sparse model from the solution path; and

construct an image using the sparse model; and

a display for displaying the constructed image.

9. The system as recited in claim 8 , wherein the processor updates the model coefficient and the regularization weight together by:

updating the model coefficient by training a Least Absolute Shrinkage and Selection Operator problem with a fixed regularization weight and solving the Least Absolute Shrinkage and Selection Operator problem for the model coefficient; and

updating the regularization weight by training a constrained optimization problem with a fixed model coefficient and solving the constrained optimization problem for the regularization weight.

10. The system as recited in claim 8 , further comprising a signal recorder for recording the input signal.

11. The system as recited in claim 10 , wherein the processor, by performing the multitask regression, increases a global regularization strength across the at least two tasks.

12. The system as recited in claim 8 , wherein updating the equality norm constraint decreases a number of input signal features in the produced model.

13. The system as recited in claim 12 , wherein the processor is further configured to repeat the process of updating the model coefficient and regularization weight together under an updated equality norm constraint until a model is produced that is without features in the input signal.

14. The system as recited in claim 8 , wherein the processor is further configured to perform a cross-validation of all of the sparse models in the solution path prior to selecting a sparse model from the solution path.

15. A non-transitory computer-readable storage medium including a computer-readable program for assigning labels to an object, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:

receiving, into a memory, the input signal, the input signal including data that includes a plurality of features;

determining at least two computational tasks to analyze within the input signal;

regularizing all of the at least two tasks using shared adaptive weights;

performing, using a processor, a multitask regression on the input signal to create a solution path for all of the at least two tasks, wherein the multitask regression includes:

updating a model coefficient and a regularization weight together under an equality norm constraint until convergence is reached to produce a sparse model in the solution path;

updating the model coefficient and regularization weight together under an updated equality norm constraint that has a greater l 1 -penalty than the previous equality norm constraint until convergence is reached to produce another sparse model in the solution path; and

gradually strengthening a global magnitude of the l 1 -penalty by adjusting the equality norm constraint on the regularization weight by annealing to improve the convergence and reduce a sensitivity on initial conditions,

wherein the shared adaptive weights w=[w 1 , w 2 , . . . , w D ] T are used for the regularizing over all K tasks, as

min

w

,

B

⁢

∑

K

k

=

1

⁢

(



X

k

⁢

β

k

-

y

k



2

2

+



w

-

γ

⊙

β

k



)

,

s.t. Σ d w d =ω, w d ≥0, wherein β k ∈ D×1 represents the model coefficient for a k th task for k=1, 2, . . . , K, and B=[β 1 , β 2 , . . . , β k ];

selecting a sparse model from the solution path;

constructing an image using the sparse model; and

displaying the image on a display.

16. The computer-readable storage medium as recited in claim 15 , wherein the updating the model coefficient and the regularization weight further comprises:

updating the model coefficient by training a Least Absolute Shrinkage and Selection Operator problem with a fixed regularization weight and solving the Least Absolute Shrinkage and Selection Operator problem for the model coefficient; and

updating the regularization weight by training a constrained optimization problem with a fixed model coefficient and solving the constrained optimization problem for the regularization weight.

17. The computer-readable storage medium as recited in claim 15 , further comprising recording the input signal, using a signal recorder, prior to receiving the input signal.

18. The computer-readable storage medium as recited in claim 17 , wherein the performing the multitask regression increases a global regularization strength across the at least two tasks.

19. The computer-readable storage medium as recited in claim 15 , wherein updating the equality norm constraint decreases a number of input signal features in the produced model.

20. The computer-readable storage medium as recited in claim 19 , wherein the updating the model coefficient and regularization weight together under an updated equality norm constraint is repeated until a model is produced that is without features in the input signal.