Method and apparatus for identifying local features

1. A method of local feature identification and description, comprising the steps of:

retrieving data representing a lightfield, wherein the lightfield is represented by a plenoptic function which comprises at least three dimensions;

forming an epipolar volume (V) from the retrieved data;

applying a transform to epipolar planes retrieved from said epipolar volume, so as to represent said epipolar planes in a new space in which detection of epipolar lines is easier;

identifying a plurality of epipolar lines in the new space;

identifying local features (fp) by selecting a point along at least some of said epipolar lines.

2. The method of claim 1 , wherein said transform is a Radon transform.

3. The method of claim 1 , wherein said transform is a Hough transform.

4. The method of claim 1 , wherein said transform is based on the relation

λ y +λ ln λ x=μ

wherein x and y are coordinates in the epipolar plane, λ is the compressed slope and μ is the outcome of the transform.

5. The method of claim 4 , comprising a step of identifying a line and dividing all the corresponding λi by the largest one, λi is the compressed slope of epipolar line i.

6. The method of claim 4 , further comprising a step of computing a scale-invariant value (g(s)) as an integral over λ wherein λ is the compressed slope of an epipolar line.

7. The method of claim 1 , wherein said transform is a Mellin transform.

8. The method of claim 1 , comprising a step of applying a low-pass filter to data representing said epipolar plane.

9. The method of claim 1 , comprising a step of applying an edge detection algorithm to data representing said epipolar plane.

10. The method of claim 1 , comprising a step of identifying prospective feature points with a corner/feature method applied to an initial image corresponding to said lightfield, and only considering said lines in said new space that correspond to said prospective feature points as possible feature points.

11. The method of claim 1 , comprising a step of determining from said new space lines (l) in said epipolar plane.

12. The method of claim 11 , comprising selecting points in said new space with an intensity higher than a predefined threshold, and determining lines in said epipolar plane that correspond to said selected point.

13. The method of claim 11 , comprising a step of pruning the set of detected lines and only retaining the most stable ones, for example the lines longer than a threshold τ2.

14. The method of claim 1 , comprising a post-processing step applied after said transform step in order to ignore lines for which variance of intensity or color along the points is above some threshold, wherein the post-processing step comprises, comparing the variance of intensity or color along said points with a threshold, identify points which have variance of intensity or color which is below the threshold; omitting those point which are identified.

15. The method of claim 1 , comprising a step of retrieving one point along at least one selected epipolar line and this point to identify a local feature.

16. The method of claim 15 , wherein said local feature is described with a feature vector (fp).

17. The method of claim 16 , wherein said local feature is described with a three dimensional SIFT or BRIEF feature vector (fp).

18. The method of claim 15 , wherein said local feature is described with a two dimensional feature vector (fp) by considering only the image plane in which the feature point falls.

19. The method of claim 1 , further comprising computing a significance ratio for at least some prospective feature points, and omitting those feature points whose significance ration is lower than a threshold.

20. The method of claim 19 , wherein the significance ratio for a prospective feature point depends on the number and/or orientation or epipolar lines passing through a neighborhood of said prospective feature point.

21. The method of claim 1 , comprising a step of matching said local features (fp) with local features extracted from a 2D, 2D and half or 3D image.

22. The method of claim 1 , further comprising a steps of, detecting the dominant orientation of an image which corresponds to said light field, and rotating each of a plurality of images, each of which corresponds to a light field, so that each of the plurality of image has its orientation aligned to a reference orientation computed based on the detected dominant orientation.

23. The method of claim 1 , further comprising steps of, applying a tilting transform to an image which corresponds to said light field, wherein the step of applying a tilting transform to an image comprises scaling and translating the image in one dimension.

24. The method according to claim 23 wherein the tilting transform comprises a Fourier and/or a Mellin transforms.

25. A method according to claim 9 further comprising the step of applying a STILT transform to the data representing said epipolar plane after the edge detection algorithm has been applied.

26. An apparatus for identifying local features in lightfield representing data, comprising a lightfield data retrieving device for retrieving data representing a lightfield; and data processing means programmed for:

forming an epipolar volume (V) from the retrieved data;

applying a transform to epipolar planes retrieved from said epipolar volume, so as to represent said epipolar planes in a new space;

identifying a plurality of epipolar lines;

identifying local features (fp) based on at least some of said epipolar lines.

27. A non-transitory machine readable medium with program code being stored arranged for causing a processor to carry out a method of, local feature identification and description, when said processor executes said program code, said method of local feature identification and description, comprising the steps of:

retrieving data representing a lightfield;

forming an epipolar volume (V) from the retrieved data;

applying a transform to epipolar planes retrieved from said epipolar volume, so as to represent said epipolar planes in a new space;

identifying a plurality of epipolar lines;

identifying local features (fp) based on at least some of said epipolar lines.

28. A method of local feature identification and description, comprising the steps of:

retrieving data representing a lightfield, wherein the light field is represented by a plenoptic function which comprises at least three dimensions;

forming an epipolar volume (V) from the retrieved data;

applying a transform to epipolar planes retrieved from said epipolar volume, so as to represent said epipolar planes in a new space in which detection of epipolar lines is easier;

identifying a plurality of epipolar lines in the new space;

identifying local features (fp) by selecting a point along at least some of said epipolar tines,

wherein said transform comprises the relation

λ y +λ ln λ x=μ

wherein one side of the epipolar plane lies on a first axis and a second side of the epipolar plane lies on a second axis, and wherein x and y are coordinates in the first and second axes respectively and thus are coordinates in the epipolar plane, and wherein λ is the compressed slope and μ is the outcome of the transform; and wherein the method further comprises the step of,

compensating for scaling of the plenoptic images in red data, by carrying out a mathematical operation on the transform of epipolar plane.

29. A method according to claim 1 further comprising the steps of, comparing the length of the epipolar lines which are identified with a predefined threshold length, and omitting epipolar lines which have a length less than a predefined threshold length to ensure that only stable epipolar lines remain.

30. A method according to claim 1 further comprising the steps of, determining coordinates having at least three dimensions for the identified local feature (fp), wherein said three dimensions include at least x and y coordinates for the selected point in the epipolar plane.