Granted Patent B2

US 12,379,775 · App. 18/479,995 · Granted Aug 5, 2025

Inventor: Sergey Bashkirov (San Mateo, CA)

Assignee: Sony Interactive Entertainment Inc.

G06F3/012G01S5/16G06F3/0325

Abstract

Claims (51)

1. A method comprising:

energizing plural lamps one at a time;

detecting events from energizing the lamps using an event detector sensor (EDS); and

using output of the EDS to generate simultaneous localization and mapping (SLAM) information in an augmented reality (AR) environment at least in part by generating, from the SLAM information, plural motion hypotheses; and

selecting one of the hypotheses for use of the one of the hypotheses to execute SLAM.

2. The method of claim 1 , comprising movably mounting the lamps.

3. The method of claim 1 , comprising movably mounting the EDS.

4. The method of claim 1 , comprising movably mounting the EDS and the lamps.

5. The method of claim 1 , wherein the lamps comprise light emitting diodes (LED).

6. The method of claim 5 , wherein the EDS is a first EDS and method comprises using a second EDS to detect events generated by the LEDs for determining stereo SLAM.

7. The method of claim 1 , wherein the EDS is a first EDS, and the method comprises using information only from the first EDS and no other EDS to determine monocular SLAM.

8. The method of claim 1 , comprising:

determining initial triangulation and pose estimation at least in part by:

receiving at least first and second frames from the EDS generated using plural events from different LEDs;

determining, from the first and second frames, a fundamental matrix;

computing an essential matrix from the fundamental matrix;

decomposing the essential matrix into four motion hypotheses;

selecting a first one of the four motion hypotheses based on the first motion hypothesis having no outliers; and

execute SLAM by triangulating new LED positions using the first motion hypothesis as a reference.

9. A system comprising:

at least one light emitting diode (LED) assembly configured to cause individual LEDs in the assembly to emit light;

at least one event driven sensor (EDS) configured to output signals representing changes in illumination intensity responsive to LED emissions; and

at least one processor configured with instructions to:

determine initial triangulation and pose estimation at least in part by:

receive at least first and second frames from the EDS generated using plural events from different LEDs;

determine, using the first and second frames, at least one matrix;

decompose the matrix into plural motion hypotheses;

select a first one of the plural motion hypotheses; and

use the first motion hypothesis execute simultaneous localization and mapping (SLAM).

10. The system of claim 9 , wherein the EDS is moveable, and the LED assembly is stationary.

11. The system of claim 9 , wherein the EDS is not moveable, and the LED assembly is moveable.

12. The system of claim 9 , wherein the EDS and LED assembly are both moveable.

13. The system of claim 9 , wherein the EDS is a first EDS and the system comprises a second EDS to facilitate the processor determining stereo simultaneous localization and mapping (SLAM).

14. The system of claim 9 , wherein the EDS is a first EDS and the processor receives information only from the first EDS and no other EDS to determine monocular simultaneous localization and mapping (SLAM).

15. A device comprising:

at least one processor system configured for:

energizing plural lamps one at a time;

detecting events from energizing the lamps using an event detector sensor (EDS); and

using output of the EDS to generate simultaneous localization and mapping (SLAM) information in an augmented reality (AR) environment at least in part by generating, from the SLAM information, plural motion hypotheses; and

selecting one of the hypotheses for use of the one of the hypotheses to execute SLAM.

16. The device of claim 15 , wherein the processor system is configured for moving the lamps.

17. The device of claim 15 , wherein the processor system is configured for moving the EDS.

18. The device of claim 15 , wherein the processor system is configured for moving the EDS and the lamps.

19. The device of claim 15 , wherein the EDS is a first EDS, and the processor system is configured for using information only from the first EDS and no other EDS to determine monocular SLAM.

20. The device of claim 15 , wherein the processor system is configured for:

determining initial triangulation and pose estimation at least in part by:

receiving at least first and second frames from the EDS generated using plural events from different LEDs;

determining, from the first and second frames, a fundamental matrix;

computing an essential matrix from the fundamental matrix;

decomposing the essential matrix into four motion hypotheses; selecting a first one of the four motion hypotheses based on the first motion hypothesis having no outliers; and

execute SLAM by triangulating new LED positions using the first motion hypothesis as a reference.

Assignments (1)

Continuity (3)

References Cited (49)