Granted Patent B2

US 12,405,160 · App. 17/610,530 · Granted Sep 2, 2025

Inventor: Tetsugo Inada (Tokyo, JP)

Assignee: Sony Interactive Entertainment Inc.

G01J1/42H04N25/50H04N25/702

Abstract

Claims (64)

1. A system comprising:

an event-driven vision sensor including a sensor array that includes a first sensor and a second sensor arrayed in a predetermined pattern, wherein,

the first sensor and the second sensor are alternately arrayed in each of a first direction and a second direction orthogonal to the first direction, and configured to detect object movement,

the first sensor being configured to generate: (i) a first event signal when detecting a change in intensity of light larger than a first threshold, (ii) the first event signal having no different characteristics when the change in intensity of light is of a magnitude larger than any and all intensities above the first threshold, and (iii) no first event signal when the change in intensity of light is not larger than the first threshold,

the second sensor being configured to generate: (i) a second event signal when detecting a change in intensity of light larger than a second threshold that is larger than the first threshold, (ii) the second event signal having no different characteristics when the change in intensity of light is of a magnitude larger than any and all intensities above the second threshold, and (iii) no second event signal when the change in intensity of light is not larger than the second threshold; and

an information processing device including one or more processors configured to:

execute first processing when the first event signal generated by the first sensor is received and the second event signal generated by the second sensor is not received, the first processing indicating a first movement of a first object based on the change in intensity of light being larger than the first threshold; and

a execute second processing different from the first processing when the second event signal generated by the second sensor is received, the second processing indicating a second movement of a second object based on the change in intensity of light being larger than the second threshold.

2. The system according to claim 1 , wherein

the sensor array further includes a third sensor arrayed together with the first sensor and the second sensor, the third sensor configured to generate a third event signal when detecting a change in intensity of light larger than a third threshold that is larger than the second threshold, and

wherein the the one or more processors are further configured to execute third processing based on the third event signal.

3. The system according to claim 1 , wherein

the one or more processors are configured to identify a first event having a first label, and

a second event having a second label, and

wherein the one or more processors are further configured to determine, in a case where a number of the second events is equal to or larger than a predetermined threshold, a location of a device having mounted thereon the event-driven vision sensor only using the second event, and to determine a location of the device using the first event and the second event in a case where the number of the second events is smaller than the predetermined threshold.

4. The system of claim 1 , wherein the first threshold corresponds to a luminance change amount associated with a movement of a non-light emitting body, and the second threshold corresponds to a luminance change amount associated with a movement or one or more flashes of a light emitting body.

5. The system of claim 1 , wherein, the one or more processors are configured to determine confidence of an event using the second event signal from the second sensor upon detection of the first event signal from the first sensor near the second sensor to achieve high event resolution.

6. The system of claim 1 , wherein the first sensor and the second sensor are unevenly arranged.

7. The system of claim 3 , wherein the device is a game controller.

8. An information processing device comprising:

memory storing instructions;

one or more processors, that upon execution of the instructions, configured to:

receive one or more event signals, including at least one of a first event signal from a first sensor or a second event signal from a second sensor;

configured to execute first processing when a first event signal is received and a second event signal is not received and further configured to second processing different from the first processing when the second event signal is received, wherein:

the first sensor is configured to generate: (i) the first event signal when detecting a change in intensity of light larger than a first threshold, the first event signal having no different characteristics when the change in intensity of light is of a magnitude larger than any and all intensities above the first threshold, and (ii) no first event signal when the change in intensity of light is not larger than the first threshold,

the second sensor is configured to generate: (i) the second event signal when detecting a change in intensity of light larger than a second threshold that is larger than the first threshold, the second event signal having no different characteristics when the change in intensity of light is of a magnitude larger than any and all intensities above the second threshold, and (ii) no second event signal when the change in intensity of light is not larger than the second threshold, and

the first sensor and the second sensor being alternately arrayed in each of a first direction and a second direction orthogonal to the first direction; and

wherein the first processing is indicative of a first movement of a first object based on the change in intensity of light being larger than the first threshold, and the second processing is indicative of a second movement of a second object based on the change in intensity of light being larger than the second threshold.

9. The information processing device of claim 8 , wherein the one or more processors are configured to identify a first event having a first label, and a second event having a second label, and

wherein the one or more processors are further configured to determine, in a case where a number of the second events is equal to or larger than a predetermined threshold, a location of a device having mounted thereon the event-driven vision sensor only using the second event, and to determine a location of the device using the first event and the second event in a case where the number of the second events is smaller than the predetermined threshold.

10. The information processing device of claim 8 , wherein the first threshold corresponds to a luminance change amount associated with a movement of a non-light emitting body, and the second threshold corresponds to a luminance change amount associated with a movement or one or more flashes of a light emitting body.

11. The information processing device of claim 8 , wherein, the one or more processors are configured to determine confidence of an event using the second event signal from the second sensor upon detection of the first event signal from the first sensor near the second sensor to achieve high event resolution.

12. An information processing method comprising:

receiving one or more event signals, including at least one of a first event signal from a first sensor or a second event signal from a second sensor;

executing first processing upon receiving the first event signal without the second event signal; and

executing second processing different from the first processing when the second event signal is received,

wherein:

the first sensor is configured to generate: (i) the first event signal when detecting a change in intensity of light larger than a first threshold, the first event signal having no different characteristics when the change in intensity of light is of a magnitude larger than any and all intensities above the first threshold, and (ii) no first event signal when the change in intensity of light is not larger than the first threshold,

the second sensor is configured to generate: (i) the second event signal when detecting a change in intensity of light larger than a second threshold that is larger than the first threshold, the second event signal having no different characteristics when the change in intensity of light is of a magnitude larger than any and all intensities above the second threshold, and (ii) no second event signal when the change in intensity of light is not larger than the second threshold, and

the first sensor and the second sensor being alternately arrayed

in each of a first direction and a second direction orthogonal to the first direction, and

wherein the first processing is indicative of a first movement of a first object based on the change in intensity of light being larger than the first threshold, and the second processing is indicative of a second movement of a second object based on the change in intensity of light being larger than the second threshold.

13. The information processing method of claim 12 , further comprising:

identifying a first event having a first label, and a second event having a second label;

determining, in a case where a number of the second events is equal to or larger than a predetermined threshold, a location of a device having mounted thereon the event-driven vision sensor only using the second event; and

determining a location of the device using the first event and the second event in a case where the number of the second events is smaller than the predetermined threshold.

14. The information processing method of claim 12 , wherein the first threshold corresponds to a luminance change amount associated with a movement of a non-light emitting body, and the second threshold corresponds to a luminance change amount associated with a movement or one or more flashes of a light emitting body.

15. The information processing method of claim 12 , further comprising determining confidence of an event using the second event signal from the second sensor upon detection of the first event signal from the first sensor near the second sensor to achieve high event resolution.

16. A non-transitory, computer readable storage medium containing a program, which when executed by a computer, causes the computer to perform an information processing method by carrying out actions, comprising:

receiving one or more event signals, including at least one of a first event signal from a first sensor or a second event signal from a second sensor;

executing first processing upon receiving first event signal without the second event signal; and

executing second processing different from the first processing when the second event signal is received,

wherein:

the first sensor is configured to generate: (i) the first event signal when detecting a change in intensity of light larger than a first threshold, the first event signal having no different characteristics when the change in intensity of light is of a magnitude larger than any and all intensities above the first threshold, and (ii) no first event signal when the change in intensity of light is not larger than the first threshold,

the second sensor is configured to generate: (i) the second event signal when detecting a change in intensity of light larger than a second threshold that is larger than the first threshold, the second event signal having no different characteristics when the change in intensity of light is of a magnitude larger than any and all intensities above the second threshold, and (ii) no second event signal when the change in intensity of light is not larger than the second threshold, and

the first sensor and the second sensor being alternately arrayed

in each of a first direction and a second direction orthogonal to the first direction, and

wherein the first processing is indicative of a first movement of a first object based on the change in intensity of light being larger than the first threshold, and the second processing is indicative of a second movement of a second object based on the change in intensity of light being larger than the second threshold.

17. The non-transitory, computer readable storage medium of claim 16 , wherein the actions further comprise:

identifying a first event having a first label, and a second event having a second label;

determining, in a case where a number of the second events is equal to or larger than a predetermined threshold, a location of a device having mounted thereon the event-driven vision sensor only using the second event; and

determining a location of the device using the first event and the second event in a case where the number of the second events is smaller than the predetermined threshold.

18. The non-transitory, computer readable storage medium of claim 16 , wherein the first threshold corresponds to a luminance change amount associated with a movement of a non-light emitting body, and the second threshold corresponds to a luminance change amount associated with a movement or one or more flashes of a light emitting body.

19. The non-transitory, computer readable storage medium of claim 16 , wherein the actions further comprise determining confidence of an event using the second event signal from the second sensor upon detection of the first event signal from the first sensor near the second sensor to achieve high event resolution.

Assignments (1)

Continuity (1)

References Cited (32)