IP Library › Granted Patent US 12,084,208
Granted Patent B2
US 12,084,208 · App. 17/966,944 · Granted Sep 10, 2024

Audio-based assistance during extravehicular activity

Inventors: Hannes Bergkvist (Rydeback, SE); Peter Exner (Malmö, SE)
Assignee: Sony Group Corporation
B64G6/00H04R5/02H04S7/303
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,084,208
App. No.
17/966,944
Granted
Sep 10, 2024
Kind
B2
Abstract

A device is configured to assist an individual during extravehicular activity. The device obtains and operates on first data indicative of a detection space around the individual, and second data indicative of a pose of the head of the individual relative to a helmet, which is worn over the head of the individual with spacing to the head. The device is configured to process the first data to detect an object in the detection space and determine a first position of the object with respect to the helmet; determine, based on the first position and the second data, a second position of the object with respect to the head, and cause an array of speakers in the helmet to generate an audible sound inside the helmet with a spatial origin given by the second position.

Claims (20)

1. A device for assisting an individual during extravehicular activity, said device comprising: a first input for first data indicative of a detection space around the individual; a second input for second data indicative of a pose of the head of the individual relative to a helmet, which is worn over the head of the individual with spacing to allow the head to move inside the helmet; and processor circuitry configured to: obtain the first data on the first input; obtain the second data on the second input; process the first data to detect an object of relevance to the individual in the detection space and determine a first position of the object with respect to the helmet; determine, based on the first position and the second data, a second position of the object with respect to the head, and cause an array of speakers in the helmet to generate an audible sound inside the helmet with a spatial origin given by the second position.

2. The device of claim 1 , wherein the processor circuitry is configured to cause the array of speakers to generate the audible sound with a discriminable characteristic that is indicative of the object.

3. The device of claim 2 , wherein the discriminable characteristic is an audible property of the audible sound other than the spatial origin.

4. The device of claim 2 , wherein the processor circuitry is configured to process the first data for detection of a hazard associated with the object, and wherein the discriminable characteristic is indicative of the hazard.

5. The device of claim 4 , wherein the processor circuitry is further configured to process the first data for classification of the hazard into a hazard category among a plurality of a hazard categories, wherein the discriminable characteristic is indicative of the hazard category and differs between the hazard categories.

6. The device of claim 2 , wherein the processor circuitry is configured to process the first data for classification of the object into an object category among a plurality of object categories, wherein the discriminable characteristic is indicative of the object category and differs between the object categories.

7. The device of claim 2 , wherein the processor circuitry is configured to modify the discriminable characteristic as a function of a distance between the object and the individual or another individual in the detection space and/or a movement direction of the object in relation to the individual or said another individual in the detection space.

8. The device of claim 2 , wherein the processor circuitry is configured to modify the discriminable characteristic as a function of the pose of the head.

9. The device of claim 2 , wherein the processor circuitry is configured to modify the discriminable characteristic in dependence of a relation between the object and a field of vision of the individual.

10. The device of claim 9 , wherein the processor circuitry is configured to calculate the field of vision of the individual through a visor of the helmet based on the second data, and determine the relation between the object and the field of vision.

11. The device of claim 9 , wherein the processor circuitry is configured to generate the audible sound with different discriminable characteristics if the object is within or outside the field of vision.

12. The device of claim 2 , wherein the processor circuitry is configured to estimate, based on the first data, a trajectory of the object, and to modify the discriminable characteristic as a function of the trajectory.

13. The device of claim 1 , wherein the processor circuitry is configured to determine, based on the first data, a waypoint with respect to the helmet on a path from a first location of the individual to a second location, determine a position of the waypoint with respect to the head based on the second data, and generate a further audible sound with a spatial origin given by the position of the waypoint with respect to the head.

14. The device of claim 13 , wherein the processor circuitry is further configured to cause the array of speakers to verbalize the path.

15. The device of claim 1 , wherein the processor circuitry is configured to cause the array of speakers to verbalize object data representative of the object.

16. The device of claim 15 , wherein the object data comprises one or more of: a characterization of the object, a distance to the object, a direction to the object, or a movement instruction to the individual.

17. The device of claim 1 , further comprising at least one of the helmet, the array of speakers, a sensor arrangement configured to generate the first data, or a head tracking device configured to generate the second data.

18. The device of claim 17 , wherein the sensor arrangement is configured to detect one or more of a shape, a distance, a temperature, or a composition.

19. A space suit for extravehicular activity, said space suit comprising a device according to claim 1 .

20. A computer-implemented method for assisting an individual that wears a helmet during extravehicular activity, the helmet allowing the individual to move its head within the helmet, said computer-implemented method comprising: obtaining first data indicative of a detection space around the individual; obtaining second data indicative of a pose of the head of the individual relative to the helmet; detecting, based on the first data, an object of relevance to the individual in the detection space; determining, based on the first data, a first position of the object with respect to the helmet; determining, based on the first position and the second data, a second position of the object with respect to the head; and causing an array of speakers in the helmet to generate an audible sound with a spatial origin given by the second position.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2023
From: BERGKVIST, HANNES; EXNER, PETER
To: SONY EUROPE BV
Reel/Frame 065261/0094 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2023
From: SONY EUROPE BV
To: SONY GROUP CORPORATION
Reel/Frame 065261/0146 →
Priority Claims (1)
SE 2151348-6 · Nov 3, 2021 · national
Continuity (1)
Related Publication 20230139295A1 · May 4, 2023