IP Library Granted Patent US 11,983,922
Granted Patent B2
US 11,983,922 · App. 17/585,778 · Granted May 14, 2024

Smart home management system for generating augmented reality scene of potentially hazardous condition

Inventor: Serhad Doken (Bryn Mawr, PA)
Assignee: ROVI GUIDES, INC.
G06V20/20G06T11/00G06V20/52G06V40/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,983,922
App. No.
17/585,778
Filed
Jan 27, 2022
Granted
May 14, 2024
Kind
B2
Examiner
GE, JIN
Art Unit
2612
USPC
345/633
Abstract

Systems and methods are described for identifying, using a sensor, a location of an object in an environment, and determining a classification of the object. A human may be identified in proximity with the object, an identity of the human may be determined, and a determination may be made that a hazardous condition may occur, based on a combination of the location of the object, the classification of the object, and the identity of the human in proximity with the object. In response to determining that the hazardous condition may occur, an augmented reality scene associated with the potentially hazardous condition associated with the object may be generated for presentation, at a user device.

Claims (72)

1. A method comprising:

identifying, using a sensor, a location of an object in an environment;

determining a classification of the object;

identifying a human in proximity with the object;

determining that a hazardous condition may occur, based on a combination of the location of the object, the classification of the object, and the identified human;

and

in response to determining that the hazardous condition may occur:

identifying a mitigating object that is not present in the environment by determining that the mitigating object, if present in the environment, would decrease the risk of the hazardous condition occurring for the identified human; and

generating for presentation, at a user device, an augmented reality scene that comprises an AR overlay of an image of the mitigating object over the environment.

2. The method of claim 1 , wherein the user device is associated with a user different from the identified human or is associated with the identified human.

3. The method of claim 1 , further comprising:

determining that a weather event is associated with the environment; and

determining that the hazardous condition may occur based at least in part of the determined weather event.

4. The method of claim 1 , wherein the augmented reality scene comprises one or more of:

an indication of an accident that may occur as a result of the potentially hazardous condition; or

a recommendation of a corrective action to ameliorate the potentially hazardous condition.

5. The method of claim 1 , wherein determining that the hazardous condition may occur further comprises:

determining that the object is in proximity to an additional object; and

determining that a database stores an indication that the object being in proximity to the additional object is indicative that the hazardous condition may occur.

6. The method of claim 1 , further comprising:

receiving input to mark an object as being associated with a potentially hazardous condition; and

causing a database to store an indication that the marked object is associated with a potentially hazardous condition.

7. The method of claim 1 , further comprising:

generating a plurality of user profiles, wherein at least one of the user profiles is associated with a respective user device;

wherein identifying the human in proximity with the object comprises:

determining a current location in the environment of the user device associated with a user profile of the human; and

determining that the current location in the environment of the user device associated with the user profile is proximate to the object associated with the potentially hazardous condition.

8. The method of claim 7 , wherein generating for presentation, at the user device, the augmented reality scene further comprises:

activating a camera of the user device in response to determining that the current location in the environment of the user device associated with the user profile is proximate to the object associated with the potentially hazardous condition; and

capturing, using the camera, an image of the object, and enhancing the image of the object with an augmented reality effect to render the augmented reality scene.

9. The method of claim 1 , further comprising:

identifying a vulnerability of the identified human based on age and health; and

determining that a hazardous condition may occur based on the vulnerability of the identified human.

10. The method of claim 1 , further comprising:

determining a frequency that the identified human is in proximity with the object; and

determining that a hazardous condition may occur based on the frequency that the identified human is in proximity with the object.

11. The method of claim 1 , wherein:

the augmented reality scene comprises a video that provides instructions for how to install the mitigating object in the environment.

12. A system comprising:

a sensor; and

control circuitry configured to:

identify, using the sensor, a location of an object in an environment;

determine a classification of the object;

identify a human in proximity with the object;

determine that a hazardous condition may occur, based on a combination of the location of the object, the classification of the object, and the identified human;

and

in response to determining that the hazardous condition may occur:

identify a mitigating object that is not present in the environment by determining that the mitigating object, if present in the environment, would decrease the risk of the hazardous condition occurring for the identified human; and

generate for presentation, at a user device, an augmented reality scene that comprises an AR overlay of an image of the mitigating object over the environment.

13. The system of claim 12 , wherein the user device is associated with a user different from the identified human or is associated with the identified human.

14. The system of claim 12 , further comprising:

determining that a weather event is associated with the environment; and

determining that the hazardous condition may occur based at least in part of the determined weather event.

15. The system of claim 12 , wherein the augmented reality scene comprises one or more of:

an indication of an accident that may occur as a result of the potentially hazardous condition; or

a recommendation of a corrective action to ameliorate the potentially hazardous condition.

16. The system of claim 12 , wherein the control circuitry is configured to determine that the hazardous condition may occur by:

determining that the object is in proximity to an additional object; and

determining that a database stores an indication that the object being in proximity to the additional object is indicative that the hazardous condition may occur.

17. The system of claim 12 , wherein the control circuitry is further configured to:

receive input to mark an object as being associated with a potentially hazardous condition; and

cause a database to store an indication that the marked object is associated with a potentially hazardous condition.

18. The system of claim 12 , wherein the control circuitry is further configured to:

generate a plurality of user profiles, wherein at least one of the user profiles is associated with a respective user device; and

identify the human in proximity with the object by:

determining a current location in the environment of the user device associated with a user profile of the human; and

determining that the current location in the environment of the user device associated with the user profile is proximate to the object associated with the potentially hazardous condition.

19. The system of claim 18 , wherein the control circuitry is configured to generate for presentation, at the user device, the augmented reality scene by:

activating a camera of the user device in response to determining that the current location in the environment of the user device associated with the user profile is proximate to the object associated with the potentially hazardous condition; and

capturing, using the camera, an image of the object, and enhancing the image of the object with an augmented reality effect to render the augmented reality scene.

20. The system of claim 12 , wherein:

the augmented reality scene comprises a video that provides instructions for how to install the mitigating object in the environment.

Assignments (3)
CHANGE OF NAME Recorded Oct 4, 2024
From: ROVI GUIDES, INC.
To: ADEIA GUIDES INC.
Reel/Frame 069113/0406 →
SECURITY INTEREST Recorded May 3, 2023
From: ADEIA GUIDES INC.; ADEIA IMAGING LLC; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA SOLUTIONS LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063529/0272 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2022
From: DOKEN, SERHAD
To: ROVI GUIDES, INC.
Reel/Frame 058942/0031 →
Continuity (1)
Related Publication 20230237797A1 · Jul 27, 2023