Scene graph-based scene re-identification
Implementations disclosed herein provide systems and methods that match a current scene graph associated with a user's current environment to a prior scene graph for a prior environment to determine that the user is in the same environment. The current scene graph is compared with the prior scene graph based on matching the objects within the current scene graph with objects within the prior scene graph.
1. A method comprising:
at a processor:
generating a scene graph for a current environment based on one or more images of the current environment, the scene graph for the current environment representing objects within the current environment and relationships between the objects within the current environment
comparing the scene graph for the current environment with a scene graph for a prior environment based on matching the objects within the scene graph for the current environment with objects within the scene graph for the prior environment;
determining that the current environment matches the prior environment based on the matching; and
based on said determining that the current environment matches the prior environment, determining that a current location associated with the user is within the prior environment.
2. The method of claim 1 further comprising comparing object metadata associated with the objects within the scene graph for the current environment with object metadata associated with the objects within the scene graph for the prior environment.
3. The method of claim 1 further comprising comparing object-specific meshes associated with the objects within the scene graph for the current environment with object-specific meshes associated with the objects within the scene graph for the prior environment.
4. The method of claim 1 further comprising comparing object-specific keyframes associated with the objects within the scene graph for the current environment with object-specific keyframes associated with the objects within the scene graph for the prior environment.
5. The method of claim 1 further comprising comparing object location information or relationships associated with the objects within the scene graph for the current environment with object location information or relationships associated with the objects within the scene graph for the prior environment.
6. The method of claim 1 , wherein one or more types of objects are excluded from the comparison of the scene graph for the current environment with the scene graph for the prior environment.
7. The method of claim 6 , wherein object types that are transient are excluded from the comparison of the scene graph for the current environment with the scene graph for the prior environment.
8. The method of claim 1 , wherein matching the objects within the scene graph for the current environment with the objects within the scene graph for the prior environment includes a multi-step matching process.
9. The method of claim 8 , wherein the multi-step matching process includes refining a matching score.
10. The method of claim 1 , wherein the multi-step matching process is discontinued based on criteria or a low score.
11. The method of claim 1 further comprising identifying that a user is in the same room based on the determination that the current environment matches the prior environment.
12. The method of claim 1 further comprising identifying that a user is in a similar room based on the determination that the current environment matches the prior environment.
13. The method of claim 1 further comprising enhancing the current environment based on the prior environment and the determination that the current environment matches the prior environment.
14. A system comprising:
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
generating a scene graph for a current environment based on one or more images of the current environment, the scene graph for the current environment representing objects within the current environment and relationships between the objects within the current environment;
comparing the scene graph for the current environment with a scene graph for a prior environment based on matching the objects within the scene graph for the current environment with objects within the scene graph for the prior environment;
determining that the current environment matches the prior environment based on the matching; and
based on said determining that the current environment matches the prior environment, determining that a current location associated with the user is within the prior environment.
15. The system of claim 14 , the operations further comprising comparing object metadata associated with the objects within the scene graph for the current environment with object metadata associated with the objects within the scene graph for the prior environment.
16. The system of claim 14 , the operations further comprising comparing object-specific meshes associated with the objects within the scene graph for the current environment with object-specific meshes associated with the objects within the scene graph for the prior environment.
17. The system of claim 14 , the operations further comprising comparing object-specific keyframes associated with the objects within the scene graph for the current environment with object-specific keyframes associated with the objects within the scene graph for the prior environment.
18. The system of claim 14 , the operations further comprising comparing object location information or relationships associated with the objects within the scene graph for the current environment with object location information or relationships associated with the objects within the scene graph for the prior environment.
19. The system of claim 14 , wherein one or more types of objects are excluded from the comparison of the scene graph for the current environment with the scene graph for the prior environment.
20. A non-transitory computer-readable storage medium, storing program instructions computer-executable on a computer to perform operations comprising:
generating a scene graph for a current environment based on one or more images of the current environment, the scene graph for the current environment representing objects within the current environment and relationships between the objects within the current environment; and
comparing the scene graph for the current environment with a scene graph for a prior environment based on matching the objects within the scene graph for the current environment with objects within the scene graph for the prior environment;
determining that the current environment matches the prior environment based on the matching; and
based on said determining that the current environment matches the prior environment, determining that a current location associated with the user is within the prior environment.