Optimal assistance for object-rearrangement tasks in augmented reality

1. A computer-implemented method comprising:

identifying, via an artificial reality system, a plurality of physical objects in a real-world environment of a user;

inferring, based at least on identifying the plurality of objects, a user goal with respect to the plurality of objects;

defining, based on identifying the plurality of objects and the inferred user goal, an object-manipulation objective for manipulating at least one of the plurality of objects;

determining an action sequence that defines a sequence of action steps for manipulating the at least one of the plurality of objects to complete the object-manipulation objective; and

presenting, via the artificial reality system, a notification to the user indicative of the action sequence.

2. The method of claim 1 , wherein the object-manipulation objective defines a desired end state for the at least one of the plurality of objects and the action sequence includes an action step for manipulating the at least one of the plurality of objects to produce the desired end state.

3. The method of claim 1 , further comprising:

monitoring, via the artificial reality system, a progress of the action sequence;

detecting, while monitoring the progress, a deviation from the action sequence; and

updating the action sequence based on the detected deviation.

4. The method of claim 1 , wherein:

inferring the user goal is further based on contextual signals; and

defining the object-manipulation objective further comprises

selecting, based on the inferred user goal, the object-manipulation objective from a library of objectives.

5. The method of claim 4 , wherein the contextual signals include at least one of a time of day, a user input, or a view of the user.

6. The method of claim 4 , wherein:

inferring the user goal further comprises inferring a plurality of user goals with respect to the plurality of objects; and

selecting the object-manipulation objective further comprises selecting a plurality of object-manipulation objectives from the library of objectives based on the plurality of user goals.

7. The method of claim 1 , further comprising identifying a current state of the object-manipulation objective.

8. The method of claim 7 , wherein the current state includes an uncertain state for a physical object associated with the object-manipulation objective.

9. The method of claim 8 , wherein the action sequence includes an action step that is expected to resolve the uncertain state.

10. The method of claim 1 , wherein the plurality of objects includes known objects and unknown objects.

11. The method of claim 1 , wherein the object-manipulation objective includes an object-rearrangement task for relocating the at least one of the plurality of objects to a desired location and the action sequence includes at least one action step for moving the at least one of the plurality of objects to the desired location.

12. The method of claim 1 , wherein the notification includes at least one of a visual notification, a haptic notification, or an audible notification.

13. A system comprising:

at least one physical processor;

physical memory comprising computer-executable instructions that, when executed by the physical processor, cause the physical processor to:

identify, via an artificial reality system, a plurality of physical objects in a real-world environment of a user;

infer, based at least on identifying the plurality of objects, a user goal with respect to the plurality of objects;

define, based on identifying the plurality of objects and the inferred user goal, an object-manipulation objective for manipulating at least one of the plurality of objects;

determine an action sequence that defines a sequence of action steps for manipulating the at least one of the plurality of objects to complete the object-manipulation objective; and

present, via the artificial reality system, a notification to the user indicative of the action sequence.

14. The system of claim 13 , wherein the object-manipulation objective defines a desired end state for the at least one of the plurality of objects and the action sequence includes an action step for manipulating the at least one of the plurality of objects to produce the desired end state.

15. The system of claim 13 , further comprising instructions that cause the physical processor to:

monitor, via the artificial reality system, a progress of the action sequence;

detect, while monitoring the progress, a deviation from the action sequence; and

update the action sequence based on the detected deviation.

16. The system of claim 13 , wherein the instructions for defining the object-manipulation objective further cause the processor to:

infer, further based on contextual signals, the user goal with respect to the plurality of objects; and

select, based on the inferred user goal, the object-manipulation objective from a library of objectives.

17. A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:

identify, via an artificial reality system, a plurality of physical objects in a real-world environment of a user;

infer, based at least on identifying the plurality of objects, a user goal with respect to the plurality of objects;

define, based on identifying the plurality of objects and the inferred user goal, an object-manipulation objective for manipulating at least one of the plurality of objects;

determine an action sequence that defines a sequence of action steps for manipulating the at least one of the plurality of objects to complete the object-manipulation objective; and

present, via the artificial reality system, a notification to the user indicative of the action sequence.

18. The non-transitory computer-readable medium of claim 17 , wherein the object-manipulation objective defines a desired end state for the at least one of the plurality of objects and the action sequence includes an action step for manipulating the at least one of the plurality of objects to produce the desired end state.

19. The non-transitory computer-readable medium of claim 17 , further comprising instructions that cause the computing device to:

monitor, via the artificial reality system, a progress of the action sequence;

detect, while monitoring the progress, a deviation from the action sequence; and

update the action sequence based on the detected deviation.

20. The non-transitory computer-readable medium of claim 17 , wherein the instructions for defining the object-manipulation objective further cause the computing device to:

infer, further based on contextual signals, the user goal with respect to the plurality of objects; and

select, based on the inferred user goal, the object-manipulation objective from a library of objectives.