IP Library Granted Patent US 12,032,728
Granted Patent B2
US 12,032,728 · App. 16/973,419 · Granted Jul 9, 2024

Machine interaction

Inventors: Robert Jason Munro (Auckland, NZ); Shane Allan Blackett (Auckland, NZ); Mark Andrew Sagar (Auckland, NZ)
Assignee: SOUL MACHINES LIMITED
G06F3/011G06F2009/45562
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,032,728
App. No.
16/973,419
Granted
Jul 9, 2024
Kind
B2
Abstract

Interaction with a Computer is provided via an autonomous virtual embodied Agent. The Computer outputs Digital Content, which includes any content that exists in the form of digital data and is representable to a User. A subset of, or all, Digital Content is configured as Shared Digital Content which is representable to both the User and to the Agent.

Claims (38)

1. A method for visualising an interaction between an embodied artificial agent and digital content on an end user display device of an electronic computing device the method comprising the steps of:

creating an agent virtual environment, having virtual environment coordinates;

simulating the digital content in the agent virtual environment;

simulating the embodied artificial agent in the agent virtual environment;

enabling the embodied artificial agent to interact with the simulated digital content; and

triggering display on the end user display device the interaction between the embodied artificial agent and the digital content,

wherein when the end user makes a change to the digital content via a user interface, reflecting the change in the simulated digital content in the agent virtual environment.

2. The method of claim 1 wherein the virtual environment is a 3D virtual space, and the virtual environment coordinates are 3D coordinates.

3. The method of claim 2 wherein the interaction is one or more of the group comprising: moving towards the digital content, looking at the digital content, gesturing towards a digital content, or touching the digital content item by moving a multi joint effector.

4. The method of claim 3 wherein digital content is simulated on a plane within the agent virtual environment.

5. A method for interacting with digital content on an electronic computing device via an embodied artificial agent the method comprising the steps of:

triggering display of the digital content to a user on a user interface on the electronic computing device;

creating an agent virtual environment, having virtual environment coordinates;

simulating the digital content in the agent virtual environment;

simulating the embodied artificial agent in the agent virtual environment;

enabling the embodied artificial agent to interact with the simulated digital content;

translating the interaction into an actuation or manipulation of the digital content and triggering actuation or manipulation of the digital content on the user interface; and

triggering display of the interaction by overlaying, underlaying, and/or blending the embodied artificial agent with digital content on the user interface,

wherein when the end user makes a change to the digital content via a user interface, reflecting the change in the simulated digital content in the agent virtual environment.

6. The method of claim 5 wherein the digital content is simulated in the agent virtual environment as a pixel in the agent virtual environment, each pixel having a coordinate location in the agent virtual environment.

7. The method of claim 6 wherein the interaction is an intersection between the coordinates of a pixel corresponding to digital content and a body of the agent.

8. The method of claim 7 wherein translating the simulated embodied interaction includes mapping the embodied interaction to a human input device event.

9. The method of claim 8 wherein the human input device event is one or more of the group comprising: a mouse event, keyboard event or touchscreen event.

10. A system for facilitating interaction with an electronic computing device, the system including:

at least one processor device;

at least one memory device in communication with the at least one processor;

an agent simulator module executable by the processor arranged to simulate an embodied agent in an agent virtual environment;

an interaction module executable by the processor arranged to simulate an embodied agent; an interaction module executable by the processor arranged to translate digital content into conceptual objects perceivable by the embodied agent and enable the embodied agent to interact with the digital content;

a rendering module executable by the processor arranged to render the digital content, the embodied agent and the interaction of the embodied agent with the digital content;

a user interface,

wherein when the end user makes a change to the digital content via a user interface, reflecting the change in the simulated digital content in the agent virtual environment.

11. The system of claim 10 wherein the interaction module is further arranged to translate embodied agent actions on the conceptual object into changes to the digital content.

12. The system of claim 11 wherein the interaction module associates conceptual objects with coordinates representing a location of the conceptual objects relative to the embodied agent.

13. The system of claim 10 wherein the interaction module associates conceptual objects with one or more affordances of the corresponding digital content.

14. The system of claim 13 wherein the computing device provides a web content and the interaction module is a JavaScript code integrated into the web content.

15. The method of claim 1 , wherein the embodied artificial agent operates autonomously.

16. The method of claim 5 , wherein the embodied artificial agent operates autonomously.

17. The system of claim 10 , wherein the embodied agent operates autonomously.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 24, 2026
From: SOUL MACHINES LIMITED
To: APPDIRECT, INC.
Reel/Frame 076058/0831 →
Priority Claims (2)
NZ 744410 · Jul 19, 2018 · national
NZ 755122 · Jul 3, 2019 · national
Continuity (1)
Related Publication 20210216349A1 · Jul 15, 2021
Cited By (5)
US 12,254,128 US 12,518,307 US 12,626,470 US 12,645,288 US 12,646,256