Granted Patent B2

US 12,437,484 · App. 18/186,478 · Granted Oct 7, 2025

Inventors: Zachary A. Silverstein (Georgetown, TX); Logan Bailey (Atlanta, GA); Dylan Zucker (New York, NY); Thomas Jefferson Sandridge (Tampa, FL)

Assignee: International Business Machines Corporation

G06T19/006

Abstract

Claims (50)

1. A processor implemented method for mixed-reality peripheral interfacing, the method comprising:

recording, with a camera, video footage of a peripheral held by a user participating in a mixed-reality environment;

extracting, by a processor, peripheral data from the video footage;

responsive to determining that the peripheral does not match a default peripheral, modifying a pre-existing virtual representation of the peripheral to reflect a visual appearance and input means comprising the peripheral; and

responsive to determining that one or more keybindings do not match the peripheral, rebinding the one or more keybindings of the peripheral based on the peripheral data.

2. The method of claim 1 , further comprising:

displaying a virtual representation of the peripheral to the user within the mixed-reality environment based on the peripheral data.

3. The method of claim 2 , further comprising:

extracting input data entered into the peripheral by the user; and wherein the virtual representation visually represents user inputs in real time based on the input data.

4. The method of claim 2 , wherein the displaying comprises modifying a pre-existing visual representation of the peripheral based on the peripheral data.

5. The method of claim 1 , further comprising:

extracting interface data from the video footage;

extracting input data entered into the peripheral by the user; and

displaying a virtual representation of the user interacting with the virtual representation of the peripheral within the mixed-reality environment based on the interface data and the input data.

6. The method of claim 1 , further comprising:

responsive to determining that the user has navigated to a keybinding reference screen, rendering a visual representation of the peripheral for a duration of time that the user remains on the keybinding reference screen based on the peripheral data.

7. A computer system for mixed-reality peripheral interfacing, the computer system comprising:

one or more processors, one or more computer readable memories and one or more computer readable storage media;

program instructions, stored on at least one of the one or more storage media for execution by at least one of the one or more processors via at least one of the one or more memories, the program instructions executable to:

record, with a camera, video footage of a peripheral held by a user participating in a mixed-reality environment;

extract, by a processor, peripheral data from the video footage; and

responsive to determining that the peripheral does not match a default peripheral, modify a pre-existing virtual representation of the peripheral to reflect a visual appearance and input means comprising the peripheral; and

responsive to determining that one or more keybindings do not match the peripheral, rebind the one or more keybindings of the peripheral based on the peripheral data.

8. The computer system of claim 7 , further comprising:

program instructions to display a virtual representation of the peripheral to the user within the mixed-reality environment based on the peripheral data.

9. The computer system of claim 8 , further comprising:

program instructions to extract input data entered into the peripheral by the user; and wherein the virtual representation visually represents user inputs in real time based on the input data.

10. The computer system of claim 8 , wherein the program instruction to display comprises modifying a pre-existing visual representation of the peripheral based on the peripheral data.

11. The computer system of claim 7 , further comprising:

program instructions to extract interface data from the video footage;

program instructions to extract input data entered into the peripheral by the user; and

program instructions to display a virtual representation of the user interacting with the virtual representation of the peripheral within the mixed-reality environment based on the interface data and the input data.

12. The computer system of claim 7 , further comprising:

program instructions to, responsive to determining that the user has navigated to a keybinding reference screen, render a visual representation of the peripheral for a duration of time that the user remains on the keybinding reference screen based on the peripheral data.

13. A computer program product for mixed-reality peripheral interfacing, the computer program product comprising:

one or more computer readable storage media;

program instructions, stored on at least one of the one or more storage media, the program instructions executable by one or more processors, the program instructions executable to:

record, with a camera, video footage of a peripheral held by a user participating in a mixed-reality environment;

extract, by a processor, peripheral data from the video footage;

responsive to determining that the peripheral does not match a default peripheral, modify a pre-existing virtual representation of the peripheral to reflect a visual appearance and input means comprising the peripheral; and

responsive to determining that one or more keybindings do not match the peripheral, rebind the one or more keybindings of the peripheral based on the peripheral data.

14. The computer program product of claim 13 , further comprising:

program instructions to display a virtual representation of the peripheral to the user within the mixed-reality environment based on the peripheral data.

15. The computer program product of claim 14 , further comprising:

program instructions to extract input data entered into the peripheral by the user; and wherein the virtual representation visually represents user inputs in real time based on the input data.

16. The computer program product of claim 14 , wherein the program instruction to display comprises modifying a pre-existing visual representation of the peripheral based on the peripheral data.

17. The computer program product of claim 13 , further comprising:

program instructions to extract interface data from the video footage;

program instructions to extract input data entered into the peripheral by the user; and

program instructions to display a virtual representation of the user interacting with the virtual representation of the peripheral within the mixed-reality environment based on the interface data and the input data.

Assignments (1)

Continuity (1)

References Cited (11)