IP Library Granted Patent US 12,019,841
Granted Patent B2
US 12,019,841 · App. 18/178,475 · Granted Jun 25, 2024

Apparatus management and control method and device

Inventors: Yong Duan (Beijing, CN); Xiao Meng (Beijing, CN)
Assignee: LENOVO (BEIJING) LIMITED
G06F3/04815G02B27/0179G06F3/012G06F3/0486G06F3/1454G06T7/55G06T7/70G02B2027/0187G06T2207/10028G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,019,841
App. No.
18/178,475
Granted
Jun 25, 2024
Kind
B2
Abstract

An apparatus management and control method includes obtaining an operation position of a target operation of a user at a holographic screen, determining a target apparatus located in a same space as the holographic screen based on the operation position, and controlling the target apparatus based on the target operation.

Claims (71)

1. An apparatus management and control method comprising:

obtaining an operation position of a target operation of a user at a holographic screen, the target operation including a drag operation;

determining a target apparatus located in a same space as the holographic screen based on the operation position; and

controlling the target apparatus based on the target operation, including:

based on the drag operation, displaying, on the holographic screen, a display content that is displayed by a screen of the terminal apparatus; or

based on the drag operation, displaying, on the screen of the terminal apparatus, a display content that is displayed by the holographic screen.

2. The method of claim 1 , wherein determining the target apparatus located in the same space with the holographic screen based on the operation position includes:

obtaining a line of sight position of a user; and

matching an apparatus position of an apparatus located in the same space with the holographic screen based on the operation position and the line of sight position to determine the target apparatus.

3. The method of claim 2 , further comprising, before obtaining the operation position of the target operation of the user at the holographic screen:

obtaining line of sight positions of the user at at least two different positions;

determining the operation position at the holographic screen corresponding to the user when the user points to the apparatus in the same space with the same holographic screen at each of the positions;

determining a mapping relationship set based on the line of sight position and associated operation position corresponding to each of the positions; and

determining the apparatus position of the apparatus based on at least two mapping relation sets.

4. The method of claim 3 , wherein obtaining the line of sight positions of the user at the at least two different positions includes:

collecting at least two depth images corresponding to the user located at the at least two different positions; and

determining the line of sight positions of the user at the at least two different positions based on the at least two depth images.

5. The method of claim 1 , wherein determining the target apparatus located in the same space with the holographic screen based on the operation position includes:

obtaining a line of sight position of the user;

collecting an operation image in a predetermined range determined based on the operation position based on the line of sight position of the user; and

identifying the target apparatus in the operation image.

6. The method of claim 1 , wherein controlling the target apparatus based on the target operation includes:

displaying a functional icon corresponding to the target apparatus on the holographic screen based on the target apparatus;

obtaining a functional operation of the user based on the functional icon; and

controlling the target apparatus based on the functional operation.

7. The method of claim 1 , wherein controlling the target apparatus based on the target operation includes:

obtaining a gesture operation of the user on the target apparatus; and

controlling the target apparatus based on the gesture operation.

8. The method of claim 1 , wherein controlling the target apparatus based on the target operation includes:

connecting the target apparatus with an internet of things (IOT) control function corresponding to the target apparatus; and

based on the target operation, controlling the target apparatus by using the IOT control function corresponding to the target apparatus.

9. An apparatus management and control device, comprising:

a first acquisition module configured to obtain an operation position of a target operation of a user at a holographic screen, the target operation including a drag operation;

a first determination module configured to determine a target apparatus located in a same space as the holographic screen based on an operation position; and

a management and control module configured to control the target apparatus based on the target operation, wherein controlling the target apparatus based on the target operation includes:

based on the drag operation, displaying, on the holographic screen, a display content that is displayed by a screen of the terminal apparatus; or

based on the drag operation, displaying, on the screen of the terminal apparatus, a display content that is displayed by the holographic screen.

10. The device of claim 9 , wherein the first determination module is further configured to:

obtain a line of sight position of a user; and

match an apparatus position of an apparatus located in the same space with the holographic screen based on the operation position and the line of sight position to determine the target apparatus.

11. The device of claim 10 , further comprising an acquisition module configured to:

obtain line of sight positions of the user at at least two different positions;

determine the operation position at the holographic screen corresponding to the user when the user points to the apparatus in the same space with the same holographic screen at each of the positions;

determine a mapping relationship set based on the line of sight position and associated operation position corresponding to each of the positions; and

determine the apparatus position of the apparatus based on at least two mapping relation sets.

12. The device of claim 11 , wherein the acquisition module further includes a collection sub-module configured to:

collect at least two depth images corresponding to the user located at the at least two different positions; and

determine the line of sight positions of the user at the at least two different positions based on the at least two depth images.

13. The device of claim 9 , wherein the first determination module is configured to:

obtain a line of sight position of the user;

collect an operation image in a predetermined range determined based on the operation position based on the line of sight position of the user; and

identify the target apparatus in the operation image.

14. The device of claim 9 , wherein the management and control module is further configured to:

display a functional icon corresponding to the target apparatus on the holographic screen based on the target apparatus;

obtain a functional operation of the user based on the functional icon; and

control the target apparatus based on the functional operation.

15. The device of claim 9 , wherein the management and control module is further configured to:

obtain a gesture operation of the user on the target apparatus; and

control the target apparatus based on the gesture operation.

16. The device of claim 9 , wherein the management and control module is further configured to:

connect the target apparatus with an internet of things (IOT) control function corresponding to the target apparatus; and

based on the target operation, control the target apparatus by using the IOT control function corresponding to the target apparatus.

17. A non-transitory computer-readable storage medium storing a program that, when executed by a processor, causes the processor to:

obtain an operation position of a target operation of a user at a holographic screen, the target operation including a drag operation;

determine a target apparatus located in a same space as the holographic screen based on the operation position; and

control the target apparatus based on the target operation, including:

based on the drag operation, displaying, on the holographic screen, a display content that is displayed by a screen of the terminal apparatus; or

based on the drag operation, displaying, on the screen of the terminal apparatus, a display content that is displayed by the holographic screen.

18. The storage medium of claim 17 , wherein the processor is further configured to:

obtain a line of sight position of a user; and

match an apparatus position of an apparatus located in the same space with the holographic screen based on the operation position and the line of sight position to determine the target apparatus.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 3, 2023
From: DUAN, YONG; MENG, XIAO
To: LENOVO (BEIJING) LIMITED
Reel/Frame 062881/0473 →
Priority Claims (1)
CN 202210220374.9 · Mar 8, 2022 · national
Continuity (1)
Related Publication 20230289031A1 · Sep 14, 2023
Cited By (1)
US 12,355,842