IP Library › Granted Patent US 12,724,478
Granted Patent B2
US 12,724,478 · App. 18/810,812 · Granted Sep 1, 2026

Information processing device, head mounted display, and information processing method

Inventors: Takeshi Omata (Kanagawa, JP); Yusuke Kazami (Kanagawa, JP); Kenya Fukabori (Tokyo, JP)
Assignee: Canon Kabushiki Kaisha
G06F3/012G06F3/04842
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,724,478
App. No.
18/810,812
Granted
Sep 1, 2026
Kind
B2
Abstract

An information processing device includes a processor; and a memory storing a program which causes the information processing device to: select a virtual object present at a line-of-sight position of a user as a control object; determine whether or not the user performed a predetermined operation for visually recognizing a rear of the control object; and execute control that the control object is moved and displayed such that the rear of the control object can be visually recognized, wherein a direction in which the control object is to be moved is determined based on at least either one of a moving direction and a rotating direction of a head part of the user, or a distance for which the control object is to be moved is determined based on at least either one of a movement amount and a rotation amount of the head part of the user.

Claims (39)

1 . An information processing device comprising:

a processor; and

a memory storing a program which, when executed by the processor, causes the information processing device to:

(1) execute selecting processing to select, as a control object, a virtual object present at a line-of-sight position of a user wearing a head mounted display;

(2) execute first determining processing to determine whether or not the user performed a predetermined operation for visually recognizing a rear of the control object;

(3) execute second determining processing to determine that the predetermined operation has finished in a case where at least either one of a movement amount and a rotation amount of a head part of the user from a positional attitude of the head part of the user when it is determined that the predetermined operation has been performed is larger than a threshold value; and

(4) execute control processing to execute control that the control object is moved and displayed such that a portion hidden by the control object can be visually recognized in response to determining in the first determining processing that the predetermined operation has been performed by the user, and to execute control that the control object is returned to an original position in response to determining in the second determining processing that the predetermined operation has finished,

wherein in the control processing, (1) a direction in which the control object is to be moved is determined based on at least either one of a moving direction and a rotating direction of the head part of the user, or (2) a distance for which the control object is to be moved is determined based on at least either one of a movement amount and a rotation amount of the head part of the user.

2 . The information processing device according to claim 1 , wherein in the determining processing, whether or not the user performed the predetermined operation is determined based on at least any one of the moving direction, the movement amount, the rotating direction, and the rotation amount of the head part of the user.

3 . The information processing device according to claim 2 , wherein in the determining processing, it is determined that the user performed the predetermined operation in a case where the movement amount of the head part of the user is a distance within a predetermined range.

4 . The information processing device according to claim 2 , wherein in the determining processing, it is determined that the user performed the predetermined operation in a case where the rotation amount of the head part of the user is an angle within a predetermined range.

5 . The information processing device according to claim 1 , wherein in the control processing, at least any one of display control of movement, size reduction, transparency, and non-display to the control object is executed in response to performance of the predetermined operation by the user.

6 . The information processing device according to claim 5 , wherein in the control processing, in a case where there is another virtual object in the direction in which the control object is to be moved, control of size reduction is executed such that the control object does not overlap the another virtual object.

7 . The information processing device according to claim 6 , wherein in the control processing, in a case where the control object overlaps the another virtual object even if the size of the control object is reduced at a predetermined size-reduction rate, control is executed such that the control object is made transparent.

8 . The information processing device according to claim 5 , wherein a correspondence relation between a type of an object and a type of the display control is set in advance, and

wherein in the control processing, display control corresponding to a type of the control object in the movement, the size reduction, the transparency, and the non-display is executed to the control object.

9 . The information processing device according to claim 1 , wherein in the control processing, control is executed such that highlighted display of the control object selected in the selecting processing is executed.

10 . The information processing device according to claim 9 , wherein in the control processing, control is executed such that the highlighted display of the control object is performed for a period since the control object was selected in the selecting processing until display control of the control object is started or the predetermined operation is finished.

11 . A head mounted display comprising:

the information processing device according to claim 1 ; and

a display which displays an image in which display of the control object is controlled.

12 . An information processing method comprising:

a selecting step of selecting, as a control object, a virtual object present at a line-of-sight position of a user wearing a head mounted display;

a first determining step of determining whether or not the user performed a predetermined operation for visually recognizing a rear of the control object;

a second determining step of determining that the predetermined operation has finished in a case where at least either one of a movement amount and a rotation amount of a head part of the user from a positional attitude of the head part of the user when it is determined that the predetermined operation has been performed is larger than a threshold value; and

a control step of executing control that the control object is moved and displayed such that a portion hidden by the control object can be visually recognized in response to determining in the first determining step that the predetermined operation has been performed by the user, and executing control that the control object is returned to an original position in response to determining in the second determining step that the predetermined operation has finished,

wherein in the control step, (1) a direction in which the control object is to be moved is determined based on at least either one of a moving direction and a rotating direction of the head part of the user, or (2) a distance for which the control object is to be moved is determined based on at least either one of a movement amount and a rotation amount of the head part of the user.

13 . A non-transitory computer-readable medium storing a program that causes a computer to execute an information processing method, the information processing method comprising:

a selecting step of selecting, as a control object, a virtual object present at a line-of-sight position of a user wearing a head mounted display;

a first determining step of determining whether or not the user performed a predetermined operation for visually recognizing a rear of the control object;

a second determining step of determining that the predetermined operation has finished in a case where at least either one of a movement amount and a rotation amount of a head part of the user from a positional attitude of the head part of the user when it is determined that the predetermined operation has been performed is larger than a threshold value; and

a control step of executing control that the control object is moved and displayed such that a portion hidden by the control object can be visually recognized in response to determining in the first determining step that the predetermined operation has been performed by the user, and executing control that the control object is returned to an original position in response to determining in the second determining step that the predetermined operation has finished,

wherein in the control step, (1) a direction in which the control object is to be moved is determined based on at least either one of a moving direction and a rotating direction of the head part of the user, or (2) a distance for which the control object is to be moved is determined based on at least either one of a movement amount and a rotation amount of the head part of the user.

14 . The information processing device according to claim 1 , wherein the control object is stored in correspondence with metadata indicating display control to be applied to the control object in response to the predetermined operation, and

wherein in the control processing, at least any one of display control of movement, size reduction, transparency, and non-display to the control object is executed based on the metadata, in response to performance of the predetermined operation by the user.

15 . The information processing device according to claim 1 , wherein the program, when executed by the processor, further causes the information processing device to execute storing processing to store the positional attitude of the head part of the user when it is determined that the predetermined operation has been performed.

16 . The information processing device according to claim 1 , wherein in the control processing:

when the user performs, as the predetermined operation, an operation of looking into from a right side of the control object, the control object is moved to a left side, and

when the user performs, as the predetermined operation, an operation of looking into from the left side of the control object, the control object is moved to the right side.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 27, 2024
From: OMATA, TAKESHI; KAZAMI, YUSUKE; FUKABORI, KENYA
To: CANON KABUSHIKI KAISHA
Reel/Frame 068715/0485 →
Priority Claims (1)
JP 2023-142131 · Sep 1, 2023 · national
Continuity (1)
Related Publication 20250076970A1 · Mar 6, 2025
References Cited (46)
US 6222465B1 · Kumar · 2001 [cited by examiner]
US 6972734B1 · Ohshima · 2005 [cited by examiner]
US 9122053B2 · Geisner · 2015 [cited by examiner]
US 9122321B2 · Perez · 2015 [cited by examiner]
US 9292085B2 · Bennett · 2016 [cited by examiner]
US 9292973B2 · Bar-Zeev · 2016 [cited by examiner]
US 9846305B2 · Nishizawa · 2017 [cited by applicant]
US 9995936B1 · Macannuco · 2018 [cited by examiner]
US 10140768B2 · Takahashi · 2018 [cited by applicant]
US 10373381B2 · Nuernberger · 2019 [cited by examiner]
US 10406432B2 · Watanabe · 2019 [cited by examiner]
US 10429923B1 · Johnston · 2019 [cited by examiner]
US 12140758B2 · Roimi · 2024 [cited by examiner]
US 20020033845A1 · Elber · 2002 [cited by examiner]
US 20110107216A1 · Bi · 2011 [cited by examiner]
US 20120056875A1 · Lee · 2012 [cited by examiner]
US 20120105473A1 · Bar-Zeev · 2012 [cited by examiner]
US 20140201666A1 · Bedikian · 2014 [cited by examiner]
US 20140364215A1 · Mikhailov · 2014 [cited by examiner]
US 20150169176A1 · Cohen · 2015 [cited by examiner]
US 20150261291A1 · Mikhailov · 2015 [cited by examiner]
US 20150339468A1 · Son · 2015 [cited by examiner]
US 20150379770A1 · Haley, Jr. · 2015 [cited by examiner]
US 20160026242A1 · Burns · 2016 [cited by examiner]
US 20160026253A1 · Bradski · 2016 [cited by examiner]
US 20160062514A1 · Jo · 2016 [cited by examiner]
US 20160092080A1 · Swanson · 2016 [cited by examiner]
US 20160109703A1 · Nishizawa · 2016 [cited by applicant]
US 20160110921A1 · Takahashi · 2016 [cited by applicant]
US 20160210780A1 · Paulovich · 2016 [cited by examiner]
US 20160214011A1 · Weising · 2016 [cited by examiner]
US 20160291329A1 · Kimura · 2016 [cited by examiner]
US 20180136465A1 · Chi · 2018 [cited by examiner]
US 20180150204A1 · Macgillivray · 2018 [cited by examiner]
US 20180314326A1 · Terahata · 2018 [cited by examiner]
US 20190026589A1 · Sugihara · 2019 [cited by examiner]
US 20190057531A1 · Sareen · 2019 [cited by examiner]
US 20190139308A1 · Sun · 2019 [cited by examiner]
US 20190346678A1 · Nocham · 2019 [cited by examiner]
US 20190362557A1 · Lacey · 2019 [cited by examiner]
US 20200117336A1 · Mani · 2020 [cited by examiner]
US 20200143554A1 · Toldo · 2020 [cited by examiner]
US 20210165484A1 · Suguhara · 2021 [cited by examiner]
JP 2016081339A · 2016 [cited by applicant]
JP 2016082466A · 2016 [cited by applicant]
WO 2020188655A1 · 2020 [cited by applicant]