IP Library › Granted Patent US 12,307,077
Granted Patent B1
US 12,307,077 · App. 17/239,425 · Granted May 20, 2025

Techniques for manipulating computer-generated objects in a computer graphics editor or environment

Inventor: Adam Bolton (La Pine, OR)
Assignee: Apple Inc.
G06F3/04845G06F3/04815G06F3/04842G06T19/20G06T2200/24G06T2219/2016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,307,077
App. No.
17/239,425
Granted
May 20, 2025
Kind
B1
Abstract

A computer generated 3D environment can include one or more free-space pins not permanently associated with any objects in the 3D environment. Objects in the 3D environment can be associated with one or more of the free-space pins, so that input at a selected free-space pin can cause manipulation of the objects associated with the selected free-space pin. For example, the selected free-space pin can be used as a temporary locus of manipulation (e.g., for rotation, movement, resizing, etc.) of the temporarily associated objects and/or can define a reference point for an action or behavior of one or more objects. A manipulator can be displayed when a free-space pin is selected to assist in receiving manipulation input.

Claims (67)

1. A method, comprising:

at an electronic device in communication with a display:

displaying, using the display, a computer graphics editing environment including an affordance representing a first free-space pin, wherein the affordance representing the first free-space pin is displayed anchored at a location in the computer graphics editing environment and at a distance from an object, and not anchored to the object;

while displaying the affordance representing the first free-space pin, receiving an input representing selection of the first free-space pin;

after receiving the input representing the selection of the first free-space pin, manipulating a location or orientation of the object associated with the first free-space pin using the location of the affordance representing the first free-space pin as a reference point for the manipulation, wherein the affordance representing the first free-space pin remains anchored at the location in the computer graphics editing environment during the manipulating of the location or orientation of the object, wherein the object has a first location when the manipulation of the object is complete; and

continuing displaying the first free-space pin anchored at the location in the computer graphics editing environment after disassociating the object from the first free-space pin, wherein the object is disassociated from the first free-space pin and has the first location while continuing displaying the first free-space pin anchored at the location in the computer graphics editing environment after disassociating the object from the first free-space pin.

2. The method of claim 1 , further comprising manipulating the location of the object by moving the object toward the location of the first free-space pin.

3. The method of claim 1 , further comprising manipulating the orientation of the object by orbiting the object around the location of the first free-space pin.

4. The method of claim 1 , wherein manipulating the location of the object associated with the first free-space pin comprises manipulating the object in accordance with a predetermined locational relationship between the object and the first free-space pin.

5. The method of claim 1 , further comprising:

while displaying the affordance representing the first free-space pin, displaying, using the display, an affordance representing a second free-space pin, and while displaying the affordance representing the second free-space pin, receiving an input representing selection of the second free-space pin along with the first free-space pin; and

after receiving the input representing the selection of the first free-space pin and the input representing the selection of the second free-space pin, manipulating the object in accordance with a predetermined locational relationship between two or more of the object, the first free-space pin, and the second free-space pin.

6. The method of claim 5 , further comprising manipulating the object by maintaining a sum of a distance between the object and the first free-space pin and a distance between the object and the second free-space pin as a constant value.

7. The method of claim 1 , further comprising:

associating the object with the first free-space pin;

after receiving the input representing the selection of the first free-space pin, displaying, using the display, an affordance representing a first manipulator anchored to the first free-space pin; and

while displaying the first manipulator, manipulating the object associated with the first free-space pin using the first manipulator.

8. The method of claim 7 , further comprising:

while displaying the first manipulator, receiving an input representing manipulation of the first manipulator; and

after receiving the input representing the manipulation of the first manipulator, manipulating the object associated with the first free-space pin.

9. The method of claim 7 , further comprising:

after displaying the affordance representing the first manipulator anchored to the first free-space pin, receiving an input representing selection of a second free-space pin;

after receiving the input representing the selection of the second free-space pin, displaying, using the display, an affordance representing a second manipulator anchored to the second free-space pin; and

while displaying the second manipulator, receiving an input representing manipulation of the object associated with the second free-space pin using the second manipulator.

10. The method of claim 9 , further comprising:

while displaying the second manipulator, ceasing to display the first manipulator.

11. The method of claim 1 , further comprising simultaneously manipulating a plurality of objects associated with the selected first free-space pin.

12. The method of claim 1 , wherein the first free-space pin is unanchored to any objects in the computer graphics editing environment.

13. The method of claim 1 , further comprising:

displaying, using the display, an affordance representing a free-space pin properties pane including one or more properties of each free-space pin of a plurality of free-space pins in the computer graphics editing environment.

14. The method of claim 13 , wherein the one or more properties includes a representation of a 3D location of each free-space pin of the plurality of free-space pins.

15. The method of claim 13 , wherein the one or more properties includes a representation of an orientation of each free-space pin of the plurality of free-space pins.

16. The method of claim 1 , further comprising:

receiving an input representing addition of a second free-space pin to the computer graphics editing environment; and

after receiving the input representing the addition of the second free-space pin, adding the second free-space pin to the computer graphics editing environment.

17. The method of claim 16 , further comprising:

displaying, using the display, an affordance for adding a free-space pin;

wherein receiving the input representing the addition of the second free-space pin comprises actuating the affordance for adding the free-space pin.

18. The method of claim 17 , further comprising:

in response to actuating the affordance for adding the free-space pin, displaying in the computer graphics editing environment an affordance representing a target, wherein receiving the input representing the addition of the second free-space pin further comprises receiving an input representing selection of the target.

19. The method of claim 1 , wherein manipulating the location or orientation of the object associated with the first free-space pin comprises utilizing an orientation of the first free-space pin in performing the manipulation.

20. The method of claim 1 , wherein manipulating the location or orientation of the object associated with the first free-space pin comprises scaling one or more properties of the object associated with the first free-space pin.

21. The method of claim 1 , further comprising:

after receiving the input representing the selection of the first free-space pin:

displaying a context menu including at least a first user interface element that is selectable to perform a first action associated with the first free-space pin.

22. A non-transitory computer-readable storage medium storing instructions, which when executed by one or more processors, cause the one or more processors to:

display, using a display, a computer graphics editing environment including an affordance representing a first free-space pin;

while displaying the affordance representing the first free-space pin:

receive a first input representing selection of the first free-space pin; and

receive a second input associating an object with the first free-space pin, wherein in response to the second input, the first free-space pin is associated with a plurality of objects that includes the object; and

after receiving the first input representing the selection of the first free-space pin:

simultaneously manipulate a location or orientation of the plurality of objects associated with the first free-space pin using a location of the first free-space pin as a reference point for the manipulation, wherein the simultaneous manipulation includes:

simultaneously manipulating:

the object in a first direction or first orientation relative to the location of the first free-space pin, and

a second object of the plurality of objects, different from the object, in a second direction or second orientation different from the first direction or first orientation; and

remove the object from association with the first free-space pin while maintaining the location of the first free-space pin.

23. An electronic device comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, using a display, a computer graphics editing environment including an affordance representing a first free-space pin, wherein the affordance representing the first free-space pin is displayed anchored at a location in the computer graphics editing environment and at a distance from an object, and not anchored to the object;

while displaying the affordance representing the first free-space pin, receiving an input representing selection of the first free-space pin;

after receiving the input representing the selection of the first free-space pin, manipulating a location or orientation of the object associated with the first free-space pin using the location of the affordance representing the first free-space pin as a reference point for the manipulation, wherein the affordance representing the first free-space pin remains anchored at the location in the computer graphics editing environment during the manipulating of the location or orientation of the object, wherein the object has a first location when the manipulation is complete; and

continuing displaying the first free-space pin anchored at the location in the computer graphics editing environment after disassociating the object from the first free-space pin, wherein the object is disassociated from the first free-space pin and has the first location while continuing displaying the first free-space pin anchored at the location in the computer graphics editing environment after disassociating the object from the first free-space pin.

24. The electronic device of claim 23 , wherein the one or more programs include instructions for:

after receiving the input representing the selection of the first free-space pin:

displaying a context menu including at least a first user interface element that is selectable to perform a first action associated with the first free-space pin.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 23, 2021
From: BOLTON, ADAM
To: APPLE INC.
Reel/Frame 056027/0908 →
Continuity (1)
Provisional Application 63039885 · Jun 16, 2020
References Cited (53)
US 5861889A · Wallace et al. · 1999 [cited by applicant]
US 6128631A · Wallace · 2000 [cited by examiner]
US 6308144B1 · Bronfeld · 2001 [cited by examiner]
US 6426745B1 · Isaacs et al. · 2002 [cited by applicant]
US 6496870B1 · Faustini · 2002 [cited by applicant]
US 6781597B1 · Vrobel et al. · 2004 [cited by applicant]
US 10318034B1 · Hauenstein · 2019 [cited by examiner]
US 10671241B1 · Jia · 2020 [cited by examiner]
US 20010042118A1 · Miyake · 2001 [cited by examiner]
US 20020149628A1 · Smith et al. · 2002 [cited by applicant]
US 20030187532A1 · Charles · 2003 [cited by examiner]
US 20050108620A1 · Allyn · 2005 [cited by examiner]
US 20050188348A1 · Han et al. · 2005 [cited by applicant]
US 20060025679A1 · Viswanathan et al. · 2006 [cited by applicant]
US 20060041178A1 · Viswanathan · 2006 [cited by examiner]
US 20070271524A1 · Chiu · 2007 [cited by examiner]
US 20090079739A1 · Fitzmaurice et al. · 2009 [cited by applicant]
US 20110041098A1 · Kajiya · 2011 [cited by examiner]
US 20120026100A1 · Migos · 2012 [cited by examiner]
US 20120078589A1 · McDaniel · 2012 [cited by examiner]
US 20120086719A1 · Brown · 2012 [cited by examiner]
US 20120262458A1 · Fowler · 2012 [cited by examiner]
US 20130093756A1 · Davidson · 2013 [cited by applicant]
US 20130127833A1 · Davidson · 2013 [cited by applicant]
US 20130135290A1 · Davidson · 2013 [cited by applicant]
US 20130135291A1 · Davidson · 2013 [cited by applicant]
US 20130318479A1 · Porwal · 2013 [cited by examiner]
US 20140040832A1 · Regelous · 2014 [cited by applicant]
US 20140104266A1 · Stone · 2014 [cited by examiner]
US 20140129990A1 · Xin · 2014 [cited by examiner]
US 20140149944A1 · Duplessis et al. · 2014 [cited by applicant]
US 20140229871A1 · Tai et al. · 2014 [cited by applicant]
US 20140344741A1 · Newman · 2014 [cited by examiner]
US 20160092080A1 · Swanson · 2016 [cited by examiner]
US 20170329488A1 · Welker et al. · 2017 [cited by applicant]
US 20180113596A1 · Ptak et al. · 2018 [cited by applicant]
US 20190294314A1 · Tada · 2019 [cited by examiner]
US 20200082633A1 · Rom et al. · 2020 [cited by applicant]
US 20200258193A1 · Katsumata · 2020 [cited by applicant]
US 20200272303A1 · Jia et al. · 2020 [cited by applicant]
US 20200319776A1 · Natzke · 2020 [cited by applicant]
US 20200379626A1 · Guyomard et al. · 2020 [cited by applicant]
US 20210134069A1 · Sorrento · 2021 [cited by applicant]
US 20210150731A1 · Saquib et al. · 2021 [cited by applicant]
Song et al., “A Handle Bar Metaphor for Virtual Object Manipulation with Mid-Air Interaction”. CHI'12, May 5-10, 2012. (Year: 2012). [cited by examiner]
Mendes et al., “A Survey on 3D Virtual Object Manipulation: from the Desktop to Immersive Virtual Environments”, Computer Graphics Forum, Apr. 2018. (Year: 2018). [cited by examiner]
Caputo et al., “The Smart Pin: An effective tool for object manipulation in immersive virtual reality environments”, Computers & Graphics 74 (2018) 225-233, May 28, 2018 (Year: 2018). [cited by examiner]
Gloumeau et al., “PinNPivot: Object Manipulation Using Pins in Immersive Virtual Environments”, IEEE Transactions on Visualization and Computer Graphics, published on Apr. 15, 2020. (Year: 2020). [cited by examiner]
Apple, “Creating 3D Content with Reality Composer”, Available online at: <https://developer.apple.com/documentation/realitykit/creating_3d_content_with_reality_composer>, Accessed on Jul. 23, 2021, 10 pages. [cited by applicant]
Autocad, “Rotate 3D Objects”, Available online at: <http://docs.autodesk.com/ACD/2010/ENU/AutoCAD%202010%20User%20Documentation/index.html?url=WS1a9193826455f5ffa23ce210c4a30acaf-66f8.htm, topicNumber=d0e103038>, 2010, … [cited by applicant]
Rhinoceros, “Gumball”, Available online at: <https://docs.mcneel.com/rhino/6/help/en-us/commands/gumball.htm>, Accessed on Jul. 23, 2021, 9 pages. [cited by applicant]
Rhinoceros, “Object Snaps”, Available online at: <http://docs.mcneel.com/rhino/5/help/en-us/user_interface/object_snaps.htm>, Accessed on Jul. 23, 2021, 15 pages. [cited by applicant]
Rhinoceros, “Rotate3D”, Available online at: <http://docs.mcneel.com/rhino/5/help/en-us/commands/rotate3d.htm>, Accessed on Jul. 23, 2021, 1 page. [cited by applicant]
Cited By (1)
US 1,147,039