IP Library Granted Patent US 11,073,933
Granted Patent B2
US 11,073,933 · App. 16/749,760 · Granted Jul 27, 2021

Systems and methods for manipulating a virtual environment

Inventors: Ilya Daniel Rosenberg (Mountain View, CA); John Aaron Zarraga (Mountain View, CA); Alexander Meagher Grau (Mountain View, CA); Charles Robert Watson (Mountain View, CA)
Assignee: Sensel, Inc.
G06F3/04144G06F3/017G06F3/0346G06F3/045G06F3/04166G06F3/04815G06F3/04842G06F3/04845G06F3/04883G06T19/006G06F2203/04102G06F2203/04104G06F2203/04105
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,073,933
App. No.
16/749,760
Granted
Jul 27, 2021
Kind
B2
Abstract

One variation of a method for manipulating virtual objects within a virtual environment includes: determining a first position of a touch sensor within real space; based on the first position of the touch sensor within real space, bounding a virtual surface of a virtual object within the virtual environment tractable through inputs across the touch sensor; generating a first force vector comprising a magnitude related to a force magnitude of a first input on the touch sensor surface and a direction related to an orientation of the touch sensor within real space; locating an origin of the first force vector within the virtual environment based on a first location of the first input on the touch sensor surface and the first position of the touch sensor within real space; and manipulating the virtual surface of the virtual object within the virtual environment according to the first force vector.

Claims (109)

1. A method for manipulating virtual objects within a virtual environment, comprising:

during a first period of time:

prompting a user to apply a first input of a first subjective force magnitude over a first surface of a touch sensor;

generating a first touch image representing a first force distribution of the first input over the first surface of the touch sensor;

calculating a first force magnitude of the first input based on the first force distribution;

calculating a first scaling factor based on a difference between the first subjective force magnitude and the first force magnitude; and

at a second time succeeding the first period of time:

generating a second touch image representing a second force distribution of a second input on the first surface of the touch sensor;

calculating a second force magnitude of the second input based on the second force distribution;

calculating a normalized force magnitude based on the second force magnitude and the first scaling factor; and

manipulating a virtual object within the virtual environment according to the normalized force magnitude.

2. The method of claim 1 , further comprising:

during the first period of time, identifying a set of discrete input areas in the touch image based on the first force distribution; and

mapping a set of control functions to a set of locations on the first surface of the touch sensor based on the set of discrete input areas.

3. The method of claim 2 :

wherein identifying the set of discrete input areas in the touch image comprises, based on the first force distribution:

identifying a resting position of a palm of the user on the surface of the touch sensor;

identifying a resting position of a thumb of the user on the surface of the touch sensor; and

identifying a set of resting positions of fingers of the user on the surface of the touch sensor; and

wherein mapping the set of control functions to the set of locations on the first surface of the touch sensor comprises:

mapping a first subset of control functions in the set of control functions to the resting position of the palm of the user on the surface of the touch sensor;

mapping a second subset of control functions in the set of control functions to the resting position of the thumb of the user on the surface of the touch sensor; and

mapping a third subset of control functions in the subset of control functions to resting positions of fingers of the user on the surface of the touch sensor.

4. The method of claim 1 :

further comprising, during the first period of time:

prompting the user to apply a third input of a second subjective force magnitude, greater than the first subjective force magnitude, over the first surface of the touch sensor;

generating a third touch image representing a third force distribution of the third input over the first surface of the touch sensor;

calculating a third force magnitude of the third input based on the third force distribution; and

calculating an updated scaling factor based on the first scaling factor and a difference between the second subjective force magnitude and the third force magnitude; and

wherein calculating the normalized force magnitude comprises calculating the normalized force magnitude based on the second force magnitude and the updated scaling factor.

5. The method of claim 1 , further comprising:

during the first period of time:

prompting the user to apply a third input of a second subjective force magnitude, greater than the first subjective force magnitude, over the first surface of the touch sensor;

generating a third touch image representing a third force distribution of the third input over the first surface of the touch sensor; and

calculating a third force magnitude of the third input based on the third force distribution; and

modifying a threshold force magnitude, related to manipulation of the virtual object within the virtual environment, based on a difference between the second subjective force magnitude and the third force magnitude.

6. The method of claim 1 , further comprising, during the first period of time:

prompting the user to apply the first input of the first subjective force magnitude over a first surface of a touch sensor, the first surface of the touch sensor defining a first input surface;

prompting the user to apply a third input of a second subjective force magnitude over a second surface of the touch sensor, the second surface of the touch sensor defining a second input surface that is discontinuous with the first input surface;

generating a third touch image representing a third force distribution of the third input over the first surface of the touch sensor;

calculating a third force magnitude of the third input based on the third force distribution; and

calculating a second scaling factor associated with the second input surface based on a difference between the second subjective force magnitude and the third force magnitude.

7. The method of claim 1 , further comprising, during the first period of time:

prompting the user to apply the first input of the first subjective force magnitude over a first surface of the touch sensor, the first surface of the touch sensor defining a first input surface;

prompting the user to apply a third input over a second surface of the touch sensor, the second surface of the touch sensor defining a second input surface that is discontinuous with the first input surface;

generating a third touch image representing a third force distribution of the third input over the first surface of the touch sensor;

identifying a discrete input area of the third input, on the second input surface, within the third touch image based on the third force distribution; and

mapping a set of control functions to a location on the second input surface corresponding to the discrete input area.

8. The method of claim 7 :

wherein prompting the user to apply the first input of the first subjective force magnitude over the first surface of the touch sensor comprises prompting the user to apply the first input over a curvilinear grip region defining a body of the touch sensor; and

wherein prompting the user to apply the third input over the second surface sensor comprises prompting the user apply the third input over a planar input surface of the touch sensor affixed to the body of the touch sensor.

9. The method of claim 1 :

wherein prompting the user to apply the first input over the first surface of the touch sensor comprises serving a first sequence of virtual reality frames representing the first subjective force magnitude to a virtual reality headset worn by the user; and

wherein manipulating the virtual object within the virtual environment comprises serving a second sequence of virtual frames representing manipulation of the virtual object to the virtual reality headset.

10. The method of claim 1 , wherein manipulating the virtual object within the virtual environment according to the normalized force magnitude comprises:

detecting an orientation of the touch sensor within a real space;

generating a first force vector based on the second force magnitude and the orientation of the touch sensor;

generating a normalized force vector based on the first force vector and the scaling factor; and

manipulating the virtual object within the virtual environment according to the normalized force vector.

11. The method of claim 1 , wherein manipulating the virtual object within the virtual environment comprises:

identifying a position of the second input on the first surface of the touch sensor based on the second touch image;

rotating the virtual object locally about a first axis within the virtual environment based on the position of second input; and

rotating the virtual object locally about a second axis within the virtual environment at a rate proportional to the normalized force magnitude, the second axis perpendicular to the first axis.

12. The method of claim 1 , wherein prompting the user to apply the first input of the first subjective force magnitude over the first surface of the touch sensor comprises prompting the user the apply one of: a weak force input; a moderate force input; or a strong force input.

13. A system, comprising:

a body;

a first touch sensor coupled to a first surface of the body, the first touch sensor comprising an array of force-sensitive elements;

a first grip coupled to the body and configured to transmit force applied to an external surface of the first grip into the first surface of the body; and

a controller electrically coupled to the first touch sensor and configured to:

interpret a first set of touch inputs applied to the external surface of the first grip based on outputs of the first touch sensor at a first time;

generate a first touch image representing positions and force magnitudes of the first set of touch inputs on the external surface of the first grip; and

output the first touch image to a computer system configured to manipulate a virtual object within a virtual environment according to a first control scheme and touch images, the first control scheme associated with the first grip.

14. The system of claim 13 :

wherein the first touch sensor comprises:

an array of drive electrode and sense electrode pairs arranged on a substrate and coupled to the controller; and

a force-sensitive material arranged over the array of drive electrode and sense electrode pairs; and

wherein the controller is further configured to:

scan a first set of resistance values across drive electrode and sense electrode pairs in the array of drive electrode and sense electrode pairs at the first time; and

interpret the first set of touch inputs, applied to the external surface of the first grip at a first time, into the touch image based on the first set of resistance values.

15. The system of claim 13 , further comprising a kit of grips comprising the first grip, each grip in the kit of grips:

comprising a flexible material;

defining a receptacle configured to transiently accept the body and contact the first surface of the body;

configured to couple a hand of the user to the first surface of the body; and

configured to transmit forces applied to the external surface of the grip, by the hand of the user, into the first surface of the body.

16. The system of claim 13 :

wherein the external surface of the first grip defines a first input surface;

wherein the first grip comprises a second input surface adjacent and discontinuous with the first input surface; and

wherein the controller is further configured to:

interpret a second set of touch inputs applied to the second input surface of the first grip based on outputs of the first touch sensor at the first time; and

represent the second set of touch inputs in the first touch image.

17. The system of claim 13 :

wherein the first grip further comprises an adjustable strap configured to constrain a palm of the user against the exterior surface of the first grip; and

wherein the controller is further configured to:

detect withdrawal of the first set of touch inputs from the exterior surface of the first grip based on outputs of the first touch sensor at a second time succeeding the first time;

generate a second touch image representing withdrawal of the first set of touch inputs from the exterior surface of the first grip; and

output the second touch image to the external computer system, wherein the external computer system is further configured to manipulate the virtual object within the virtual environment according to the first control scheme and the second touch image.

18. The system of claim 13 , wherein the controller is further configured to:

during an initial period of time, preceding the first time:

output a prompt to the user to apply an initial input of a subjective force magnitude over the first grip;

generate an initial touch image representing an initial force distribution of the initial input, communicated through the first grip, over the first surface of the body;

calculate an initial force magnitude of the initial input based on the initial force distribution; and

calculate a scaling factor based on a difference between the subjective force magnitude and the initial force magnitude; and

modify force magnitudes of the first set of touch inputs, represented in the first touch image, according to the scaling factor.

19. The system of claim 18 , wherein the controller is further configured to:

during the first period of time:

identify a set of discrete input areas on the first surface of the body based on the first force distribution; and

map a set of control functions associated with the first control scheme to the set of discrete input areas; and

label the first touch image with a subset of control functions in the set of control functions based on correspondences between the set of discrete input areas and positions of the first set of touch inputs represented in the first touch image.

20. The system of claim 13 , wherein the first grip comprises a first hand grip overlay configured to rest in a palm of a user, to transiently install over the body, and to transmit force applied to the external surface of the first grip into the first surface of the body.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 17, 2025
From: SENSEL, INC
To: CIRQUE CORPORATION
Reel/Frame 072018/0426 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2020
From: ROSENBERG, ILYA DANIEL; ZARRAGA, JOHN AARON; GRAU, ALEXANDER MEAGHER; WATSON, CHARLES ROBERT
To: SENSEL, INC.
Reel/Frame 053322/0160 →
Continuity (6)
Continuation 16456248 · Jun 28, 2019
Continuation 15827411 · Nov 30, 2017
Continuation 15224003 · Jul 29, 2016
Continuation 15223968 · Jul 29, 2016
Provisional Application 62198222 · Jul 29, 2015
Related Publication 20200159361A1 · May 21, 2020