Surface texture detection and emulation
A hand-held device that includes sensor components that enable the detection and measurement of surface features such as roughness or texture. Motion data (e.g., velocity and trajectory) may also be captured. The hand-held device may also include components for emulating surface features. The hand-held device may also recreate, cancel, or alter curves or roughness on virtual or real surfaces, and recreate or cancel curves or roughness on real surfaces. A hand-held device may include the sensor components, the emulation components, or both. The hand-held device may, for example, be used as an accessory in an extended reality (XR) system, or may be a standalone device.
1 . A hand-held device, comprising:
one or more surface sensors configured to collect data about a surface as the hand-held device is passed over the surface;
a linear actuator configured to change a grip distance from the surface to a grippable portion of the hand-held device; and
a controller comprising one or more processors configured to:
analyze the collected data to determine features of the surface;
generate a surface description for the surface, wherein the surface description includes indications of the features of the surface; and
control the linear actuator to change the grip distance from the surface to the grippable portion of the hand-held device to simulate a curved surface or alter a curvature of the surface.
2 . The hand-held device as recited in claim 1 , wherein the one or more surface sensors include one or more of a force sensor, an accelerometer, an interferometer, a depth sensor, and a visible light camera.
3 . The hand-held device as recited in claim 1 , wherein, to analyze the collected data to determine features of the surface, the controller is configured to classify the surface as a surface type based on the determined features, wherein features of a surface include one or more of roughness, hardness, depth and width of peaks and valleys, angles, surface normals, friction, curvature, and color.
4 . The hand-held device as recited in claim 1 , wherein the one or more surface sensors include one or more of:
a high-bandwidth accelerometer capable of detecting 0 Hz to 2.5 kHz vibration signals;
a force sensor capable of 1 to 100 gram-force sensing; and
an interferometer that provides ˜100 nm resolution absolute depth sensing, 1D or 2D absolute distance, ˜100 Hz, and distance to surface of 0 to 1 cm.
5 . The hand-held device as recited in claim 1 , wherein the hand-held device further comprises one or more emulation components, and wherein the controller is further configured to recreate, alter, or cancel the features of the surface via the emulation components.
6 . The hand-held device as recited in claim 5 , wherein, to recreate, alter, or cancel the features of the surface via the emulation components, the controller is configured to:
obtain features of the surface;
determine, based at least in part on the features of the surface, one or more effects to be applied on a target surface; and
cause the one or more emulation components to apply the one or more effects as the hand-held device is passed over the target surface.
7 . The hand-held device as recited in claim 6 , wherein, to cause the one or more emulation components to apply the one or more effects as the hand-held device is passed over the target surface, the controller is configured to use motion data of the hand-held device to determine application of the one or more effects.
8 . The hand-held device as recited in claim 6 , wherein the one or more emulation components include a haptic engine, and wherein the effects include recreating, altering, or canceling the surface features on the target surface via the haptic engine.
9 . The hand-held device as recited in claim 8 , wherein the linear actuator is used in conjunction with the haptic engine to recreate, alter, or cancel the surface features on the target surface.
10 . The hand-held device as recited in claim 6 , wherein the one or more emulation components include the linear actuator, and wherein the effects include emulating or canceling a curve on the target surface via the linear actuator.
11 . The hand-held device as recited in claim 6 , further comprising a gyroscope, wherein data collected from the gyroscope is used to alter feeling of a surface depending on roll angle of the device.
12 . The hand-held device as recited in claim 1 , wherein the collected data includes velocity and trajectory of the hand-held device.
13 . The hand-held device as recited in claim 1 , wherein the hand-held device is a component of an extended reality (XR) system.
14 . A method, comprising:
collecting, by one or more surface sensors of a hand-held device comprising a linear actuator, data about a surface as the hand-held device is passed over the surface;
analyzing, by a controller comprising one or more processors, the collected data to determine features of the surface;
generating a surface description for the surface, wherein the surface description includes indications of the features of the surface; and
controlling, based at least in part on the surface description, the linear actuator to change a grip distance from the surface to a grippable portion of the hand-held device to simulate a curved surface or alter a curvature of the surface.
15 . The method as recited in claim 14 , wherein the one or more surface sensors include one or more of a force sensor, an accelerometer, an interferometer, a depth sensor, and a visible light camera.
16 . The method as recited in claim 14 , wherein, analyzing the collected data to determine features of the surface comprises classifying the surface as a surface type based on the determined features, wherein features of a surface include one or more of roughness, hardness, depth and width of peaks and valleys, angles, surface normals, friction, curvature, and color.
17 . The method as recited in claim 14 , wherein the surface sensors include one or more of:
a high-bandwidth accelerometer capable of detecting 0 Hz to 2.5 kHz vibration signals;
a force sensor capable of 1 to 100 gram-force sensing; and
an interferometer that provides ˜100 nm resolution absolute depth sensing, 1D or 2D absolute distance, ˜100 Hz, and distance to surface of 0 to 1 cm.
18 . The method as recited in claim 14 , further comprising recreating, altering or canceling the features of a surface via one or more emulation components of the device, wherein the emulation components include a haptic engine and the linear actuator.
19 . The method as recited in claim 18 , wherein recreating, altering or canceling the features of a surface comprises:
obtaining features of the surface;
determining, based at least in part on the features of the surface, one or more effects to be applied on a target surface; and
causing the one or more emulation components to apply the one or more effects as the hand-held device is passed over the target surface.
20 . A system, comprising:
a hand-held device, comprising a linear actuator and one or more surface sensors configured to collect data about a physical surface as the hand-held device is passed over the surface;
a head-mounted device (HMD), comprising a display subsystem configured to display content in a virtual environment; and
one or more processors configured to:
analyze the data collected by the surface sensors to classify the surface as a surface type;
control the linear actuator to change a grip distance from a surface to a greppable portion of the hand-held device to simulate a curved surface or alter a curvature of the surface; and
recreate the surface type on a surface in the virtual environment.