Optical tactile sensor and method for estimating shape from touch
A method performed by an optical tactile sensor comprising a camera, includes: measuring a plurality of relations between the optical tactile sensor and an object; detecting a contact of a membrane on the object; detecting a plurality of markers on the membrane; performing a plurality of first operations by using a tactile perception module; and performing a second operation by using a three-dimensional (3D) perception module, based on a result of the performing of the plurality of first operations.
1. A method performed by an optical tactile sensor comprising a camera, the method comprising:
measuring a plurality of relations between the optical tactile sensor and an object;
detecting a contact of a membrane on the object;
detecting a plurality of markers on the membrane;
performing a plurality of first operations by using a tactile perception module; and
performing a second operation by using a three-dimensional (3D) perception module, based on a result of the performing of the plurality of first operations
wherein the result of the plurality of first operations comprises a marker distance map, and
wherein the second operation comprises 3D geometry of the object, based on the marker distance map,
wherein the marker distance map comprises calculating
d
(
t
)
=
M
dist
·
(
1
-
l
(
0
)
l
(
t
)
)
wherein d(t) is a displacement of the membrane towards a camera image plane when the membrane is in contact with the object, Mdist is an initial distance between the camera and an initial membrane, and
wherein, when the membrane is not in contact with the object, the initial membrane is oriented in a direction perpendicular to an optical axis of the camera, l(t) is a projected length as measured on the camera image plane, when the membrane is deformed after the contact with the object, in the direction towards the camera image plane, and l(0) is an initial projected length as measured on the camera image plane when the membrane is not in contact with the object.
2. The method of claim 1 , wherein the plurality of relations between the optical tactile sensor and the object comprises:
a fixed distance (f) between the camera and the camera image plane; and
the distance (Mdist) between the camera and the membrane.
3. The method of claim 1 , wherein the performing of the plurality of first operations by using the tactile perception module, comprises:
computing a structure of the plurality of markers;
computing an outward distance function, based on the structure of the plurality of markers; and
generating the marker distance map, based on the outward distance function.
4. The method of claim 3 , wherein the structure of the plurality of markers comprises a plurality of medial axes of the plurality of markers.
5. The method of claim 1 , wherein the plurality of markers comprises a N×N grid.
6. The method of claim 1 , wherein the plurality of markers has a random formation.
7. An optical tactile sensor comprising:
a membrane;
a plurality of markers on the membrane;
a memory;
a camera;
at least one processor operatively connected to the memory, the at least one processor configured to:
measure a plurality of relations between the optical tactile sensor and an object,
detect a contact of the membrane on the object,
detect the plurality of markers,
perform a plurality of first operations by using a tactile perception module in the memory, and
perform a second operation by using a three-dimensional (3D) perception module in the memory, based on a result of the plurality of first operations,
wherein the result of the plurality of first operations comprises a marker distance map,
wherein the second operation comprises 3D geometry of the object, based on the marker distance map, and
wherein the marker distance map comprises calculating
d
(
t
)
=
M
dist
·
(
1
-
l
(
0
)
l
(
t
)
)
wherein d(t) is a displacement of the membrane towards a camera image plane when the membrane is in contact with the object, Mdist is an initial distance between the camera and an initial membrane, and
wherein, when the membrane is not in contact with the object, the initial membrane is oriented in a direction perpendicular to an optical axis of the camera, l(t) is a projected length as measured on the camera image plane, when the membrane is deformed after the contact with the object, in the direction towards the camera image plane, and l(0) is an initial projected length as measured on the camera image plane when the membrane is not in contact with the object.
8. The optical tactile sensor of claim 7 , wherein the plurality of relations between the optical tactile sensor and the object comprises:
a fixed distance (f) between the camera and the camera image plane; and
the distance (Mdist) between the camera and the membrane.
9. The optical tactile sensor of claim 7 , wherein the plurality of first operations by using the tactile perception module in the memory, includes:
computing a structure of the plurality of markers;
computing an outward distance function, based on the structure of the plurality of markers; and
generating the marker distance map, based on the outward distance function.
10. The optical tactile sensor of claim 9 , wherein the structure of the plurality of markers comprises a plurality of medial axes of the plurality of markers.
11. The optical tactile sensor of claim 7 , wherein the plurality of markers comprises a N×N grid.
12. The optical tactile sensor of claim 7 , wherein the plurality of markers has a random formation.
13. A method performed by an optical tactile sensor for reconstructing a three-dimensional (3D) shape of an object from touch, comprising:
inferring the 3D shape of the object from a 3D depth extracted from a deformation of markers by the object being in contact with the markers, the markers being provided on a planar surface; and
reconstructing the object's 3D shape by capturing the deformation of the markers,
wherein the deformation of the markers is associated with a motion of the planar surface,
wherein the inferring the 3D shape of the object comprises a marker distance map, and
wherein the reconstructing the object's 3D shape comprises 3D geometry of the object, based on the marker distance map,
wherein the marker distance map comprises calculating
d
(
t
)
=
M
dist
·
(
1
-
l
(
0
)
l
(
t
)
)
wherein d(t) is a displacement of the planar surface towards a camera image plane when the planar surface is in contact with the object, Mdist is an initial distance between a camera and an initial planar surface, and
wherein, when the planar surface is not in contact with the object, the initial planar surface is oriented in a direction perpendicular to an optical axis of the camera, l(t) is a projected length as measured on the camera image plane, when the planar surface is deformed after the contact with the object, in the direction towards the camera image plane, and l(0) is an initial projected length as measured on the camera image plane when the planar surface is not in contact with the object.
14. The method of claim 13 , wherein the planar surface is a surface of a membrane of the optical tactile sensor.
15. The method of claim 13 , wherein the motion of the planar surface is a fronto-parallel motion or a tilting motion.