Systems and methods for estimating tactile output based on depth data
Systems and methods for tactile output estimation are provided. In one embodiment, the system includes a depth map module, an estimation module, and a surface module. The depth map module is configured to identify a region of interest (RoI) of an object. The area of the RoI corresponds to a tactile sensor size of a tactile sensor. The depth module is further configured to receive depth data for the RoI from a depth sensor and generate a depth map for the RoI based on a volume of the depth data relative to a frame of reference of the RoI. The estimation module is configured to estimate a tactile sensor output based on the depth map. The surface module configured to determine surface properties based on the estimated tactile sensor output.
1. A system for tactile output estimation, comprising:
a depth map module configured to:
identify a region of interest (RoI) of an object, wherein an area of the RoI corresponds to a tactile sensor size of a tactile sensor,
receive depth data for the RoI from a depth sensor, and
generate a depth map for the RoI based on a volume of the depth data relative to a frame of reference of the RoI;
an estimation module configured to estimate a tactile sensor output based on the depth map; and
a surface module configured to determine surface properties based on the estimated tactile sensor output,
wherein the estimation module is configured to be trained by inputting the depth map in a network having a discriminator and a generator,
wherein the discriminator is configured to receive tactile data and the estimated tactile sensor output and determine whether the estimated tactile sensor output was received from the tactile sensor.
2. The system of claim 1 , wherein the RoI is identified based on optical data from an optical sensor.
3. The system of claim 1 , further comprising a grasping module configured to identify a location on the object to grip based on the surface properties.
4. The system of claim 1 , wherein the frame of reference is based on optical data, and wherein the depth data is registered with the optical data to form the depth map.
5. The system of claim 1 , wherein the network is a conditional generative adversarial network.
6. A method for tactile output estimation, comprising:
identifying a region of interest (RoI) of an object, wherein an area of the RoI corresponds to a tactile sensor size of a tactile sensor;
receiving depth data for the RoI from a depth sensor;
generating a depth map for the RoI based on a volume of the depth data relative to a frame of reference of the RoI;
estimating a tactile sensor output based on the depth map; and
determining surface properties based on the estimated tactile sensor output,
wherein estimating the estimated tactile sensor output includes the depth map being input to a network to generate the estimated tactile sensor output,
the method further comprising:
receiving tactile data and the estimated tactile sensor output; and
determining whether the estimated tactile sensor output was received from the tactile sensor.
7. The method of claim 6 , wherein the RoI is identified based on optical data from an optical sensor.
8. The method of claim 6 , wherein the frame of reference is based on optical data, and wherein the depth data is registered with the optical data to form the depth map.
9. The method of claim 6 , the method further comprising identifying a location on the object to grip based on the surface properties.
10. The method of claim 9 , wherein the frame of reference is based on optical data, and wherein the depth data is registered with the optical data to form the depth map.
11. A non-transitory computer readable storage medium storing instructions that when executed by a computer having a processor to perform a method for tactile output estimation, the method comprising:
identify a region of interest (RoI) of an object, wherein an area of the RoI corresponds to a tactile sensor size of a tactile sensor;
receive depth data for the RoI from a depth sensor;
generate a depth map for the RoI based on a volume of the depth data relative to a frame of reference of the RoI;
estimating a tactile sensor output based on the depth map; and
determining surface properties based on the estimated tactile sensor output,
wherein the frame of reference is based on optical data, and wherein the depth data is registered with the optical data to form the depth map,
the method further comprising:
receiving tactile data and the estimated tactile sensor output; and
determining whether the estimated tactile sensor output was received from the tactile sensor.
12. The non-transitory computer readable storage medium of claim 11 , wherein the RoI is identified based on optical data from an optical sensor.
13. The non-transitory computer readable storage medium of claim 11 , wherein the frame of reference is based on optical data, and wherein the depth data is registered with the optical data to form the depth map.
14. The non-transitory computer readable storage medium of claim 11 , the method further comprising identifying a location on the object to grip based on the surface properties.