IP Library Granted Patent US 12,358,170
Granted Patent B2
US 12,358,170 · App. 17/406,842 · Granted Jul 15, 2025

Method and apparatus for robot collision avoidance by full surface proximity detection

Inventors: Xiaoran Fan (Secaucus, NJ); Riley William Simmons-Edler (Princeton, NJ); Daewon Lee (Princeton, NJ); Lawrence Jackel (Holmdel, NJ); Daniel Dongyuel Lee (Tenafly, NJ); Richard Howard (Highland Park, NJ)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
B25J19/028B25J9/1666B25J13/086B25J13/089G06N3/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,358,170
App. No.
17/406,842
Granted
Jul 15, 2025
Kind
B2
Abstract

An apparatus for collision avoidance by surface proximity detection includes a plurality of piezoelectric elements disposed adjacent to a surface of an object, a memory storing instructions, and at least one processor configured to execute the instructions to control a first one among the piezoelectric elements to generate an acoustic wave along the surface of the object, and receive, via a second one among the piezoelectric elements, an acoustic wave signal corresponding to the generated acoustic wave. The at least one processor is further configured to execute the instructions to filter the received acoustic wave signal, using a band-pass filter for reducing noise of the received acoustic wave signal, obtain a proximity signal for proximity detection, from the filtered acoustic wave signal, using a linear time-invariant filter, and detect whether an obstacle is proximate to the surface of the object by inputting the obtained proximity signal into a neural network.

Claims (61)

1. An apparatus for collision avoidance by surface proximity detection of an obstacle relative to an object, the apparatus comprising:

a plurality of piezoelectric elements attached to a surface of the object;

a memory storing instructions; and

at least one processor configured to execute the instructions to:

control a first one among the piezoelectric elements to generate an acoustic wave along the surface of the object;

receive, via a second one among the piezoelectric elements, an acoustic wave signal corresponding to an acoustic impedance change in an acoustic pressure field perturbed by a standing wave pattern or interference pattern established by the obstacle;

reduce noise of the received acoustic wave signal by filtering, using a band-pass filter, the received acoustic wave signal;

obtain a proximity signal for proximity detection, from the filtered acoustic wave signal, using a linear time-invariant filter; and

detect whether the obstacle is adjacent to the surface of the object by inputting the obtained proximity signal into a neural network.

2. The apparatus of claim 1 , wherein the object is a robot, and

the at least one processor is further configured to execute the instructions to, based on the obstacle being detected to be adjacent to the surface of the object, control the object to avoid collision with the obstacle.

3. The apparatus of claim 2 , wherein the noise comprises electrical noise of a power supply of the robot and mechanical noise of a motor of the robot.

4. The apparatus of claim 1 , wherein the linear time-invariant filter comprises any one or any combination of a Hilbert transform and a Fourier transform of the filtered acoustic wave signal.

5. The apparatus of claim 1 , wherein the at least one processor is further configured to execute the instructions to:

truncate the received acoustic wave signal into a window; and

filter the truncated acoustic wave signal, using the band-pass filter.

6. The apparatus of claim 1 , wherein the at least one processor is further configured to execute the instructions to:

truncate the obtained proximity signal into a plurality of windows; and

detect whether the obstacle is adjacent to the surface of the object by inputting the truncated proximity signal into the neural network.

7. The apparatus of claim 1 , wherein the at least one processor is further configured to execute the instructions to:

truncate the obtained proximity signal into a plurality of windows;

input the truncated proximity signal into the neural network to obtain a plurality of predictions of whether the obstacle is adjacent to the surface of the object; and

detect whether the obstacle is adjacent to the surface of the object, based on a majority vote among the obtained plurality of predictions.

8. A method of collision avoidance by surface proximity detection of an obstacle relative to an object, the method being performed by at least one processor, and the method comprising:

controlling a first one among piezoelectric elements attached to a surface of the object, to generate an acoustic wave along the surface of the object;

receiving, via a second one among the piezoelectric elements, an acoustic wave signal corresponding to an acoustic impedance change in an acoustic pressure field perturbed by a standing wave pattern or interference pattern established by the obstacle;

reducing noise of the received acoustic wave signal by filtering, using a band-pass filter, the received acoustic wave signal;

obtaining a proximity signal for proximity detection, from the filtered acoustic wave signal, using a linear time-invariant filter; and

detecting whether the obstacle is adjacent to the surface of the object by inputting the obtained proximity signal into a neural network.

9. The method of claim 8 , wherein the object is a robot, and

the method further comprises, based on the obstacle being detected to be adjacent to the surface of the object, control the object to avoid collision with the obstacle.

10. The method of claim 9 , wherein the noise comprises electrical noise of a power supply of the robot and mechanical noise of a motor of the robot, from the received acoustic wave signal.

11. The method of claim 8 , wherein the linear time-invariant filter comprises any one or any combination of a Hilbert transform and a Fourier transform of the filtered acoustic wave signal.

12. The method of claim 8 , further comprising truncating the received acoustic wave signal into a window,

wherein the filtering comprises filtering the truncated acoustic wave signal, using the band-pass filter.

13. The method of claim 8 , further comprising truncating the obtained proximity signal into a plurality of windows,

wherein the detecting comprises detecting whether the obstacle is adjacent to the surface of the object by inputting the truncated proximity signal into the neural network.

14. The method of claim 8 , further comprising:

truncating the obtained proximity signal into a plurality of windows; and

inputting the truncated proximity signal into the neural network to obtain a plurality of predictions of whether the obstacle is adjacent to the surface of the object,

wherein the detecting comprises detecting whether the obstacle is adjacent to the surface of the object, based on a majority vote among the obtained plurality of predictions.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to:

control a first one among piezoelectric elements attached to a surface of an object, to generate an acoustic wave along the surface of the object;

receive, via a second one among the piezoelectric elements, an acoustic wave signal corresponding to an acoustic impedance change in an acoustic pressure field perturbed by a standing wave pattern or interference pattern established by an obstacle;

reduce noise of the received acoustic wave signal by filtering, by using a band-pass filter, the received acoustic wave signal;

obtain a proximity signal for proximity detection, from the filtered acoustic wave signal, using a linear time-invariant filter; and

detect whether an obstacle is adjacent to the surface of the object by inputting the obtained proximity signal into a neural network.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the object is a robot,

wherein the instructions, when executed by the at least one processor, further cause the at least one processor to, based on the obstacle being detected to be adjacent to the surface of the object, control the object to avoid collision with the obstacle, and

wherein the noise comprises electrical noise of a power supply of the robot and mechanical noise of a motor of the robot.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the linear time-invariant filter comprises any one or any combination of a Hilbert transform and a Fourier transform of the filtered acoustic wave signal.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

truncate the received acoustic wave signal into a window; and

filter the truncated acoustic wave signal, using the band-pass filter.

19. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

truncate the obtained proximity signal into a plurality of windows; and

detect whether the obstacle is adjacent to the surface of the object by inputting the truncated proximity signal into the neural network.

20. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:

truncate the obtained proximity signal into a plurality of windows;

input the truncated proximity signal into the neural network to obtain a plurality of predictions of whether the obstacle is adjacent to the surface of the object; and

detect whether the obstacle is adjacent to the surface of the object, based on a majority vote among the obtained plurality of predictions.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2021
From: FAN, XIAORAN; SIMMONS-EDLER, RILEY WILLIAM; LEE, DAEWON; JACKEL, LAWRENCE; LEE, DANIEL DONGYUEL; HOWARD, RICHARD
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 057233/0914 →
Continuity (2)
Provisional Application 63155126 · Mar 1, 2021
Related Publication 20220274269A1 · Sep 1, 2022
References Cited (55)
US 5483501A · Park et al. · 1996 [cited by applicant]
US 5515341A · Toda · 1996 [cited by examiner]
US 8742645B2 · Lee et al. · 2014 [cited by applicant]
US 8994248B2 · Lee · 2015 [cited by examiner]
US 10031111B2 · Clark · 2018 [cited by examiner]
US 10416629B1 · Bingham et al. · 2019 [cited by applicant]
US 10789824B1 · Cheng · 2020 [cited by examiner]
US 10795342B1 · Cai · 2020 [cited by examiner]
US 20030133362A1 · Fortuna et al. · 2003 [cited by applicant]
US 20090059726A1 · Okuda · 2009 [cited by examiner]
US 20090168603A1 · Okuda · 2009 [cited by examiner]
US 20120147708A1 · Lee · 2012 [cited by examiner]
US 20150016225A1 · Steckel · 2015 [cited by examiner]
US 20180250824A1 · Hannya · 2018 [cited by examiner]
US 20200001460A1 · Sato · 2020 [cited by examiner]
US 20200210726A1 · Yang et al. · 2020 [cited by applicant]
US 20200302949A1 · Jeong · 2020 [cited by examiner]
US 20200367977A1 · Liu et al. · 2020 [cited by applicant]
US 20220036245A1 · Muller · 2022 [cited by examiner]
CN 110954251A · 2020 [cited by applicant]
JP 2005161450A · 2005 [cited by applicant]
JP 2010128543A · 2010 [cited by applicant]
JP 4511216B2 · 2010 [cited by applicant]
KR 1020120063743A · 2012 [cited by applicant]
KR 101655302B1 · 2016 [cited by applicant]
KR 102177711B1 · 2020 [cited by applicant]
Communication dated May 24, 2022 issued by the International Searching Authority in counterpart Application No. PCT/KR2022/002517 (PCT/ISA/220, PCT/ISA/210, and PCT/ISA/237). [cited by applicant]
Cho, Il-Joo et al., “Compliant Ultrasound Proximity Sensor for the Safe Operation of Human Friendly Robots Integrated with Tactile Sensing Capability”, Journal of Electrical Engineering and Technology, 2017, vol. 12, No… [cited by applicant]
Geiger, Martin et al., “160-GHz Radar Proximity Sensor With Distributed and Flexible Antennas for Collaborative Robots”, IEEE Access, Feb. 8, 2019, vol. 7, pp. 14977-14984. (8 pages total). [cited by applicant]
Benet, G. et al., “Using infrared sensors for distance measurement in mobile robots”, Robotics and Autonomous Systems, vol. 1006, 2002, pp. 1-12. [cited by applicant]
Amzajerdian, Farzin et al., “Imaging Flash Lidar for Autonomous Safe Landing and Spacecraft Proximity Operation”, in AIAA SPACE 2016, 2016. (12 pages total). [cited by applicant]
Hashimoto, Ken-Ya, “Surface Acoustic Wave Devices in Telecommunications”, . Springer, 2000. (9 pages total). [cited by applicant]
Alvarez, H. et al., “Collision Avoidance for Quadrotors with a Monocular Camera”, Experimental Robotics, Springer, 2016, pp. 1-14. [cited by applicant]
Chen, Gang et al., “Bio-inspired Obstacle Avoidance for Flying Robots with Active Sensing”, arXiv:2010.04977v1 [cs.RO], Oct. 10, 2020. (8 pages total). [cited by applicant]
Schmidt, Bernard et al., “Depth camera based collision avoidance via active robot control”, Journal of Manufacturing Systems, 2014. (8 pages total). [cited by applicant]
Fan, Xiaoran et al., “Towards Flexible Wireless Charging for Medical Implants Using Distributed Antenna System”, MobiCom 20, Sep. 21-25, 2020, ACM MOBICOM, 2020. (15 pages total). [cited by applicant]
Poeppel, Alexander et al., “Robust Distance Estimation of Capacitive Proximity Sensors in HRI using Neural Networks”, 2020 Fourth IEEE International Conference on Robotic Computing (IRC), pp. 344-351. [cited by applicant]
“Kinova Jaco Prosthetic robotic arm,” KINOVA, Webpage, 2018, https://www.kinovarobotics.com/sites/default/files/PS-PRA-JAC-UG-INT-EN%20201804-1.0%20%28KINOVA%20JACO%E2%84%A2%20Prosthetic%20robotic%20arm%20user%20guide%2… [cited by applicant]
“VNA-UHF—Two Port Vector Network Analyzer, 5 kHz to 1200 MHz”, Array Solutions, Webpage, 2021, https://www.arraysolutions.com/vna-uhf. (3 pages total). [cited by applicant]
“Noico 80 mil Sound Deadner”, Noico Solutions, Webpage, 2020, https://noico.info/products/noico-80-mil-10-sqft-car-sound-deadening-mat/. (3 pages total). [cited by applicant]
Erceg, Vinko et al., “An Empirically Based Path Loss Model for Wireless Channels in Suburban Environments”, IEEE Journal on Selected Areas in Communications, vol. 17, No. 7, Jul. 1999, pp. 1205-1211. (7 pages total). [cited by applicant]
Fan, Xiaoran et al., “Acoustic Collision Detection and Localization for Robot Manipulators”, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 25-29, 2020, pp. 9529-9536. (8 pages tot… [cited by applicant]
Granjon, Pierre, “The CuSum algorithm—a small review”, HAL Open Science, Mar. 13, 2014. (23 pages total). [cited by applicant]
“CPT-2065-L100 Piezo Element,” CUDEVICES, Webpage, 2021, https://www.cuidevices.com/product/resource/cpt-2065-1100.pdf. (4 pages total). [cited by applicant]
“SMSL M100 USB DAC”, Webpage, 2021, https://www.amazon.com/AK4452-DSD512-Coaxial-Optical-Decoder/dp/B0868PW3YZ. (8 pages total). [cited by applicant]
“Massdrop X Alex Cavalli Liquid Carbon X Amp”, Webpage, https://drop.com/buy/massdrop-x-alex-cavalli-liquid-carbon-x-amp, 2021. (9 pages total). [cited by applicant]
“Zoom F8n MultiTrack Field Recorder”, ZOOMCORP, Webpage, https://zoomcorp.com/en/us/field-recorders/field-recorders/f8n/, 2015. (34 pages total). [cited by applicant]
“Intel NUC7i7BNH”, Webpage, https://ark.intel.com/content/www/us/en/ark/products/95065/intel-nuc-kit-nuc7i7bnh.html, 2019. (4 pages total). [cited by applicant]
Paszke, Adam et al., “PyTorch: An Imperative Style, High-Performance Deep Learning Library”, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019). (12 pages totals). [cited by applicant]
Kingma, Diederik P., “ADAM: A Method for Stochastic Optimization”, Published as a conference paper at ICLR 2015, arXiv:1412.6980v9 [cs.LG], Jan. 30, 2017. (15 pages total). [cited by applicant]
Ioffe, Sergey et al., “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”, Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 2015, JMLR: W&CP… [cited by applicant]
Fan, Xiaoran et al., “AuraSense: Robot Collision Avoidance by Full Surface Proximity Detection”, arXiv:2108.04867v1 [cs.RO], Aug. 10, 2021. (8 pages total). [cited by applicant]
Kohji Toda et al., “Ultrasonic imaging system using a leaky surface acoustic wave transducer composed of piezoelectric ceramic and fused quartz”, Journal of Applied Physics, vol. 69, No. 1, Jan. 1, 1991, pp. 103-108, DO… [cited by applicant]
Communication issued on Feb. 22, 2024 by the European Patent Office for European Patent Application No. 22763516.6. [cited by applicant]
Communication dated May 21, 2025 issued by the European Patent Office in European Patent Application No. 22763516.6. [cited by applicant]