Digital twin of an automated non-destructive ultrasonic testing system
Systems and methods are provided for generating a digital twin representation of an ultrasonic testing environment. In one aspect, probe position data received from one or more controllers and coordinate system data generated by one or more sensors can be used to generate a digital twin representation of an ultrasonic testing environment. In another aspect, generating the digital twin representation can include determining probe position measurements and a plurality of probe paths to be included in the digital twin representation. In one aspect the system can include a vision system to enable remote operation of the ultrasonic testing environment based on the generated digital twin representation.
1. A system, comprising:
a probe assembly coupled to one or more probe assembly controllers configured to control the probe assembly, the probe assembly including an ultrasonic probe coupled to one or more probe controllers configured to control the ultrasonic probe, the ultrasonic probe configured to transmit ultrasonic signals and receive reflected ultrasonic signals;
a plurality of sensors configured to generate coordinate system data associated with an ultrasonic testing environment, the coordinate system data corresponding to one or more geometric features of the ultrasonic testing environment and a test piece within the ultrasonic testing environment;
a vision system including a camera or an ultrasound transducer affixed to the probe assembly and configured to generate a live-view image of the ultrasonic probe and the test piece; and
a processor configured with executable instructions which when executed cause the processor to,
receive three-dimensional probe position data, from the one or more probe assembly controllers and the one or more probe controllers,
receive coordinate system data associated with the ultrasonic testing environment,
receive the live-view image from the vision system,
generate a digital twin representation of the ultrasonic testing environment corresponding to the coordinate system data and the probe position data, wherein the generated digital twin representation is a three-dimensional digital model of the ultrasonic testing environment and generating the digital twin representation includes determining a plurality of probe position measurements and a plurality of paths to be included in the digital twin representation, the plurality of paths including one or more paths along which the probe assembly and/or the ultrasonic probe can be configured to traverse within the ultrasonic testing environment,
generate a graphical user interface (GUI) including the live-view image of the ultrasonic probe including a probe orientation indicator and textual data characterizing a current location of the ultrasonic probe relative to the probe assembly based on the determined plurality of probe position measurements,
receive a user interaction within the GUI moving the probe orientation indicator to a desired position within the coordinate system of the generated digital twin representation, and
positioning, via the one or more probe assembly controllers and the one or more probe controllers, the ultrasonic probe at a physical location in the ultrasonic testing environment corresponding to the desired position within the coordinate system of the generated digital twin representation.
2. The system of claim 1 , wherein the GUI further includes the plurality of probe position measurements and one or more test piece position measurements, the one or more test piece position measurements determined based on the coordinate system data.
3. The system of claim 1 , wherein the plurality of probe position measurements include a probe elevation measurement corresponding to a distance measured between a transducer surface of the ultrasonic probe and a surface of the test piece from which ultrasonic signals are reflected to the ultrasonic probe.
4. The system of claim 1 , wherein the plurality of probe position measurements include a probe angle measurement corresponding to an angle measured between an axis of the ultrasonic probe and a surface of the test piece from which ultrasonic signals are reflected to the ultrasonic probe.
5. The system of claim 1 , wherein the coordinate system data includes one or more of cartesian coordinate data, polar coordinate data, cylindrical coordinate data, spherical coordinate data, homogenous coordinate data, curvilinear coordinate data, log-polar coordinate data, or Plücker coordinate data.
6. The system of claim 1 , wherein the plurality of sensors include tactile and non-tactile sensors.
7. The system of claim 6 , wherein the plurality of sensors are positioned inside or outside the ultrasonic testing environment.
8. The system of claim 1 , wherein the ultrasonic test environment can be operated based on the digital twin representation to perform ultrasonic scans of the test piece from a location remote to a location of the ultrasonic test environment.
9. The system of claim 1 , wherein the processor is further configured to generate the digital twin representation based on receiving test piece coordinate data.
10. The system of claim 1 , wherein the plurality of paths includes one or more of a linear scan path, an irregular scan path, and/or a scan path determined to avoid collision of the ultrasonic probe and the ultrasonic testing environment or test piece.
11. A method comprising:
receiving, by a processor of a computing device, three-dimensional probe position data from one or more probe assembly controllers coupled to the computing device and from one or more probe controllers, the one or more probe controllers coupled to the computing device and to an ultrasonic probe included in a probe assembly, wherein the one or more probe assembly controllers are configured to control the probe assembly, the one or more probe controllers are configured to control the ultrasonic probe, and the ultrasonic probe is configured to transmit ultrasonic signals and to receive reflected ultrasonic signals;
receiving, by the processor of the computing device, coordinate system data associated with an ultrasonic testing environment and corresponding to one or more geometric features of the ultrasonic testing environment and a test piece within the ultrasonic testing environment, the coordinate system data generated by a plurality of sensors coupled to the computing device;
receiving, by the processor of the computing device, a live-view image of the ultrasonic probe and the test piece from vision system including a camera or an ultrasound transducer affixed to the probe assembly;
generating, by the processor of the computing device, a digital twin representation of the ultrasonic testing environment corresponding to the coordinate system data and to the three-dimensional probe position data, wherein the generated digital twin representation is a three-dimensional digital model of the ultrasonic testing environment and generating the digital twin representation includes determining a plurality of probe position measurements and a plurality of paths to be included in the digital twin representation, the plurality of paths including one or more paths along which the probe assembly and/or the ultrasonic probe can be configured to traverse within the ultrasonic testing environment;
generating, by the processor of the computing device, a graphical user interface (GUI), on a display of the computing device, the GUI including the live-view image of the ultrasonic probe including a probe orientation indicator and textual data characterizing a current location of the ultrasonic probe relative to the probe assembly based on the determined plurality of probe position measurements;
receiving a user interaction within the GUI moving the probe orientation indicator to a desired position within the coordinate system of the generated digital twin representation; and
positioning, via the one or more probe assembly controllers and the one or more probe controllers, the ultrasonic probe at a physical location in the ultrasonic testing environment corresponding to the desired position within the coordinate system of the generated digital twin representation.
12. The method of claim 11 , wherein the GUI further includes the plurality of probe position measurements and one or more test piece position measurements, the one or more test piece position measurements determined based on the coordinate system data.
13. The method of claim 11 , wherein the plurality of probe position measurements include a probe elevation measurement corresponding to a distance measured between a transducer surface of the ultrasonic probe and a surface of the test piece from which ultrasonic signals are reflected to the ultrasonic probe.
14. The method of claim 11 , wherein the plurality of probe position measurements include a probe angle measurement corresponding to an angle measured between an axis of the ultrasonic probe and a surface of the test piece from which ultrasonic signals are reflected to the ultrasonic probe.
15. The method of claim 11 , wherein the coordinate system data includes one or more of cartesian coordinate data, polar coordinate data, cylindrical coordinate data, spherical coordinate data, homogenous coordinate data, curvilinear coordinate data, log-polar coordinate data, or Plücker coordinate data.
16. The method of claim 11 , wherein the plurality of sensors include tactile and non-tactile sensors.
17. The method of claim 16 , wherein the plurality of sensors are positioned inside or outside the ultrasonic testing environment.
18. The method of claim 11 , wherein the processor of the computing device is configured to operate the ultrasonic test environment based on the digital twin representation to perform ultrasonic scans of the test piece from a location remote from a location of the ultrasonic test environment.
19. The method of claim 11 , wherein the processor of the computing device is further configured to generate the digital twin representation based on receiving test piece coordinate data.
20. The method of claim 11 , wherein the plurality of paths includes one or more of a linear scan path, an irregular scan path, and/or a scan path determined to avoid collision of the ultrasonic probe and the ultrasonic testing environment or test piece.
21. A device comprising:
a display;
one or more probe controllers coupled to a probe assembly including an ultrasonic probe, the one or more probe controllers configured to control the ultrasonic probe, wherein the ultrasonic probe is configured to transmit ultrasonic signals and to receive reflected ultrasonic signals;
one or more probe assembly controllers configured to control the probe assembly;
a processor coupled to the display, the one or more probe controllers and the one or more probe assembly controllers; and
a memory coupled to the processor and storing computer-readable executable instructions, which when executed by the processor cause the processor to perform operations including,
receiving three-dimensional probe position data, from the one or more probe assembly controllers and by the one or more probe controllers;
receiving coordinate system data associated with an ultrasonic testing environment and corresponding to one or more geometric features of the ultrasonic testing environment and a test piece within the ultrasonic testing environment, the coordinate system data generated by a plurality of sensors coupled to the processor;
receiving a live-view image of the ultrasonic probe and the test piece from a vision system including a camera or an ultrasound transducer affixed to the probe assembly;
generating a digital twin representation of the ultrasonic testing environment corresponding to the coordinate system data and the probe position data, wherein the generated digital twin representation is a three-dimensional digital model of the ultrasonic testing environment and generating the digital twin representation includes determining a plurality of probe position measurements and a plurality of paths to be included in the digital twin representation, the plurality of paths including one or more paths along which the probe assembly and/or the ultrasonic probe can be configured to traverse within the ultrasonic testing environment;
providing a graphical user interface (GUI) to the display, the GUI including the live-view image of the ultrasonic probe including a probe orientation indicator and textual data characterizing a current location of the ultrasonic probe relative to the probe assembly based on the determined plurality of probe position measurements;
receiving a user interaction within the GUI moving the probe orientation indicator to a desired position within the coordinate system of the generated digital twin representation; and
positioning, via the one or more probe assembly controllers and the one or more probe controllers, the ultrasonic probe at a physical location in the ultrasonic testing environment corresponding to the desired position within the coordinate system of the generated digital twin representation.