IP Library Granted Patent US 11,521,512
Granted Patent B2
US 11,521,512 · App. 16/695,027 · Granted Dec 6, 2022

Systems for simulating joining operations using mobile devices

Inventors: Pedro Gerardo Marquinez Torrecilla (Annapolis, MD); William Joshua Becker (Manitowoc, WI); Justin Monroe Blount (Whitefish Bay, WI); Mitchell James Muske (Watertown, SD); Jessica Marie Marhefke (Neenah, WI); Pavel Gunia (Huelva, ES)
Assignees: Illinois Tool Works Inc.; Seabery North America Inc.
G09B19/24G06F1/163G06F1/1637G06F1/1686G09B9/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,521,512
App. No.
16/695,027
Granted
Dec 6, 2022
Kind
B2
Abstract

Systems are disclosed relating to a mobile device mounted to a welding helmet such that a wearer of the welding helmet can see a display of the mobile device when wearing the welding helmet. In some examples, the mobile device is mounted such that a camera of the mobile device is unobscured and positioned at approximately eye level, facing the same way the wearer's eyes are facing. In some examples, the simulated training environment may be presented to the user via the display screen of the mobile device, using images captured by the camera of the mobile device, when the mobile device is so mounted to the welding helmet.

Claims (59)

1. A weld training system, comprising:

a mobile device, the mobile device comprising:

a housing,

a display screen configured to display welding simulation images and a user-selectable element from a first side of the housing,

a camera configured to capture welding environment images of a welding environment from a second side of the housing opposite the first side of the housing,

processing circuitry in the housing and coupled to the display and the camera, and

a computer readable storage device comprising computer readable instructions which, when executed, cause the processing circuitry to:

execute a welding simulation that generates the welding simulation images based on the welding environment images captured via the camera,

determine, during the welding simulation, a workpiece position or a workpiece orientation of a workpiece in the welding environment, or a tool position or a tool orientation of a welding tool in the welding environment, based on the welding environment images captured by the camera,

in response to receiving a selection input, determine whether the user-selectable element has been selected based on the workpiece position, the workpiece orientation, the tool position, or the tool orientation, and

in response to determining the user-selectable element has been selected, perform a task associated with a selection of the user-selectable element.

2. The weld training system as defined in claim 1 , wherein the welding environment images comprise an image of the workpiece, the welding simulation images comprise a simulation workpiece image, and the computer readable instructions, when executed, cause the processor to:

generate the simulation workpiece image based on the workpiece position, or the workpiece orientation, and the image of the workpiece, the user-selectable element being anchored to the simulation workpiece image.

3. The weld training system as defined in claim 2 , further comprising a welding helmet, wherein the mobile device is mounted to the welding helmet.

4. The weld training system as defined in claim 3 , wherein the selection input comprises an indication of a trigger pull of the welding tool.

5. The weld training system as defined in claim 3 , wherein the selection input comprises a plurality of successive trigger pulls of a trigger of the welding tool.

6. The weld training system as defined in claim 1 , wherein the selection input comprises a speech input.

7. The weld training system as defined in claim 1 , wherein the computer readable instructions, when executed, further cause the processor to:

access a remote computing system to identify a simulated welding task based on a logged in user of the welding simulation; and

conduct the simulated welding mode based on the simulated welding task.

8. The weld training system as defined in claim 7 , wherein the remote computing system comprises a weld training management system.

9. The weld training system as defined in claim 7 , wherein the computer readable instructions, when executed, further cause the processor to:

download the identified simulated welding task from the remote computing system;

determine a representative result of the simulated welding task based on inputs received via the mobile device during the simulated welding mode; and

upload the representative result to the remote computing system.

10. The weld training system as defined in claim 7 , wherein the computer readable instructions, when executed, further cause the processor to:

monitor activities associated with the simulated welding task;

compare the activities to an activity goal; and

provide feedback via the display screen based on the comparison.

11. The weld training system as defined in claim 10 , wherein the activity goal is obtained from a remote computing system.

12. A weld training system, comprising:

a mobile device configured to conduct a welding simulation, the mobile device comprising:

processing circuitry, and

a computer readable storage device comprising computer readable instructions which, when executed, cause the processing circuitry to:

execute an application associated with the welding simulation,

detect insertion of the mobile device into an accessory, and

in response to the detection, transition the application to a simulated welding mode, and suppress one or more functions of one or more input devices of the mobile device.

13. The weld training system as defined in claim 12 , wherein the one or more functions comprise one or more of a touch sensitive input function, a power off function, a power restart function, or a volume control function.

14. The weld training system as defined in claim 12 , wherein the computer readable instructions, when executed, further cause the processor to:

detect removal of the mobile device from the accessory based on at least one of a sensor input, a physical button press, an input to a touchscreen of the mobile device, a voice command, or a menu selection using at least one of a welding tool communicatively coupled to the mobile device or a voice command.

15. The weld training system as defined in claim 12 , wherein the computer readable instructions, when executed, cause the processor to detect insertion of the mobile device into the accessory based on at least one of: recognition of a marker in one or more captured images or processed images, a sensor input, a physical button press, physical connection of the mobile device to the accessory, or a menu selection using at least one of a welding tool communicatively coupled to the mobile device, a touchscreen device input, or a voice command.

16. A weld training system, comprising:

a mobile device configured to conduct a welding simulation, the mobile device comprising:

one or more cameras configured to capture an image,

a display configured to display images of the welding simulation,

processing circuitry, and

a computer readable storage device comprising computer readable instructions which, when executed, cause the processing circuitry to:

determine a resolution of the display, a resolution of the one or more cameras, a size of the image captured by the one or more cameras, a processing capability of the mobile device, a number of the one or more cameras, or a measurement of an accelerometer, inertial measurement unit, or gyroscope of the mobile device, and

execute the welding simulation based on the resolution of the display, the resolution of the one or more cameras, the size of the image captured by the one or more cameras, the processing capability of the mobile device, the number of the one or more cameras, or the measurement of the accelerometer, inertial measurement unit, or gyroscope of the mobile device.

17. The weld training system as defined in claim 16 , wherein the computer readable instructions, when executed, cause the processor to:

determine the size of the image captured by the one or more cameras, the resolution of the one or more cameras, or the resolution of the display, and

convert the image captured by the one or more cameras to a threshold size when the size of the image is greater than the threshold size or the resolution is higher than a threshold resolution.

18. The weld training system as defined in claim 17 , wherein the computer readable instructions, when executed, cause the processor to convert the images by at least one of bilinear interpolation or nearest neighbors interpolation.

19. The weld training system as defined in claim 16 , wherein the computer readable instructions, when executed, cause the processor to:

determine whether the processing capability of the mobile device is sufficient to perform image recognition or image rendering given the size of the image captured by the one or more cameras, the resolution of the one or more cameras, or the resolution of the display, and

in response to determining the processing capability of the mobile device is not sufficient, configure a size, resolution, brightness, or color property of the images displayed by the display of the mobile device.

20. The weld training system as defined in claim 16 , wherein the computer readable instructions, when executed, cause the processor to:

determine the processing capability of the mobile device, and

configure an output image size, output image resolution, or output image frame rate of a welding simulation running on the mobile device based on the processing capability of the mobile device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 23, 2022
From: TORRECILLA, PEDRO GERARDO MARQUINEZ; BECKER, WILLIAM JOSHUA; BLOUNT, JUSTIN MONROE; MUSKE, MITCHELL JAMES; MARHEFKE, JESSICA MARIE; GUNIA, PAVEL
To: ILLINOIS TOOL WORKS INC.; SEABERY NORTH AMERICA INC.
Reel/Frame 061190/0445 →
Continuity (2)
Provisional Application 62807666 · Feb 19, 2019
Related Publication 20200265750A1 · Aug 20, 2020
Cited By (8)
US 1,127,682 US 12,240,634 US 12,371,198 US 12,454,375 US 12,486,017 US 12,522,384 US 12,591,247 US 12,722,812