IP Library Granted Patent US 12,250,362
Granted Patent B1
US 12,250,362 · App. 18/675,092 · Granted Mar 11, 2025

Hybrid display system for a force measurement assembly, an exercise device, or an interactive game

Inventor: Necip Berme (Worthington, OH)
Assignee: Bertec Corporation
H04N13/361G06T19/006H04N13/156H04N13/243H04N13/279H04N13/282H04N13/332H04N13/351H04N13/383
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,250,362
App. No.
18/675,092
Granted
Mar 11, 2025
Kind
B1
Abstract

A hybrid display system for a force measurement assembly, an exercise device, or an interactive game is disclosed herein. The hybrid display system includes a head-mounted visual display device having an output screen; at least one camera or a substantially transparent lens for enabling a system user to see one or more images of a real-world environment outside of the head-mounted visual display device; and a data processing device configured to generate a virtual reality environment and display the virtual reality environment on the output screen of the head-mounted visual display device, and the one or more images of the real-world environment being visible through the substantially transparent lens or the data processing device further configured to project at least a portion of the one or more images of the real-world environment captured by the at least one camera into the display of the virtual reality environment.

Claims (31)

1. A hybrid display system for a force measurement assembly, an exercise device, or an interactive game, the hybrid display system comprising:

a head-mounted visual display device having an output screen, the head-mounted visual display device configured to display one or more images on the output screen so that the one or more images are viewable by a system user;

at least one camera attached to the head-mounted visual display device or a substantially transparent lens for enabling the system user to see one or more images of a real-world environment outside of the head-mounted visual display device, the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, or the substantially transparent lens being provided on the head-mounted visual display device so that the system user is able to see the one or more images of the real-world environment outside of the head-mounted visual display device through the substantially transparent lens; and

a data processing device operatively coupled to the head-mounted visual display device, the data processing device configured to generate a virtual reality environment and display the virtual reality environment on the output screen of the head-mounted visual display device, and the one or more images of the real-world environment being visible through the substantially transparent lens or the data processing device further configured to project at least a portion of the one or more images of the real-world environment captured by the at least one camera into the display of the virtual reality environment so that the system user is able to have a frame of reference in the real-world environment while he or she is immersed in the virtual reality environment;

wherein the system user is: (i) disposed on an exercise device that relates to the virtual reality environment on the output screen of the head-mounted visual display device, or (ii) disposed on a force measurement assembly; and

wherein the one or more images of the real-world environment projected into the display of the virtual reality environment by the data processing device include a top portion of the exercise device or the force measurement assembly that is within a visibility range of the at least one camera, or the one or more images of the real-world environment being visible through the substantially transparent lens include a top portion of the exercise device or the force measurement assembly that is within a visibility range of the substantially transparent lens, so as to enable the system user to have a sense of his or her position on the exercise device or the force measurement assembly while being immersed in the virtual reality environment.

2. The hybrid display system according to claim 1 , wherein the system user is disposed on the exercise device that relates to the virtual reality environment on the output screen of the head-mounted visual display device, and the one or more images of the real-world environment projected into the display of the virtual reality environment by the data processing device include a top portion of the exercise device or the one or more images of the real-world environment being visible through the substantially transparent lens include a top portion of the exercise device so as to enable the system user to have a sense of his or her position on the exercise device while being immersed in the virtual reality environment.

3. The hybrid display system according to claim 2 , wherein the exercise device on which the system user is disposed is selected from the group consisting of: (i) a treadmill, (ii) an exercise bike, (iii) a rowing machine, (iv) an elliptical machine, and (v) another type of exercise device.

4. The hybrid display system according to claim 1 , wherein the system user is disposed on the force measurement assembly, and the one or more images of the real-world environment projected into the display of the virtual reality environment by the data processing device include a top portion of the force measurement assembly or the one or more images of the real-world environment being visible through the substantially transparent lens include a top portion of the force measurement assembly so as to enable the system user to have a sense of his or her position on the force measurement assembly while being immersed in the virtual reality environment.

5. The hybrid display system according to claim 4 , wherein the force measurement assembly on which the system user is disposed is selected from the group consisting of: (i) a force plate, (ii) a balance plate, (iii) an instrumented treadmill, and (iv) another type of force measurement assembly that is configured to measure one or more forces and/or moments exerted thereon by the system user.

6. The hybrid display system according to claim 5 , wherein the force measurement assembly comprises:

a top surface for receiving a body of the system user; and

at least one force transducer, the at least one force transducer configured to sense one or more measured quantities and output one or more signals that are representative of forces and/or moments being applied to the top surface of the force measurement assembly by the system user.

7. The hybrid display system according to claim 1 , wherein the hybrid display system is provided with the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, and the one or more images of the real-world environment captured by the at least one camera and projected into the display of the virtual reality environment by the data processing device include the top portion of the exercise device or the force measurement assembly so as to enable the system user to have a sense of his or her position on the exercise device or the force measurement assembly while being immersed in the virtual reality environment.

8. The hybrid display system according to claim 1 , wherein the one or more images of the real-world environment are intermixed with the display of the virtual reality environment on the output screen of the head-mounted visual display device.

9. The hybrid display system according to claim 1 , wherein the virtual reality environment on the output screen of the head-mounted visual display device is located in a first section of the output screen of the head-mounted visual display device, the one or more images of the real-world environment are located in a second section of the output screen of the head-mounted visual display device, and the first section of the output screen of the head-mounted visual display device is separate from the second section of the output screen of the head-mounted visual display device.

10. The hybrid display system according to claim 1 , wherein the hybrid display system is provided with the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, and the at least one camera is disposed on the head-mounted visual display device.

11. The hybrid display system according to claim 1 , wherein the hybrid display system is provided with at least one additional camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, and the at least one additional camera is a part of a motion capture system that is separate from the head-mounted visual display device.

12. The hybrid display system according to claim 1 , wherein the head-mounted visual display device is in the form of an augmented reality headset, a virtual reality headset, or a mixed reality headset.

13. A hybrid display system for a force measurement assembly, an exercise device, or an interactive game, the hybrid display system comprising:

a head-mounted visual display device having an output screen, the head-mounted visual display device configured to display one or more images on the output screen so that the one or more images are viewable by a system user;

at least one camera or a substantially transparent lens for enabling the system user to see one or more images of a real-world environment outside of the head-mounted visual display device, the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, or the substantially transparent lens being provided on the head-mounted visual display device so that the system user is able to see the one or more images of the real-world environment outside of the head-mounted visual display device through the substantially transparent lens; and

a data processing device operatively coupled to the head-mounted visual display device, the data processing device configured to generate a virtual reality environment and display the virtual reality environment on the output screen of the head-mounted visual display device, and the one or more images of the real-world environment being visible through the substantially transparent lens or the data processing device further configured to project at least a portion of the one or more images of the real-world environment captured by the at least one camera into the display of the virtual reality environment so that the system user is able to have a frame of reference in the real-world environment while he or she is immersed in the virtual reality environment;

wherein the system user is: (i) disposed on an exercise device that relates to the virtual reality environment on the output screen of the head-mounted visual display device, (ii) disposed on a force measurement assembly, or (iii) playing an interactive game;

wherein the one or more images of the real-world environment projected into the display of the virtual reality environment by the data processing device include one or more boundaries of an allowable interaction space in the real-world environment, or the one or more images of the real-world environment being visible through the substantially transparent lens include the one or more boundaries of the allowable interaction space in the real-world environment, so as to enable the system user to have a sense of his or her position relative to the one or more boundaries of the allowable interaction space in the real-world environment while being immersed in the virtual reality environment; and

wherein, when the system user comes within a predetermined distance of at least one of the one or more boundaries of the allowable interaction space in the real-world environment, the real-world environment begins to appear in the virtual reality environment displayed on the head-mounted visual display device so that the system user does not collide with the at least one of the one or more boundaries of the allowable interaction space.

14. The hybrid display system according to claim 13 , wherein the system user is disposed on the exercise device that relates to the virtual reality environment on the output screen of the head-mounted visual display device, and the exercise device on which the system user is disposed is selected from the group consisting of: (i) a treadmill, (ii) an exercise bike, (iii) a rowing machine, (iv) an elliptical machine, and (v) another type of exercise device.

15. The hybrid display system according to claim 13 , wherein the system user is disposed on the force measurement assembly, and the force measurement assembly on which the system user is disposed is selected from the group consisting of: (i) a force plate, (ii) a balance plate, (iii) an instrumented treadmill, and (iv) another type of force measurement assembly that is configured to measure one or more forces and/or moments exerted thereon by the system user.

16. The hybrid display system according to claim 13 , wherein the system user is playing the interactive game, and the one or more boundaries of the allowable interaction space in the real-world environment comprise one or more game boundaries of the allowable interaction game space in the real-world environment.

17. The hybrid display system according to claim 13 , wherein the hybrid display system is provided with the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, and the one or more images of the real-world environment projected into the display of the virtual reality environment by the data processing device include one or more boundaries of an allowable interaction space in the real-world environment so as to enable the system user to have a sense of his or her position relative to the one or more boundaries of the allowable interaction space in the real-world environment while being immersed in the virtual reality environment.

18. The hybrid display system according to claim 13 , wherein the head-mounted visual display device is in the form of an augmented reality headset, a virtual reality headset, or a mixed reality headset.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 27, 2024
From: BERME, NECIP, DR.
To: BERTEC CORPORATION
Reel/Frame 067532/0888 →
Continuity (9)
Continuation In Part 18138112 · Apr 23, 2023
Continuation In Part 17718290 · Apr 11, 2022
Continuation In Part 17114375 · Dec 7, 2020
Continuation In Part 16785960 · Feb 10, 2020
Continuation In Part 16550219 · Aug 24, 2019
Continuation In Part 16283685 · Feb 22, 2019
Continuation In Part 15908323 · Feb 28, 2018
Continuation In Part 15242558 · Aug 21, 2016
Provisional Application 62208671 · Aug 22, 2015
References Cited (180)
US 5205800A · Grant · 1993 [cited by applicant]
US 5375610A · LaCourse et al. · 1994 [cited by applicant]
US 5562572A · Carmein · 1996 [cited by applicant]
US 5577981A · Jarvik · 1996 [cited by applicant]
US 5611174A · Hayashi · 1997 [cited by applicant]
US 5961195A · Yoshimatsu et al. · 1999 [cited by applicant]
US 6038488A · Barnes et al. · 2000 [cited by applicant]
US 6102832A · Tani · 2000 [cited by examiner]
US 6113237A · Ober et al. · 2000 [cited by applicant]
US 6152564A · Ober et al. · 2000 [cited by applicant]
US 6295878B1 · Berme · 2001 [cited by applicant]
US 6354155B1 · Berme · 2002 [cited by applicant]
US 6389883B1 · Berme et al. · 2002 [cited by applicant]
US 6600475B2 · Gutta et al. · 2003 [cited by applicant]
US 6606111B1 · Kondo et al. · 2003 [cited by applicant]
US 6738065B1 · Even-Zohar · 2004 [cited by applicant]
US 6774885B1 · Even-Zohar · 2004 [cited by applicant]
US 6936016B2 · Berme et al. · 2005 [cited by applicant]
US 6944581B2 · Creek · 2005 [cited by applicant]
US 7931604B2 · Even-Zohar et al. · 2011 [cited by applicant]
US 8181541B2 · Berme · 2012 [cited by applicant]
US 8315822B2 · Berme et al. · 2012 [cited by applicant]
US 8315823B2 · Berme et al. · 2012 [cited by applicant]
US D689388S · Berme · 2013 [cited by applicant]
US D689389S · Berme · 2013 [cited by applicant]
US 8543540B1 · Wilson et al. · 2013 [cited by applicant]
US 8544347B1 · Berme · 2013 [cited by applicant]
US 8643669B1 · Wilson et al. · 2014 [cited by applicant]
US 8700569B1 · Wilson et al. · 2014 [cited by applicant]
US 8704855B1 · Berme et al. · 2014 [cited by applicant]
US 8764532B1 · Berme · 2014 [cited by applicant]
US 8847989B1 · Berme et al. · 2014 [cited by applicant]
US D715669S · Berme · 2014 [cited by applicant]
US 8902249B1 · Wilson et al. · 2014 [cited by applicant]
US 8915149B1 · Berme · 2014 [cited by applicant]
US 9032817B2 · Berme et al. · 2015 [cited by applicant]
US 9043278B1 · Wilson et al. · 2015 [cited by applicant]
US 9066667B1 · Berme et al. · 2015 [cited by applicant]
US 9081436B1 · Berme et al. · 2015 [cited by applicant]
US 9111498B2 · Border et al. · 2015 [cited by applicant]
US 9159152B1 · Glover · 2015 [cited by examiner]
US 9168420B1 · Berme et al. · 2015 [cited by applicant]
US 9173596B1 · Berme et al. · 2015 [cited by applicant]
US 9200897B1 · Wilson et al. · 2015 [cited by applicant]
US 9277857B1 · Berme et al. · 2016 [cited by applicant]
US D755067S · Berme et al. · 2016 [cited by applicant]
US 9404823B1 · Berme et al. · 2016 [cited by applicant]
US 9414784B1 · Berme et al. · 2016 [cited by applicant]
US 9468370B1 · Shearer · 2016 [cited by applicant]
US 9517008B1 · Berme et al. · 2016 [cited by applicant]
US 9526443B1 · Berme et al. · 2016 [cited by applicant]
US 9526451B1 · Berme · 2016 [cited by applicant]
US 9558399B1 · Jeka et al. · 2017 [cited by applicant]
US 9568382B1 · Berme et al. · 2017 [cited by applicant]
US 9622686B1 · Berme et al. · 2017 [cited by applicant]
US 9763604B1 · Berme et al. · 2017 [cited by applicant]
US 9770203B1 · Berme et al. · 2017 [cited by applicant]
US 9778119B2 · Berme et al. · 2017 [cited by applicant]
US 9814430B1 · Berme et al. · 2017 [cited by applicant]
US 9818230B2 · Moravetz · 2017 [cited by applicant]
US 9829311B1 · Wilson et al. · 2017 [cited by applicant]
US 9854997B1 · Berme et al. · 2018 [cited by applicant]
US 9916011B1 · Berme et al. · 2018 [cited by applicant]
US 9927312B1 · Berme et al. · 2018 [cited by applicant]
US 10010248B1 · Shearer · 2018 [cited by applicant]
US 10010286B1 · Berme et al. · 2018 [cited by applicant]
US 10085676B1 · Berme et al. · 2018 [cited by applicant]
US 10117602B1 · Berme et al. · 2018 [cited by applicant]
US 10126186B2 · Berme et al. · 2018 [cited by applicant]
US 10216262B1 · Berme et al. · 2019 [cited by applicant]
US 10231662B1 · Berme et al. · 2019 [cited by applicant]
US 10264964B1 · Berme et al. · 2019 [cited by applicant]
US 10331324B1 · Wilson et al. · 2019 [cited by applicant]
US 10342473B1 · Berme et al. · 2019 [cited by applicant]
US 10390736B1 · Berme et al. · 2019 [cited by applicant]
US 10413230B1 · Berme et al. · 2019 [cited by applicant]
US 10463250B1 · Berme et al. · 2019 [cited by applicant]
US 10527508B2 · Berme et al. · 2020 [cited by applicant]
US 10555688B1 · Berme et al. · 2020 [cited by applicant]
US 10646153B1 · Berme et al. · 2020 [cited by applicant]
US 10722114B1 · Berme et al. · 2020 [cited by applicant]
US 10736545B1 · Berme et al. · 2020 [cited by applicant]
US 10765936B2 · Berme et al. · 2020 [cited by applicant]
US 10803990B1 · Wilson et al. · 2020 [cited by applicant]
US 10853970B1 · Akbas et al. · 2020 [cited by applicant]
US 10856796B1 · Berme et al. · 2020 [cited by applicant]
US 10860843B1 · Berme et al. · 2020 [cited by applicant]
US 10945599B1 · Berme et al. · 2021 [cited by applicant]
US 10966606B1 · Berme · 2021 [cited by applicant]
US 11033453B1 · Berme et al. · 2021 [cited by applicant]
US 11052288B1 · Berme et al. · 2021 [cited by applicant]
US 11054325B2 · Berme et al. · 2021 [cited by applicant]
US 11074711B1 · Akbas et al. · 2021 [cited by applicant]
US 11097154B1 · Akbas et al. · 2021 [cited by applicant]
US 11158422B1 · Berme et al. · 2021 [cited by applicant]
US 11182924B1 · Akbas et al. · 2021 [cited by applicant]
US 11262231B1 · Berme et al. · 2022 [cited by applicant]
US 11262258B2 · Berme et al. · 2022 [cited by applicant]
US 11301045B1 · Berme et al. · 2022 [cited by applicant]
US 11311209B1 · Berme et al. · 2022 [cited by applicant]
US 11321868B1 · Akbas et al. · 2022 [cited by applicant]
US 11337606B1 · Berme et al. · 2022 [cited by applicant]
US 11348279B1 · Akbas et al. · 2022 [cited by applicant]
US 11458362B1 · Berme et al. · 2022 [cited by applicant]
US 11521373B1 · Akbas et al. · 2022 [cited by applicant]
US 11540744B1 · Berme · 2023 [cited by applicant]
US 11604106B2 · Berme et al. · 2023 [cited by applicant]
US 11631193B1 · Akbas et al. · 2023 [cited by applicant]
US 11688139B1 · Karagoz et al. · 2023 [cited by applicant]
US 11705244B1 · Berme · 2023 [cited by applicant]
US 11712162B1 · Berme et al. · 2023 [cited by applicant]
US 11790536B1 · Berme et al. · 2023 [cited by applicant]
US 11798182B1 · Karagoz et al. · 2023 [cited by applicant]
US 11816258B1 · Berme et al. · 2023 [cited by applicant]
US 11826601B1 · Berme · 2023 [cited by applicant]
US 11850078B1 · Berme · 2023 [cited by applicant]
US 11857331B1 · Berme et al. · 2024 [cited by applicant]
US 11865407B1 · Berme et al. · 2024 [cited by applicant]
US 11911147B1 · Berme et al. · 2024 [cited by applicant]
US 11992746B1 · Berme · 2024 [cited by applicant]
US 12013542B1 · Berme · 2024 [cited by examiner]
US 20020149752A1 · Courchesne · 2002 [cited by applicant]
US 20030216656A1 · Berme et al. · 2003 [cited by applicant]
US 20030223113A1 · Starkweather · 2003 [cited by applicant]
US 20050148432A1 · Carmein · 2005 [cited by applicant]
US 20060247104A1 · Grabiner et al. · 2006 [cited by applicant]
US 20060264786A1 · Nashner · 2006 [cited by applicant]
US 20080221487A1 · Evev-Zohar et al. · 2008 [cited by applicant]
US 20080228110A1 · Berme · 2008 [cited by applicant]
US 20100131113A1 · Even-Zohar · 2010 [cited by applicant]
US 20110277562A1 · Berme · 2011 [cited by applicant]
US 20120266648A1 · Berme · 2012 [cited by examiner]
US 20120271565A1 · Berme · 2012 [cited by examiner]
US 20130107371A1 · DeVaul · 2013 [cited by examiner]
US 20130225370A1 · Flynt et al. · 2013 [cited by applicant]
US 20140074179A1 · Heldman et al. · 2014 [cited by applicant]
US 20140098128A1 · Fein · 2014 [cited by examiner]
US 20140274564A1 · Greenbaum · 2014 [cited by applicant]
US 20140364212A1 · Osman et al. · 2014 [cited by applicant]
US 20150049113A1 · Rahman et al. · 2015 [cited by applicant]
US 20150094142A1 · Stafford · 2015 [cited by examiner]
US 20150096387A1 · Berme et al. · 2015 [cited by applicant]
US 20150269780A1 · Herman et al. · 2015 [cited by applicant]
US 20150348327A1 · Zalewski · 2015 [cited by examiner]
US 20160054837A1 · Stafford · 2016 [cited by examiner]
US 20160245711A1 · Berme et al. · 2016 [cited by applicant]
US 20160334288A1 · Berme et al. · 2016 [cited by applicant]
US 20170263056A1 · Leppanen · 2017 [cited by examiner]
US 20180024015A1 · Berme et al. · 2018 [cited by applicant]
US 20180028896A1 · Ray · 2018 [cited by examiner]
US 20180356718A1 · Nashida · 2018 [cited by examiner]
US 20190078951A1 · Berme et al. · 2019 [cited by applicant]
US 20200139229A1 · Berme et al. · 2020 [cited by applicant]
US 20200408625A1 · Berme et al. · 2020 [cited by applicant]
US 20210154529A1 · Barr · 2021 [cited by examiner]
US 20210333163A1 · Berme et al. · 2021 [cited by applicant]
US 20220178775A1 · Berme et al. · 2022 [cited by applicant]
US 20220233938A1 · Morin · 2022 [cited by examiner]
US 20220384027A1 · Kaleal, III · 2022 [cited by examiner]
US 20240202834A1 · Crabtree · 2024 [cited by examiner]
CN 201880270U · 2011 [cited by applicant]
First office action on the merits (Non-Final Rejection) in U.S. Appl. No. 15/242,558, mailed on Oct. 24, 2016. [cited by applicant]
Second office action on the merits (Final Rejection) in U.S. Appl. No. 15/242,558, mailed on Feb. 24, 2017. [cited by applicant]
Third office action on the merits (Non-Final Rejection) in U.S. Appl. No. 15/242,558, mailed on May 26, 2017. [cited by applicant]
Fourth office action on the merits (Final Rejection) in U.S. Appl. No. 15/242,558, mailed on Aug. 15, 2017. [cited by applicant]
Notice of Allowance in U.S. Appl. No. 15/242,558, mailed on Oct. 30, 2017. [cited by applicant]
First office action on the merits (Non-Final Rejection) in U.S. Appl. No. 15/908,323, mailed on Apr. 12, 2018. [cited by applicant]
Notice of Allowance in U.S. Appl. No. 15/908,323, mailed on Oct. 2, 2018. [cited by applicant]
Notice of Allowance in U.S. Appl. No. 16/283,685, mailed on Apr. 15, 2019. [cited by applicant]
Notice of Allowance in U.S. Appl. No. 16/550,219, mailed on Oct. 1, 2019. [cited by applicant]
Notice of Allowance in U.S. Appl. No. 16/785,960, mailed on Aug. 3, 2020. [cited by applicant]
First office action on the merits (Non-Final Rejection) in U.S. Appl. No. 17/114,375, mailed on Feb. 1, 2021. [cited by applicant]
Second office action on the merits (Final Rejection) in U.S. Appl. No. 17/114,375, mailed on Apr. 22, 2021. [cited by applicant]
Third office action on the merits (Non-Final Rejection) in U.S. Appl. No. 17/114,375, mailed on Aug. 19, 2021. [cited by applicant]
Notice of Allowance in U.S. Appl. No. 17/114,375, mailed on Dec. 2, 2021. [cited by applicant]
X. Zhou, M. Zhu, K. Derpanis, and K. Daniilidis. Sparseness meets deepness: 3D human pose estimation from monocular video. In IEEE Conference on Computer Vision and Pattern Recognition. (Apr. 2016) pp. 1-10. [cited by applicant]
First office action on the merits (Non-Final Rejection) in U.S. Appl. No. 17/718,290, mailed on Aug. 31, 2022. [cited by applicant]
Second office action on the merits (Final Rejection) in U.S. Appl. No. 17/718,290, mailed on Dec. 23, 2022. [cited by applicant]
First office action on the merits (Non-Final Rejection) in Appl. No. 18/138,112, mailed on Aug. 17, 2023. [cited by applicant]
Notice of Allowance in Appl. No. 18/138,112, mailed on Jan. 22, 2024. [cited by applicant]
Cited By (3)
US 12,494,085 US 12,535,679 US 12,725,366