Granted Patent B2

US 9,361,522 · App. 14/792,543 · Granted Jun 7, 2016

Inventors: Michael Bentley (San Diego, CA); Ryan Kaps (San Diego, CA); Bhaskar Bose (San Diego, CA); Sheehan Alam (San Diego, CA); Michael Gillen (San Diego, CA); Mazen Abdel-Rahman (San Diego, CA)

Assignee: BLAST MOTION INC.

G06K9/00744A63F13/00G06F17/3079G06K9/00342G06K9/00711G06K9/00758G06T7/20G11B27/022G11B27/17G11B31/006H04N7/18A63F2300/105A63F2300/69G06K2009/00738G06T2207/30221G08B21/0476

Abstract

Claims (135)

1. A motion event recognition and video synchronization system comprising:

at least one motion capture element configured to couple with a user or piece of equipment or mobile device coupled with the user, wherein said at least one motion capture element comprises

a sensor configured to capture one or more values associated with an orientation, position, velocity and acceleration of said at least one motion capture element;

a first communication interface;

a microcontroller coupled with said memory, said sensor and said first communication interface, wherein said microcontroller is configured to

collect data that comprises sensor values from said sensor;

optionally analyze said data and recognize an event within said data to determine event data;

transmit said data or said event data associated with said event or both said data and said event data via said first communication interface;

said microcontroller configured to transmit said data or said event data to a computer comprising

a memory;

a second communication interface configured to

communicate with said first communication interface to obtain said data or said event data associated with said event or both said data and said event data;

wherein said computer is coupled with said memory and said second communication interface, wherein said computer is configured to

receive

said data from said second communication interface and

analyze said data and recognize an event within said data to determine event data

 or

said event data from said second communication interface

 or

both said data and said event data from said second communication interface;

analyze said event data to form motion analysis data;

store said event data, or said motion analysis data, or both said event data and said motion analysis data in said memory;

obtain an event start time and an event stop time from said event data;

request image data captured with a camera comprising a video captured during a timespan from said event start time to said event stop time;

display an event video on a display comprising both of

said event data, said motion analysis data or any combination thereof that occurs during said timespan from said event start time to said event stop time,

 and

said video captured during said timespan from said event start time to said event stop time; and,

wherein said computer is further configured to command said camera to discard at least a portion of said video outside of said event start time to said event stop time and store said event video without said portion of said video outside of said event start time to said event stop time.

2. The system of claim 1 wherein said computer is further configured to determine a clock difference between said at least one motion capture element and said computer and synchronize said motion analysis data with said video.

3. The system of claim 1 wherein said computer is further configured to command said camera to save said video from said event start time to said event stop time with said motion analysis data that occurs from said event start time to said event stop time without said portion of said video outside of said event start time to said event stop time to a server computer external to said computer.

4. The system of claim 1 wherein said system further comprises a server computer remote to said computer and wherein said server computer is configured to save said video from said event start time to said event stop time with said motion analysis data that occurs from said event start time to said event stop time and return said video captured during said timespan from said event start time to said event stop time to said computer in said computer.

5. The system of claim 1 wherein said computer is further configured while a communication link is open between said at least one motion capture element and said computer to command said camera to discard at least a portion of said video outside of said event start time to said event stop time, and to save said video from said event start time to said event stop time with said motion analysis data that occurs from said event start time to said event stop time.

6. The system of claim 1 wherein said computer is further configured while a communication link is not open between said at least one motion capture element and said computer to save said video when said camera is coupled with said computer, and after said event is received after said communication link is open, to then command said camera to discard at least a portion of said video outside of said event start time to said event stop time and to save said video from said event start time to said event stop time with said motion analysis data that occurs from said event start time to said event stop time.

7. The system of claim 1 wherein said video is obtained from a camera coupled with said computer or remote from said computer.

8. The system of claim 1 wherein said video is obtained from a server computer remote to said computer.

9. The system of claim 1 wherein said at least one motion capture element is further configured to be worn near the user's head and wherein said recognize said event within said data comprises calculation of a location of impact on the user's head.

10. The system of claim 1 wherein said at least one motion capture element is further configured to couple with a helmet or hat or cap or mouthpiece.

11. The system of claim 1 wherein said at least one motion capture element is further configured to couple with a helmet and wherein said recognize said event within said data comprises calculation of the location of impact on the user's head based on the physical geometry of the helmet.

12. The system of claim 1 further comprising an isolator configured to surround said at least one motion capture element to approximate physical acceleration dampening of cerebrospinal fluid around said user's brain to minimize translation of linear acceleration and rotational acceleration of said event data to obtain an observed linear acceleration and an observed rotational acceleration of the user's brain.

13. The system of claim 1 wherein said computer is further configured to synchronize said video and said event data, or said motion analysis data, via image analysis to more accurately determine a start event frame or stop event frame in said video or both, that is most closely associated with said event start time or said event stop time or both.

14. The system of claim 1 wherein said system further comprises a server computer, wherein said server computer is configured to synchronize said video and said event data, or said motion analysis data, via image analysis to more accurately determine a start event frame or stop event frame in said video or both, that is most closely associated with said event start time or said event stop time or both.

15. The system of claim 1 wherein said computer is further configured to

access previously stored event data or motion analysis data associated with said user or piece of equipment;

wherein said display said event video further comprises a presentation of said event data associated with said user on a display based on

said event data or motion analysis data associated with said user or piece of equipment

and

said previously stored event data or motion analysis data associated with said user or piece of equipment.

16. The system of claim 1 wherein said computer is further configured to

access previously stored event data or motion analysis data associated with said user or piece of equipment;

wherein said display said event video further comprises a presentation of said event data associated with said user on a display based on

said event data or motion analysis data associated with said user or piece of equipment

and

said previously stored event data or motion analysis data associated with at least one other user or other piece of equipment.

17. The system of claim 1 wherein said microcontroller in said at least one motion capture element is further configured to transmit said event to at least one other motion capture element or at least one other computer or any combination thereof, and wherein said at least one other motion capture element or said at least one other computer or any combination thereof is configured to save data associated with said event based on said event start time and said event stop time.

18. The system of claim 1 further comprising:

an identifier coupled with said at least one motion capture element or said user or said piece of equipment;

wherein said computer is further configured to

receive said identifier; and

associate said identifier with said event data and said motion analysis data.

19. The system of claim 1 wherein said at least one motion capture element further comprises a light emitting element configured to output light if said event occurs.

20. The system of claim 1 wherein said at least one motion capture element further comprises an audio output element configured to output sound if said event occurs or if said at least one motion capture element is out of range of said computer or wherein said computer is configured to display and alert if said at least one motion capture element is out of range of said computer, or any combination thereof.

21. The system of claim 1 wherein said at least one motion capture element further comprises a location determination element configured to determine a location that is coupled with said microcontroller and wherein said microcontroller is further configured to transmit said location to said computer, or wherein said system further comprises a server computer and wherein said microcontroller is further configured to transmit said location to said server computer and wherein said computer or server computer is further configured to form at least one event video from portions of at least one video based on said location and said event start time and said event stop time.

22. The system of claim 1 wherein said microcontroller or said computer is further configured to determine a location of said event or wherein said microcontroller and said computer are further configured to determine the location of said event and correlate the location.

23. The system of claim 1 wherein said computer is further configured to request at least one image or video that contains said event from at least one camera proximal to said event or from a server computer.

24. The system of claim 1 wherein said computer is further configured to broadcast a request for camera locations proximal to said event or oriented to view said event.

25. The system of claim 1 wherein said computer is further configured to display a list of one or more times at which said event has occurred or wherein one or more events has occurred.

26. The system of claim 1 wherein said at least one motion capture element is physically coupled with said computer.

27. The system of claim 1 wherein the microcontroller is coupled with a temperature sensor and wherein said microcontroller is further configured to transmit a temperature obtained from the temperature sensor as a temperature event.

28. The system of claim 1 wherein said event data comprises

motion data associated with said at least one motion capture element coupled with any combination of said user, or said piece of equipment or said mobile device or said computer

or

motion data indicative of standing, walking, falling, heat stroke, a seizure, violent shaking, a concussion, a collision, abnormal gait, abnormal or non-existent breathing

or

any combination thereof.

29. A motion event recognition and video synchronization system comprising:

at least one motion capture element configured to couple with a user or piece of equipment or mobile device coupled with the user, wherein said at least one motion capture element comprises

a sensor configured to capture one or more values associated with an orientation, position, velocity and acceleration of said at least one motion capture element;

a first communication interface;

a microcontroller coupled with said memory, said sensor and said first communication interface, wherein said microcontroller is configured to

collect data that comprises sensor values from said sensor;

optionally analyze said data and recognize an event within said data to determine event data;

transmit said data or said event data associated with said event or both said data and said event data via said first communication interface;

said microcontroller configured to transmit said data or said event data to a computer comprising

a memory;

a second communication interface configured to

communicate with said first communication interface to obtain said data or said event data associated with said event or both said data and said event data;

wherein said computer is coupled with said memory and said second communication interface, wherein said computer is configured to

receive

said data from said second communication interface and

analyze said data and recognize an event within said data to determine event data

 or

said event data from said second communication interface

 or

both said data and said event data from said second communication interface;

analyze said event data to form motion analysis data;

store said event data, or said motion analysis data, or both said event data and said motion analysis data;

obtain an event start time and an event stop time from said event data;

request image data captured with a camera comprising a video captured during a timespan from said event start time to said event stop time;

display an event video on a display comprising both of

said event data, said motion analysis data or any combination thereof that occurs during said timespan from said event start time to said event stop time,

 and

said video captured during said timespan from said event start time to said event stop time; and,

a server computer remote to said computer, wherein said server computer is configured to obtain said event video without said portion of said video outside of said event start time to said event stop time without at least a portion of said video outside of said event start time to said event stop, and

return said video captured during said timespan from said event start time to said event stop time to another computer.

30. A motion event recognition and video synchronization system comprising:

at least one motion capture element configured to couple with a user or piece of equipment or mobile device coupled with the user, wherein said at least one motion capture element comprises

a sensor configured to capture one or more values associated with an orientation, position, velocity and acceleration of said at least one motion capture element;

a first communication interface;

a microcontroller coupled with said memory, said sensor and said first communication interface, wherein said microcontroller is configured to

collect data that comprises sensor values from said sensor;

optionally analyze said data and recognize an event within said data to determine event data;

transmit said data or said event data associated with said event or both said data and said event data via said first communication interface;

said microcontroller configured to transmit said data or said event data to a computer comprising

a second communication interface configured to

communicate with said first communication interface to obtain said data or said event data associated with said event or both said data and said event data;

wherein said computer is coupled with said second communication interface, wherein said computer is configured to

receive

said data from said second communication interface and

analyze said data and recognize an event within said data to determine event data

 or

said event data from said second communication interface

 or

both said data and said event data from said second communication interface;

analyze said event data to form motion analysis data;

store said event data, or said motion analysis data, or both said event data and said motion analysis data;

obtain an event start time and an event stop time from said event data;

request image data captured with a camera comprising a video captured during a timespan from said event start time to said event stop time;

display an event video on a display comprising both of

said event data, said motion analysis data or any combination thereof that occurs during said timespan from said event start time to said event stop time,

 and

said video captured during said timespan from said event start time to said event stop time; and

while a communication link is open between said at least one motion capture element and said computer, to command said camera to discard at least a portion of said video outside of said event start time to said event stop, and to save said video from said event start time to said event stop time with said motion analysis data that occurs from said event start time to said event stop time

or

while a communication link is not open between said at least one motion capture element and said computer, to save said video when said camera is coupled with said computer, and after said event is received after said communication link is open, to then command said camera to discard at least a portion of said video outside of said event start time to said event stop time and to save said video from said event start time to said event stop time with said motion analysis data that occurs from said event start time to said event stop time.

Assignments (10)

Continuity (11)