Determination of spin rate and spin axis of a ball in flight
Embodiments are disclosed for determining the spin rate and the spin axis of a ball in flight. In some embodiments, a method comprises: capturing, with an image sensor, a time series of images of a ball in flight and corresponding image capture times; inputting, with at least one processor, the image frames and the image capture times into a machine learning model; and predicting, with the at least one processor, a spin axis and a spin rate of the ball based on the machine learning model.
1 . A method comprising:
capturing, with an image sensor, a time series of images of a ball in flight and corresponding image capture times;
inputting, with at least one processor, the image frames and the image capture times into a machine learning model;
predicting, with the at least one processor, a spin axis and a spin rate of the ball based on the machine learning model;
wherein the machine learning model includes:
a first neural network trained to predict seam norms of the ball in 3D space based on a training set of ball images;
a second neural network trained to predict the spin rate of the ball based on the predicted seam norms and time differences obtained from the training set of ball images; and
a third neural network trained to predict the spin axis of the ball based on the predicted seam norms and ratios of the time differences.
2 . The method of claim 1 , wherein the training set of ball images include synthetic ball images of different types of balls, and wherein the ball images have non-uniform lighting, complex backgrounds, occlusions, or artifacts.
3 . The method of claim 1 , wherein the training set of ball images include augmented ball images of different types of balls, and wherein the ball images have at least one of random color, random contrast, random brightness, random sharpness, random shift or random noise level.
4 . The method of claim 1 , wherein randomly generated synthetic seam norms are used to train the first neural network.
5 . The method of claim 1 , wherein the first, second and third neural networks are trained using batch processing, stochastic gradient descent, Nesterov momentum without dampening, a weight decay or a cosine-shaped learning rate.
6 . The method of claim 5 , wherein the first neural network comprises M blocks, where each block includes a transition unit to downsample a feature size, followed by N residual units, where N and M are integer values greater than zero.
7 . The method of claim 5 , wherein the second and third neural networks each include M blocks, each block having a fully connected layer to increase a number of inputs, followed by N residual units, where each residual unit includes a first fully connected layer, followed by a first activation function, followed by a second fully connected layer, followed by a second activation function, where N and M are integer values greater than zero.
8 . The method of claim 1 , wherein the first, second and third neural networks are regression neural networks.
9 . The method of claim 1 , wherein the ball is a cricket ball.
10 . A system comprising:
at least one camera;
at least one processor;
memory storing instructions that when executed by the at least one processor, cause the at least one processor to perform operations comprising:
capturing, with the at least one camera, a time series of images of a ball in-flight and corresponding image capture times;
inputting the image frames and image capture times into a machine learning model;
predicting, with the machine learning model, a spin axis and a spin rate of the ball;
wherein the machine learning model includes:
a first neural network trained to predict seam norms of the ball in 3D space based on a training set of ball images;
a second neural network trained to predict the spin rate of the ball based on the predicted seam norms and time differences obtained from the training set of ball images; and
a third neural network trained to predict the spin axis of the ball based on the predicted seam norms and ratios of the time differences.
11 . The system of claim 10 , wherein the training set of ball images include synthetic ball images of different types of balls, and wherein the ball images have non-uniform lighting, complex backgrounds, occlusions, or artifacts.
12 . The system of claim 10 , wherein the training set of ball images include augmented ball images of different types of balls, and wherein the ball images have at least one of random color, random contrast, random brightness, random sharpness, random shift or random noise level.
13 . The system of claim 10 , wherein randomly generated synthetic seam norms are used to train the first neural network.
14 . The system of claim 10 , wherein the first, second and third neural networks are trained using batch processing, stochastic gradient descent, Nesterov momentum without dampening, a weight decay or a cosine-shaped learning rate.
15 . The system of claim 10 , wherein the first, second and third neural networks are regression neural networks.
16 . The system of claim 15 , wherein the first neural network comprises M blocks, where each block includes a transition unit to down sample a feature size, followed by N residual units, where M and N are integer values greater than zero.
17 . The system of claim 15 , wherein the second and third neural networks each include M blocks, each block having one fully connected layer to increase a number of inputs, followed by N residual units, where each residual unit includes a first fully connected layer, followed by a first activation function, followed by a second fully connected layer, followed by a second activation function, where M and N are integer values greater than zero.
18 . The system of claim 10 , wherein the ball is a cricket ball.
19 . An apparatus comprising:
at least one camera;
at least one processor;
memory storing instructions that when executed by the at least one processor, cause the at least one processor to perform operations comprising:
capturing, with the at least one camera, a time series of images of a ball in-flight and corresponding image capture times;
inputting the image frames and image capture times into a machine learning model; and
predicting, with the machine learning model, a spin axis and a spin rate of the ball;
wherein the machine learning model includes:
a first neural network trained to predict seam norms of the ball in 3D space based on a training set of ball images;
a second neural network trained to predict the spin rate of the ball based on the predicted seam norms and time differences obtained from the training set of ball images; and
a third neural network trained to predict the spin axis of the ball based on the predicted seam norms and ratios of the time differences.
20 . The apparatus of claim 19 , wherein the training set of ball images include synthetic ball images of different types of balls, and wherein the ball images have non-uniform lighting, complex backgrounds, occlusions, or artifacts.