Systems and methods for eyelid localization
Systems and methods for localizing an upper eyelid in an image of a subject are provided. An image of an eye of the subject is obtained in electronic format. The image is inputted into a trained neural network comprising at least 10,000 parameters, thereby obtaining a set of coordinates for an upper eyelid in the image. This obtaining and inputting can be repeated over the course of a non-zero duration thereby obtaining a corresponding set of coordinates for the upper eyelid in each image in a plurality of images. Each corresponding set of coordinates for the upper eyelid from each image in the plurality of images can be used to determine whether the subject is afflicted with a neurological condition.
1 . A method for localizing an upper eyelid in an image of a subject, the method comprising:
(a) obtaining the image of an eye of the subject in electronic format;
(b) inputting the image into a trained neural network comprising at least 10,000 parameters, thereby obtaining a set of coordinates for an upper eyelid in the image;
(c) repeating the obtaining (a) and inputting (b) over the course of a non-zero duration thereby obtaining a corresponding set of coordinates for the upper eyelid in each image in a plurality of images; and
(d) using each corresponding set of coordinates for the upper eyelid from each image in the plurality of images to determine whether the subject is afflicted with a neurological condition.
2 . The method of claim 1 , wherein the neurological condition is a result of a traumatic event, a head impact, or a mild traumatic brain injury.
3 . The method of claim 1 , wherein the plurality of images is taken upon stimulation of at least one facial region of the subject using at least one stimulator so as to cause an involuntary blink response in the subject and wherein each image in the plurality of images is an image of the involuntary blink response.
4 . The method of claim 3 , wherein the at least one facial region is selected from the group consisting of the temple, the outer canthus, and the eye.
5 . The method of claim 3 , wherein the stimulator is provided in proximity of the left eye, the right eye, or both.
6 . The method of claim 1 , wherein the plurality of images is taken for the left eye of the subject, the right eye of the subject, or both.
7 . The method of claim 3 , wherein the at least one stimulator is selected from the group consisting of a puff of fluid, a mechanical contact, one or more flashes of light, an electrical current, and a sound.
8 . The method of claim 3 , further comprising using each corresponding set of coordinates for the upper eyelid in the plurality of images to obtain a first measurement of a characteristic of the involuntary blink response in the subject.
9 . The method of claim 8 , wherein the characteristic of the involuntary blink response is selected from the group consisting of individual latency, differential latency, number of oscillations, change in tonic lid position, horizontal lid velocity, vertical lid velocity, time to close, time to open, total blink time, and time under threshold.
10 . The method of claim 9 , further comprising using each corresponding set of coordinates for the upper eyelid in the plurality of images to obtain a second measurement of the characteristic of the involuntary blink response for the subject, wherein the second measurement represents a baseline condition for the subject;
comparing the first measurement and the second measurement; and
when the difference between the first measurement and the second measurement exceeds a predetermined threshold value, determining that the subject is afflicted with a neurological condition.
11 . The method of claim 1 , wherein the trained neural network comprises:
a plurality of convolutional layers, wherein each convolutional layer in the plurality of convolutional layers comprises one or more filters, a respective size, and a respective stride; and
one or more pooling layers, wherein each pooling layer in the one or more pooling layers comprises a respective size and a respective stride.
12 . The method of claim 1 , wherein the trained neural network is LeNet, AlexNet, VGGNet 16 , GoogLeNet, ResNet, SE-ResNeXt, MobileNet, or EfficientNet.
13 . The method of claim 1 , further comprising:
obtaining a corresponding set of coordinates for a lower eyelid in each image in the plurality of images, thereby localizing a lower eyelid in each image in the plurality of images; and
using each corresponding set of coordinates for both the upper eyelid and the lower eyelid in the plurality of images to determine whether the subject is afflicted with the neurological condition.
14 . The method of claim 1 , wherein the image of the eye of the subject comprises a corresponding plurality of pixels and one or more pixel values for each pixel in the corresponding plurality of pixels.
15 . The method of claim 14 , wherein an edge length, in pixels, of the image consists of between 164 pixels and 1024 pixels.
16 . The method of claim 14 , wherein
the trained neural network comprises an initial convolutional neural network layer that receives a grey-scaled pixel value for each pixel in the corresponding plurality of pixels as input into the neural network,
the initial convolutional neural network layer includes a first activation function, and
the initial convolutional neural network layer convolves the corresponding plurality of pixels into more than 10 separate parameters for each pixel in the plurality of pixels.
17 . The method of claim 16 , wherein the trained neural network further comprises a pooling layer that pools the more than 10 separate parameters for each pixel in the plurality of pixels outputted by the initial convolutional neural network layer.
18 . The method of claim 16 , wherein the initial convolutional neural network layer has a stride of two or more.
19 . The method of claim 17 , wherein the trained neural network further comprises a plurality of intermediate blocks including a first intermediate block and a final intermediate block, wherein
the first intermediate block takes as input the output of the pooling layer;
each intermediate block in the plurality of intermediate blocks other than the first intermediate block and the final intermediate block takes, as input, an output of another intermediate block in the plurality of intermediate blocks and has an output that serves as input to another intermediate block in the plurality of intermediate blocks, and wherein each intermediate block comprises a respective first convolutional layer comprising more than 1000 parameters, wherein the respective convolutional layer has a corresponding activation function.
20 . The method of claim 19 , wherein each intermediate block in the plurality of intermediate blocks comprises a corresponding second convolutional layer that takes, as input, an output of the respective first convolutional layer.
21 . The method of claim 20 , wherein each intermediate block in the plurality of intermediate blocks comprises a merge layer that merges (i) an output of the respective second convolutional layer and (ii) an output of a preceding intermediate block in the plurality of intermediate blocks.
22 . The method of claim 21 , wherein:
each intermediate block in the plurality of intermediate blocks has a corresponding input size and a corresponding output size, and,
when the corresponding input size of a respective intermediate block differs from the corresponding output size, the respective intermediate block further comprises a corresponding third convolutional layer that receives, as input, the (ii) output of the preceding intermediate block, wherein the corresponding third convolutional layer convolves the (ii) output of the preceding intermediate block prior to the merging (i) and (ii) by the merge layer.
23 . The method of claim 19 , wherein the final intermediate block takes, as input, an output of another intermediate block in the plurality of intermediate blocks and produces, as output, a flattened data structure comprising a predetermined plurality of values.
24 . The method of claim 23 , wherein the neural network further comprises a regressor block including a first dropout layer, a first linear layer, and a corresponding activation function, wherein the regressor block takes, as input, the flattened data structure comprising the predetermined plurality of values.
25 . The method of claim 24 , wherein the first dropout layer removes a first subset of values from the plurality of values in the flattened data structure, based on a first dropout rate.
26 . The method of claim 25 , wherein the first linear layer applies a first linear transformation to the plurality of values in the flattened data structure.
27 . The method of claim 26 , wherein the regressor block further includes a second dropout layer, wherein the second dropout layer removes a second subset of values from the plurality of values in the flattened data structure, based on a second dropout rate.
28 . The method of claim 26 , wherein the regressor block further includes a second linear layer, wherein the second linear layer applies a second linear transformation to the plurality of values in the flattened data structure.
29 . The method of claim 16 , wherein the first activation function is tanh, sigmoid, softmax, Gaussian, Boltzmann-weighted averaging, absolute value, linear, rectified linear unit (ReLU), bounded rectified linear, soft rectified linear, parameterized rectified linear, average, max, min, sign, square, square root, multiquadric, inverse quadratic, inverse multiquadric, polyharmonic spline, swish, mish, Gaussian error linear unit (GeLU), or thin plate spline.
30 . A computing system, comprising:
one or more processors;
memory storing one or more programs to be executed by the one or more processors, the one or more programs comprising instructions for localizing an upper eyelid in an image of a subject by a method comprising:
(a) obtaining the image of an eye of the subject in electronic format;
(b) inputting the image into a trained neural network comprising at least 10,000 parameters, thereby obtaining a set of coordinates for an upper eyelid in the image;
(c) repeating the obtaining (a) and inputting (b) over the course of a non-zero duration thereby obtaining a corresponding set of coordinates for the upper eyelid in each image in a plurality of images; and
(d) using each corresponding set of coordinates for the upper eyelid from each image in the plurality of images to determine whether the subject is afflicted with a neurological condition.
31 . A non-transitory computer readable storage medium storing one or more programs for localizing an upper eyelid in an image of a subject, the one or more programs configured for execution by a computer, wherein the one or more programs comprise instructions for:
(a) obtaining the image of an eye of the subject in electronic format;
(b) inputting the image into a trained neural network comprising at least 10,000 parameters, thereby obtaining a set of coordinates for an upper eyelid in the image;
(c) repeating the obtaining (a) and inputting (b) over the course of a non-zero duration thereby obtaining a corresponding set of coordinates for the upper eyelid in each image in a plurality of images; and
(d) using each corresponding set of coordinates for the upper eyelid from each image in the plurality of images to determine whether the subject is afflicted with a neurological condition.