User input utilizing dual line scanner apparatus and method
View Patent ↗A user input method and apparatus may comprise a two line object imaging sensor having a primary line scan-sensor providing a primary line scan-sensor output and a secondary line scan-sensor providing a secondary line scan-sensor output, representing pixels in a current primary scan row and a current secondary scan row, and adapted to scan an object; storing for each scan time each current primary line scan-sensor output and each current secondary line scan-sensor output and a correlation unit correlating at least one of the current representations of pixels in a primary line sensor output with stored representations and the current representations of pixels in a secondary line sensor output with stored representations and, the correlation unit providing as an output a motion indicator.
1. A user input apparatus comprising:
a two line object imaging sensor having a primary line scan-sensor providing a primary line scan-sensor output and a secondary line scan-sensor providing a secondary line scan-sensor output, each representing, respectively, the pixels in a current primary scan row of pixels and a current secondary scan row of pixels, the primary line scan-sensor and the secondary line scan-sensor each adapted to scan an object;
a storage apparatus storing a selected portion of a current primary line scan-sensor output to maintain a plurality of stored representations of the current primary line scan-sensor output and a plurality of past primary line scan-sensor outputs and storing a selected portion of a current secondary line scan-sensor output to maintain a plurality of stored representations of the current secondary line scan-sensor output and a plurality of past secondary line scan-sensor outputs;
a correlation unit configured to correlate at least one of the representation of the current primary line scan-sensor output with respective ones of the plurality of representations of past secondary line scan-sensor outputs and the representation of the current secondary line scan-sensor output with respective ones of the plurality of representations of past primary line scan-sensor outputs to find a respective match; and,
the correlation unit configured to provide as an output a motion indicator comprising an amplitude of the motion of the object being scanned in a coordinate system coplanar with the primary and secondary line scan-sensors and aligned to the primary line scan sensor and the secondary line scan-sensor, based at least in part on at least one of (1) a temporal separation of the primary line scan-sensor output and the secondary line scan-sensor output found to match or (2a) a line count between the primary line scan-sensor output and the secondary line scan-sensor output found to match and (2b) a spatial separation between the primary line scan sensor and the secondary line scan-sensor.
2. The user input apparatus of claim 1 further comprising:
the correlation unit configured to correlate according to an individual pixel location matching algorithm at each corresponding pixel location for each of one or more pixel shifts between the one of the current representation of the primary line scan-sensor output and the respective one of the plurality of representations of the past secondary line scan-sensor outputs and the current representation of the secondary line scan-sensor output and the respective one of the plurality of representations of the past primary line scan-sensor outputs.
3. The user input apparatus of claim 1 further comprising:
the correlation unit configured to correlate according to a full line statistical analysis of each of the one of the current representation of the primary line scan-sensor output and the respective one of the plurality of representations of the past secondary line scan-sensor outputs and the current representation of the secondary line scan-sensor output and the respective one of the plurality of representations of the past primary line scan-sensor outputs, for each respective pixel shift.
4. The user input apparatus of claim 1 further comprising:
the full line statistical analysis comprising one of normalized cross correlation, mean-squared-error and sum-of-absolute-differences.
5. The user input apparatus of claim 2 further comprising:
the correlation unit configured to provide an output of a motion indicator comprising a direction of motion of the object being scanned based at least in part on the pixel shift necessary to find the match.
6. The user input apparatus of claim 1 further comprising:
the correlation unit configured to provide the output of a motion indicator comprising a direction of motion of the object being scanned based at least in part on whether the match is made with the representation of the current primary line scan-sensor output or the representation of the current secondary line scan-sensor output.
7. The user input apparatus of claim 2 further comprising:
the correlation unit comprising a logic circuit configured to:
receive a selected portion of the current primary line scan-sensor output selected to comprise a representation of the primary line scan-sensor output,
store a plurality of representations of the primary line scan-sensor output,
receive a selected portion of the current secondary line scan-sensor output selected to comprise a representation of the secondary line scan-sensor output,
store a plurality of representations of the primary line scan-sensor output,
compare the representation of the current primary line scan-sensor output with a respective one of the plurality of stored representations of the secondary line scan-sensor output, for each of a selected one or more pixel shifts between the representation of the current primary line scan sensor output and the respective one of the plurality of stored representations of the secondary line scan-sensor output, for a selected number of stored secondary line scan-sensor outputs, to determine if a match can be found,
compare the representation of the current secondary line scan-sensor output with a respective one of the plurality of stored representations of the primary line scan-sensor output, for each of a selected one or more pixel shifts between the representation of the current secondary line scan-sensor output and the respective one of the plurality of stored representations of the primary line scan-sensor output, for a selected number of stored primary line scan-sensor outputs, to determine if a match can be found,
identify the current primary line scan-sensor output and the respective one of the secondary line scan-sensor outputs or the current secondary line scan-sensor output and the respective one of the primary line scan-sensor outputs for which the match exists and the respective pixel shift resulting in the match.
8. The user input apparatus of claim 3 further comprising:
the correlation unit comprising a logic circuit configured to:
receive a selected portion of the current primary line scan-sensor output selected to comprise a representation of the primary line scan-sensor output,
store a plurality of representations of the primary line scan-sensor output,
receive a selected portion of the current secondary line scan-sensor output selected to comprise a representation of the secondary line scan-sensor output,
store a plurality of representations of the primary line scan-sensor output,
align the representation of the current primary line scan-sensor output with a respective one of the plurality of stored representations of the secondary line scan-sensor output, at a pixel shift location for the alignment, for each of a selected one or more pixel shifts between the representation of the current primary line scan-sensor output and the respective one of the plurality of stored representations of the secondary line scan-sensor output, for a selected number of stored representations of the secondary line scan-sensor output, and provide the alignments to a computing device configured to determine if a match can be found utilizing the full line statistical analysis; and
align the representation of the current secondary line scan-sensor output with a respective one of the plurality of stored representations of the primary line scan-sensor output, at a pixel shift location for the alignment, for each of a selected one or more pixel shifts between the representation of the current secondary line scan-sensor output and the respective one of the plurality of stored representations of the primary line scan-sensor output, for a selected number of representations of the stored primary line scan-sensor output, and provide the alignments to a computing device configured to determine if a match can be found utilizing the full line statistical analysis.
9. The user input apparatus of claim 1 further comprising:
the correlation unit configured to correlate at least one of the representation of the current primary line scan-sensor output and a stored representation of a primary line scan-sensor output or the current secondary line scan-sensor output and a store representation of a secondary line scan-sensor output;
the correlation unit configured to assign a correlation score to each correlation for the representation of the current primary line scan-sensor output and the respective ones of the representations of the stored secondary line scan-sensor output and to save the highest score as a CorrUP correlation score and to assign a correlation score to each correlation of the representation of current secondary line scan-sensor output and the representations of the stored representations of the stored primary line scan-sensor output and to save the highest score as a CorrDOWN score, and to assign a correlation score to each correlation of either the current representations of the current primary line scan-sensor output and a representation of a stored primary line scan-sensor output of the representations of the current secondary line scan-sensor output and a representation of a stored secondary line scan-sensor output, and to save the highest score as a CorrSIDE; and
computing device configured to determine a direction of motion of the object based on the correlation scores CorrUP, CorrDOWN and CorrSIDE.
10. A method of providing a user input comprising:
providing, via a two line object imaging sensor having a primary line scan-sensor a primary line scan-sensor output, and having a secondary line scan-sensor, a secondary line scan-sensor output, each representing, respectively, the pixels in a current primary scan row of pixels and a current secondary scan row of pixels, the primary line scan-sensor and the secondary line scan-sensor each adapted to scan an object;
storing in a storage apparatus a selected portion of a current primary line scan-sensor output to maintain a plurality of stored representations of the current primary line scan-sensor output and a plurality of past primary line scan-sensor outputs and a selected portion of a current secondary line scan-sensor output to maintain a plurality of stored representations of the current secondary line scan-sensor output and a plurality of past secondary line scan-sensor outputs;
correlating, via a correlation unit, at least one of the representation of the current primary line scan-sensor output with respective ones of the plurality of representations of past secondary line scan-sensor outputs and the representation of the current secondary line scan-sensor output with respective ones of the plurality of representations of past primary line scan-sensor outputs to find a respective match; and,
the correlation unit providing as an output a motion indicator comprising an amplitude of the motion of the object being scanned in a coordinate system coplanar with the primary and secondary line scan-sensors and aligned to the primary line scan-sensor and the secondary line scan-sensor, based at least in part on at least one of (1) a temporal separation of the primary line scan-sensor output and the secondary line scan-sensor output found to match or (2a) a line count between the primary line scan-sensor output and the secondary line scan-sensor output found to match and (2b) a spatial separation between the primary line scan sensor and the secondary line scan-sensor.
11. The method of claim 10 further comprising:
the correlation unit correlating according to an individual pixel location matching algorithm at each corresponding pixel location for each of one or more pixel shifts between the one of the current representation of the primary line scan-sensor output and the respective one of the plurality of representations of the past secondary line scan-sensor outputs and the current representation of the secondary line scan-sensor output and the respective one of the plurality of representations of the past primary line scan-sensor outputs.
12. The method of claim 10 further comprising:
the correlation unit correlating according to a full line statistical analysis of each of the one of the current representation of the primary line scan-sensor output and the respective one of the plurality of representations of the past secondary line scan-sensor outputs and the current representation of the secondary line scan-sensor output and the respective one of the plurality of representations of the past primary line scan-sensor outputs, for each respective pixel shift.
13. The method of claim 12 further comprising:
the full line statistical analysis comprising one of normalized cross correlation, mean-squared-error and sum-of-absolute-differences.
14. The method of claim 10 further comprising:
providing, via the correlation unit, an output of a motion indicator comprising a direction of motion of the object being scanned based at least in part on the pixel shift necessary to find the match.
15. The method of claim 12 further comprising:
providing, via the correlation unit, the output of a motion indicator comprising a direction of motion of the object being scanned based at least in part on whether the match is made with the representation of the current primary line scan-sensor output or the representation of the current secondary line scan-sensor output.
16. The method of claim 11 further comprising:
the correlation unit comprising a logic circuit configured to:
receive a selected portion of the current primary line scan-sensor output selected to comprise a representation of the primary line scan-sensor output,
store a plurality of representations of the primary line scan-sensor output,
receive a selected portion of the current secondary line scan-sensor output selected to comprise a representation of the secondary line scan-sensor output,
store a plurality of representations of the primary line scan-sensor output,
compare the representation of the current primary line scan-sensor output with a respective one of the plurality of stored representations of the secondary line scan-sensor output, for each of a selected one or more pixel shifts between the representation of the current primary line scan sensor output and the respective one of the plurality of stored representations of the secondary line scan-sensor output, for a selected number of stored secondary line scan-sensor outputs, to determine if a match can be found,
compare the representation of the current secondary line scan-sensor output with a respective one of the plurality of stored representations of the primary line scan-sensor output, for each of a selected one or more pixel shifts between the representation of the current secondary line scan-sensor output and the respective one of the plurality of stored representations of the primary line scan-sensor output, for a selected number of stored primary line scan-sensor outputs, to determine if a match can be found,
identify the current primary line scan-sensor output and the respective one of the secondary line scan-sensor outputs or the current secondary line scan-sensor output and the respective one of the primary line scan-sensor outputs for which the match exists and the respective pixel shift resulting in the match.
17. The method of 12 further comprising:
the correlation unit comprising a logic circuit configured to:
receive a selected portion of the current primary line scan-sensor output selected to comprise a representation of the primary line scan-sensor output,
store a plurality of representations of the primary line scan-sensor output,
receive a selected portion of the current secondary line scan-sensor output selected to comprise a representation of the secondary line scan-sensor output,
store a plurality of representations of the primary line scan-sensor output, and
align the representation of the current primary line scan-sensor output with a respective one of the plurality of stored representations of the secondary line scan-sensor output, at a pixel shift location for the alignment, for each of a selected one or more pixel shifts between the representation of the current primary line scan-sensor output and the respective one of the plurality of stored representations of the secondary line scan-sensor output, for a selected number of stored representations of the secondary line scan-sensor output, and provide the alignments to a computing device configured to determine if a match can be found utilizing the full line statistical analysis; and
align the representation of the current secondary line scan-sensor output with a respective one of the plurality of stored representations of the primary line scan-sensor output, at a pixel shift location for the alignment, for each of a selected one or more pixel shifts between the representation of the current secondary line scan-sensor output and the respective one of the plurality of stored representations of the primary line scan-sensor output, for a selected number of representations of the stored primary line scan-sensor output, and provide the alignments to a computing device configured to determine if a match can be found utilizing the full line statistical analysis.
18. The method of claim 10 further comprising:
the correlating at least one of the representation of the current primary line scan-sensor output and a stored representation of a primary line scan sensor-output or the current secondary line scan-sensor output and a store representation of a secondary line scan-sensor output;
assigning a correlation score to each correlation for the representation of the current primary line scan-sensor output and the respective ones of the representations of the stored secondary line scan-sensor output and saving the highest score as a CorrUP correlation score and assigning a correlation score to each correlation of the representation of current secondary line scan-sensor output and the representations of the stored primary line scan-sensor output and saving the highest score as a CorrDOWN score, and assigning a correlation score to each correlation of either the current representation of the current primary line scan-sensor output and a representation of a stored primary line scan-sensor output of the representations of the current secondary line scan-sensor output and a representation of a stored secondary line scan-sensor output, and to save the highest score as a CorrSIDE; and
a computing device configured to determine a direction of motion of the object based on the correlation scores DorrUP, CorrDOWN and CorrSIDE.
19. A non-transitory tangible machine readable medium storing instructions that, when executed by a computing device cause the computing device to perform a method, the method comprising:
receiving from a two line object imaging sensor having a primary line scan-sensor a primary line scan-sensor output, and having a secondary line scan-sensor, a secondary line scan-sensor output, each representing, respectively, the pixels in a current primary scan row of pixels and a current secondary scan row of pixels, the primary line scan-sensor and the secondary line scan-sensor each adapted to scan an object;
storing a selected portion of a current primary line scan-sensor output to maintain a plurality of stored representations of the current primary line scan-sensor output and a plurality of past primary line scan-sensor outputs and a selected portion of a current secondary line scan-sensor output to maintain a plurality of stored representations of the current secondary line scan-sensor output and a plurality of past secondary line scan-sensor outputs;
correlating at least one of the representation of the current primary line scan-sensor output with respective ones of the plurality of representations of the past secondary line scan-sensor outputs and the representation of the current secondary line scan-sensor output with respective ones of the plurality of representations of the past primary line scan-sensor output to find a respective match; and,
the correlation unit providing as an output a motion indicator comprising an amplitude of the motion of the object being scanned in a coordinate system coplanar with the primary and secondary line scan-sensors and aligned to the primary line scan-sensor and the secondary line scan-sensor, based at least in part on at least one of (1) a temporal separation of the primary line scan-sensor output and the secondary line scan-sensor output found to match or (2a) a line count between the primary line scan-sensor output and the secondary line scan-sensor output found to match and (2b) a spatial separation between the primary line scan sensor and the secondary line scan-sensor.
20. The non-transitory tangible machine readable medium of claim 19 , the method further comprising:
the correlation unit correlating according to a full line statistical analysis of each of the one of the current representation of the primary line scan-sensor output and the respective one of the plurality of representations of the past secondary line scan-sensor outputs and the current representation of the secondary line scan-sensor output and the respective one of the plurality of representations of the past primary line scan-sensor outputs, for each respective pixel shift.