Camera device and method for precluding capture of manual data entry in image data
A camera device and method are described that perform object detection on sensor data generated by an image sensor to detect an input device within an environment outside of the camera device. The camera device and method analyze the sensor data to detect a hand of a user within the environment, and determine an impending use of the input device by the user based on a proximity of the hand to the input device and/or movement of the hand. Responsive to determining the impending use, the camera device and method generate an image data stream that includes altered image content within an area of interest that encompasses in the input device. The altered image content is not an accurate representation of the hand manipulating the input device in the environment, thereby precluding capture of manual data entry in the image data stream.
1 . A camera device comprising:
an image sensor configured to capture light propagating through a lens from an environment outside of the camera device and generate sensor data representing the environment;
a memory configured to store program instructions; and
one or more processors operably connected to the memory, wherein the program instructions are executable by the one or more processors to:
perform object detection on the sensor data to detect an input device within the environment that is configured to be manually manipulated to provide an input signal;
analyze the sensor data to detect at least one hand of a user within the environment;
determine an impending use of the input device by the user based on at least one of a proximity of the hand to the input device or movement of the hand; and
responsive to determining the impending use, generate an image data stream that includes altered image content within an area of interest (AOI) of a field of the image data stream, wherein the AOI encompasses the input device, wherein the altered image content is not an accurate representation of the hand manipulating the input device in the environment, so the image data stream lacks capture of data entry by the user into the input device.
2 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to at least one of: (i) store the image data stream on the memory or (ii) communicate the image data stream to an external computing device.
3 . The camera device of claim 1 , wherein the one or more processors are configured to directly receive the sensor data from the image sensor.
4 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to generate the image data stream to include the altered image content and accurate image content within a remainder of the field of the image data stream, wherein the one or more processors are configured to generate the accurate image content by performing at least one of lighting balance or color correction on the sensor data.
5 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to determine one or more measured vectors based on the movement of the hand, the one or more processors configured to determine the impending use of the input device by the user by comparing the one or more measured vectors to stored vectors in a gesture database.
6 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to perform the object detection on the sensor data to detect at least one of a keyboard, a keypad, a touch screen, a touch pad, or a stylus.
7 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to receive the sensor data as a sequence of sensor data frames generated by the image sensor over time, and the program instructions are executable by the one or more processors to analyze multiple sensor data frames in the sequence to determine the impending use of the input device by the user based on at least one of the proximity of the hand to the input device or the movement of the hand.
8 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to determine the impending use of the input device based on both the proximity of the hand to the input device and the movement of the hand.
9 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to determine the impending use of the input device based also on an orientation of fingers of the hand depicted in the sensor data.
10 . The camera device of claim 1 , wherein the one or more processors include an image signal processor (ISP).
11 . The camera device of claim 1 , wherein the one or more processors include a machine learning (ML) algorithm that receives the sensor data as an input, wherein the ML algorithm is trained to detect the hand of the user within the environment and analyze the movement of the hand relative to the input device based on the input, wherein the ML algorithm is configured to output a value indicating a likelihood of the impending use of the input device by the user.
12 . The camera device of claim 1 , further comprising a housing that surrounds and contains the image sensor, the memory, and the one or more processors, wherein the lens is mounted to the housing.
13 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to generate the image data stream by blurring the AOI as the altered image content.
14 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to generate the image data stream by incorporating a still image in the AOI as the altered image content.
15 . The camera device of claim 14 , wherein the still image depicts at least one of: (i) the hand of the user, (ii) a stock hand of a generic user, or (iii) the input device.
16 . The camera device of claim 1 , wherein the program instructions are executable by the one or more processors to generate the image data stream by generating, as the altered image content in the AOI, an inaccurate reconstruction of the movement of the hand relative to the input device, wherein a position of one or more fingers of the hand relative to the input device in the inaccurate reconstruction differs from an actual position of the one or more fingers relative to the input device in the environment.
17 . The camera device of claim 1 , wherein, responsive to determining that use of the input device by the user is not impending, the program instructions are executable by the one or more processors to generate the image data stream based on the sensor data so that the image data stream lacks the altered image content.
18 . A method comprising:
receiving sensor data generated by an image sensor of a camera device and representing an environment outside of the camera device, the image sensor configured to generate the sensor data based on capturing light propagating through a lens of the camera device from the environment;
performing object detection on the sensor data to detect an input device within the environment that is configured to be manually manipulated to provide an input signal;
analyzing the sensor data to detect at least one hand of a user within the environment;
determining an impending use of the input device by the user based on at least one of a proximity of the hand to the input device or movement of the hand; and
responsive to determining the impending use, generating an image data stream that includes altered image content within an area of interest (AOI) of a field of the image data stream, wherein the AOI encompasses the input device, wherein the altered image content is not an accurate representation of the hand relative to the input device in the environment, so the image data stream lacks capture of data entry by the user into the input device.
19 . The method of claim 18 , further comprising at least one of: (i) storing the image data stream on a memory of the camera device or (ii) communicating the image data stream to an external computing device.
20 . A computer program product comprising a non-transitory computer readable storage medium, the non-transitory computer readable storage medium comprising computer executable code configured to be executed by one or more processors to:
receive sensor data generated by an image sensor of a camera device and representing an environment outside of the camera device, the image sensor configured to generate the sensor data based on capturing light propagating through a lens of the camera device from the environment;
perform object detection on the sensor data to detect an input device within the environment that is configured to be manually manipulated to provide an input signal;
analyze the sensor data to detect at least one hand of a user within the environment;
determine an impending use of the input device by the user based on at least one of a proximity of the hand to the input device or movement of the hand; and
responsive to determining the impending use, generate an image data stream that includes altered image content within an area of interest (AOI) of a field of the image data stream, wherein the AOI encompasses the input device, wherein the altered image content is not an accurate representation of the hand relative to the input device in the environment, so the image data stream lacks capture of data entry by the user into the input device.