Smart optical input/output (I/O) extension for context-dependent workflows
View Patent ↗Systems, methods, and computer program products for smart, automated capture of textual information using optical sensors of a mobile device are disclosed. The textual information is provided to a mobile application or workflow without requiring the user to manually enter or transfer the data without requiring user intervention such as a copy/paste operation. The capture and provision context-aware, and can normalize or validate the captured textual information prior to entry in the workflow or mobile application. Other information necessary by the workflow and available to the mobile device optical sensors may also be captured and provided, in a single automatic process. As a result, the overall process of capturing information from optical input using a mobile device is significantly simplified and improved in terms of accuracy of data transfer/entry, speed and efficiency of workflows, and user experience.
1. A computer-implemented method, comprising:
invoking a user input interface on a mobile device;
invoking an optical input extension of the user input interface;
capturing optical input via one or more optical sensors of the mobile device;
determining textual information from the captured optical input; based on analyzing the optical input, wherein the analyzing comprises:
determining a restricted optical character recognition (OCR) alphabet based on the optical input; and
performing OCR using the restricted OCR alphabet;
identifying desired textual information among the determined textual information based on the OCR;
selectively providing the desired textual information to the user input interface without relying on feedback from the user; and
wherein the user input interface is invoked in response to detecting a user interaction with a user interface element configured to receive textual information.
2. The method as recited in claim 1 , wherein the desired textual information comprises a plurality of identifiers, each identifier corresponding to one of a plurality of user interface elements configured to receive textual information, and
wherein the providing further comprises automatically and selectively providing each of the plurality of identifiers to the corresponding user interface element without relying on feedback from the user.
3. The method as recited in claim 2 , further comprising normalizing at least one of the identifiers to conform with one or more of an expected format of the textual information and an expected range of values for the textual information.
4. The method as recited in claim 3 , further comprising determining one or more of reference content from a complementary document and business rules applicable to at least one of the identifiers,
wherein the determining is based on the user interface element corresponding to the identifier(s), and
wherein the normalizing is based on one or more of the reference content and the business rules.
5. The method as recited in claim 1 , further comprising normalizing the desired textual information to conform with one or more of an expected format of the textual information and an expected range of values for the textual information, wherein the normalizing does not rely on feedback from the user.
6. The method as recited in claim 5 , wherein the normalizing is based on one or more of reference content from a complementary document and business rules.
7. The method as recited in claim 6 , further comprising determining one or more of the complementary document and the business rules based on the element with which the user interacted.
8. The method as recited in claim 1 , further comprising automatically invoking an optical input capture interface in response to detecting the invocation of the optical input extension;
wherein the user input interface comprises a virtual keyboard displayed on the mobile device, and
wherein the optical input extension comprises a camera button displayed on the virtual keyboard.
9. A method, comprising:
invoking a user input interface on a mobile device;
invoking an optical input extension of the user input interface;
pre-analyzing optical input prior to capturing the optical input, wherein the pre-analyzing comprises:
detecting an object depicted in the optical input;
determining one or more characteristics of the object depicted in the optical input; and
determining one or more analysis parameters based at least in part on the determined characteristic(s), wherein the one or more analysis parameters comprise a restricted optical character recognition (OCR) alphabet; and
capturing the optical input via one or more optical sensors of the mobile device;
analyzing the optical input to determine textual information, wherein the analyzing comprises:
performing OCR using the restricted OCR alphabet
identifying expected textual information among the determined textual information based on the OCR; and
selectively providing the expected textual information to the user input interface without relying on feedback from the user; and
wherein the user input interface is invoked in response to detecting a user interaction with a user interface element configured to receive textual information.
10. The method as recited in claim 9 , wherein the one or more analysis parameters comprise OCR parameters; and
wherein the one or more characteristics comprise the object depicting one or more of textual information having a non-standard color profile and a background having a non-standard color profile.
11. A computer program product comprising: a computer readable storage medium having program code embodied therewith, the program code readable/executable by a processor to:
invoke a user input interface on a mobile device;
invoke an optical input extension of the user input interface;
automatically pre-analyze optical input received via one or more optical sensors of the mobile device, the pre-analysis comprising;
determining a restricted optical character recognition (OCR) alphabet based on the optical input; and
performing OCR using the restricted OCR alphabet; and
identifying textual information based on the OCR;
capture optical input via the one or more optical sensors of the mobile device in response to identifying textual information in the pre-analysis;
determine expected textual information from the identified textual information; and
selectively provide the expected textual information to the user input interface without relying on feedback from the user; and
wherein the user input interface is invoked in response to detecting a user interaction with a user interface element configured to receive textual information.
12. The computer program product as recited in claim 11 , wherein the expected textual information is selectively provided to an appropriate element of the user input interface based on determining the expected textual information includes required textual information for the appropriate element of the user input interface.
13. The computer program product as recited in claim 11 , the program code being further readable/executable by the processor to: automatically normalize, without relying on feedback from a user, at least some of the expected textual information to conform with one or more of an expected format of the expected textual information and an expected range of values for the expected textual information.
14. The computer program product as recited in claim 13 , wherein the program code readable/executable by the processor to automatically normalize, without relying on feedback from the user, the expected textual information is further readable/executable by the processor to:
determine an appropriate formatting for the expected textual information based on one or more of reference content from a complementary document and business rules; and
modify a format of the expected textual information to match the appropriate formatting.
15. The method as recited in claim 9 , wherein the pre-analyzing is performed automatically without relying on input from a user.