IP Library Granted Patent US 8,218,020
Granted Patent B2
US 8,218,020 · App. 12/455,252 · Granted Jul 10, 2012

Providing camera-based services using a portable communication device

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,218,020
App. No.
12/455,252
Granted
Jul 10, 2012
Kind
B2
Abstract

Camera-based services are provided to a user of a portable communication device by recognizing text contained in an image. An image of an environment is captured using a camera within the portable communication device so as to obtain image data. The image data is processed such that text data is recognized and extracted from the image data. Data related to the text data is then output in a form recognizable by a user of the portable communication device. The text data can be processed on the portable communication device to obtain the data related to the text data. Alternatively, the processing is performed by a processing unit external to the portable communication device. Translated and audio versions of the text data are output to the user. One camera-based service provides price and product information related to a product described in an image captured by the camera.

Claims (36)

1. A method comprising:

capturing an image of an environment using a camera within a portable communication device so as to obtain image data;

processing the image data such that text data is recognized and extracted from the image data, wherein the processing of the image data is performed by an external processing unit external to the portable communication device, the external processing unit including a first server that communicates with the portable communication device via a first communication network, and the first server processing the image data,

the external processing unit including a second server that communicates with the first server via a second communication network, wherein the second server stores the text data, wherein the first server communicates with a personal computer, and wherein the personal computer accesses the text data on the second server via the first server; and

outputting data related to the text data in a form recognizable by a user of the portable communication device.

2. A method comprising:

capturing an image of an environment using a camera within a portable communication device so as to obtain image data;

processing the image data such that text data is recognized and extracted from the image data, wherein the processing of the image data is performed by an external processing unit external to the portable communication device, the external processing unit including a first server that communicates with the portable communication device via a first communication network, and the first server processing the image data;

performing block recognition, wherein the processing the image data is performed in parallel on image data within recognized blocks; and

outputting data related to the text data in a form recognizable by a user of the portable communication device.

3. A portable communication device, comprising:

a camera adapted to capture an image of an environment, wherein image data obtained from the image is processed to generate text data;

a display adapted to display an indication of a service selectable by a user of the portable communication device;

outputting means that outputs data related to the text data in a form recognizable by the user of the portable communication device, wherein the portable communication device performs the service selected by the user;

a first worker module that displays the image on the display and that detects an orientation of text included in the image;

a second worker module that performs text recognition on the image data to generate the text data; and

a third worker module that converts the text data into speech and generates an audio output, wherein the first, second and third worker modules work independently of one other and communicate with an operation system of the portable communication device via dedicated interfaces;

wherein the first, second and third worker modules are part of a client application, wherein the portable communication device communicates with a server, and wherein the first, second and third worker modules are configured dynamically through the server without reinstalling the client application on the portable communication device.

4. The portable communication device of claim 3 , wherein each of the first, second and third worker modules also runs on the server so as to achieve a load balancing.

5. The portable communication device of claim 3 , further comprising:

a service module that communicates with a server and requests performance of the service selected by the user.

6. A method comprising:

capturing an image of an environment using a camera within a portable communication device so as to obtain image data;

processing the image data such that text data is recognized and extracted from the image data, wherein the processing of the image data is performed by an external processing unit external to the portable communication device, the external processing unit including a first server that communicates with the portable communication device via a first communication network, and the first server processing the image data, the external processing unit further including a second server that communicates with the first server via a second communication network and that stores the image data;

the portable communication device accessing the image data in real-time from the second server; and

outputting data related to the text data in a form recognizable by a user of the portable communication device.

7. The method of claim 6 , wherein the second server stores the text data, wherein the first server communicates with a personal computer, and wherein the personal computer accesses the text data on the second server via the first server.

8. The method of claim 6 , further comprising:

performing block recognition, wherein the processing the image data is performed in parallel on image data within recognized blocks.

9. The method of claim 6 , wherein the image data are stored in association with position data indicating a position where the image was captured.

10. A method comprising:

capturing an image of an environment using a camera within a portable communication device so as to obtain image data;

performing block recognition on the image data by segmenting the obtained image data into blocks thereby obtaining block image data per block;

processing the block image data such that text data is recognized and extracted from the block image data,

wherein the processing the block image data is performed such that text data is recognized and extracted from the block image data in parallel of the respective block image data of the recognized blocks; and

outputting data related to the text data in a form recognizable by a user of the portable communication device.

Assignments (8)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 15, 2024
From: KOFAX, INC.
To: TUNGSTEN AUTOMATION CORPORATION
Reel/Frame 067428/0392 →
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2022
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: KAPOW TECHNOLOGIES, INC.; KOFAX, INC.
Reel/Frame 060805/0161 →
FIRST LIEN INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jul 20, 2022
From: KOFAX, INC.; PSIGEN SOFTWARE, INC.
To: JPMORGAN CHASE BANK, N.A. AS COLLATERAL AGENT
Reel/Frame 060757/0565 →
SECURITY INTEREST Recorded Jul 20, 2022
From: KOFAX, INC.; PSIGEN SOFTWARE, INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 060768/0159 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 4, 2019
From: NUANCE COMMUNICATIONS, INC.
To: KOFAX, INC.
Reel/Frame 048234/0789 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT SUPPLEMENT Recorded Feb 4, 2019
From: KOFAX, INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 048253/0614 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2013
From: BEYO GMBH
To: NUANCE COMMUNICATIONS, INC.
Reel/Frame 031730/0362 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 29, 2009
From: TENCHIO, OLIVER; GOEKTEKIN, CUENEYT
To: BEYO GMBH
Reel/Frame 022801/0837 →