IP Library Granted Patent US 8,370,755
Granted Patent B2
US 8,370,755 · App. 11/965,136 · Granted Feb 5, 2013

User interface controlled by environmental cues

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,370,755
App. No.
11/965,136
Filed
Dec 27, 2007
Granted
Feb 5, 2013
Kind
B2
Art Unit
2171
USPC
715/764
Abstract

A method comprises receiving a signal containing an environmental cue; correlating the environmental cue to a user interface control element; and generating a user interface control command based on the environmental cue. A device may be controlled by receiving a visual image, processing the image to identify a visual cue, correlating the visual cue with a user interface control, and controlling the device using the user interface control. The visual image may be received from a camera that is part of an automobile, and the user interface control may include a control for a radio to tune to a radio station; a speed control for controlling a speed of the automobile corresponding to a speed sign visual cue; or a navigation system control for controlling a navigation system based on traffic signs. Other variations include an apparatus and computer-readable media including instructions for performing the methods.

Claims (72)

1. A method comprising:

receiving a signal including an environmental cue;

correlating the environmental cue to an interface control element from a plurality of possible interface control elements, wherein the plurality of possible interface control elements includes a control for automatically operating at least one of a radio device, a telephone device, and a navigation device; and

generating an interface control command based on the environmental cue,

wherein when the received signal comprises a visual image and the environmental cue comprises an advertising element in a user-selected portion of the visual image, generating an interface control command comprises generating a command for the radio device.

2. The method of claim 1 , wherein receiving a signal including an environmental cue comprises receiving a visual image from a camera.

3. The method of claim 2 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating a traffic sign within the visual image to a vehicle speed control.

4. The method of claim 2 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating a traffic sign within the visual image to a navigation system control.

5. The method of claim 2 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating an advertising sign within the visual image to a radio station control.

6. The method of claim 1 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises receiving user input regarding the user-selected portion of the visual image corresponding to the received signal.

7. The method of claim 6 , further comprising processing the user-selected portion of the visual image to identify the environmental cue.

8. The method of claim 7 , further comprising identifying information in the visual image indicative of a speed limit, and wherein generating the interface control command comprises generating a speed control command for a vehicle.

9. The method of claim 1 , further comprising:

displaying a visual image comprising one or more environmental cues;

receiving user input selecting a portion of the displayed visual image;

displaying a border having a first dimension around the selected portion of the displayed image;

in response to additional user input, displaying a border smaller than the first dimension around the selected portion of the displayed image; and

upon further user input or after expiration of a predetermined time period, using a portion of the image within the smaller border as the basis for comparison with a database of potentially matching environmental cues.

10. The method of claim 7 , further comprising identifying a navigation element within the visual image, and wherein generating the interface control command comprises automatically generating a detour command for the navigation device when the environmental cue comprises a traffic alert.

11. The method of claim 7 , further comprising identifying the advertising element within the visual image.

12. The method of claim 1 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises matching a visual cue within a received image to one of a plurality of potentially matching visual cues arranged by region in a database.

13. The method of claim 12 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises matching the visual cue to only those visual cues corresponding to a region in which the method is performed.

14. The method of claim 7 , further comprising identifying the advertising element within the visual image, and wherein generating the interface control command comprises causing a telephone number to be dialed corresponding to the advertising element.

15. The method of claim 1 , wherein generating the interface control command comprises generating a website address.

16. The method of claim 15 , further comprising navigating to the website address.

17. An apparatus comprising a processor and memory storing executable instructions configured to, when executed by the processor, cause the apparatus to:

receive a signal including an environmental cue;

correlate the environmental cue to an interface control element from a plurality of possible interface control elements, wherein the plurality of possible interface control elements includes a control for automatically operating at least one of a radio device, a telephone device, and a navigation device; and

generate an interface control command based on the environmental cue,

wherein when the signal comprises a visual image and the environmental cue comprises an advertising element in a user-selected portion of the visual image, generating an interface control command comprises generating a command for the radio device.

18. The apparatus of claim 17 , wherein receiving a signal containing an environmental cue comprises receiving a visual image from a camera.

19. The apparatus of claim 18 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating a traffic sign within the visual image to a vehicle speed control.

20. The apparatus of claim 18 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating a traffic sign within the visual image to a navigation system control.

21. The apparatus of claim 18 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating an advertising sign within the visual image to a radio station control.

22. The apparatus of claim 17 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises receiving user input regarding the user-selected portion of the visual image corresponding to the received signal.

23. The apparatus of claim 22 , wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to: process the user-selected portion of the visual image to identify the environmental cue.

24. The apparatus of claim 23 , wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to: identify from the visual image a speed limit, and wherein generating the interface control command comprises generating a speed control command for the vehicle.

25. The apparatus of claim 17 , wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

display a visual image comprising one or more environmental cues;

receive user input selecting a portion of the displayed visual image;

display a border having a first dimension around the selected portion of the displayed image;

in response to additional user input, display a border smaller than the first dimension around the selected portion of the displayed image; and

upon further user input or after expiration of a predetermined time period, use a portion of the image within the smaller border as the basis for comparison with a database of potentially matching environmental cues.

26. The apparatus of claim 23 , wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

identify a navigation element within the visual image, and wherein generating the interface control command comprises automatically generating a detour command for the navigation device when the environmental cue comprises a traffic alert.

27. The apparatus of claim 23 , wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

identify the advertising element within the visual image.

28. The apparatus of claim 17 , wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises matching a visual cue within a received image to one of a plurality of potentially matching visual cues arranged by region in a database.

29. The apparatus of claim 28 , wherein correlating the environmental cue from a plurality of possible interface control elements comprises matching the visual cue to only those visual cues corresponding to a region in which the method is performed.

30. The apparatus of claim 23 , wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

identify the advertising element within the visual image, and wherein generating the interface control command comprises causing a telephone number to be dialed corresponding to the advertising element.

31. The apparatus of claim 17 , wherein generating the interface control command comprises generating a website address.

32. The apparatus of claim 31 , wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

navigate to the website address.

33. A non-transitory computer-readable medium storing executable instructions configured to, when executed, cause a processor to:

receive a signal including an environmental cue;

correlate the environmental cue to an interface control element from a plurality of possible interface control elements, wherein the plurality of possible interface control elements includes a control for automatically operating at least one of a radio device, a telephone device, and a navigation device; and

generate an interface control command based on the environmental cue,

wherein when the signal comprises a visual image and the environmental cue comprises an advertising element in a user-selected portion of the visual image, generating an interface control command comprises generating a command for the radio device.

34. The computer-readable medium of claim 33 , further storing executable instructions configured to, when executed, cause the processor to:

receive user input selecting a portion of a visual image corresponding to the received signal.

35. An apparatus comprising a processor and memory storing executable instructions configured to, when executed by the processor, cause the apparatus to:

receive an image from a camera configured to capture traffic signs along a travel path of a vehicle;

display on a touch-screen display the received image;

receive user input identifying a region of the displayed image;

identify a traffic sign at least in part by processing the identified region of the displayed image and comparing it to a plurality of patterns each of which corresponds to a visual cue;

in response to determining that the identified region matches one of the plurality of patterns, generate a control signal configured to automatically control an element of the vehicle, the element being one of a plurality of possible elements of the vehicle, wherein the plurality of possible elements includes at least one of a radio device, a telephone device, and a navigation device,

wherein when the signal comprises a visual image and the environmental cue comprises an advertising element in a user-selected portion of the visual image, generating a control signal comprises generating a command for the radio device.

36. The apparatus of claim 35 , wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

display a border having a first dimension around the identified region of the displayed image;

in response to additional user input, display a border smaller than the first dimension around the identified region of the displayed image; and

upon further user input or after expiration of a predetermined time period, use a portion of the image within the smaller border as the basis for the comparison with the plurality of patterns.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 2, 2023
From: CONVERSANT WIRELESS LICENSING S.A R.L.
To: CONVERSANT WIRELESS LICENSING LTD.
Reel/Frame 063507/0900 →
RELEASE OF SECURITY INTEREST Recorded Apr 13, 2021
From: CPPIB CREDIT INVESTMENTS INC.
To: CONVERSANT WIRELESS LICENSING S.A R.L.
Reel/Frame 055910/0698 →
AMENDED AND RESTATED U.S. PATENT SECURITY AGREEMENT (FOR NON-U.S. GRANTORS) Recorded Aug 22, 2018
From: CONVERSANT WIRELESS LICENSING S.A R.L.
To: CPPIB CREDIT INVESTMENTS, INC.
Reel/Frame 046897/0001 →
CHANGE OF NAME Recorded Oct 20, 2017
From: CORE WIRELESS LICENSING S.A.R.L.
To: CONVERSANT WIRELESS LICENSING S.A R.L.
Reel/Frame 044250/0398 →