IP Library Granted Patent US 9,916,081
Granted Patent B2
US 9,916,081 · App. 13/997,225 · Granted Mar 13, 2018

Techniques for image-based search using touch controls

Inventors: Junmin Zhu (Shanghai, CN); Hongbo Min (Shanghai, CN); Ningxin Patrick Hu (Shanghai, CN); Yongsheng Zhu (Shanghair, CN); Zhiqiang Nelson Yu (Shanghai, CN)
Assignee: INTEL CORPORATION
G06F3/04883G06F3/0412G06F3/0481G06F3/04842G06F17/30864G06K9/00671G06K9/2081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,916,081
App. No.
13/997,225
Granted
Mar 13, 2018
Kind
B2
Abstract

Techniques for image-based search using touch controls are described. An apparatus may comprise: a processor circuit; a gesture component operative on the processor circuit to receive gesture information from a touch-sensitive screen displaying an image and generate a selection area corresponding to the gesture information; a capture component operative on the processor circuit to extract an image portion of the image corresponding to the selection area; and a search component operative on the processor circuit to perform an image-based search using the extracted image portion. Other embodiments are described and claimed.

Claims (57)

1. An apparatus, comprising:

a processor circuit;

a gesture component operative on the processor circuit to:

receive gesture information from a touch-sensitive screen displaying an image and at least one user interface element, and

generate a selection area corresponding to the gesture information, the gesture component to:

determine a minimized rectangle bounding a touch gesture indicated by the gesture information,

crop the minimized rectangle to lie entirely within the image and to exclude the at least one user interface element to generate a cropped minimized rectangle, and

set the selection area as the cropped minimized rectangle;

a capture component operative on the processor circuit to extract an image portion of the image corresponding to the selection area; and

a search component operative on the processor circuit to perform an image-based search using the extracted image portion.

2. The apparatus of claim 1 , the image comprising a frame from a video, the image presented as part of a playback of the video.

3. The apparatus of claim 1 , the gesture component operative to recognize a scratch gesture based on the received gesture information and generate the selection area corresponding to the scratch gesture.

4. The apparatus of claim 3 , the scratch gesture comprising a substantially-parallel substantially-simultaneous swipe of a plurality of touch-points downwards across the touch-sensitive screen.

5. The apparatus of claim 1 , the touch-sensitive screen operative to detect objects positioned on a surface of the touch-sensitive screen.

6. The apparatus of claim 4 , the gesture component to determine the minimized rectangle as an area traversing a portion of the touch-sensitive screen that minimally bounds the plurality of touch-points.

7. The apparatus of claim 4 , each swipe of the plurality of touch-points represented by a pair of points corresponding to touch points recorded for a beginning and an end of each swipe, each of the pair of points comprising an x-coordinate and a y-coordinate.

8. The apparatus of claim 7 , the gesture component to:

determine a horizontal extent of the minimized triangle to span from a minimum x-coordinate of a plurality of x-coordinates of pairs of points for the scratch gesture to a maximum x-coordinate for the plurality of x-coordinates, and

determine a vertical extent of the minimized triangle to span from a minimum y-coordinate of a plurality of y-coordinates of the pairs of points for the scratch gesture to a maximum y-coordinate for the plurality of y-coordinates.

9. A computer-implemented method, comprising:

displaying an image on a touch-sensitive screen, the image displayed in combination with at least one user interface element;

receiving gesture information from the touch-sensitive screen;

generating a selection area corresponding to the received gesture information by;

determining a minimized rectangle bounding a touch gesture indicated by the gesture information,

cropping the minimized rectangle to lie entirely within the image and to exclude the at least one user interface element to generate a cropped minimized rectangle, and

setting the selection area as the cropped minimized rectangle;

extracting an image portion of the image corresponding to the selection area;

performing an image-based search using the extracted image portion to produce search results; and

displaying the search results on the touch-sensitive screen.

10. The method of claim 9 , the image comprising a frame from a video, the image presented as part of a playback of the video.

11. The method of claim 9 , comprising:

recognizing a scratch gesture based on the received gesture information; and

generating the selection area corresponding to the scratch gesture.

12. The method of claim 11 , the scratch gesture comprising a substantially-parallel substantially-simultaneous swipe of a plurality of touch-points downwards across the touch-sensitive screen.

13. The method of claim 12 , comprising determining the minimized rectangle as an area traversing a portion of the touch-sensitive screen that minimally bounds the plurality of touch-points.

14. The method of claim 12 , each swipe of the plurality of touch-points represented by a pair of points corresponding to touch points recorded for a beginning and an end of each swipe, each of the pair of points comprising an x-coordinate and a y-coordinate.

15. The method of claim 14 , comprising:

determining a horizontal extent of the minimized triangle to span from a minimum x-coordinate of a plurality of x-coordinates of pairs of points for the scratch gesture to a maximum x-coordinate for the plurality of x-coordinates, and

determining a vertical extent of the minimized triangle to span from a minimum y-coordinate of a plurality of y-coordinates of the pairs of points for the scratch gesture to a maximum y-coordinate for the plurality of y-coordinates.

16. At least one non-transitory computer-readable storage medium comprising instructions that, when executed, cause a system to:

receive gesture information from a touch-sensitive screen displaying a video;

generate a selection area corresponding to the received gesture information by;

determining a minimized rectangle bounding a touch gesture indicated by the gesture information,

cropping the minimized rectangle to lie entirely within the image and to exclude the at least one user interface element to generate a cropped minimized rectangle, and

setting the selection area as the cropped minimized rectangle;

extract an image portion of the video corresponding to the selection area, the image portion displayed in combination with user interface elements;

request an image-based search from an image-based search system based on the extracted image portion; and

receive search results from the image-based search system.

17. The non-transitory computer-readable storage medium of claim 16 , further comprising instructions that when executed cause the system to:

recognize a scratch gesture based on the received gesture information; and

generate the selection area corresponding to the scratch gesture.

18. The non-transitory computer-readable storage medium of claim 17 , the scratch gesture comprising a substantially-parallel substantially-simultaneous swipe of a plurality of touch-points downwards across the touch-sensitive screen.

19. The non-transitory computer-readable storage medium of claim 18 , further comprising instructions that when executed cause the system to determine the minimized rectangle as an area traversing a portion of the touch-sensitive screen that minimally bounds the plurality of touch-points.

20. The non-transitory computer-readable storage medium of claim 18 , each swipe of the plurality of touch-points represented by a pair of points corresponding to touch points recorded for a beginning and an end of each swipe, each of the pair of points comprising an x-coordinate and a y-coordinate.

21. The non-transitory computer-readable storage medium of claim 20 , further comprising instructions that when executed cause the system to:

determine a horizontal extent of the minimized triangle to span from a minimum x-coordinate of a plurality of x-coordinates of pairs of points for the scratch gesture to a maximum x-coordinate for the plurality of x-coordinates, and

determine a vertical extent of the minimized triangle to span from a minimum y-coordinate of a plurality of y-coordinates of the pairs of points for the scratch gesture to a maximum y-coordinate for the plurality of y-coordinates.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 2, 2016
From: ZHU, JUNMIN; MIN, HONGBO; HU, NINGXIN PATRICK; ZHU, YONGSHENG; YU, ZHIQIANG NELSON
To: INTEL CORPORATION
Reel/Frame 037395/0506 →
Continuity (1)
Related Publication 20150052431A1 · Feb 19, 2015