IP Library Granted Patent US 10,867,124
Granted Patent B2
US 10,867,124 · App. 16/112,492 · Granted Dec 15, 2020

Manual annotations using clustering, anchoring, and transformation

Inventors: Jiva Gandhara DeVoe (Vancouver, WA); John Henry Sturgeon (Camas, WA); James Vernon Van Boxtel (Ridgefield, WA); Elden G. Wood, III (Portland, OR); Kevin Raemon Glyn Smyth (Portland, OR); Evan Sean Torchin (Austin, TX)
Assignee: Apple Inc.
G06F40/169G06F40/106G06F40/171
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,867,124
App. No.
16/112,492
Granted
Dec 15, 2020
Kind
B2
Abstract

Embodiments are disclosed in which a process receives annotation inputs for annotations of a document. The annotation inputs may be grouped together into clusters based at least in part on a functional relationship between the timing of annotation inputs and the paths of the annotation inputs across an electronic display. The annotation inputs may be associated with portions of the document based at least in part on the shape of the paths of the annotation inputs, the sequence of annotation inputs, or the locations of the annotation inputs. Additionally, the paths of the annotation inputs to portions of the document may be modified in response to modifications of the portion of the document data.

Claims (70)

1. A tangible, non-transitory, machine-readable medium, comprising machine-readable instructions that when executed by one or more processors, cause the one or more processors to:

group one or more annotation inputs into one or more clusters by:

determining a padding area around a respective path for each of the one or more annotation inputs based on a function relating a padding distance and an elapsed time since a lift-off event of the respective annotation input; and

grouping each set of the one or more annotation inputs having overlapping padding areas into a respective cluster of the one or more clusters;

associate each of the one or more clusters with an object of one or more objects of document data; and

display annotations on a graphical representation of the one or more objects, wherein each displayed annotation corresponds to a cluster of the one or more clusters.

2. The machine-readable medium of claim 1 , comprising machine-readable instructions that, when executed by the one or more processors, cause the one or more processors to adjust the padding distance in response to a user input.

3. The machine-readable medium of claim 1 , comprising machine-readable instructions that, when executed by the one or more processors, cause the one or more processors to associate a first cluster of the one or more clusters with a first anchor location of the graphical representation, wherein the first anchor location corresponds to a first object of the one or more objects within the graphical representation.

4. The machine-readable medium of claim 3 , comprising machine-readable instructions that, when executed by the one or more processors, cause the one or more processors to:

receive input to move the first object a first distance within the graphical representation;

move the first anchor location the first distance within the graphical representation; and

move the annotations corresponding to the first cluster with the first anchor location.

5. The machine-readable medium of claim 1 , comprising machine-readable instructions that, when executed by the one or more processors, cause the one or more processors to:

anchor a first cluster of the one or more clusters to a first object of the one or more objects within the graphical representation, wherein the first object comprises a range of text;

receive input to add characters or to remove characters within the range of text of the first object; and

expand a path of a respective annotation input of the first cluster if the input adds characters within the range of text or contract a path of a respective annotation input of the first cluster if the input removes characters within the range of text.

6. A processor-implemented method for processing annotation inputs, comprising:

receiving annotation inputs, wherein each of the annotation inputs comprises a path defined by a corresponding touch-down event and lift-off event relative to an image shown on a display;

determining a padding area around the path for each annotation input, wherein the padding area for each annotation input is based on a function relating a padding distance and an elapsed time since the lift-off event of the respective annotation input;

identifying annotation inputs with overlapping padding areas;

grouping annotation inputs with overlapping padding areas into a cluster; and

displaying, at an associated location relative to an anchor location within the image the cluster as an annotation comprising an aggregated path of each respective annotation input of the cluster.

7. The processor-implemented method of claim 6 , wherein the function comprises a continuous function that inversely relates the padding distance with the elapsed time.

8. The processor-implemented method of claim 6 , wherein the padding distance is zero pixels when the elapsed time is greater than or equal to a predefined time limit.

9. The processor-implemented method of claim 6 , comprising anchoring the cluster to an object at the associated location relative to the anchor location within the image, wherein the object comprises an anchor range defined by the path of an identifying annotation input of the annotation inputs grouped into the cluster.

10. The processor-implemented method of claim 9 , wherein the path of the identifying annotation input comprises an underline, a strikethrough, or an enclosure.

11. The processor-implemented method of claim 9 , comprising:

expanding the path of the identifying annotation input of the cluster when characters are added within the anchor range; or

contracting the path of the identifying annotation input of the cluster when characters are removed from the anchor range.

12. The processor-implemented method of claim 6 , comprising determining the anchor location based on an average location within the image of the aggregated path of the annotation inputs of the cluster and a body region of the image, wherein the anchor location comprises a location of an object within the body region when the average location is within the body region.

13. A tangible, non-transitory, machine-readable medium, comprising machine-readable instructions that when executed by one or more processors, cause the one or more processors to:

display an image of a document via a display of an electronic device;

display first annotations associated with a first portion of the document via the display of the electronic device, wherein the first annotations correspond to a cluster of annotation inputs formed based at least in part on overlapping padding areas around one or more first paths of the annotation inputs, wherein each of the padding areas of the annotation inputs is determined based on a function relating a padding distance and an elapsed time since a lift-off event of a respective annotation input;

receive input to:

move the first portion within the document;

expand the first portion of the document; or

contract the first portion of the document;

adjust the first portion of the document based on the received input; and

adjust the display of the first annotations for the first portion of the document based at least in part on the received input to modify the first portion of the document, by:

moving one or more first paths of the first annotations;

expanding one or more first paths of the first annotations;

contracting one or more first paths of the first annotations; or

any combination thereof.

14. The machine-readable medium of claim 13 , wherein the first annotations are associated with the first portion of the document based at least in part on:

a path shape of one or more identifying annotation inputs of the annotation inputs;

an earliest annotation input of the annotation inputs; or

any combination thereof.

15. The machine-readable medium of claim 13 , comprising machine-readable instructions that when executed by the one or more processors, cause the one or more processors to:

receive second annotation inputs;

associate the second annotation inputs with a second portion of the document, wherein each annotation input of the second annotation inputs comprises a respective second path; and

display second annotations on the display within the second portion of the document, wherein the second annotations correspond to the respective second paths of the second annotation inputs.

16. A processor-implemented method for processing annotation inputs, comprising:

displaying an image of a document via a display of an electronic device;

grouping a first group of annotation inputs into a first cluster by:

determining a padding area around a respective path for each of the annotation inputs based on a function relating a padding distance and an elapsed time since a lift-off event of the respective annotation input; and

grouping one or more annotation inputs having overlapping padding areas into the first cluster;

associating the first group of annotation inputs with a first associated location relative to a first anchor location of the image of the document based at least in part on:

a path shape of one or more identifying annotation inputs of the first group of annotation inputs;

an earliest annotation input of the first group of annotation inputs;

an average location within the image of the paths of the annotation inputs of the first group of annotation inputs; or

a combination thereof; and

displaying first annotations for the first group of annotation inputs via the display of the electronic device at the first associated location, wherein the displayed first annotations correspond to the first cluster.

17. The processor-implemented method of claim 16 , comprising:

associating a second group of annotation inputs to a second anchor location of the image of the document based at least in part on a second location of the second group of annotation inputs, wherein the second location corresponds to an average location of second paths of the second group of annotation inputs that is in a header of the image of the document or a footer of the image of the document;

displaying second annotations for the second group of annotation inputs via the display of the electronic device at the second anchor location;

receiving input to move one or more objects of the document relative to the second anchor location; and

maintaining the second anchor location for the second group of annotation inputs despite the received input to modify the document.

18. The machine-readable medium of claim 1 , wherein the function comprises a continuous function that inversely relates the padding distance with the elapsed time.

19. The machine-readable medium of claim 13 , wherein the function comprises a continuous function that inversely relates the padding distance with the elapsed time.

20. The processor-implemented method of claim 16 , wherein the function comprises a continuous function that inversely relates the padding distance with the elapsed time.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2018
From: DEVOE, JIVA GANDHARA; STURGEON, JOHN HENRY; VAN BOXTEL, JAMES VERNON; WOOD, ELDEN G., III; SMYTH, KEVIN RAEMON GLYN; TORCHIN, EVAN SEAN
To: APPLE INC.
Reel/Frame 046805/0767 →
Continuity (2)
Provisional Application 62648301 · Mar 26, 2018
Related Publication 20190294660A1 · Sep 26, 2019