IP Library Granted Patent US 10,567,726
Granted Patent B2
US 10,567,726 · App. 13/509,796 · Granted Feb 18, 2020

Method, apparatus or system for image processing

Inventors: Jian Ma (Beijing, CN); Ye Tian (Chongqing, CN); Xiaogang Yang (Beijing, CN); Wendong Wang (Beijing, CN)
Assignee: Nokia Technologies Oy
H04N9/8205
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,567,726
App. No.
13/509,796
Granted
Feb 18, 2020
Kind
B2
Abstract

A method, apparatus, system for image processing is provided that creates an object note for a picture. The method, apparatus or system includes: automatically obtaining picture description information from a picture source, wherein the picture description information at least partly have been formed automatically; obtaining annotation information from at least two object sources, wherein the two object sources are different from the picture sources, automatically fusing the said annotation information from the two object sources to form fused annotation information, and attaching said fused annotation information to the picture to create the object note for the picture.

Claims (74)

1. A method comprising:

obtaining picture description information from a picture source, wherein the picture description information having been formed by adding at least one of a time, a location, or user information to the picture description information,

obtaining annotation information from at least two object sources, wherein the at least two object sources are different from said picture source,

fusing said annotation information from said at least two object sources to form fused annotation information, wherein fusing the annotation information includes:

selecting content for annotation from the annotation information from the at least two object sources by parsing the annotation information into a text format and selecting relevant text,

assigning a score value to each sentence from the selected relevant text based upon relevancy and redundancy in comparison with other sentences from the selected relevant text;

filtering, using assigned score values for the selected content, to reduce irrelevant or redundant information by removing stigma words and repetitive information, wherein repetitive information comprises selected content that overlaps in subject matter, and

enhancing the cohesion or coherence of the content by at least arranging the text with topically related themes to reduce non-fluency, and

attaching said fused annotation information to a picture to create an object note for said picture.

2. A method according to claim 1 , further comprising:

forming a hyper-object-link between said picture and at least one object source, wherein said hyper-object-link comprises a link to an object in said object source, and

attaching said link to said object to said picture to create an object note for a picture.

3. A method according to claim 1 , further comprising:

forming relevance information by automatically analyzing information from said two sources against information from said picture source, and

obtaining said annotation information from said at least two sources based on said relevance information.

4. A method according to claim 3 , further comprising:

forming said relevance information by determining a correlation between the picture and the at least two sources by determining their similarity using at least one of the group of time information, location information, event information and person information, and

forming a weighted similarity indicator by using the at least one of the group of time information, location information, event information and person information.

5. A method according to claim 1 , wherein said at least two sources comprise at least two of the group of email messages, short messages, multimedia messages, instant messages, calendar entries, contact cards, blog entries, wiki entries and social network service entries.

6. A method according to claim 1 , further comprising:

clustering pictures based on said annotation information from said at least two sources.

7. A method according to claim 1 , further comprising:

receiving filter information or source selection information from the user for restricting the data from said at least two sources.

8. A method according to claim 1 , wherein forming the fused annotation information comprises:

generating summarization of the content through natural language processing.

9. The method according to claim 1 , wherein filtering the selected content to reduce irrelevant or redundant information comprises:

determining a ratio of overlap in subject matter of the selected content, and

removing content from at least one of the at least two object sources, in an instance in which the ratio of overlap satisfies a specified degree.

10. An apparatus comprising at least one processor, and memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:

obtain picture description information from a picture source, wherein the picture description information having been formed by adding at least one of a time, a location, or user information to the picture description information,

obtain annotation information from at least two object sources, wherein the at least two object sources are different from said picture source,

fuse said annotation information from said at least two object sources to form fused annotation information, wherein fusing the annotation information includes:

select content for annotation from the annotation information from the at least two object sources by parsing the annotation information into a text format and selecting relevant text,

assign a score value to each sentence from the selected relevant text based upon relevancy and redundancy in comparison with other sentences from the selected relevant text;

filter, using assigned score values for the selected content, to reduce irrelevant or redundant information by removing stigma words and repetitive information, wherein repetitive information comprises selected content that overlaps in subject matter, and

enhance the cohesion or coherence of the content by at least arranging the text with topically related themes to reduce non-fluency, and

attach said fused annotation information to a picture to create an object note for said picture.

11. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:

form a hyper-object-link between said picture and at least one object source, wherein said hyper-object-link comprises a link to an object in said object source, and

attach said link to said object to said picture to create an object note for a picture.

12. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:

form relevance information by automatically analyzing information from said two sources against information from said picture source, and

obtain said annotation information from said at least two sources based on said relevance information.

13. An apparatus according to claim 12 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:

form said relevance information by determining a correlation between the picture and the at least two sources by determining their similarity using at least one of the group of time information, location information, event information and person information, and

form a weighted similarity indicator by using the at least one of the group of time information, location information, event information and person information.

14. An apparatus according to claim 10 , wherein said at least two sources comprise at least two of the group of email messages, short messages, multimedia messages, instant messages, calendar entries, contact cards, blog entries, wiki entries and social network service entries.

15. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:

cluster pictures based on said annotation information from said at least two sources.

16. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:

receive filter information or source selection information from the user for restricting the data from said at least two sources.

17. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following for forming the fused annotation information:

generate summarization of the content through natural language processing.

18. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following for filtering the selected content to reduce irrelevant or redundant information:

determining a ratio of overlap in subject matter of the selected content, and

removing content from at least one of the at least two object sources, in an instance in which the ratio of overlap satisfies a specified degree.

19. A computer program product comprising at least one non-transitory computer readable medium having computer executable program code portions stored thereon, the computer executable program code portions comprising program instructions configured for:

obtaining picture description information from a picture source, wherein the picture description information having been formed by adding at least one of a time, a location, or user information to the picture description information,

obtaining annotation information from at least two object sources, wherein the at least two object sources are different from said picture source,

fusing said annotation information from said at least two object sources to form fused annotation information, wherein fusing the annotation information includes:

selecting content for annotation from the annotation information from the at least two object sources by parsing the annotation information into a text format and selecting relevant text,

assigning a score value to each sentence from the selected relevant text based upon relevancy and redundancy in comparison with other sentences from the selected relevant text;

filtering, using assigned score values for the selected content, to reduce irrelevant or redundant information by removing stigma words and repetitive information, wherein repetitive information comprises selected content that overlaps in subject matter, and

enhancing the cohesion or coherence of the content by at least arranging the text with topically related themes to reduce non-fluency, and

attaching said fused annotation information to a picture to create an object note for said picture.

20. A network service comprising at least one server and at least one user device providing to a user at least the following:

picture description information from a picture source, wherein the picture description information having been formed by adding at least one of a time, a location, or user information to the picture description information,

annotation information from at least two object sources, wherein the at least two object sources are different from said picture source,

fused annotation information formed by fusing said annotation information from said at least two object sources, wherein fusing the annotation information includes:

selecting content for annotation from the annotation information from the at least two object sources by parsing the annotation information into a text format and selecting relevant text,

assigning a score value to each sentence from the selected relevant text based upon relevancy and redundancy in comparison with other sentences from the selected relevant text;

filtering, using assigned score values for the selected content, to reduce irrelevant or redundant information by removing stigma words and repetitive information, wherein repetitive information comprises selected content that overlaps in subject matter, and

enhancing the cohesion or coherence of the content by at least arranging the text with topically related themes to reduce non-fluency, and

an object note for a picture by attaching said fused annotation information to said picture.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2023
From: NOKIA TECHNOLOGIES OY
To: PIECE FUTURE PTE LTD
Reel/Frame 063889/0841 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2015
From: NOKIA CORPORATION
To: NOKIA TECHNOLOGIES OY
Reel/Frame 035512/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 1, 2014
From: MA, JIAN; TIAN, YE; YANG, XIAOGANG; WANG, WENDONG
To: NOKIA CORPORATION
Reel/Frame 033863/0951 →