Photo content clustering for digital picture frame display and automated frame storytelling
A method and system for automated routing of pictures taken on mobile electronic devices to a digital picture frame including a camera, microphone, and speaker integrated with the frame, and a network connection module allowing the frame for direct contact and upload of photos from electronic devices or from photo collections of community members. Clustering photos by content is used to improve display and to respond to photo viewer desires. Trends or patterns can be detected from the photo collections and that information used for various purposes beyond photo display. The frame includes a conversational intelligence that provides a verbal communication with a viewer, such as for determining an identity or preferences of the frame viewer, determining photos to display for the viewer, discussing displayed photos with the viewer, or telling stories to the viewer based upon photo content.
1 . A method of displaying a digital photo collection on a digital picture frame including a digital display mounted within a frame, a microphone and speaker connected to the frame, and a network connection module, the method comprising:
automatically clustering original photos within the digital photo collection into a plurality of sub-clusters, each for a corresponding common detected content in the original photos selected from a person, a thing, a place, an activity, or combinations thereof, wherein each of the plurality of sub-clusters is formed for the corresponding common detected content of interest at a similar time and/or age;
extracting one or more photo features from the corresponding common detected content within the each of the sub-clusters of the original photos to form a cluster representation, wherein the photo features are predetermined as having the limited change over a timeframe;
comparing the cluster representation of the each of the sub-clusters to new photos added to the digital photo collection, to determine a corresponding sub-cluster for each of the new photos;
linking the each of the new photos to the corresponding sub-cluster for display;
automatically displaying a slideshow on the digital display of the original photos and the new photos from the digital photo collection as a function of the clustering;
receiving through the microphone a spoken story request from a viewer of the digital picture frame;
automatically searching for and/or displaying one or more digital photos of the digital photo collection matching the spoken story request; and
automatically generating a new and unique story narrative in response to the spoken story request, wherein the new and unique story narrative is spoken by the digital picture frame through the speaker and matches the spoken story request and the one or more digital photos, wherein the new and unique story narrative is guided by displayed image content, photo metadata, and settings provided by the viewer.
2 . The method of claim 1 , further comprising receiving the spoken story request in response to a photo being displayed on the digital display.
3 . The method of claim 1 , further comprising displaying a newly and automatically generated slideshow of photos related to the new and unique story narrative as the new and unique story narrative progresses.
4 . The method of claim 1 , further comprising annotating the one or more digital photos with overlaid text or images related to the new and unique story narrative.
5 . The method of claim 1 , further comprising selecting a spoken dialect for the new and unique story narrative that corresponds to a person, location, or activity of the one or more photos.
6 . The method of claim 1 , further comprising:
receiving responsive vocal utterances or voice prompts from the viewer during the new and unique story narrative; and
automatically adjusting a storyline of the new and unique story narrative in response to the responsive vocal utterances or voice prompts.
7 . The method of claim 6 , further comprising automatically searching for and displaying a further photo relevant to the adjusted storyline.
8 . The method of claim 1 , wherein automatically generating the new and unique story narrative comprises analyzing metadata for the one or more digital photos.
9 . The method of claim 1 , wherein the settings provided by the viewer include a desired sentiment or motivation for the new and unique story narrative.
10 . The method of claim 1 , wherein generating the new and unique story narrative comprises an initial verbal interactive conversation with or about a content of the displayed photo, wherein the initial verbal interaction conversation comprises a plurality of automated back-and-forth conversational iterations to establish the identity of the user and/or a desired narrative type, and the new and unique story narrative comprises a person, an activity, a location, or combinations thereof, identified from keywords detected in the initial verbal interaction conversation.
11 . The method of claim 1 , further comprising automatically determining with automated questions an identity of a viewer of the digital picture frame.
12 . The method of claim 1 , wherein the new and unique story narrative is developed from content clustered photographs of the digital photo collection, clustered according to a person, an activity, a location, or combinations thereof, and/or developed from metadata associated with photos in the digital photo collection.
13 . The method of claim 1 , further comprising comparing metadata of the original and new photos to identify or confirm the original or new photos in the sub-clusters, the metadata including photograph dates.
14 . The method of claim 1 , wherein the cluster representation comprises a cluster centroid of averaged photo features within each of the plurality of sub-clusters, and further comprising:
comparing the centroids to the new photos; and
linking the new photos for display as a function of a similar centroid.
15 . The method of claim 1 , further comprising:
automatically extracting content features from photos of the digital photo collection;
automatically extracting photo image features from photos of the digital photo collection;
creating metadata tags for the photos as a function of the extracted content features and the extracted photo image features;
storing tagged photos in a database; and
automatically searching the digital photo collection for metadata tags and digital photos matching a search parameter for the spoken story request.
16 . The method of claim 1 , further comprising:
downloading a digital picture frame interaction application to an electronic device of a viewer of the digital display;
automatically scanning pictures of the electronic device with the downloaded digital picture frame interaction application; and
clustering photos of the electronic device according to content in the plurality of clusters within the digital picture frame interaction application.
17 . The method of claim 1 , further comprising:
automatically providing a verbal interaction with the viewer of the digital picture frame via the microphone and speaker;
automatically determining a search parameter from the verbal interaction;
automatically connecting to the digital photo collection;
automatically searching the digital photo collection for digital photos matching the search parameter; and
automatically displaying on the digital display matching photos obtained from the digital photo collection.
18 . The method of claim 1 , wherein the new and unique story narrative is fictional and guided by displayed image content, photo metadata, or settings provided by the viewer.
19 . The method of claim 1 , wherein the new and unique story narrative has a fictitious storyline spoken through the speaker, the fictitious storyline corresponding to and based upon displayed photo content.
20 . The method of claim 1 , further comprising:
automatically detecting a sentiment or mood of the viewer from the spoken story request or a further verbal interaction with the viewer; and
automatically adjusting a storyline of the new and unique story narrative in response to the detected sentiment or mood.