- Number
- 9652685
- Published
- 2017-05-16
- Filed
- 2014-09-30
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Sigal; Leonid, Kim; Gunhee
- CPC
- G06V10/763; G06F18/2321; G06V20/30
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Generates a story graph structuring activities, events, and locations from a large collection of photos taken by different people.
Abstract
Embodiments presented herein describe techniques for generating a story graph using a collection of digital media, such as images and video. The story graph presents a structure for activities, events, and locales commonly occurring in sets of photographs taken by different individuals across a given location (e.g., a theme park, tourist attraction, convention, etc.). To build a story graph, streams from sets of digital media are generated. Each stream corresponds to media (e.g., images or video) taken in sequence at the location by an individual (or related group of individuals) over a period of time. For each stream, features from each media are extracted relative to the stream. Clusters of media are generated and are connected by directed edges. The connections indicate a path observed to have occurred in the streams from one cluster to another cluster.
Background
BRIEF DESCRIPTION OF THE DRAWINGS(1) So thatthe manner in which the above recited features of the present disclosure can be understood in detail, a more particular description of the disclosure, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only exemplary embodiments and are therefore not to be considered limiting of its scope, may admit to other equally effective embodiments.(2) FIG. 1 illustrates an example computing environment, according to one embodiment.(3) FIG. 2 illustrates the application depicted in FIG. 1, according to one embodiment.(4) FIG. 3 illustrates an example theme park map used to localize images of a photo stream, according to one embodiment.(5) FIG. 4 illustrates a method for classifying photo streams used in story graphs, according to one embodiment.(6) FIG. 5 illustrates a method for extracting semantic information from photo streams, according to one embodiment.(7) FIG. 6 illustrates a method for generating a story graph using photo streams, according to one embodiment.(8) FIG. 7 illustrates an example sub-graph of a story graph, according to one embodiment.(9) FIG. 8 illustrates an example of a story graph interface used to recommend theme park attractions to a visitor of the theme park, according to one embodiment.(10) FIG. 9 illustrates an example of a story graph interface used to provide temporal analysis
Claims
1. A method for generating a representation of activities occurring within agiven location having one or more landmarks based on a collection of digital media, the method comprising: generating, via a processor, a plurality of streams from the collection of media, each stream corresponding to an ordered sequence of media captured by an individual over a given period of time; for each stream, extracting features from one or more of the media in the stream; generating, from the plurality of streams, one or more clusters of the collection of the media based on the extracted features, wherein each cluster indicates a landmark depicted in the media of the cluster; and connecting clusters with directededges, wherein each directed edge indicates a path from a first landmark to a second landmark based on the ordered sequence of images from each stream, wherein the clusters and the directed edges are defined hierarchically, and wherein the directed edges are weighted based on a probability that connected clusters co-occur sequentially across one or more of the streams.
2. A method for generating a representation of activities occurring within a given location having one or more landmarks based on a collection of digital media, the method comprising: generating, via a processor, a plurality of streams from the collection of media, each stream corresponding to an ordered sequence of media captured by an individual over a given period of time; for each stream, extracting features from one or more of the media in the stream; generating, from the plurality of streams, one or more clusters ofthe collection of the media based on the extracted features, wherein each cluster indicates a landmark depicted in the media of the cluster; connecting clusters with directed edges, wherein each directed edge indicates a path from a first landmark to a second landmark based on the ordered sequence of images from each stream; and presenting recommendations to a guest at a theme park for activities, events, and attractions to visit based on the connected clusters.
5. A system, comprising: a processor; and a memory storing one or more application programs configured to perform an operation for generating a representation of activities occurring within a given location having one or more landmarks based on a collection of digital media, the operation comprising: generating a plurality of streams from the collection of media, each stream corresponding to an ordered sequence of media capturedby an individual over a given period of time, for each stream, extracting features from one or more of the media in the stream, generating, from the plurality of streams, one or more clusters of the collection of the media based on the extracted features, wherein each cluster indicates a landmark depicted in the media of the cluster, connecting clusters with directed edges, wherein each directed edge indicates a path from a first landmark to a second landmark based on the ordered sequence of images from each stream, wherein the clusters and directed edges are defined hierarchically, and wherein the directed edges are weighted based on a probability that connected clusters co-occur sequentially across one or more of the streams, and removing directed edges having a low weight determined based on a maximum likelihood estimation.
6. A system, comprising: a processor; and a memory storing one or more application programs configured to perform an operation for generating a representation of activities occurring within a given location having one or more landmarks based on a collection of digital media, the operation comprising: generating a plurality of streams from the collection of media, each stream corresponding to an ordered sequence of media captured by an individual over a given period of time, for each stream, extracting features from one or more of the media in the stream, generating, from the plurality of streams, one or more clusters of the collection of the media based on the extracted features, wherein each cluster indicates a landmark depicted in the media of the cluster, connecting clusters with directed edges, wherein each directed edge indicates a path from a first landmark to a second landmark based on the ordered sequence of images from each stream, wherein the clusters and directed edges are defined hierarchically, and wherein the directed edges are weighted based on a probability that connected clusters co-occur sequentially across one or more of the streams.