- Number
- 11055538
- Published
- 2021-07-06
- Filed
- 2017-03-31
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Koperski; Michal, Bak; Slawomir W., Carr; G. Peter K.
- CPC
- G06V20/52; G06T7/292; G06V10/761
- Verdict
- Set aside generic CV re-identification research
- Source
- Google Patents · FreePatentsOnline
Abstract
Techniques for object re-identification based on temporal context. Embodiments extract, from a first image corresponding to a first camera device and a second image corresponding to a second camera device, a first plurality of patch descriptors and a second plurality of patch descriptors, respectively. A measure of visual similarity between the first image and the second image is computed, based on the first plurality of patch descriptors and the second plurality of patch descriptors. A temporal cost between the first image and the second image is computed, based on a first timestamp at which the first image was captured and a second timestamp at which the second image was captured. The measure of visual similarity and the temporal cost are combined into a single cost function, and embodiments determine whether the first image and the second image depict a common object, using the single cost function.
Background
BACKGROUND Field of the Invention (1) The present disclosure relates to digital image processing, and more specifically, to techniques for unsupervised learning for recognizing objects appearing across multiple, non-overlapping images. Description of the Related Art (2) Object recognition is useful in many different fields. However, many conventional techniques require affirmative interaction with one or more electronic devices, and thus are ill-suited for situations in which objects such as people, animals, vehicles and other things are moving quickly through a location (e.g., an airport). Image processing techniques can be used to recognize objects within frames captured by a camera. One challenge when detecting objects across multiple cameras is handling variations in lighting, camera position and object pose across the cameras. For instance, an object may have a certain set of color characteristics as in an image captured by a first camera, and may be captured with different color characteristics in an image captured by a different camera. As a result, conventional techniques for determining that an object captured by multiple cameras are indeed the same object may be inaccurate. SUMMARY (3) Embodiments provide a method, system and computer readable medium for re-identifying objects depicted in images captured by two non-overlapping cameras. A first image corresponding to a first camera and a second image corresponding to a second camera, a first plurality of patch descri
Claims
1. A computer-implemented method of object identification, the computer-implemented method comprising: capturing, with a first camera at a first point in time, a first image depicting a first scene; capturing, with a second camera at a second point in time different from the first point in time, a second image depicting a second scene that does not overlap with the first scene; extracting a first patch from the first image; extracting a second patch from the second image; extracting a first patch descriptor from the first patch; extracting a second patch descriptor from the second patch; mapping the first and second patch descriptors to a concatenated codeword in a clustered codebook learned via coupled clustering of a set of concatenated features, wherein at least one concatenated feature of the set of concatenated features is generated by concatenating corresponding patch descriptors from the first and second cameras, the concatenated codeword comprising dimensions of cluster centers of coupled clusters to which the first and second patch descriptors map; computing an appearance cost comprising a measure of visual dissimilarity between the first and second images, based on the concatenated codeword and by operation of one or more computer processors; computing a temporal cost between the first and second images, using a temporal context model and based on the first and second points in time; combining the appearance cost and the temporal cost into a single cost function; and determining whether the first and second images depict a common object, using the single cost function, after which an indication of whether the first and second images depict a common object is output. ||
15. A method, comprising: extracting, from a first image corresponding to a first camera device and a second image corresponding to a second camera device, a first plurality of patch descriptors and a second plurality of patch descriptors, respectively; computing a measure of visual similarity between the first image and the second image, based on the first plurality of patch descriptors and the second plurality of patch descriptors and by operation of one or more computer processors; computing a temporal cost between the first image and the second image, based on a first timestamp at which the first image was captured and a second timestamp at which the second image was captured; combining the measure of visual similarity and the temporal cost into a single cost function; determining a measure of confidence for the measure of visual similarity between the first image and the second image; upon determining that the measure of confidence exceeds a predefined threshold level of confidence, updating a temporal context model based on the first timestamp and the second timestamp; determining whether the first image and the second image depict a common object, using the single cost function; and continuing to refine the temporal context model over time, until an amount of change from each refinement is less than a threshold amount of change. ||
16. A non-transitory computer-readable medium containing computer program code executable to perform an operation for object identification, the operation comprising: capturing, with a first camera at a first point in time, a first image depicting a first scene; capturing, with a second camera at a second point in time different from the first point in time, a second image depicting a second scene that does not overlap with the first scene; extracting a first patch from the first image; extracting a second patch from the second image; extracting a first patch descriptor from the first patch; extracting a second patch descriptor from the second patch; mapping the first and second patch descriptors to a concatenated codeword in a clustered codebook learned via coupled clustering of a set of concatenated features, wherein at least one concatenated feature of the set of concatenated features is generated by concatenating corresponding patch descriptors from the first and second cameras, the concatenated codeword comprising dimensions of cluster centers of coupled clusters to which the first and second patch descriptors map; computing an appearance cost comprising a measure of visual dissimilarity between the first and second images, based on the concatenated codeword and by operation of one or more computer processors when executing the computer program code; computing a temporal cost between the first and second images, using a temporal context model and based on the first and second points in time; combining the appearance cost and the temporal cost into a single cost function; and determiningwhetherthefirstandsecondimagesdepictacommonobject,usingthe single cost function, after which an indication of whether the first and second images depict a common object is output. ||
17. A computer-implemented method to facilitate object identification, the computer-implemented method comprising: extracting a first patch descriptor from a first image depicting a first scene captured by a first camera device; extracting a second patch descriptor from a second image depicting a second scene captured by a second camera device, wherein the first and second scenes are non-overlapping, wherein each of the first and second images has a respective timestamp; concatenating corresponding patch descriptors from the first and second camera devices into at least a concatenated patch descriptor of a set of concatenated patch descriptors; generating, by operation of one or more computer processors, a clustered codebook based on coupled clustering of the set of concatenated patch descriptors, the clustered codebook including a concatenated codeword, the concatenated codeword comprising dimensions of cluster centers of coupled clusters that the concatenated patch descriptor maps to; and generating a temporal context model based on the respective timestamp of each of the first and second images, wherein based on the clustered codebook and the temporal context model, an appearance cost and a temporal cost between a subsequent plurality of images are computed and combined into a single cost function, wherein based on the single cost function, the subsequent plurality of images is determined to depict a common object.