In some embodiments, a method determines an anchor image and a first mapping of a grid to the anchor image. The anchor image includes a first value of a characteristic and the grid includes a second value. Input images and second mappings between respective input images and the anchor image is determined. The method determines a query value for the grid based on features extracted from a location in the anchor image that is determined from the first mapping of the grid. Keys and values for each of the input images is determined based on respective input images and respective second mappings. The method weights values of the input images based on comparing the query value and the respective keys to generate weighted values. The weighted values are used to generate a feature value in a feature map. The method generates an output image from the feature map.
BACKGROUND
Increasing the resolution of images and frames is a common problem in media production, and this continues to be a problem despite better and better cameras that are capable of capturing high quality video. This is due to several reasons: keeping the video processing pipeline in lower resolution for compute and storage reasons, change of framing due to artistic considerations, or even bad capture conditions (low light, incorrect camera setting, etc.). In addition to this, enhancing a legacy low quality video catalog to meet modern standards is required in some applications.
1. A method comprising: determining an anchor image and a first mapping of a grid to the anchor image, wherein the anchor image includes a first value of a characteristic and the grid includes a second value of the characteristic; determining a set of input images and a set of second mappings between respective input images in the set of input images and the anchor image; determining a query value for the grid based on features extracted from a location in the anchor image that is determined from the first mapping of the grid; determining keys and values for each of the set of input images based on respective input images and respective second mappings; weighting values of the set of input images based on comparing the query value and the respective keys to generate weighted values, wherein the weighted values are used to generate a feature value in a feature map; and generating an output image from the feature map, wherein the output image is a resampling of the anchor image from the first value of the characteristic to the second value. ||
17. A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for: determining an anchor image and a first mapping of a grid to the anchor image, wherein the anchor image includes a first value of a characteristic and the grid includes a second value of the characteristic; determining a set of input images and a set of second mappings between respective input images in the set of input images and the anchor image; determining a query value for the grid based on features extracted from a location in the anchor image that is determined from the first mapping of the grid; determining keys and values for each of the set of input images based on respective input images and respective second mappings; weighting values of the set of input images based on comparing the query value and the respective keys to generate weighted values, wherein the weighted values are used to generate a feature value in a feature map; and generating an output image from the feature map, wherein the output image is a resampling of the anchor image from the second value of the characteristic to the first value. ||
20. An apparatus comprising: one or more computer processors; and a computer-readable storage medium comprising instructions for controlling the one or more computer processors to be operable for: determining an anchor image and a first mapping of a grid to the anchor image, wherein the anchor image includes a first value of a characteristic and the grid includes a second value of the characteristic; determining a set of input images and a set of second mappings between respective input images in the set of input images and the anchor image; determining a query value for the grid based on features extracted from a location in the anchor image that is determined from the first mapping of the grid; determining keys and values for each of the set of input images based on respective input images and respective second mappings; weighting values of the set of input images based on comparing the query value and the respective keys to generate weighted values, wherein the weighted values are used to generate a feature value in a feature map; and generating an output image from the feature map, wherein the output image is a resampling of the anchor image from the first value of the characteristic to the second value.