Image and Video Upscaling from Local Self-Examples

Gilad Freeman and Raanan Fattal

 

Abstract

We propose a new high-quality and efficient single-image upscaling technique that extends existing example-based super-resolution frameworks. In our approach we do not rely on an external example database or use the whole input image as a source for example patches. Instead, we follow a local self-similarity assumption on natural images and extract patches from extremely localized regions in the input image. This allows us to reduce considerably the nearest-patch search time without compromising quality in most images. Tests, that we perform and report, show that the local-self similarity assumption holds better for small scaling factors where there are more example patches of greater relevance. We implement these small scalings using dedicated novel non-dyadic filter banks, that we derive based on principles that model the upscaling process. Moreover, the new filters are nearly-biorthogonal and hence produce high-resolution images that are highly consistent with the input image without solving implicit back-projection equations. The local and explicit nature of our algorithm makes it simple, efficient and allow a trivial parallel implementation on a GPU. We demonstrate the new method ability to produce high-quality resolution enhancement, its application to video sequences with no algorithmic modifications, and its efficiency to perform real-time enhancement of low-resolution video standard into recent high-definition formats.

 

 

 

Links:

  • Manuscript, Full ACM TOG 2010 article
  • Results page
  • NTSC (640) to full HD (1920) conversion demo, (large file 635MB) and another demo, (large file 742MB)
  • SIG '11 presentation ppt, video1, video2
  • Information about licencing this technology can be found here.

Bibtex reference:

@article{FreFat10,   
author = {Freedman, Gilad and Fattal, Raanan},   
title = {Image and Video Upscaling from Local Self-Examples},   
journal = {ACM Trans. Graph.},   
volume = {28},   
number = {3},   
year = {2010},   
issn = {0730-0301},   
pages = {1--10},   
doi = {http://doi.acm.org/10.1145/1531326.1531328},   
publisher = {ACM},   
address = {New York, NY, USA},   
}