ICCP 2010

Image Upsampling via Texture Hallucination

Yoav HaCohen Raanan Fattal Dani Lischinski
Hebrew University of Jerusalem, Israel

Abstract

Image upsampling is a common yet challenging task, since it is severely underconstrained. While considerable progress was made in preserving the sharpness of salient edges, current methods fail to reproduce the fine detail typically present in the textured regions bounded by these edges, resulting in unrealistic appearance.
In this paper we address this fundamental shortcoming by integrating higher-level image analysis and custom lowlevel image synthesis. Our approach extends and refines the patch-based image model of Freeman et al. [10] and interprets the image as a tiling of distinct textures, each of which is matched to an example in a database of relevant textures. The matching is not done at the patch level, but rather collectively, over entire segments. Following this model fitting stage, which requires some user guidance, a higher-resolution image is synthesized using a hybrid approach that incorporates principles from examplebased texture synthesis. We show that for images that comply with our model, our method is able to reintroduce consistent fine-scale detail, resulting in enhanced appearance textured regions.

Full Paper as PDF

This page contains:
1. Comparison of our texture hallucination method with other upsampling methods
2. Segmentation levels
3. Texture database

Click on the method name at the left of each image in order to toggle between the results.
To see the visualization/effect of some components of our method please click the image descriptions to the right of each image.


1. Comparison of our texture hallucination method to other image upsampling methods

We compare our image upsampling method with results obtained via the following methods:

[Bicubic] Adobe Photoshop's bicubic interpolation
[Freeman et al.] W. T. Freeman, T. R. Jones, and E. C. Pasztor. Example-based super-resolution. IEEE Computer Graphics and Applications, 22(2):56-65, 2002.
[Kim et al.] K. I. Kim and Y. Kwon. Example-based learning for single-image super-resolution and JPEG artifact removal. Technical Report 173, 08 2008.
[Fattal] R. Fattal. Image upsampling via imposed edge statistics. ACM Trans. Graphics (Proc. SIGGRAPH 2007), 26(3):95-102, 2007.
[Glasner et al.] D. Glasner, S. Bagon, and M. Irani. Super-resolution from a single image. In Proc. ICCV, 2009.
[Shan et al.] Q. Shan, Z. Li, J. Jia, and C.-K. Tang. Fast image/video upsampling. ACM Trans. Graphics, 27(5):1–7, 2008.
[Genuine Fractals] A state-of-the-art image enlarging commercial product.


Bicubic
Genuine Fractals
Freeman et al.
Fattal
Kim et al.
Glasner et al.
Our Result

Magnification X4
Loading image... Bicubic
Categories: fabric, face

Textures Model
Without self similarity
Predicted high band without self similarity
Predicted high band with self similarity
Self similarity blend map

Bicubic
Genuine Fractals
Fattal
Glasner et al.
Our Result

Magnification X4
Loading image... Bicubic
Categories: fur, leaves, trunk

Textures Model
Predicted high band without self-similarity
Predicted high band blended with self similarity
Self similarity blend map

Bicubic
Fattal
Genuine Fractals
Our Result

Magnification X4
Loading image...

Bicubic
Categories: grass, stone

Texture model
Synthesis without segmentation and classification steps
Self-similarity blend map
Different texture examples

Bicubic
Genuine Fractals
Fattal
Shan et al.
Our Result

Magnification X4
Loading image...

Bicubic
Categories: pumpkins, dust


Bicubic
Glasner et al.
Our Result

Magnification X3
Loading image...

Bicubic
Categories: sunflowers


Bicubic
Fattal
Shan et al.
Genuine Fractals
Our Result

Magnification X4
Loading image...

Bicubic
Categories: sky, trees, bricks, stones, leaves, dry-grass

2. Segmentation levels


Three levels of segmentation for the koala image

3. Textures database


Images below show cropped versions of some of the example textures we have in our database (all the textures needed to produce the examples in this page are included here).
Please click the thumbnails to open the full images.
Click at the name of each class in order to obtain the full set of example textures.

Stones
Grass
Face
Fabric
Fur
Leaves
Sunflowers
Pumpkins
Dirt
Dry Grass
Bricks
Sky
Trees