[GIS] How are GeoTIFF pyramids/overviews standardised

file formatsgdalgdaladdogeotiff-tiffimage-pyramids

While pyramids/overviews are not part of the GeoTIFF standard, many tools support creating them. For example vips/nip2, Orfeo Toolbox (otb) and ossim which all promise some support for creating those. But I have no idea if they would all create files in a format that the others support. Looking through the documentation of the tools does not really mention anything about that.

vips is not a geospatial tool and I could not find any enduser-friendly documentation but the IIPImage docs mention its support for "Tiled Pyramidal TIFF" : http://iipimage.sourceforge.net/documentation/images/

otb does not mention anything about a format or spec for its "Multi Resolution Pyramids": https://www.orfeo-toolbox.org/CookBook/CookBooksu65.html

ossim says it supports various output formats for its "reduced resolution data sets" but I don't know what they actually mean: https://trac.osgeo.org/ossim/wiki/img2rr

GDAL also does not really specify things about its "overview images": http://www.gdal.org/gdaladdo.html

So they all have pyramids/overviews but it is not clear if they are cross-compatible.

On more general pages I found the following quotes:

http://iipimage.sourceforge.net/documentation/images/ says

Tiled Multi-Resolution (or Tiled Pyramidal) TIFF is simply a tiled
multi-page TIFF image, with each resolution stored as a separate layer
within the TIFF. This is a standard TIFF extension and is supported by
most image processing applications including Photoshop, GIMP, VIPS and
ImageMagick. The libtiff codec library is also perfectly capable of
reading and writing such images.

Is that what everyone uses?

The Library of Congress also has some info: http://www.digitalpreservation.gov/formats/fdd/fdd000237.shtml
They note:

Pyramid TIFF files created by different applications are not
necessarily the same in structure. In particular, judging from
analysis with JHOVE and the identify command in ImageMagick, Adobe's
Photoshop and Image Magick generate files with different internal TIFF
structures; in both cases, most software that can handle TIFFs appears
to recognize the primary (full-size) TIFF without problem.

So, are those formats standardised and specified and documented somewhere? How can I find out which tool can produce them in compatible ways to others? Are there any geospatial tags in overviews/pyramids or could I use any software to create them on the image data?

Best Answer

Short answer: I suspect there is no such standard for TIFF or GeoTIFF overviews. There are multiple implementations, methods and formats to define overviews for GeoTIFFs.

GeoTIFF is based on the TIFF format (PDF specification for Revision 6.0, from 1992). The format has support for multi-page documents or subfiles, similar to a multi-page PDF.

A GeoTIFF file with 4 internal overviews is essentially a 5-page TIFF file, as viewed with identify:

$ identify -quiet file.tif
file.tif[0] TIFF 2027x2823 2027x2823+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.000
file.tif[1] TIFF 1014x1412 1014x1412+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.009
file.tif[2] TIFF 507x706 507x706+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.009
file.tif[3] TIFF 254x353 254x353+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.009
file.tif[4] TIFF 127x177 127x177+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.009

You can split these into separate TIFF files with (e.g.) convert file.tif file%d.tif, or GIMP.

A GeoTIFF file with an external overview uses a sidecar file. There is no standard approach how external overview files are created or used. For example, sometimes you may see a so-called ".ovr" file, which is a multi-page TIFF document:

$ identify -quiet file.tif
file.tif TIFF 2027x2823 2027x2823+0+0 8-bit Grayscale DirectClass 132KB 0.000u 0:00.000
$ identify -quiet file.tif.ovr
file.tif.ovr[0] TIFF 1014x1412 1014x1412+0+0 8-bit Grayscale DirectClass 2.101MB 0.000u 0:00.000
file.tif.ovr[1] TIFF 507x706 507x706+0+0 8-bit Grayscale DirectClass 2.101MB 0.000u 0:00.000
file.tif.ovr[2] TIFF 254x353 254x353+0+0 8-bit Grayscale DirectClass 2.101MB 0.000u 0:00.000
file.tif.ovr[3] TIFF 127x177 127x177+0+0 8-bit Grayscale DirectClass 2.101MB 0.000u 0:00.009

Other external overviews may use a ".rdd" or ".aux" sidecar file, which is a HFA/Erdas Imagine image file format. This is probably the most esoteric of the overview formats, since it is not even a TIFF file.

Related Solutions

[GIS] Merging multiple 16-bit image bands to create a True Color Tiff

Hopefully this is useful. Take note, it is written for python3 and needs specific libraries.

First you would need to read each tiff band as a numpy array using something like this:

import numpy as np
from osgeo import gdal
import os.path

def file_test(input_file):
    return os.path.isfile(input_file)

def open_tiff(tiff_file):
    print(tiff_file)
    if file_test(tiff_file):
        return gdal.Open(tiff_file)
    else:
        print("Error!")

def tiff_to_array(tiff_file, lon_offset_px=0, lat_offset_px=0):
    gdo = open_tiff(tiff_file)
    tiff_array = gdo.ReadAsArray(lon_offset_px,lat_offset_px,1000,1000)
    tiff_array = np.array(bytescale(tiff_array, low=0, high=255), dtype=np.int16)
    gdo = None
    return tiff_array

This is used to create image array. I use the scikit-image library to do some enhancements.

import numpy as np
from scipy.misc import bytescale
from skimage import exposure
from matplotlib import pyplot as plt

def create_composite(red_band, green_band, blue_band):
    img_dim = red.shape
    img = np.zeros((img_dim[0], img_dim[1], 3), dtype=np.float)
    img[:,:,0] = red
    img[:,:,1] = green
    img[:,:,2] = blue
    p2, p98 = np.percentile(img, (2, 98))
    img_rescale = exposure.rescale_intensity(img, in_range=(p2, p98))
    return bytescale(img_rescale)

image = create_composite(red, green, blue)
plt.imshow(image)

You could convert each band to 8-bit using scipy.misc.bytescale first before passing it to the function.

[GIS] Comparing use of GDAL to build raster pyramids or overviews versus ArcMap

I have worked with external and internal to the large extent . External pyramids are little bit slow then internal one's because the data has to be ready from the external file . GDAL has to read .ovr file.

Pros of external overview:-

Less space to store.
Handy and can be deelte anytime if not required.

Cons of external overview:-

Less Speed 2. takes 15% more size w.r.t. actual data size on hard disk.

Cons of internal overview:-

takes 35% more size w.r.t. actual data size on hard disk.
once created you can remove internal pyramids but space used for internal pyramids will not come back. Ex:- if 300 MB original dataset , we build internal it becomes 415MB , when we remove internal pyramids, size of raster will remains 415MB.
can only be build if data set size don't exceed 4 GB

Pros of internal overview:-

Faster Speed.

Best Answer

Related Solutions

[GIS] Merging multiple 16-bit image bands to create a True Color Tiff

[GIS] Comparing use of GDAL to build raster pyramids or overviews versus ArcMap

Related Question