Introduction to Common Image Formats

Background

With the advancement of terminal devices, the production of high-definition and ultra-high-definition images has become increasingly convenient, leading to rising bandwidth costs for image uploading, transmission, and downloading. Common image compression methods like PNG and JPEG can achieve some degree of compression, but their compression ratios still fall short of meeting the growing size of image files. As a result, new image compression formats such as HEIC, WebP, and AVIF have been introduced. These new formats can achieve greater compression while maintaining the same image quality as JPEG and PNG.

This article first outlines the development history of image compression formats and then focuses on introducing the mainstream formats currently in use. It primarily covers the compression methods and performance comparisons of different formats, aiming to provide a clearer understanding of various image formats and help users choose the most suitable compression format based on their needs.

Development History of Image Formats

The above diagram shows that JPEG and PNG are relatively old compression formats, while WebP, HEIF, and AVIF are newer formats that have emerged recently. However, due to their superior compression ratios, they are gradually replacing JPEG and PNG in various terminal devices and browsers. Below, we will introduce the compression methods and key features of these different image formats step by step.

Introduction to Image Formats

PNG Introduction

Portable Network Graphics (PNG) is a lossless compressed bitmap graphics format that supports three color schemes—indexed, grayscale, and RGB—as well as features like Alpha channels.

PNG defines two types of data chunks: critical chunks, which are standard and mandatory, and ancillary chunks, which are optional. Critical chunks include four standard data blocks that every PNG file must contain, and all PNG software must support these chunks.

Here, we introduce the role of the IHDR chunk in standard data blocks: the header chunk (IHDR) contains basic information about the image data stored in the PNG file and must appear as the first chunk in the PNG data stream. Moreover, a PNG data stream can only have one header chunk. The header chunk consists of 13 bytes:

Compression Process

PNG compression is entirely lossless, meaning the compressed file can accurately reconstruct the original image. The PNG compression process mainly includes:

Channel extraction can break a large image into several smaller ones.

Filtering

The PNG format uses a delta encoding method called "filtering." Essentially, for each pixel scanline, the current pixel is encoded based on its relationship with the left pixel, the above pixel, and the above-left pixel.

Assume there is an image block as shown above, and we need to predict the unknown value "X" using a filter. PNG allows selecting one of five modes per row:

No filtering

The difference between X and A

The difference between X and B

The difference between X and (A + B)/2 (i.e., the average)

Paeth predictor (a linear function of A, B, and C)

Each row can choose the best filtering method for itself, producing the fewest unique symbols (the more repeated symbols, the better the compression). These filters are applied per channel, not per pixel. This means the filter is applied to all red values in a scanline, then separately to all blue values.

Once filtering is applied to the scanline, it is passed to a variant of the LZ77 algorithm called DEFLATE, which combines LZ77 encoding with a Huffman encoder.

JPEG Introduction

JPEG stands for Joint Photographic Experts Group, a joint committee of ISO and IEC responsible for developing standards for static image compression. The algorithm developed by this group is called the JPEG algorithm, which has become a universal standard known as the JPEG standard. JPEG compression is lossy.

Compression Process

The steps in JPEG compression that result in quality loss occur during YUV sampling and quantization. YUV sampling is based on the human eye's lower sensitivity to chrominance compared to luminance. Sampling YUV444 to YUV420 reduces file size by 50%. Quantization is the lossy step in encoding; larger quantization steps result in smaller file sizes but poorer image quality. Different quantization matrices can be selected to control the compression quality.

WebP Introduction

WebP (pronounced "weppy") is an image file format that supports both lossy and lossless (reversible) compression. WebP supports a maximum pixel count of 16383x16383. Lossy WebP only supports 8-bit YUV 4:2:0 format, while lossless WebP supports VP8L encoding and 8-bit ARGB color space. WebP's lossy compression algorithm is based on intra-frame encoding of the VP8 video format, using RIFF as the container format.

RIFF: Resource Interchange File Format (RIFF) is a file format standard that stores data in tagged chunks.

Compression Process

Block size: Maximum block size 16x16, minimum 4x4

Prediction modes: Four intra-frame prediction modes

H_PRED (horizontal prediction): Uses the left column L to fill each column in the block.

V_PRED (vertical prediction): Uses the top row A to fill each row in the block.

DC_PRED (DC prediction): Uses the average of all pixels in L and A as a single value to fill the block.

TM_PRED (TrueMotion prediction):

HEIF Introduction

High Efficiency Image File Format (HEIF, pronounced "heef") was developed by the Moving Picture Experts Group (MPEG) in 2013 and is based on the ISOBMFF standard. HEIF is a container image format that can contain images and image sequences (a single file can contain multiple images). Current encoding formats include HEVC and H.264/MPEG-4 AVC, with potential for new formats in the future. .heic is just one extension of the HEIF file format. .heif and .avci are also part of the HEIF file format, though .heif and .heic are the most common, while .avci is rare.

ISOBMFF standard (=ISO Base Media File Format)

The HEIF format is designed based on this standard, which is used to store time-based or non-time-based media data. Its predecessor was Apple's QuickTime File Format (.mov).

Design philosophy—container structure:

1. Defines a box-structured file, meaning data is allocated within these boxes.

2. Allows boxes to contain other boxes.

3. Each box's header contains information about the box type and its size, enabling readers to skip unnecessary or unsupported boxes.

4. Uses 4 ASCII characters as identifiers for box types, often referred to as FourCC or 4CC.

Compression Process

HEIF's compression process is similar to WebP's, but it uses HEVC's intra-frame encoding technology, featuring HEVC's intra-frame encoding toolset, achieving higher compression ratios and better image quality.

Block size: Encoding unit sizes up to 64x64, minimum 4x4

Prediction modes: 35 intra-frame prediction modes

AVIF Introduction

AV1 Image File Format (AVIF) is an open, royalty-free image file format specification developed by the Alliance for Open Media (aomedia.org) in collaboration with Google, Cisco, and Xiph.org. It stores images or image sequences compressed with AV1 in the HEIF container format. AVIF supports full-resolution 10-bit and 12-bit color, HDR, and both lossy and lossless encoding.

Compression Process

AVIF's compression process is also similar to WebP's, but it uses AV1's intra-frame encoding technology, featuring AV1's intra-frame encoding toolset, offering higher compression capabilities and better image detail.

More flexible encoding tree partitioning: Encoding unit sizes up to 128x128, with up to 10 block partitioning methods.

More precise intra-frame prediction modes: 56 angular modes, intra copy palette mode, and other precise intra-frame prediction modes.

More advanced filtering techniques: Constrained Directional Enhancement Filter (CDEF), loop restoration filtering, etc., effectively improving image quality.