Skip to content

API.md

Latest commit

 

History

History
455 lines (348 loc) · 16.2 KB

API.md

File metadata and controls

455 lines (348 loc) · 16.2 KB
Raw

heic_decoder API Guide

This document is the practical API reference for integrating heic_decoder in apps and automation.

Scope

  • Decode HEIF/HEIC/AVIF to RGBA (u8 or u16 storage).
  • Decode HEVC-coded HEIC/HEIF straight to RGB8 when alpha is not needed.
  • Decode from bytes, Read, BufRead, and file paths.
  • Optional image crate integration through hook registration.
  • Keep default decode output aligned with primary item transforms (clap/irot/imir) and expose explicit EXIF orientation helpers.

Feature Flags

  • default: no optional features.
  • image-integration: enables heic_decoder::image_integration module and hook APIs.

Cargo.toml:

[dependencies]
heic_decoder = { path = "../heic-decoder" }

With image integration:

[dependencies]
heic_decoder = { path = "../heic-decoder", features = ["image-integration"] }
image = { version = "0.25", default-features = false, features = ["png"] }

Core Types

  • DecodeGuardrails
    • max_input_bytes: Option<u64>
    • max_pixels: Option<u64>
    • max_temp_spool_bytes: Option<u64>
    • temp_spool_directory: Option<PathBuf>
  • DecodedRgbaImage
    • width, height
    • source_bit_depth
    • pixels: DecodedRgbaPixels (U8(Vec<u8>) or U16(Vec<u16>))
    • icc_profile: Option<Vec<u8>>
  • DecodedRgbImage
    • width, height
    • source_bit_depth
    • pixels: Vec<u8> (RGB8 interleaved)
    • icc_profile: Option<Vec<u8>>
  • ExifOrientationHint
    • exif_orientation: Option<u8>
    • primary_item_has_orientation_transform: bool
    • should_apply_exif_orientation()
    • orientation_to_apply()
  • DecodeError, DecodeErrorCategory, DecodeGuardrailError

Core Decode Entry Points

Return type for RGBA decode functions:

Result<DecodedRgbaImage, DecodeError>

RGBA output

  • decode_bytes_to_rgba_with_guardrails(input, guardrails)
  • decode_read_to_rgba_with_guardrails(reader, guardrails)
  • decode_bufread_to_rgba_with_guardrails(bufread, guardrails)
  • decode_path_to_rgba_with_guardrails(path, guardrails)

Non-guardrailed convenience variants also exist (decode_*_to_rgba) and use DecodeGuardrails::default().

RGB8 output

For callers that only need RGB, such as pipelines that would otherwise call DynamicImage::into_rgb8 on the RGBA result:

  • decode_bytes_to_rgb8_with_guardrails(input, guardrails)
  • decode_path_to_rgb8_with_guardrails(path, guardrails)

Both return Result<DecodedRgbImage, DecodeError>. decode_bytes_to_rgb8 and decode_path_to_rgb8 use DecodeGuardrails::default(). There are no Read/BufRead variants.

Contract:

  • Only HEVC-coded primary items (hvc1/hev1, or a grid of them) are supported. AVIF and uncompressed (unci) HEIF return an error; use the RGBA entry points for those.
  • Calling RGB8 is an explicit request to drop alpha. Auxiliary alpha is not decoded and no RGBA buffer is allocated. On large images this roughly halves peak memory compared with decoding RGBA and then calling into_rgb8.
  • pixels holds width * height * 3 bytes, row-major with no padding. Primary item transforms (clap/irot/imir) are applied and EXIF orientation is not, as with RGBA. DecodedRgbImage has no orientation helper.
  • Output is always 8-bit; source_bit_depth reports the coded bit depth.
  • When the RGBA decode of the same input succeeds, pixels is byte-for-byte equal to that RGBA output converted with DynamicImage::into_rgb8. For high-bit-depth input this means each RGBA16 sample v becomes (v + 128) / 257.
  • icc_profile matches the RGBA output's.

PNG output

  • decode_bytes_to_png_with_guardrails(input, output_path, guardrails)
  • decode_read_to_png_with_guardrails(reader, output_path, guardrails)
  • decode_bufread_to_png_with_guardrails(bufread, output_path, guardrails)
  • decode_path_to_png_with_guardrails(input_path, output_path, guardrails)

Legacy aliases:

  • decode_file_to_rgba -> decode_path_to_rgba
  • decode_file_to_png -> decode_path_to_png

Advanced Decode APIs

These are useful for tooling and specialized pipelines, but most applications should use the core entry points above.

  • decode_primary_avif_to_image(input) -> Result<DecodedAvifImage, DecodeAvifError>
  • assemble_primary_heic_hevc_stream(input) -> Result<Vec<u8>, DecodeHeicError>
  • decode_primary_heic_to_image(input) -> Result<DecodedHeicImage, DecodeHeicError>
  • decode_primary_heic_to_metadata(input) -> Result<DecodedHeicImageMetadata, DecodeHeicError>
  • decode_primary_uncompressed_to_image(input) -> Result<DecodedUncompressedImage, DecodeUncompressedError>

Also note:

  • isobmff and source modules are public and available for advanced use.
  • These low-level APIs expose more container/codec details and have a larger surface for malformed-input handling.

Framing note

Assembled HEVC streams are decoded with fixed four-byte length-prefixed framing. Streams whose first NAL unit is 1 byte or 256-511 bytes long were previously re-detected as Annex B start-code data, which could decode hidden parameter sets embedded anywhere in the stream and bypass pixel limits; such streams now decode strictly by their length prefixes and either decode their visible parameter sets or fail cleanly.

Input Path Selection

Use this decision order:

  1. If you already have &[u8], use decode_bytes_to_rgba_with_guardrails.
  2. If you have a file path, use decode_path_to_rgba_with_guardrails.
  3. If you have Read/BufRead (non-seek stream), use decode_read_to_rgba_with_guardrails or decode_bufread_to_rgba_with_guardrails.

Important behavior:

  • Read/BufRead inputs are spooled once to a temporary file to enable random-access decode.
  • Path and seekable decode paths avoid full-file in-memory buffering.

Guardrail Presets

Use explicit guardrails in production.

Example preset:

use heic_decoder::DecodeGuardrails;

let guardrails = DecodeGuardrails {
    max_input_bytes: Some(128 * 1024 * 1024),
    max_pixels: Some(64_000_000),
    max_temp_spool_bytes: Some(128 * 1024 * 1024),
    temp_spool_directory: None,
};

Notes:

  • max_input_bytes applies to all entry points.
  • max_temp_spool_bytes protects non-seek Read/BufRead ingestion.
  • max_pixels checks AV1 and HEVC coded frame area before pixel decoding, including auxiliary alpha frames and HEIC grid tiles. Container dimensions and post-decode geometry checks still apply.
  • Auxiliary alpha must match the primary's visible geometry. Its coded frame, including any conformance-window padding, counts against max_pixels and the decoded-byte allowance below, so cropping cannot hide an oversized coded frame.
  • HEVC preflight estimates decoded planes and normalized bitstream storage. The byte allowance is 12 bytes per allowed pixel plus 64 MiB, with the primary area limiting auxiliary frames and the composed output area limiting grids. A grid dimension covered by a single tile row or column counts as at least 512 pixels when tiles are stacked along the other dimension, matching the fixed tile span encoders use for narrow images. Every grid tile, including the first, and the sum of all tile estimates must fit this allowance. This estimate is not a process RSS limit.

Error Handling Contract

Top-level errors are DecodeError. Use category() for stable handling:

use heic_decoder::{DecodeError, DecodeErrorCategory};

fn classify(err: &DecodeError) -> DecodeErrorCategory {
    err.category()
}

Stable categories:

  • Io
  • Parse
  • MalformedInput
  • UnsupportedFeature
  • ResourceLimit
  • DecoderBackend
  • OutputEncoding

Typical policy mapping:

  • ResourceLimit: return 413 / reject input.
  • MalformedInput or Parse: return 422.
  • UnsupportedFeature: return 415.
  • Io or DecoderBackend: retry/fallback/log depending on context.

Working With Decoded Pixels

Inspect storage and branch explicitly:

use heic_decoder::DecodedRgbaPixels;

match &decoded.pixels {
    DecodedRgbaPixels::U8(p) => {
        // p: &[u8], RGBA8 interleaved
    }
    DecodedRgbaPixels::U16(p) => {
        // p: &[u16], RGBA16 interleaved
    }
}

Helpers on DecodedRgbaImage / DecodedRgbaPixels:

  • storage_bit_depth()
  • as_rgba8(), as_rgba16()
  • into_rgba8(), into_rgba16()
  • apply_exif_orientation(exif_orientation)

EXIF Orientation Handling

Default decode behavior:

  • Decode applies HEIF primary item transforms from clap/irot/imir.
  • Decode does not implicitly apply EXIF orientation.
  • This keeps output stable for libheif parity and avoids hidden double-rotation behavior.

Orientation APIs:

  • exif_orientation_hint(input_bytes) -> ExifOrientationHint
  • exif_orientation_hint_from_path(input_path) -> Result<ExifOrientationHint, DecodeError>
  • primary_exif_orientation(input_bytes) -> Option<u8>
  • primary_exif_orientation_from_path(input_path) -> Result<Option<u8>, DecodeError>
  • primary_item_has_orientation_transform(input_bytes) -> bool
  • path_extension_is_heif(path) -> bool
  • path_extension_is_heif_family(path) -> bool
  • DecodedRgbaImage::apply_exif_orientation(exif_orientation) -> Result<DecodedRgbaImage, DecodeError>

Recommended pattern:

use heic_decoder::{decode_path_to_rgba, exif_orientation_hint};
use std::path::Path;

fn decode_with_explicit_orientation(path: &Path) -> Result<(), heic_decoder::DecodeError> {
    let input = std::fs::read(path).map_err(heic_decoder::DecodeError::Io)?;
    let hint = exif_orientation_hint(&input);

    let decoded = decode_path_to_rgba(path)?;
    let decoded = if let Some(orientation) = hint.orientation_to_apply() {
        decoded.apply_exif_orientation(orientation)?
    } else {
        decoded
    };

    eprintln!("{}x{}", decoded.width, decoded.height);
    Ok(())
}

Path-only variant (no full input read, no image decode):

use heic_decoder::{exif_orientation_hint_from_path, path_extension_is_heif};
use std::path::Path;

fn orientation_to_apply(path: &Path) -> Result<Option<u8>, heic_decoder::DecodeError> {
    if !path_extension_is_heif(path) {
        return Ok(None);
    }
    Ok(exif_orientation_hint_from_path(path)?.orientation_to_apply())
}

image Integration

Requires image-integration feature.

Hook decodes buffer the entire encoded input in memory before decoding; in exchange, pixels decode straight into the buffer image hands over, without an additional full-frame owned RGBA allocation. Codec-native planes and a single grid-tile scratch buffer may still be allocated. register_image_decoder_hooks() caps the encoded input buffer at DEFAULT_HOOK_MAX_INPUT_BYTES (128 MiB); register with explicit guardrails to choose a different max_input_bytes bound (or None for unbounded).

1) Register hooks once at startup

use heic_decoder::DecodeGuardrails;
use heic_decoder::image_integration::register_image_decoder_hooks_with_guardrails;

let registration = register_image_decoder_hooks_with_guardrails(DecodeGuardrails {
    max_input_bytes: Some(128 * 1024 * 1024),
    max_pixels: Some(64_000_000),
    max_temp_spool_bytes: Some(128 * 1024 * 1024),
    temp_spool_directory: None,
});

assert!(registration.any_decoder_hook_registered());

2) Decode using image::ImageReader

use image::ImageReader;
use heic_decoder::exif_orientation_hint;
use heic_decoder::image_integration::apply_exif_orientation_dynamic;

let bytes = std::fs::read("input.heic").map_err(image::ImageError::IoError)?;
let hint = exif_orientation_hint(&bytes);

let img = ImageReader::new(std::io::Cursor::new(&bytes))
    .with_guessed_format()?
    .decode()?;

let img = if let Some(orientation) = hint.orientation_to_apply() {
    apply_exif_orientation_dynamic(img, orientation)
} else {
    img
};

The registered hook decoder also exposes the primary item's EXIF block through the standard ImageDecoder::exif_metadata/ImageDecoder::orientation methods, so generic image code (decoder.orientation() + DynamicImage::apply_orientation) works without crate-specific helpers. EXIF orientation stays unapplied in the pixels; applying it remains the caller's choice. When the primary item carries irot/imir container transforms — which decode does bake into the pixels — orientation() reports NoTransforms (matching ExifOrientationHint::should_apply_exif_orientation) so callers do not rotate twice; exif_metadata still returns the full block.

Conversion Helpers (image module)

Under image-integration, DecodedRgbaImage provides conversion helpers:

  • into_image_buffer_with_metadata()
  • into_image_buffer()
  • into_dynamic_image_with_metadata()
  • into_dynamic_image()

Metadata-preserving wrappers:

  • ImageBufferWithMetadata
  • DynamicImageWithMetadata

Possible conversion failures:

  • ImageConversionError::SampleCountOverflow
  • ImageConversionError::SampleCountMismatch

Additional helper under image-integration:

  • apply_exif_orientation_dynamic(image, exif_orientation)

AI/Automation Usage Rules

For reliable agent behavior:

  1. Always use *_with_guardrails APIs for untrusted input.
  2. Treat Read/BufRead as temp-spooled random-access decode, not progressive decode.
  3. Branch on decoded.storage_bit_depth() and avoid implicit down-conversion.
  4. Match DecodeError::category() for stable retry/reject decisions.
  5. Register image hooks exactly once during process startup.
  6. Keep guardrail values centralized in one config module for consistency.

Minimal End-to-End Example

use heic_decoder::{decode_path_to_rgba_with_guardrails, DecodeGuardrails};
use std::path::Path;

fn decode(path: &Path) -> Result<(), heic_decoder::DecodeError> {
    let guardrails = DecodeGuardrails {
        max_input_bytes: Some(128 * 1024 * 1024),
        max_pixels: Some(64_000_000),
        max_temp_spool_bytes: Some(128 * 1024 * 1024),
        temp_spool_directory: None,
    };

    let decoded = decode_path_to_rgba_with_guardrails(path, guardrails)?;
    eprintln!(
        "decoded {}x{}, storage={}bit, source={}bit",
        decoded.width,
        decoded.height,
        decoded.storage_bit_depth(),
        decoded.source_bit_depth
    );
    Ok(())
}

Experimental incremental bounded decoding

Enable the incremental-experiment Cargo feature to use decode_incremental_experiment(BoundedInput, BoundedDecodeOptions). It returns (BoundedRgbImage, [u64; 3]); the diagnostic counters are reconstructed rows, SAO edge components, and SAO band components across all coded items. This feature and entry point are experimental. Existing decode functions retain their behavior and do not select this path automatically.

The incremental path supports opaque 8-bit 4:2:0 HEVC still images with one IDR slice per coded item, including non-grid images and uniform grids whose individual tiles exceed the output cap. WPP, HEVC tiles, multiple slices, higher bit depths, alpha, other chroma formats, and unsupported color profiles return errors. The output is display-oriented sRGB RGB8, using area averaging when reduced, with a maximum side from 1 through 6000. Known depth, semantic matte, and gain-map auxiliaries may accompany the SDR base image; this does not apply their effects or provide HDR output.

Reconstruction, deblocking, and SAO finish in rolling bands before conversion and reduction. Large coded images use a bounded two-thread pipeline. The working state grows with coded width and CTU size, while output storage is capped by max_side; it does not retain a full source raster or use temporary files or repeated reconstruction. A caller's borrowed compressed input is separate from decoder-owned memory. Admission is conservative and may reject budgets smaller than its reserved workspace, including a worker stack. Allocator overhead and process RSS are not the same as requested allocation payloads. Use Path input when avoiding a caller-owned compressed input buffer is also important.

This first subset retains limits of 16384 per coded side, 256 million display pixels, bounded metadata, and a 64 KiB slice-header prefix. It supports one clean-aperture crop before orientation, with chroma interpolation for odd origins on a single coded image or one-tile grid. Odd-origin crops across multiple grid items and repeated crops or crops after orientation are unsupported. Exif orientation applies when no effective container orientation is present. decoder-tracing disables bounded decoding.