Skip to content

DECIMAL columns lose precision (rendered as float64) — likely across all backends #934

Description

@paddymul

Problem

DECIMAL / high-precision fixed-point columns lose precision when rendered. Numeric cell values and numeric summary stats are transported and rendered as JS number (IEEE-754 float64), so any decimal whose significand exceeds float64's 53-bit mantissa — e.g. DECIMAL(38,9) — is rounded on the way to the grid. BIGINT is already protected (bigintToCell / the parseParquetRow BigInt path preserves it as a string), but there is no analogous handling for DECIMAL.

This was identified while designing the DuckDB/Node backend (#930), where @duckdb/node-api returns DECIMAL as an exact DuckDBDecimalValue{value: bigint, scale}. The #930 plan deliberately casts DECIMAL → DOUBLE for v1 and defers exactness — this issue tracks the real fix.

I suspect this is not DuckDB-specific. The pandas, polars (native Decimal dtype), and ibis/xorq backends all funnel numeric values and stats through the same float displayer / float64 render path, so the same rounding should occur there. Worth verifying across backends rather than fixing only the DuckDB path.

Suggested fix

Preserve exact decimals end to end the way BIGINT is preserved — carry a lossless representation (string, or scaled-int + scale) through serialization and add a displayer (or extend the float displayer) that renders the exact value. Confirm reproduction on pandas/polars first to size the cross-backend scope.

Context

Identified during the #930 DuckDB/Node backend design. v1 of #930 ships DECIMAL → DOUBLE as a known, documented limitation pending this fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions