Repository navigation
Description
Description
While testing RowBinaryFormatWriter for our use case, we identified several bottlenecks in its current serialization path. The writer initially took approximately 684 seconds to process the workload. After addressing these bottlenecks, the execution time decreased to approximately 462.47 seconds, representing a 32.39% reduction in runtime.
The main bottlenecks identified are:
Schema-dependent operations are repeated for every column of every row, even though the TableSchema does not change during the writer’s lifetime.
Primitive setters such as setInteger(), setLong(), setFloat(), and setDouble() delegate to setValue(int, Object), causing primitive boxing.
Values for the entire row are staged in an Object[] until commitRow() is called.
The stored values are later unboxed during serialization.
These operations add avoidable CPU, allocation, memory-access, and garbage-collection overhead for large, wide RowBinary workloads.
The proposed improvement is to compile and cache a reusable serialization plan when the writer is constructed. A primitive-specialized or sequential writing path should also be considered to avoid unnecessary Object[] staging, boxing, and unboxing.
Affected component
client-v2
com.clickhouse.client.api.data_formats.RowBinaryFormatWriter
com.clickhouse.client.api.data_formats.RowBinaryFormatSerializer
com.clickhouse.client.api.data_formats.internal.SerializerUtils
Current behavior
Typed setter methods eventually delegate to the generic object setter:
public void setLong(int colIndex, long value) {
setValue(colIndex, value);
}
public void setValue(int colIndex, Object value) {
row[colIndex - 1] = value;
if (!rowStarted) {
rowStarted = true;
}
}
Because setValue() accepts Object, primitive values are autoboxed
The values are retained in an Object[] until commitRow() is called.
commitRow() then performs another traversal of the row:
List columnList = tableSchema.getColumns();
for (int i = 0; i < row.length; i++) {
ClickHouseColumn column = columnList.get(i);
if (column.hasDefault()&& column.getDefaultValue() != ClickHouseColumn.DefaultValue.DEFAULT) {
continue;
}
if (RowBinaryFormatSerializer.writeValuePreamble(out, defaultSupport, column, row[i])) {
SerializerUtils.serializeData(out, row[i], column);
}
}
Potential bottlenecks
1: Primitive boxing
Typed methods such as setInteger(), setLong(), and setDouble() delegate to:
setValue(int colIndex, Object value)
This boxes primitives and stores them in an Object[], adding allocation, reference indirection, later unboxing, and possible GC overhead.
2: Multiple passes over every row
Each row is:
- Written into Object[]
- Traversed again by commitRow()
- Cleared using Arrays.fill(row, null)
A schema-ordered writer could serialize values directly without staging the complete row.
Repeated type dispatch
3: For every cell, commitRow() calls:
SerializerUtils.serializeData(out, row[i], column);
The column type is static, but the serializer path is selected repeatedly for every row.
4: Repeated schema metadata processing
commitRow() repeatedly performs:
tableSchema.getColumns();
columnList.get(i);
column.hasDefault();
column.getDefaultValue();
Writable columns and other static metadata could be resolved once when the writer is created.
Activity
Harry262530 commentedon Nov 3, 2026
I have also attached a benchmarking script that can be usefull to demonstrate the improved performance
rowbinary-writer-benchmark.zip
bineetsingh30 commentedon Nov 4, 2026
Hi, I am part of the same company as @Harry262530. We would like to contribute to the issue. @chernser can we improve this and raise a merge request