[IMP] export: reduce JSON size by VincentSchippefilt · Pull Request #7756 · odoo/o-spreadsheet

VincentSchippefilt · 2026-01-13T07:41:36Z

Description:

Reduce the JSON file size of spreadsheets by storing only changes to formulae and, in a second pass,
collecting all the formulae or changes that are the same and storing them under one single key in the JSON file.

Task: 5489478

API changes:

compiler.ts/compile(formula: string) --> CompiledFormula.Compile(formula: string, sheetId: UID, getters: CoreGetters)
fomulas/helpers.ts/getFunctionsFromTokens(tokens: Tokens[], functionNames: string[]) --> fomulas/helpers.ts/getFunctionsFromTokens(compiledFormula: CompiledFormula, functionNames: string[])
removed RangeCompiledFormula --> CompiledFormula
model.ts/export() --> export(shouldSquish?: boolean = false)
cell.ts/getTranslatedFormulaCell(sheetId: UID, offsetX: number, offsetY: number, tokens: Token[]) --> getTranslatedFormulaCell(sheetId: UID,offsetX: number,offsetY: number,compiledFormula: CompiledFormula | SerializedCompiledFormula )
removed plugins/core/cell.ts/ReferenceToken class --> it is implemented in CompiledFormula formulas/helper.ts/isExportableToExcel&getFunctionsFromTokens --> CompiledFormula.areAllFunctionsExportableToExcel&getFunctionsFromTokens`
removed FormulaCellWithDependencies and ReferenceToken

Observed performance gains on large dashboard:

importing sheets:
~2300ms --> ~680 ms
1560 MB ram --> 580 MB

JSON Size:
40 MB -> 147 KB
4,4 MB -> 3KB
1,6 MB-> 43 KB

How does it work:

Shared formula

   A1: "My Text",
   A2: "=SUM(B2:B9)",
   A3: "=SUM(B2:B9)",

As we can see, A2 and A3 share the same formula. They can be rewritten as

   A1: "My Text",
   'A2:A3': "=SUM(B2:B9)",

removing duplication completely

Formula with small differences

Usually formula do not repeat exactly the same, but offset their dependencies (being either references to other cells, numbers or strings) slightly. We can rewrite:

   A1: "My Text",
   A2: "=CONCAT(B2, $C$2, D2, "hello"),
   A3: "=CONCAT(B3, $C$2, D3, "hello"),
   A4: "=CONCAT(B4, $C$2, D4, "hello"),

to

   A1: "My Text",
   A2: "=CONCAT(B2, $C$2, D2, "hello"),
   A3: { R : "+R1|=|+R1" },
   A4: { R : "+R1|=|+R1" },
   ...

now we see that A3 and A4 have the same transformation, so we can rewrite them to

   A1: "My Text",
   A2: "=CONCAT(B2, $C$2, D2, "hello"),
   "A3:A4": { R : "+R1|=|+R1" },

We apply the same for number arguments and string arguments.
A formula with all the arguments slightly changed might look like

   A1: "=MY_FORMULA(1, B1, "coucou"),
   A2: { R : "+R1", N: "+1", S: ["hello"] },
   A3:A4: { R : "+R1", N: "+1"}

can be read as

   A1: "=MY_FORMULA(1, B1, "coucou"),
   A2: "=MY_FORMULA(2, B2, "hello"),
   A3: "=MY_FORMULA(3, B3, "hello"),
   A4: "=MY_FORMULA(4, B4, "hello"),

When reading the compressed JSON, the order of the keys in the JSON itself do not matter, they are sorted on the first part of the key, so either the cell reference or the left part of the range.

robodoo · 2026-01-13T07:41:39Z

LucasLefevre

first review 👌

main point (but not the easiest): BananaCompiledFormula should be type-safe to raise at compile time instead of raising in production

pro-odoo

Nice :D
Didn't fully read the squisher yet

hokolomopo

👋

Mostly small things. I didn't read the details of the squisher/unsquisher. Really cool feature 🥳

VincentSchippefilt · 2026-02-11T15:01:26Z

robodoo rebase-ff

robodoo · 2026-02-11T15:01:29Z

Merge method set to rebase and fast-forward.

LucasLefevre

deeper look at the squish/unsquish

rrahir · 2026-02-19T11:57:05Z

+  const endPos = toCartesian(end);
+  for (
+    let col = Math.min(startPos.col, endPos.col);
+    col <= Math.max(startPos.col, endPos.col);


can be evaluated a single time; same for the condition below

rrahir · 2026-02-19T12:10:33Z

+    numbers: string[],
+    strings: string[],
+    references: string[]
+  ): Partial<SquishedFormula> {


useless Partial

rrahir · 2026-02-19T12:44:37Z

robodoo r+

- it manages the tokens privately - it tells you if your formula is valid or not - it replaces 2 classes: FormulaCellWithDependencies and RangeReferenceToken - it is required for the next commit: allow to create multiple formulas based the same formula with different parameters compiler.ts/`compile(formula: string)` --> `CompiledFormula.Compile(formula: string, sheetId: UID, getters: CoreGetters)` fomulas/helpers.ts/`getFunctionsFromTokens(...)` --> `compiledFormula.getFunctionsFromTokens(functionNames: string[], getters: CoreGetters)` removed `RangeCompiledFormula` --> `CompiledFormula` removed plugins/core/cell.ts/`ReferenceToken` class --> it is implemented in CompiledFormula` formulas/helper.ts/`isExportableToExcel` & `getFunctionsFromTokens` --> compiledFormula.`areAllFunctionsExportableToExcel` & `getFunctionsFromTokens` removed FormulaCellWithDependencies and ReferenceToken Task: 5489478 Part-of: #7756 Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>

Today all cells have the property `content` that contain the raw content of the cell: either its value, its formula text or the markdown link. Having the content re-create the formula from its parts is costly, and having the convenience of defining the property `content` on an object, either through a dedicated class or though a closure is very expensive (100 MB on large models). This PR move the property `content` out of Cell, in `LiteralCell` so it is no longer available on FormulaCell. Update all the usages of `content` to be specific to LiteralCell (!FormulaCell). Mostly, content was used to check if a cell is empty. This check was `if(!cell?.content)` and becomes `if (!cell?.isFormula && !cell?.content)` Task: 5489478 Part-of: #7756 Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>

Description: Reduce the JSON file size of spreadsheets by storing only changes to formulae and, in a second pass, collecting all the formulae or changes that are the same and storing them under one single key in the JSON file. importing sheets: ~2300ms --> ~680 ms 1560 MB ram --> 580 MB JSON Size: 40 MB -> 147 KB 4,4 MB -> 3KB 1,6 MB-> 43 KB ```cells: { A1: "My Text", A2: "=SUM(B2:B9)", A3: "=SUM(B2:B9)", ``` As we can see, A2 and A3 share the same formula. They can be rewritten as ```cells: { A1: "My Text", 'A2:A3': "=SUM(B2:B9)", ``` removing duplication completely Usually formula do not repeat exactly the same, but offset their dependencies (being either references to other cells, numbers or strings) slightly. We can rewrite: ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), A3: "=CONCAT(B3, $C$2, D3, "hello"), A4: "=CONCAT(B4, $C$2, D4, "hello"), ``` to ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), A3: { R : "R+1|=|R+1" }, A4: { R : "R+1|=|R+1" }, ... ``` now we see that A3 and A4 have the same transformation, so we can rewrite them to ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), "A3:A4": { R : "R+1|=|R+1" }, ``` We apply the same for number arguments and string arguments. A formula with all the arguments slightly changed might look like ```cells: { A1: "=MY_FORMULA(1, B1, "coucou"), A2: { R : "R+1", N: "+1", S: ["hello"] }, A3:A4: { R : "R+1", N: "+1"} ``` can be read as ```cells: { A1: "=MY_FORMULA(1, B1, "coucou"), A2: "=MY_FORMULA(2, B2, "hello"), A3: "=MY_FORMULA(3, B3, "hello"), A4: "=MY_FORMULA(4, B4, "hello"), ``` closes #7756 Task: 5489478 Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>

robodoo · 2026-02-19T13:13:07Z

@VincentSchippefilt @rrahir staging failed: ci/runbot (view more at https://runbot.odoo.com/runbot/batch/2378120/build/101554677)

- it manages the tokens privately - it tells you if your formula is valid or not - it replaces 2 classes: FormulaCellWithDependencies and RangeReferenceToken - it is required for the next commit: allow to create multiple formulas based the same formula with different parameters compiler.ts/`compile(formula: string)` --> `CompiledFormula.Compile(formula: string, sheetId: UID, getters: CoreGetters)` fomulas/helpers.ts/`getFunctionsFromTokens(...)` --> `compiledFormula.getFunctionsFromTokens(functionNames: string[], getters: CoreGetters)` removed `RangeCompiledFormula` --> `CompiledFormula` removed plugins/core/cell.ts/`ReferenceToken` class --> it is implemented in CompiledFormula` formulas/helper.ts/`isExportableToExcel` & `getFunctionsFromTokens` --> compiledFormula.`areAllFunctionsExportableToExcel` & `getFunctionsFromTokens` removed FormulaCellWithDependencies and ReferenceToken Task: 5489478

Today all cells have the property `content` that contain the raw content of the cell: either its value, its formula text or the markdown link. Having the content re-create the formula from its parts is costly, and having the convenience of defining the property `content` on an object, either through a dedicated class or though a closure is very expensive (100 MB on large models). This PR move the property `content` out of Cell, in `LiteralCell` so it is no longer available on FormulaCell. Update all the usages of `content` to be specific to LiteralCell (!FormulaCell). Mostly, content was used to check if a cell is empty. This check was `if(!cell?.content)` and becomes `if (!cell?.isFormula && !cell?.content)` Task: 5489478

Description: Reduce the JSON file size of spreadsheets by storing only changes to formulae and, in a second pass, collecting all the formulae or changes that are the same and storing them under one single key in the JSON file. importing sheets: ~2300ms --> ~680 ms 1560 MB ram --> 580 MB JSON Size: 40 MB -> 147 KB 4,4 MB -> 3KB 1,6 MB-> 43 KB ```cells: { A1: "My Text", A2: "=SUM(B2:B9)", A3: "=SUM(B2:B9)", ``` As we can see, A2 and A3 share the same formula. They can be rewritten as ```cells: { A1: "My Text", 'A2:A3': "=SUM(B2:B9)", ``` removing duplication completely Usually formula do not repeat exactly the same, but offset their dependencies (being either references to other cells, numbers or strings) slightly. We can rewrite: ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), A3: "=CONCAT(B3, $C$2, D3, "hello"), A4: "=CONCAT(B4, $C$2, D4, "hello"), ``` to ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), A3: { R : "R+1|=|R+1" }, A4: { R : "R+1|=|R+1" }, ... ``` now we see that A3 and A4 have the same transformation, so we can rewrite them to ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), "A3:A4": { R : "R+1|=|R+1" }, ``` We apply the same for number arguments and string arguments. A formula with all the arguments slightly changed might look like ```cells: { A1: "=MY_FORMULA(1, B1, "coucou"), A2: { R : "R+1", N: "+1", S: ["hello"] }, A3:A4: { R : "R+1", N: "+1"} ``` can be read as ```cells: { A1: "=MY_FORMULA(1, B1, "coucou"), A2: "=MY_FORMULA(2, B2, "hello"), A3: "=MY_FORMULA(3, B3, "hello"), A4: "=MY_FORMULA(4, B4, "hello"), ``` Task: 5489478

LucasLefevre · 2026-02-19T14:39:46Z

robodoo r+

- it manages the tokens privately - it tells you if your formula is valid or not - it replaces 2 classes: FormulaCellWithDependencies and RangeReferenceToken - it is required for the next commit: allow to create multiple formulas based the same formula with different parameters compiler.ts/`compile(formula: string)` --> `CompiledFormula.Compile(formula: string, sheetId: UID, getters: CoreGetters)` fomulas/helpers.ts/`getFunctionsFromTokens(...)` --> `compiledFormula.getFunctionsFromTokens(functionNames: string[], getters: CoreGetters)` removed `RangeCompiledFormula` --> `CompiledFormula` removed plugins/core/cell.ts/`ReferenceToken` class --> it is implemented in CompiledFormula` formulas/helper.ts/`isExportableToExcel` & `getFunctionsFromTokens` --> compiledFormula.`areAllFunctionsExportableToExcel` & `getFunctionsFromTokens` removed FormulaCellWithDependencies and ReferenceToken Task: 5489478 Part-of: #7756 Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com>

Today all cells have the property `content` that contain the raw content of the cell: either its value, its formula text or the markdown link. Having the content re-create the formula from its parts is costly, and having the convenience of defining the property `content` on an object, either through a dedicated class or though a closure is very expensive (100 MB on large models). This PR move the property `content` out of Cell, in `LiteralCell` so it is no longer available on FormulaCell. Update all the usages of `content` to be specific to LiteralCell (!FormulaCell). Mostly, content was used to check if a cell is empty. This check was `if(!cell?.content)` and becomes `if (!cell?.isFormula && !cell?.content)` Task: 5489478 Part-of: #7756 Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com>

Description: Reduce the JSON file size of spreadsheets by storing only changes to formulae and, in a second pass, collecting all the formulae or changes that are the same and storing them under one single key in the JSON file. importing sheets: ~2300ms --> ~680 ms 1560 MB ram --> 580 MB JSON Size: 40 MB -> 147 KB 4,4 MB -> 3KB 1,6 MB-> 43 KB ```cells: { A1: "My Text", A2: "=SUM(B2:B9)", A3: "=SUM(B2:B9)", ``` As we can see, A2 and A3 share the same formula. They can be rewritten as ```cells: { A1: "My Text", 'A2:A3': "=SUM(B2:B9)", ``` removing duplication completely Usually formula do not repeat exactly the same, but offset their dependencies (being either references to other cells, numbers or strings) slightly. We can rewrite: ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), A3: "=CONCAT(B3, $C$2, D3, "hello"), A4: "=CONCAT(B4, $C$2, D4, "hello"), ``` to ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), A3: { R : "R+1|=|R+1" }, A4: { R : "R+1|=|R+1" }, ... ``` now we see that A3 and A4 have the same transformation, so we can rewrite them to ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), "A3:A4": { R : "R+1|=|R+1" }, ``` We apply the same for number arguments and string arguments. A formula with all the arguments slightly changed might look like ```cells: { A1: "=MY_FORMULA(1, B1, "coucou"), A2: { R : "R+1", N: "+1", S: ["hello"] }, A3:A4: { R : "R+1", N: "+1"} ``` can be read as ```cells: { A1: "=MY_FORMULA(1, B1, "coucou"), A2: "=MY_FORMULA(2, B2, "hello"), A3: "=MY_FORMULA(3, B3, "hello"), A4: "=MY_FORMULA(4, B4, "hello"), ``` closes #7756 Task: 5489478 Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com>

- it manages the tokens privately - it tells you if your formula is valid or not - it replaces 2 classes: FormulaCellWithDependencies and RangeReferenceToken - it is required for the next commit: allow to create multiple formulas based the same formula with different parameters compiler.ts/`compile(formula: string)` --> `CompiledFormula.Compile(formula: string, sheetId: UID, getters: CoreGetters)` fomulas/helpers.ts/`getFunctionsFromTokens(...)` --> `compiledFormula.getFunctionsFromTokens(functionNames: string[], getters: CoreGetters)` removed `RangeCompiledFormula` --> `CompiledFormula` removed plugins/core/cell.ts/`ReferenceToken` class --> it is implemented in CompiledFormula` formulas/helper.ts/`isExportableToExcel` & `getFunctionsFromTokens` --> compiledFormula.`areAllFunctionsExportableToExcel` & `getFunctionsFromTokens` removed FormulaCellWithDependencies and ReferenceToken Task: 5489478 Part-of: #7756 Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com>

Today all cells have the property `content` that contain the raw content of the cell: either its value, its formula text or the markdown link. Having the content re-create the formula from its parts is costly, and having the convenience of defining the property `content` on an object, either through a dedicated class or though a closure is very expensive (100 MB on large models). This PR move the property `content` out of Cell, in `LiteralCell` so it is no longer available on FormulaCell. Update all the usages of `content` to be specific to LiteralCell (!FormulaCell). Mostly, content was used to check if a cell is empty. This check was `if(!cell?.content)` and becomes `if (!cell?.isFormula && !cell?.content)` Task: 5489478 Part-of: #7756 Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com>

Description: Reduce the JSON file size of spreadsheets by storing only changes to formulae and, in a second pass, collecting all the formulae or changes that are the same and storing them under one single key in the JSON file. importing sheets: ~2300ms --> ~680 ms 1560 MB ram --> 580 MB JSON Size: 40 MB -> 147 KB 4,4 MB -> 3KB 1,6 MB-> 43 KB ```cells: { A1: "My Text", A2: "=SUM(B2:B9)", A3: "=SUM(B2:B9)", ``` As we can see, A2 and A3 share the same formula. They can be rewritten as ```cells: { A1: "My Text", 'A2:A3': "=SUM(B2:B9)", ``` removing duplication completely Usually formula do not repeat exactly the same, but offset their dependencies (being either references to other cells, numbers or strings) slightly. We can rewrite: ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), A3: "=CONCAT(B3, $C$2, D3, "hello"), A4: "=CONCAT(B4, $C$2, D4, "hello"), ``` to ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), A3: { R : "R+1|=|R+1" }, A4: { R : "R+1|=|R+1" }, ... ``` now we see that A3 and A4 have the same transformation, so we can rewrite them to ```cells: { A1: "My Text", A2: "=CONCAT(B2, $C$2, D2, "hello"), "A3:A4": { R : "R+1|=|R+1" }, ``` We apply the same for number arguments and string arguments. A formula with all the arguments slightly changed might look like ```cells: { A1: "=MY_FORMULA(1, B1, "coucou"), A2: { R : "R+1", N: "+1", S: ["hello"] }, A3:A4: { R : "R+1", N: "+1"} ``` can be read as ```cells: { A1: "=MY_FORMULA(1, B1, "coucou"), A2: "=MY_FORMULA(2, B2, "hello"), A3: "=MY_FORMULA(3, B3, "hello"), A4: "=MY_FORMULA(4, B4, "hello"), ``` closes #7756 Task: 5489478 Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com>

VincentSchippefilt changed the base branch from 19.0 to master January 13, 2026 07:42

VincentSchippefilt commented Jan 13, 2026

View reviewed changes

Comment thread packages/o-spreadsheet-engine/src/formulas/tokenizer.ts Outdated

VincentSchippefilt commented Jan 13, 2026

View reviewed changes

Comment thread packages/o-spreadsheet-engine/src/helpers/misc.ts Outdated

VincentSchippefilt commented Jan 13, 2026

View reviewed changes

Comment thread packages/o-spreadsheet-engine/src/plugins/core/cell.ts Outdated

VincentSchippefilt force-pushed the master-squish-vsc branch 4 times, most recently from 7516ed3 to 489a7be Compare January 15, 2026 12:26

VincentSchippefilt marked this pull request as ready for review January 15, 2026 12:28

VincentSchippefilt force-pushed the master-squish-vsc branch from 489a7be to d6405b8 Compare January 15, 2026 12:31

VincentSchippefilt commented Jan 15, 2026

View reviewed changes

Comment thread tests/clipboard/clipboard_plugin.test.ts Outdated

pro-odoo reviewed Jan 15, 2026

View reviewed changes

Comment thread packages/o-spreadsheet-engine/src/formulas/compiler.ts Outdated

VincentSchippefilt force-pushed the master-squish-vsc branch from d6405b8 to e3bdd35 Compare January 15, 2026 14:31

LucasLefevre reviewed Jan 19, 2026

View reviewed changes

pro-odoo reviewed Jan 20, 2026

View reviewed changes

VincentSchippefilt force-pushed the master-squish-vsc branch 2 times, most recently from 3224c45 to 1f3038b Compare January 20, 2026 15:55

VincentSchippefilt changed the title ~~reduce json file size~~ [IMP] export: reduce JSON size Jan 20, 2026

hokolomopo reviewed Jan 21, 2026

View reviewed changes

LucasLefevre reviewed Jan 21, 2026

View reviewed changes

VincentSchippefilt force-pushed the master-squish-vsc branch 6 times, most recently from e1b7c7f to 0adc821 Compare January 28, 2026 13:49

VincentSchippefilt force-pushed the master-squish-vsc branch 3 times, most recently from 4791b75 to c277d63 Compare February 3, 2026 13:46

LucasLefevre reviewed Feb 12, 2026

View reviewed changes

VincentSchippefilt force-pushed the master-squish-vsc branch 6 times, most recently from 7c961d0 to 3313428 Compare February 19, 2026 08:09

rrahir approved these changes Feb 19, 2026

View reviewed changes

VincentSchippefilt added 3 commits February 19, 2026 14:41

VincentSchippefilt force-pushed the master-squish-vsc branch from 3313428 to a34c02c Compare February 19, 2026 13:42

robodoo temporarily deployed to merge February 19, 2026 14:41 Inactive

robodoo added the 19.2 label Feb 19, 2026

robodoo closed this Feb 19, 2026

fw-bot deleted the master-squish-vsc branch February 26, 2026 14:45

Conversation

VincentSchippefilt commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description:

API changes:

Observed performance gains on large dashboard:

How does it work:

Shared formula

Formula with small differences

Uh oh!

robodoo commented Jan 13, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LucasLefevre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pro-odoo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hokolomopo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

VincentSchippefilt commented Feb 11, 2026

Uh oh!

robodoo commented Feb 11, 2026

Uh oh!

LucasLefevre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

VincentSchippefilt commented Jan 13, 2026 •

edited

Loading