Why ObjectTextSizeOz is bigger than ObjectTextSizeO0 ？

[sparks-code]

❓ Questions and Help

I tested the code size in relation to the platform and this is my code. My compilergym version is 0.25 and the benchmark i used is cbench.

cbench = env1.datasets["benchmark://cbench-v1"]
cbench_list = list(cbench.benchmark_uris())
for bench in cbench_list:
    with compiler_gym.make("llvm-autophase-codesize-v0") as env:  
        env.reset(benchmark = bench)
        print(env.benchmark)
        print(f'''ObjectTextSize = {env.observation.ObjectTextSizeBytes()},
        ObjectTextSizeO0 = {env.observation.ObjectTextSizeO0()},
        ObjectTextSizeO3= {env.observation.ObjectTextSizeO3()},
        ObjectTextSizeOz = {env.observation.ObjectTextSizeOz()},
        ''')

this is result

I want to know why ObjectTextSizeOz is bigger than ObjectTextSizeO0 ？
I optimize adpcm program with -O3 and -Oz in the command line

~/.local/share/compiler_gym/llvm-v0/bin/clang -O0 -c adpcm.bc -o adpcm_O0.o
~/.local/share/compiler_gym/llvm-v0/bin/clang -Oz -c adpcm.bc -o adpcm_Oz.o
~/.local/share/compiler_gym/llvm-v0/bin/clang -O3 -c adpcm.bc -o adpcm_O3.o

and the result is

why the result is different from compilergym？Is this normal?

Additional Context

[sparks-code]

benchmark://npb-v0/1
ObjectTextSize = 1833,
        ObjectTextSizeO0 = 1833,
        ObjectTextSizeO3= 1849,
        ObjectTextSizeOz = 1849,
        
benchmark://npb-v0/2
ObjectTextSize = 3204,
        ObjectTextSizeO0 = 3204,
        ObjectTextSizeO3= 4751,
        ObjectTextSizeOz = 3833,
        
benchmark://npb-v0/3
ObjectTextSize = 2644,
        ObjectTextSizeO0 = 2644,
        ObjectTextSizeO3= 3031,
        ObjectTextSizeOz = 2735,
        
benchmark://npb-v0/4
ObjectTextSize = 4531,
        ObjectTextSizeO0 = 4531,
        ObjectTextSizeO3= 4973,
        ObjectTextSizeOz = 4341,
        
benchmark://npb-v0/5
ObjectTextSize = 4279,
        ObjectTextSizeO0 = 4279,
        ObjectTextSizeO3= 6679,
        ObjectTextSizeOz = 6262,
        
benchmark://npb-v0/6
ObjectTextSize = 4796,
        ObjectTextSizeO0 = 4796,
        ObjectTextSizeO3= 6755,
        ObjectTextSizeOz = 6737,
        
benchmark://npb-v0/7
ObjectTextSize = 665,
        ObjectTextSizeO0 = 665,
        ObjectTextSizeO3= 1576,
        ObjectTextSizeOz = 1692,
        
benchmark://npb-v0/8
ObjectTextSize = 5237,
        ObjectTextSizeO0 = 5237,
        ObjectTextSizeO3= 4350,
        ObjectTextSizeOz = 3921,
        
benchmark://npb-v0/9
ObjectTextSize = 603,
        ObjectTextSizeO0 = 603,
        ObjectTextSizeO3= 1328,
        ObjectTextSizeOz = 658,
        
benchmark://npb-v0/10
ObjectTextSize = 170,
        ObjectTextSizeO0 = 170,
        ObjectTextSizeO3= 189,
        ObjectTextSizeOz = 189,
        
benchmark://npb-v0/11
ObjectTextSize = 2640,
        ObjectTextSizeO0 = 2640,
        ObjectTextSizeO3= 5382,
        ObjectTextSizeOz = 3384,
        
benchmark://npb-v0/12
ObjectTextSize = 16140,
        ObjectTextSizeO0 = 16140,
        ObjectTextSizeO3= 30611,
        ObjectTextSizeOz = 26386,
        
benchmark://npb-v0/13
ObjectTextSize = 4423,
        ObjectTextSizeO0 = 4423,
        ObjectTextSizeO3= 24981,
        ObjectTextSizeOz = 15896,
        
benchmark://npb-v0/14
ObjectTextSize = 1809,
        ObjectTextSizeO0 = 1809,
        ObjectTextSizeO3= 1738,
        ObjectTextSizeOz = 1738,
        
benchmark://npb-v0/15
ObjectTextSize = 398,
        ObjectTextSizeO0 = 398,
        ObjectTextSizeO3= 330,
        ObjectTextSizeOz = 378,
        
benchmark://npb-v0/16
ObjectTextSize = 97,
        ObjectTextSizeO0 = 97,
        ObjectTextSizeO3= 112,
        ObjectTextSizeOz = 112,
        
benchmark://npb-v0/17
ObjectTextSize = 4279,
        ObjectTextSizeO0 = 4279,
        ObjectTextSizeO3= 6679,
        ObjectTextSizeOz = 6262,
        
benchmark://npb-v0/18
ObjectTextSize = 1148,
        ObjectTextSizeO0 = 1148,
        ObjectTextSizeO3= 4132,
        ObjectTextSizeOz = 3051,
        
benchmark://npb-v0/19
ObjectTextSize = 13025,
        ObjectTextSizeO0 = 13025,
        ObjectTextSizeO3= 12016,
        ObjectTextSizeOz = 10906,
        
benchmark://npb-v0/20
ObjectTextSize = 170,
        ObjectTextSizeO0 = 170,
        ObjectTextSizeO3= 189,
        ObjectTextSizeOz = 189,
        
benchmark://npb-v0/21
ObjectTextSize = 398,
        ObjectTextSizeO0 = 398,
        ObjectTextSizeO3= 330,
        ObjectTextSizeOz = 378,
        
benchmark://npb-v0/22
ObjectTextSize = 8268,
        ObjectTextSizeO0 = 8268,
        ObjectTextSizeO3= 11602,
        ObjectTextSizeOz = 10180,
        
benchmark://npb-v0/23
ObjectTextSize = 911,
        ObjectTextSizeO0 = 911,
        ObjectTextSizeO3= 646,
        ObjectTextSizeOz = 644,
        
benchmark://npb-v0/24
ObjectTextSize = 170,
        ObjectTextSizeO0 = 170,
        ObjectTextSizeO3= 189,
        ObjectTextSizeOz = 189,
        
benchmark://npb-v0/25
ObjectTextSize = 1833,
        ObjectTextSizeO0 = 1833,
        ObjectTextSizeO3= 1849,
        ObjectTextSizeOz = 1849,
        
benchmark://npb-v0/26
ObjectTextSize = 1809,
        ObjectTextSizeO0 = 1809,
        ObjectTextSizeO3= 1738,
        ObjectTextSizeOz = 1738,
        
benchmark://npb-v0/27
ObjectTextSize = 398,
        ObjectTextSizeO0 = 398,
        ObjectTextSizeO3= 330,
        ObjectTextSizeOz = 378,
        
benchmark://npb-v0/28
ObjectTextSize = 3884,
        ObjectTextSizeO0 = 3884,
        ObjectTextSizeO3= 5680,
        ObjectTextSizeOz = 5133,
        
benchmark://npb-v0/29
ObjectTextSize = 11026,
        ObjectTextSizeO0 = 11026,
        ObjectTextSizeO3= 14282,
        ObjectTextSizeOz = 12264,
        
benchmark://npb-v0/30
ObjectTextSize = 17889,
        ObjectTextSizeO0 = 17889,
        ObjectTextSizeO3= 23847,
        ObjectTextSizeOz = 21185,
        
benchmark://npb-v0/31
ObjectTextSize = 170,
        ObjectTextSizeO0 = 170,
        ObjectTextSizeO3= 189,
        ObjectTextSizeOz = 189,
        
benchmark://npb-v0/32
ObjectTextSize = 398,
        ObjectTextSizeO0 = 398,
        ObjectTextSizeO3= 330,
        ObjectTextSizeOz = 378,
        
benchmark://npb-v0/33
ObjectTextSize = 915,
        ObjectTextSizeO0 = 915,
        ObjectTextSizeO3= 712,
        ObjectTextSizeOz = 664,
        
benchmark://npb-v0/34
ObjectTextSize = 13357,
        ObjectTextSizeO0 = 13357,
        ObjectTextSizeO3= 9921,
        ObjectTextSizeOz = 9208,
        
benchmark://npb-v0/35
ObjectTextSize = 3647,
        ObjectTextSizeO0 = 3647,
        ObjectTextSizeO3= 3256,
        ObjectTextSizeOz = 3256,
        
benchmark://npb-v0/36
ObjectTextSize = 1133,
        ObjectTextSizeO0 = 1133,
        ObjectTextSizeO3= 4096,
        ObjectTextSizeOz = 3036,
        
benchmark://npb-v0/37
ObjectTextSize = 561,
        ObjectTextSizeO0 = 561,
        ObjectTextSizeO3= 1136,
        ObjectTextSizeOz = 566,
        
benchmark://npb-v0/38
ObjectTextSize = 4134,
        ObjectTextSizeO0 = 4134,
        ObjectTextSizeO3= 8208,
        ObjectTextSizeOz = 7100,
        
benchmark://npb-v0/39
ObjectTextSize = 14460,
        ObjectTextSizeO0 = 14460,
        ObjectTextSizeO3= 24136,
        ObjectTextSizeOz = 22412,
        
benchmark://npb-v0/40
ObjectTextSize = 1355,
        ObjectTextSizeO0 = 1355,
        ObjectTextSizeO3= 1082,
        ObjectTextSizeOz = 1048,
        
benchmark://npb-v0/41
ObjectTextSize = 170,
        ObjectTextSizeO0 = 170,
        ObjectTextSizeO3= 189,
        ObjectTextSizeOz = 189,
        
benchmark://npb-v0/42
ObjectTextSize = 18330,
        ObjectTextSizeO0 = 18330,
        ObjectTextSizeO3= 11743,
        ObjectTextSizeOz = 11560,
        
benchmark://npb-v0/43
ObjectTextSize = 1803,
        ObjectTextSizeO0 = 1803,
        ObjectTextSizeO3= 1934,
        ObjectTextSizeOz = 1839,
        
benchmark://npb-v0/44
ObjectTextSize = 1683,
        ObjectTextSizeO0 = 1683,
        ObjectTextSizeO3= 1580,
        ObjectTextSizeOz = 1580,
        
benchmark://npb-v0/45
ObjectTextSize = 943,
        ObjectTextSizeO0 = 943,
        ObjectTextSizeO3= 3125,
        ObjectTextSizeOz = 2647,
        
benchmark://npb-v0/46
ObjectTextSize = 832,
        ObjectTextSizeO0 = 832,
        ObjectTextSizeO3= 798,
        ObjectTextSizeOz = 798,
        
benchmark://npb-v0/47
ObjectTextSize = 3749,
        ObjectTextSizeO0 = 3749,
        ObjectTextSizeO3= 6345,
        ObjectTextSizeOz = 6068,
        
benchmark://npb-v0/48
ObjectTextSize = 321,
        ObjectTextSizeO0 = 321,
        ObjectTextSizeO3= 260,
        ObjectTextSizeOz = 260,
        
benchmark://npb-v0/49
ObjectTextSize = 911,
        ObjectTextSizeO0 = 911,
        ObjectTextSizeO3= 646,
        ObjectTextSizeOz = 644,
        
benchmark://npb-v0/50
ObjectTextSize = 13047,
        ObjectTextSizeO0 = 13047,
        ObjectTextSizeO3= 12093,
        ObjectTextSizeOz = 10991

In addition to Cbench, a similar situation occurs with many benchmarks in NPB

[sparks-code]

@Celebio @hughleat @ryanrussell @colesbury @ChrisCummins

[sparks-code]

On the other hand, I use 2 ways to test ,
1.
~/.local/share/compiler_gym/llvm-v0/bin/opt adpcm.bc -O0 -o adpcm_opt_O0.bc
~/.local/share/compiler_gym/llvm-v0/bin/clang -c adpcm_opt_O0.bc -o adpcm_opt_O0.o
~/.local/share/compiler_gym/llvm-v0/bin/llvm-size adpcm_opt_O0.o
text data bss dec hex filename
1901 432 0 2333 91d adpcm_opt_O0.o

~/.local/share/compiler_gym/llvm-v0/bin/opt adpcm.bc -Oz -o adpcm_opt_Oz.bc
~/.local/share/compiler_gym/llvm-v0/bin/clang -c adpcm_opt_Oz.bc -o adpcm_opt_Oz.o
~/.local/share/compiler_gym/llvm-v0/bin/llvm-size adpcm_opt_Oz.o
text data bss dec hex filename
2386 0 0 2386 952 adpcm_opt_Oz.o

~/.local/share/compiler_gym/llvm-v0/bin/opt adpcm.bc -O3 -o adpcm_opt_O3.bc
~/.local/share/compiler_gym/llvm-v0/bin/clang -c adpcm_opt_O3.bc -o adpcm_opt_O3.o
~/.local/share/compiler_gym/llvm-v0/bin/llvm-size adpcm_opt_O3.o
text data bss dec hex filename
2725 0 0 2725 aa5 adpcm_opt_O3.o

[sparks-code]

~/.local/share/compiler_gym/llvm-v0/bin/clang -c -O0 adpcm.bc -o adpcm_O0.o
~/.local/share/compiler_gym/llvm-v0/bin/llvm-size adpcm_O0.o
text data bss dec hex filename
1901 432 0 2333 91d adpcm_O0.o

~/.local/share/compiler_gym/llvm-v0/bin/clang -c -Oz adpcm.bc -o adpcm_Oz.o
~/.local/share/compiler_gym/llvm-v0/bin/llvm-size adpcm_Oz.o
text data bss dec hex filename
1629 0 0 1629 65d adpcm_Oz.o

~/.local/share/compiler_gym/llvm-v0/bin/clang -c -O3 adpcm.bc -o adpcm_O3.o
~/.local/share/compiler_gym/llvm-v0/bin/llvm-size adpcm_O3.o
text data bss dec hex filename
1648 0 0 1648 670 adpcm_O3.o
Why are the text section sizes of these two methods so different?

[yingluosanqian]

I optimize adpcm program with -O3 and -Oz in the command line

~/.local/share/compiler_gym/llvm-v0/bin/clang -O0 -c adpcm.bc -o adpcm_O0.o
~/.local/share/compiler_gym/llvm-v0/bin/clang -Oz -c adpcm.bc -o adpcm_Oz.o
~/.local/share/compiler_gym/llvm-v0/bin/clang -O3 -c adpcm.bc -o adpcm_O3.o

why the result is different from compilergym？Is this normal?

IIUC, compiler_gym behave like below, this may explain why these two results are different.
(See 179 line in this file CompilerGym/compiler_gym/envs/llvm/llvm_benchmark.py for details)

clang -O1 -Xclang -disable-llvm-passes -Xclang -disable-llvm-optzns -c adpcm.bc
opt -O0 -c adpcm.bc -o adpcm_O0.bc
opt -Oz -c adpcm.bc -o adpcm_Oz.bc
opt -O3 -c adpcm.bc -o adpcm_O3.bc

However, it also confused me that ObjectTextSizeOz > ObjectTextSizeO0. I don't know what happend...

[ChrisCummins]

Yes @yingluosanqian you are correct, the LlvmEnv first uses clang to lower to a bitcode, then uses opt to do the actual optimization. Unfortunately, doing this does not get the same results as running a single clang invocation. This is kind of frustrating, but I don't think there is a simple workaround, since clang doesn't expose pass-ordering flags. It'd require a fairly substantial change to clang to expose the action space. Bottom line, these janky results are sadly unavoidable :(

Cheers,
Chris