What command does write_bitcode execute?

[zcfh]

❓ Questions and Help

import gym
bench = make_benchmark("path/to/file.bc")
env2 = gym.make("llvm-autophase-ic-v0", benchmark=bench, reward_space="IrInstructionCountO3")
env2.reset()
env2.write_bitcode("path/to/out.bc")

For env that has not executed any action, call write_bitcode directly. The generated bitcode is different from the original bitcode. I want to know what happened during this period? Was opt executed again?
I set COMPILER_GYM_DEBUG=99. According to the log, it seems that the first time llvm::parseBitcodeFile(83068 bits) loaded the original bitcode, and then llvm::parseBitcodeFile(77648) was executed again without performing any operations. bitcode, here is the last written bitcode, but I haven’t found where it is called.

Additional Context

I1016 20:41:39.214298 14281 CompilerGymServiceImpl.h:192] AddBenchmark()
I1016 20:41:39.214318 14281 BenchmarkCache.cc:54] Cached benchmark benchmark://file-v0/home/relay/wuminghui03/workspace/ppo/ppo_compiler_gym_clang10/bzip2/build_ppo_to_obj/bzip2.c.ori.bc (83197 bytes). Cache size = 83197 bytes, 1 items
I1016 20:41:39.214601 14256 CompilerGymServiceImpl.h:60] StartSession(id=0, benchmark=benchmark://file-v0/home/xxx/workspace/ppo/ppo_compiler_gym_clang10/bzip2/build_ppo_to_obj/bzip2.c.ori.bc), 1 active sessions
I1016 20:41:39.214673 14256 BenchmarkFactory.cc:41] BenchmarkFactory initialized
I1016 20:41:39.214681 14256 BenchmarkFactory.cc:68] LLVM benchmark cache miss, add bitcode: benchmark://file-v0/home/xxx/workspace/ppo/ppo_compiler_gym_clang10/bzip2/build_ppo_to_obj/bzip2.c.ori.bc
I1016 20:41:39.214809 14256 Benchmark.cc:106] llvm::parseBitcodeFile(83068 bits)
I1016 20:41:47.993764 14256 BenchmarkFactory.cc:143] Cached LLVM benchmark: benchmark://file-v0/home/xxx/workspace/ppo/ppo_compiler_gym_clang10/bzip2/build_ppo_to_obj/bzip2.c.ori.bc. Cache size = 1 items
I1016 20:41:47.993835 14256 BenchmarkFactory.cc:59] LLVM benchmark cache hit: benchmark://file-v0/home/xxx/workspace/ppo/ppo_compiler_gym_clang10/bzip2/build_ppo_to_obj/bzip2.c.ori.bc
I1016 20:41:48.035897 14256 Benchmark.cc:106] llvm::parseBitcodeFile(77648 bits)

[zcfh]

I debuged it.
I debugged it
The second time seems to be because Benchmark::clone regenerated a bitcode instead of directly copying it. Why is this happening?

#0  compiler_gym::llvm_service::makeModule (context=..., bitcode=...,
    name="benchmark://file-v0/home/relay/workspace/ppo/ppo_compiler_gym_clang10/bzip2/build_ppo_to_obj/bzip2.c.ori.bc",
    status=0x7f888f49df48) at compiler_gym/envs/llvm/service/Benchmark.cc:109
#1  0x0000000000e0c2bc in compiler_gym::llvm_service::(anonymous namespace)::makeModuleOrDie (context=..., bitcode=...,
    name="benchmark://file-v0/home/relay/workspace/ppo/ppo_compiler_gym_clang10/bzip2/build_ppo_to_obj/bzip2.c.ori.bc")
    at compiler_gym/envs/llvm/service/Benchmark.cc:51
#2  0x0000000000e0c0fd in compiler_gym::llvm_service::Benchmark::Benchmark (this=0x7f8888227cc0,
    name="benchmark://file-v0/home/relay/workspace/ppo/ppo_compiler_gym_clang10/bzip2/build_ppo_to_obj/bzip2.c.ori.bc", bitcode=...,
    dynamicConfig=..., workingDirectory=..., baselineCosts=...)
    at compiler_gym/envs/llvm/service/Benchmark.cc:147
#3  0x0000000000e105fc in std::make_unique<compiler_gym::llvm_service::Benchmark, std::string const&, llvm::SmallString<0u>&, compiler_gym::BenchmarkDynamicConfig const&, boost::filesystem::path const&, std::array<double, 9ul> const&> (__args=..., __args=..., __args=..., __args=..., __args=...)
    at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/unique_ptr.h:831
#4  0x0000000000e0cc60 in compiler_gym::llvm_service::Benchmark::clone (this=0x7f8888217a20, workingDirectory=...)
    at compiler_gym/envs/llvm/service/Benchmark.cc:185
#5  0x0000000000db0845 in compiler_gym::llvm_service::BenchmarkFactory::getBenchmark (
    this=0x318e040 <compiler_gym::llvm_service::BenchmarkFactory::getSingleton(boost::filesystem::path const&, std::optional<std::mersenne_twister_engine<unsigned long, 64ul, 312ul, 156ul, 31ul, 13043109905998158313ul, 29ul, 6148914691236517205ul, 17ul, 8202884508482404352ul, 37ul, 18444473444759240704ul, 43ul, 6364136223846793005ul> >, unsigned long)::instance>, benchmarkMessage=..., benchmark=0x7f888f49f2b8)
    at compiler_gym/envs/llvm/service/BenchmarkFactory.cc:60
#6  0x0000000000db133d in compiler_gym::llvm_service::BenchmarkFactory::getBenchmark (
    this=0x318e040 <compiler_gym::llvm_service::BenchmarkFactory::getSingleton(boost::filesystem::path const&, std::optional<std::mersenne_twister_engine<unsigned long, 64ul, 312ul, 156ul, 31ul, 13043109905998158313ul, 29ul, 6148914691236517205ul, 17ul, 8202884508482404352ul, 37ul, 18444473444759240704ul, 43ul, 6364136223846793005ul> >, unsigned long)::instance>, benchmarkMessage=..., benchmark=0x7f888f49f2b8)
    at compiler_gym/envs/llvm/service/BenchmarkFactory.cc:94
#7  0x0000000000d6e2d0 in compiler_gym::llvm_service::LlvmSession::init (this=0x7f8888006d90, actionSpace=..., benchmark=...)
    at compiler_gym/envs/llvm/service/LlvmSession.cc:90
#8  0x0000000000d3e31c in compiler_gym::runtime::CompilerGymService<compiler_gym::llvm_service::LlvmSession>::StartSession (this=0x7ffc64ed7218,
    context=0x7f8888001800, request=0x7f888800ad30, reply=0x7f888f4a01b0)
    at compiler_gym/service/runtime/CompilerGymServiceImpl.h:82
#9  0x00000000025ba26c in compiler_gym::CompilerGymService::Service::Service()::$_2::operator()(compiler_gym::CompilerGymService::Service*, grpc::ServerContext*, compiler_gym::StartSessionRequest const*, compiler_gym::StartSessionReply*) const (this=0x3736928, service=0x7ffc64ed7218,
    ctx=0x7f8888001800, req=0x7f888800ad30, resp=0x7f888f4a01b0)

[ChrisCummins]

The second time seems to be because Benchmark::clone regenerated a bitcode instead of directly copying it. Why is this happening?

Hi @zcfh, that's a good question! Tbh it's a long time since I worked on this bit of the codebase and I can't remember. Perhaps it's to decouple the cloned version from the LLVM context?