[question] How to package a C++ library with all dependencies for non-Conan consumers

[sachatt]

What is your question?

Context

I am developing a C++ library (e.g. MyLib) that will be consumed by a React Native iOS application. The consumer cannot use Conan because React Native for iOS relies on CocoaPods/Swift Package Manager, which does not support Conan integration. Therefore, I need to create a fully self-contained package that includes all necessary headers and binaries.

The Problem

My library (MyLib) depends on third-party libraries (e.g., boost, nlohmann_json, etc.) and exposes their types in my public API headers in certain contexts. I need to distribute a pre-compiled binary package that includes:

• My library, MyLib, headers and binaries
• All necessary third-party headers and binaries used in MyLib's public API

In addition even if react supported conan I still rather package these third-party libs together to avoid any ABI mismatches that may not directly appear or even fail, and avoid doing something like adding a package_info for requirements.

Example of the concern:

// MyLib.h (Public Header)
#pragma once
#include <fmt/core.h>

namespace mylib {
    struct LogContext {
        fmt::memory_buffer buffer;  // Size/alignment depends on how fmt was compiled
    };

    void process(LogContext& ctx);
}

If the consumer were using Conan, they might rebuild fmt with different compiler flags, causing ABI mismatches with my pre-compiled binary. Since memory layout of fmt::memory_buffer could vary based on compilation options, this would lead to undefined behavior.

However, since my consumer cannot use Conan at all, I need to bundle everything together into a self-contained package.

Attempted Solutions

Option 1: Using conan deploy with custom deployer

1. Created a separate conanfile.py with only MyLib as a requirement

from pathlib import Path
from conan import ConanFile


class DeployConan(ConanFile):
    settings = "os", "compiler", "build_type", "arch"

    def set_version(self):
        # Read version from parent directory's VERSION file
        version_file = Path(self.recipe_folder).parent / "VERSION"
        self.my_lib_version = version_file.read_text().strip()

    def requirements(self):
        self.requires(f"MyLib/{self.my_lib_version}")

2. This allowed Conan to resolve and fetch my library within the final output as well as all transitive dependencies
3. Implemented a custom deploy.py script to selectively copy only the dependencies exposed in public headers

from pathlib import Path
import shutil

PACKAGES_TO_DEPLOY = [
    "MyLib",
    "boost",
    "range-v3",
    "fmt",
    "nlohmann_json",
]

def deploy(graph, output_folder, **kwargs):
    """Custom deployer that only copies specified packages."""
    output_path = Path(output_folder)
    deployed = set()

    # Use graph.nodes to access ALL packages (including hidden/transitive)
    for node in graph.nodes:
        if node.ref is None:
            continue

        pkg_folder = node.conanfile.package_folder
        if pkg_folder is None:
            continue

        pkg_name = node.ref.name

        if pkg_name in PACKAGES_TO_DEPLOY:
            src = Path(pkg_folder)
            dest = output_path / pkg_name

            if dest.exists():
                shutil.rmtree(dest)

            shutil.copytree(src, dest)
            deployed.add(pkg_name)
            print(f"Deployed: {pkg_name} -> {dest}")

    # Verify all required packages were deployed
    missing = set(PACKAGES_TO_DEPLOY) - deployed
    if missing:
        raise RuntimeError(f"Failed to deploy packages (not found in graph): {', '.join(sorted(missing))}")

4. Deploy specifying275 the custom deploy.py script shown above with a local folder

5. Problem: This approach did not include header-only libraries, and we couldn't find a clean (non-hacky) way to handle them. It would however include properly all build libraries that were not header only.

Option 2: Modify MyLib's conanfile.py package method

1. Instead of using conan deploy, modify MyLib's own conanfile.py
2. In the package() method, copy third-party headers and libraries directly

    def package(self):
        cmake = CMake(self)
        cmake.install()
        self._copy_third_party_headers()
        self._copy_third_party_libraries()

3. Advantage: In this context, we have direct access to dependency information
4. Question: Is this the recommended approach?

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @sachatt

Thanks for your question.

If the consumer were using Conan, they might rebuild fmt with different compiler flags, causing ABI mismatches with my pre-compiled binary. Since memory layout of fmt::memory_buffer could vary based on compilation options, this would lead to undefined behavior.

That is up to the consumer.
In general, injecting arbitrary compiler flags that change the ABI without a proper custom settings model is not recommended. And when defining a custom new setting, then it will force a rebuild of your "mylib" too. In general the binary model of Conan is flexible and powerful enough to model down to the detail that the user wants, but it is always a tradeoff.
So that shouldn't be a big concern in practice, unless you want to distribute mylib only as binary and closed source, and that wouldn't allow the consumers to rebuild mylib from source if they want.

In addition even if react supported conan I still rather package these third-party libs together to avoid any ABI mismatches that may not directly appear or even fail, and avoid doing something like adding a package_info for requirements.

Still, creating bundles of binaries that include dependencies binaries too when desired is a valid and common use case. Conan is mainly a "development time" dependency and package management, but for final deployment, there are often other mechanisms.

Have you considered using the vendor=True approach?
At the end of the day, you want to create a bundle that includes mylib and the dependencies, and such a bundle to be deployed in the react app might also be versioned and conveniently managed as a Conan package itself.
See:

• https://blog.conan.io/2024/07/09/Introducing-vendoring-packages.html
• https://docs.conan.io/2/devops/vendoring.html

then, the same recipe that you use for vendor=True and collecting the dependencies binaries and re-packaging (copying them), can easily use a deploy() method to install things.

So probably if "mylib" is a regular C++ library, then its recipe and package should probably be agnostic of the deployment details, and that responsibility should be moved to other place. So option 2 seems a bit less idiomatic to me.

The separate deploy deployer might make a bit more sense if it can be reused for deploying other applications, but if it is just mainly one, maybe option 1, with a separate conanfile.py that vendors and re-package mylib and its transitive dependencies for the final deployment could make more sense.

Please let me know what you think.

[sachatt]

@memsharded thank you!

So that shouldn't be a big concern in practice, unless you want to distribute mylib only as binary and closed source, and that wouldn't allow the consumers to rebuild mylib from source if they wan

I understand what you mean, although my concern comes from past experience.
There was a case where I changed a compiler-related setting in one of my Conan dependencies and Conan didn’t fully detect the change in its binary ID. The result was that the package didn’t rebuild correctly until I manually cleared the local cache and forced a rebuild + publish. Unfortunately I don’t remember the exact configuration that triggered it, but it left me feeling a bit uneasy about depending on consumers to always rebuild dependencies with perfectly matching settings. My understanding is that the “best practice” approaches here are typically:

• avoid exposing third-party types through public headers, or

• ensure everything is rebuilt together so ABI is always aligned, or

• use purely header-only libraries.

I might be mistaken, and I realize the proper solution in my old issue would have been to fix the recipe so the settings were correctly captured. But it’s part of why I prefer bundling all required libraries together.

Have you considered using the vendor=True approach?

I actually didn’t know this existed — thank you for pointing it out.
In our case everything is packaged internally, so visibility of third-party dependencies isn’t a problem. What I do want to avoid is having consumers rebuild those dependencies externally. The consuming project is a simple React Native library, and we’ve had issues keeping its third-party environment stable, so we try to keep that side as simple as possible. Vendoring seems like it might be exactly what we need.

One question:
If I use vendor=True, do the vendored dependencies still remain usable by the consumer (e.g., if they want to include <boost/...>), and the feature simply bundles everything into a single package? Something like:

from conan import ConanFile
from pathlib import Path

class MyLibBundle(ConanFile):
    name = "MyLibBundle"
    version = "1.0"
    settings = "os", "compiler", "build_type", "arch"

    def requirements(self):
        version_file = Path(self.recipe_folder).parent / "VERSION"
        my_version = version_file.read_text().strip()
        self.requires(f"MyLib/{my_version}", transitive=True)

    vendor = True

If this approach still exposes the libraries but just packages them together in a stable way, that would fit our use case very well.

Let me know if I’m understanding it correctly.

[memsharded]

There was a case where I changed a compiler-related setting in one of my Conan dependencies and Conan didn’t fully detect the change in its binary ID. The result was that the package didn’t rebuild correctly until I manually cleared the local cache and forced a rebuild + publish. Unfortunately I don’t remember the exact configuration that triggered it, but it left me feeling a bit uneasy about depending on consumers to always rebuild dependencies with perfectly matching settings. My understanding is that the “best practice” approaches here are typically:

Was this in Conan 1, or using Conan 2? Conan 2 has a way more restricted policy for rebuilding, requires rebuilding binaries way more often and in more cases than Conan 1.
Still in Conan 2, it is necessary to do the right version bumping, for example, if modifying the public headers it is necessary to bump the version, see for example https://docs.conan.io/2/devops/versioning/default.html

Also if the change of the setting happened exclusively for the dependency, but not for the consumer, then it is true, it might not require the rebuilding of the consumer. For example if the consumer is a static library, and the public interface of the dependency doesn't change at all, then it is typically not necessary to re-build that consumer static library, I don't know if it could be one edge case around this.

This is why I was suggesting that the default binary model is good for a vast majority of cases, but it is still the consumers responsibility to know what they are doing, like changing only one package settings with mypkg/*:mysetting=mynewvalue might not be enough to trigger the build of some consumers, and then it is necessary to be aware of the binary implications of such definition.

In our case everything is packaged internally, so visibility of third-party dependencies isn’t a problem. What I do want to avoid is having consumers rebuild those dependencies externally. The consuming project is a simple React Native library, and we’ve had issues keeping its third-party environment stable, so we try to keep that side as simple as possible. Vendoring seems like it might be exactly what we need.

Yes, vendoring packages do not take into account their dependencies, or the settings of its dependencies. As long as a binary for that package for that configuration exists, it is possible to install it without even expanding the dependency graph. But it is still convenient for managing new versions/releases of it, having them in a structured way in the server, etc.

If I use vendor=True, do the vendored dependencies still remain usable by the consumer (e.g., if they want to include <boost/...>), and the feature simply bundles everything into a single package? Something like:

Yes, exactly. It is the responsibility of the vendor=True package to decide what it wants to repackage, in some cases it will be all headers and static libs, in other cases it might be only the shared runtime libs of dependencies, depending on the use case. The docs and blog post have examples about this. The idea is that the recipe is a very convenient way to create new releases of the bundle, by triggering a new conan create + conan upload of that package, so it will expand the graph and collect and re-package everything it wants from the dependencies, but when using it as conan install --requires=mybundle/version it will not expand the dependencies at all, it will just retrieve that binary and deploy it if you want, with full isolation of the dependencies.

[sachatt]

Thanks again for the detailed clarification, I just want to restate my understanding to make sure I’m following correctly.

Was this in Conan 1 or Conan 2? Conan 2 has much stricter rebuild rules…

I was using Conan 2. I unfortunately don’t remember the exact option that caused the issue, but it was a build flag added to one of the dependencies that didn’t trigger a rebuild, and I had to manually clear the cache to force it. I completely agree that if the change only affects the consumer it shouldn’t require a rebuild; my concern was more about cases where a dependency exposes non–header-only third-party types in its public API. If those types depend on build flags or configuration that isn’t fully captured by the recipe, there’s always a small risk of layout mismatches. Thus we either need to hide those third party dependencies within our cpp or allow a full rebuild so we assure a proper match.

It is the responsibility of the vendor=True package…

Perfect, thank you. I followed the docs and implemented vendor=True directly inside mylib’s own recipe. Here’s the approach I ended up with:

class MyLibConan(ConanFile):
    name = "myLib"
    ...
    vendor = True  # NEW: bundle all dependencies into this package

    def _package_deps(self, package_name, include_dest, lib_dest):
        package = self.dependencies[package_name]
        if package.cpp_info.includedir:
            copy(self, "*", src=package.cpp_info.includedir, dst=include_dest)

        if package.cpp_info.libdirs:
            copy(self, "*.a",   src=package.cpp_info.libdir, dst=lib_dest)
            copy(self, "*.lib", src=package.cpp_info.libdir, dst=lib_dest)

    def package(self):
        lib_dest = os.path.join(self.package_folder, "lib")
        include_dest = os.path.join(self.package_folder, "include")
        os.makedirs(lib_dest, exist_ok=True)
        os.makedirs(include_dest, exist_ok=True)

        deps = ["nlohmann_json", "range-v3", "boost", "fmt"]
        for dep in deps:
            self._package_deps(dep, include_dest, lib_dest) # New to manually package requirements

        cmake = CMake(self)
        cmake.install()

    def package_info(self):
        self.cpp_info.includedirs = ["include"]
        self.cpp_info.libdirs = ["lib"]
        self.cpp_info.libs = ["myLib", "fmt", "boost"]  # New to specify what libs we depend on but due to vendor its assumed they are packaged

From what I understand, this ends up being a cleaner version of my earlier “option 2.” Because of vendor=True, these dependencies won’t be resolved separately at install time ,they must already be in the package, which is exactly what we want.

…with a separate conanfile.py that vendors and repackages mylib…

Right. After bundling everything into myLib itself, I can still create a lightweight deployer recipe for environments that do not support Conan directly:

from pathlib import Path
from conan import ConanFile

class DeployConan(ConanFile):
    settings = "os", "compiler", "build_type", "arch"

    def set_version(self):
        version_file = Path(self.recipe_folder).parent / "VERSION"
        self.version = version_file.read_text().strip()

    def requirements(self):
        self.requires(f"myLib/{self.version}")

Running a deploy with this simply extracts the fully self-contained myLib package, which already includes all the third-party headers and libraries, and makes it ready for use in environments like React Native iOS that don’t support Conan integration.

[memsharded]

I unfortunately don’t remember the exact option that caused the issue, but it was a build flag added to one of the dependencies that didn’t trigger a rebuild, and I had to manually clear the cache to force it. I completely agree that if the change only affects the consumer it shouldn’t require a rebuild; my concern was more about cases where a dependency exposes non–header-only third-party types in its public API. If those types depend on build flags or configuration that isn’t fully captured by the recipe, there’s always a small risk of layout mismatches. Thus we either need to hide those third party dependencies within our cpp or allow a full rebuild so we assure a proper match.

For the case of package_type = "header-library" the package id mode is always "embed mode". That means that the consumers of the header only library are always affected down to the package-id of the consumer, no matter what changes in the header library, it will always force the re-build of the consumers.
The only requirement is that it has to be a "modeled" change, like a custom setting, option, version bump, etc. Changing a compiler flag is not enough, because that is not part of the default binary model directly, because there are many compiler flags (such as -Wxxx that don't affect the binary model), and this is why those compiler flags changes are recommended to work together with custom settings, or at least with tools.info.package_id:confs = ["tools.build:cxxflags"] (not very clean and recommended, but possible) to make it part of the binary model.

From what I understand, this ends up being a cleaner version of my earlier “option 2.” Because of vendor=True, these dependencies won’t be resolved separately at install time ,they must already be in the package, which is exactly what we want.

Ok, yes, I was a bit confused with the steps and the fact that Option 2 also needed a deployer later. But yes, it is true that vendor=True is closer to your initial option 2, sounds good

Right. After bundling everything into myLib itself, I can still create a lightweight deployer recipe for environments that do not support Conan directly:

Maybe with the:

conan install --requires=mylib/version --deployer=full_deploy

or something in that line, you don't even need a consumer recipe. It can be convenient to automate your VERSION reading from filesystem, but otherwise, it wouldn't be necessary

Running a deploy with this simply extracts the fully self-contained myLib package, which already includes all the third-party headers and libraries, and makes it ready for use in environments like React Native iOS that don’t support Conan integration.

Yes, sounds good. Also adding a deploy() method to your mylib package sounds valid too.

[sachatt]

@memsharded thanks so much, this seems to have worked and fixed our issues following that exact approach! Much appreciated

[memsharded]

Great, always happy to help, thanks to you for the feedback!