[question] How to use a editable dependency that uses a different profile to the main consumer

[KingKiller100]

What is your question?

Conan: 1.66.0
OS: Windows 11

Hi, I am having trouble when installing a consumer package that uses a dependency that is built with a different profile (namely settings.compiler).

My situation is that the dependency built with a different profile than the consumer outputs a DLL. The DLL exposes C API to communicate, so there are no ABI issues between the DLL user and its internals. Because the DLL needs to work on Windows XP and newer environments and VS ended XP support with the v141_xp msvc toolset, v141_xp is the latest compiler the project can use and have designed a build profile to match this requirement. Some consumer projects for this DLL package only run on newer Windows environments so they can use new compilers and we've written profiles for this.

We designed the DLL package's recipe to accommodate these requirements. This can be seen below when the compiler and requirements are removed from consideration when calculating the package ID. When using a real package from the cache, this works as expected. However, when using this project as an editable, the consumers that use different profiles fail when installing. I'm not sure why this is an issue for the editable case.

Can the issue be explained and is there a solution for this issue without any major changes needed for the DLL package

DLL package's profile

[settings]
os=Windows
os_build=Windows
arch=x86
arch_build=x86_64
compiler=msvc
compiler.version=191
compiler.toolset=v141_xp
build_type=Debug
compiler.runtime=static
compiler.runtime_type=Debug
compiler.cppstd=17
[options]
[conf]
tools.microsoft.msbuild:verbosity=Normal
[build_requires]
[env]
CONAN_DISABLE_STRICT_MODE=1

Dll package's recipe

from conan import ConanFile
from conan.tools.microsoft import MSBuild, MSBuildToolchain, MSBuildDeps, vs_layout, check_min_vs
from conans.client.tools import latest_vs_version_installed
from conan.tools.files import copy
from conan.tools.env import Environment
import os 

required_conan_version = ">=1.62.0"
class DllPackage(ConanFile):
...
	def configure(self):
		if self.settings.get_safe("compiler") == "msvc":
			if self.conf.get("tools.microsoft.msbuild:vs_version", check_type=str) == None:
				vs_version = latest_vs_version_installed(self.output)
				self.conf["tools.microsoft.msbuild:vs_version"] = vs_version
		self.options["boost"].shared = "False"

	def requirements(self):
		self.requires("boost/1.81.0@#9bd7ae86a881631b6cc76590621e470b", private=True)
		self.requires("minizip/1.2.11@#08bf40c8f7adc31a842a76f60ec3b5a7", private=True)
		self.requires("zlib/1.2.13@#e377bee636333ae348d51ca90874e353", override=True)
		self.requires("bzip2/1.0.8#411fc05e80d47a89045edc1ee6f23c1d", override=True)

	def build_requirements(self):
		self.test_requires("doctest/2.4.11@#a4211dfc329a16ba9f280f9574025659")

	def package_id(self):
		# dll with a c interface, doesn't matter what it's compiled with or depends on
		del self.info.settings.compiler
		self.info.requires.unrelated_mode()

	def package_info(self):
		# c interface, no libs
		if self.in_local_cache:
			self.cpp_info.bindirs = []

		self.cpp_info.libdirs = []
		self.cpp_info.libs = []
....

Example consumer profile

[settings]
os=Windows
os_build=Windows
arch=x86
arch_build=x86_64
build_type=Debug
compiler=msvc
compiler.cppstd=20
compiler.version=193
compiler.runtime=static
compiler.runtime_type=Debug
[options]
[conf]
tools.microsoft.msbuild:verbosity=Normal
[build_requires]
[env]
CONAN_DISABLE_STRICT_MODE=1

Error message

Cross-build from 'Windows:x86_64' to 'Windows:x86'
Installing (downloading, building) binaries...
ERROR: Missing binary: libb64/1.2.1+3@inseinc/stable:511f262523f082a841eca3ffbc4dded6a164cb00
ERROR: Missing binary: minizip/1.2.11:e2a2958b12da4f7b59aa733a4a285c00354de1e3

libb64/1.2.1+3@inseinc/stable: WARN: Can't find a 'libb64/1.2.1+3@inseinc/stable' package for the specified settings, options and dependencies:
- Settings: arch=x86, build_type=Debug, compiler=msvc, compiler.cppstd=20, compiler.runtime=static, compiler.runtime_type=Debug, compiler.version=193, os=Windows
- Options:
- Dependencies:
- Requirements:
- Package ID: 511f262523f082a841eca3ffbc4dded6a164cb00

ERROR: Missing prebuilt package for 'libb64/1.2.1+3@inseinc/stable', 'minizip/1.2.11'
Use 'conan search libb64/1.2.1+3@inseinc/stable --table=table.html -r=remote' and open the table.html file to see available packages
Or try to build locally from sources with '--build=libb64 --build=minizip'

More Info at 'https://docs.conan.io/en/latest/faq/troubleshooting.html#error-missing-prebuilt-package'

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @KingKiller100

Thanks for your question.

In general, when a particular package gets some settings different than the main project ones, the approach is to use per-package settings, see for example https://docs.conan.io/en/1.66/reference/profiles.html#package-settings-and-env-vars

So something like:

[settings]
os=Windows
os_build=Windows
arch=x86
arch_build=x86_64
build_type=Debug
compiler=msvc
compiler.cppstd=20
compiler.version=193
compiler.runtime=static
compiler.runtime_type=Debug

# The ``mypkg`` dependency specific settings
mypkg*:compiler.version=191
mypkg*:compiler.toolset=v141_xp
mypkg*:compiler.cppstd=17

Note that if you override the mypkg*:compiler=othercompiler then it would be necessary to re-define all sub-settings too.

As side feedback, I guess that you might be aware that things like os_build=Windows are very legacy nowadays, even in Conan 1, dropping those and using the --profile:build approach is the recommended one for being Conan 2-ready. Please take into account that Conan 2 is already 2 years old, and Conan 1, even if it keeps running is considered fully legacy.

[KingKiller100]

Thank you for the quick reply as always @memsharded.

This is a solution to my problem, but it requires the consumer to be aware of the mypkg's settings. Is there any solutions that can be handled just within the DLL package's recipe or settings with the consumer none the wiser?

And yes, I am aware of the build profile feature of conan 1.x and 2.x. It is something my team need to tinker with to understand before enforcing the company's dev workflow. We're slowly trying to transition our workflow to conan 2.x by making sure our conan 1.x recipes use the conan 2.x recommended features. We're also aware that ConanCenter no longer supports 1.x packages so we will need to move if we want newer packages

[memsharded]

This is a solution to my problem, but it requires the consumer to be aware of the mypkg's settings. Is there any solutions that can be handled just within the DLL package's recipe or settings with the consumer none the wiser?

I am not sure if I understood. In the original question you said:

Can the issue be explained and is there a solution for this issue without any major changes needed for the DLL package

So it seemed that it was desired to not modify the DLL recipe/package.

If you want to modify it, then what you were suggesting above, like more aggressive package_id() information erasure, that could work. You could for example completely wipe all information of the DLL package, except maybe os and arch. But remove information for options, and other possible settings.

I think in general it is better to leave the package_id() without information erasure and:

• Explicit the exact settings/options desired for a specific package in the profile
• In Conan 2, the new more advanced compatibility mechanisms can define rules for binary compatibility outside the recipes

What is not possible neither in Conan 1 nor in Conan 2 is to modify settings of DLL from a consumer conanfile recipe. It can only be done in profile files.

[KingKiller100]

Oh I see. Little misunderstanding. I'm willing to make some changes to the dll's recipe like the one you mentioned since there aren't many significant changes and branches needed to support both editable and real behaviours in the same recipe.

I think its worth testing the more aggressive package_id approach. What would need to be added to test that with my editable? I thought this snippet in the recipe would be enough

	def package_id(self):
		# dll with a c interface, doesn't matter what it's compiled with or depends on
		del self.info.settings.compiler
		self.info.requires.unrelated_mode()

[memsharded]

think its worth testing the more aggressive package_id approach. What would need to be added to test that with my editable? I thought this snippet in the recipe would be enough

While it is in editable, not much, as the binary location is mostly defined in the layout() and there is no really package_id() involved there, the pacakge_id serves as a way to distinguish accross any number of different binaries in servers and in Conan cache.

If you wanted to do the right changes to the package_id() to work on the Conan cache, you probably need to clear the options (try self.options.clear().

For things to work fine when editable, the important point is to define a correct layout() method, with the right parameterization to locate the built artifacts in the build-layout folder structure, see https://docs.conan.io/en/1.66/developing_packages/package_layout.html