Cannot find boost after migration to manifest mode

[seigeweapon]

Describe the bug
We used to build our app against vcpkg in classic mode. It worked. Now we want to change to manifest mode. But after changing, we got an error:

CMake Error at C:/Program Files/CMake/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Boost (missing: program_options filesystem graph system
  unit_test_framework) (found version "1.80.0")
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  C:/Program Files/CMake/share/cmake-3.26/Modules/FindBoost.cmake:2377 (find_package_handle_standard_args)
  D:/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package)
  CMakeLists.txt:92 (find_package)

This error persists, even when I force boost's version to the same to classic mode.

The vcpkg.json and vcpkg-configuration.json is here:
vcpkg.json
vcpkg-configuration.json

Environment

• OS: Win10
• Compiler: visual studio 2019

To Reproduce
Steps to reproduce the behavior:

1. under vcpkg dir:
   put vcpkg.json & vcpkg-configuration.json under vcpkg dir
   .\vcpkg.exe integrate install
   .\vcpkg.exe install
2. under app dir:
   cmake -Bbuild -S. -DCMAKE_TOOLCHAIN_FILE:STRING=D:\vcpkg\scripts\buildsystems\vcpkg.cmake

[Osyotr]

put vcpkg.json & vcpkg-configuration.json under vcpkg dir

You should put these files in the same folder your root CMakeLists.txt is located.

.\vcpkg.exe integrate install
.\vcpkg.exe install

These steps are unnecessary. Vcpkg toolchain will call vcpkg install automatically.

[seigeweapon]

PS D:\xliu\fvs\FVSCore\3rdparty\colmap> cmake -B build_debug -S . -DCMAKE_TOOLCHAIN_FILE=D:\vcpkg\scripts\buildsystems\vcpkg.cmake
-- Selecting Windows SDK version 10.0.20348.0 to target Windows 10.0.19045.
CMake Error at CMakeLists.txt:90 (find_package):
  By not providing "FindCeres.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Ceres", but
  CMake did not find one.

  Could not find a package configuration file provided by "Ceres" with any of
  the following names:

    CeresConfig.cmake
    ceres-config.cmake

  Add the installation prefix of "Ceres" to CMAKE_PREFIX_PATH or set
  "Ceres_DIR" to a directory containing one of the above files.  If "Ceres"
  provides a separate development package or SDK, be sure it has been
  installed.


-- Configuring incomplete, errors occurred!

I put vcpkg.json & vcpkg-configuration.json under app root dir. Now I got this? Did I miss something?

[Osyotr]

Can't say without proper configure log.

[Cheney-W]

Could you please provide your CMakeLists.txt file?

[seigeweapon]

Hi, I played with it a little more, and the build passed, though I didn't know why :(

I do have some questions:

1. when I run a cmake or visualstudio project build, which has been enabled vcpkg manifest mode, it will start with intalling vcpkg packages. How does it know where's vcpkg dir? Throught VCPKG_ROOT env?
2. what happens if I use different versions of the same package. for example, in project A, I use ceres 1.x, while in project B I use ceres 2.1, and both projects build against the same VCPKG_ROOT. Since the ceres lib's file name does not have version number in it, will it confilict?

[Cheney-W]

1. For CMake, you need to pass the parameter -DCMAKE_TOOLCHAIN_FILE:STRING="VCPKG_ROOT\scripts\buildsystems\vcpkg.cmake"; for Visual Studio, you need to set Use Vcpkg Manifest to Yes in the project properties.
2. If each of your projects has a corresponding vcpkg.json, vcpkg will store the installed ports in the vcpkg_installed directory under the corresponding project, and they will not interfere with each other.

[seigeweapon]

follow up of question 1: for Visual Studio, running "vcpkg.exe integrate install" under VCPKG_ROOT is needed, right?

another question:

3. I want to shrink the size of our CI container's image. It's currently very large because of VCPKG_ROOT. Any suggestion? Maybe skipping debug build somehow? Or enabling a "--clean-after-build" like in classic mode?

[Osyotr]

1. you don't need vcpkg integrate install if you use vcpkg toolchain.
2. Don't rebuild packages each time, use binarycaching.
3. There are *-release triplets that do that.
4. VCPKG_INSTALL_OPTIONS=--clean-after-build though you probably don't want to clean downloads.

[seigeweapon]

1. but how does a visual studio project know where or which vcpkg dir is? There's only a true/false switch in project's property configurations to set/unset manifest mode, no pointer to vcpkg dir.
2. I noticed the binarycaching. My understanding is, all the binaries ever installed will be cached in vcpkg dir. Whenever a project need it by manifest mode, it'll install a copy to project root dir's vcpkg_installed folder
3. Could you please elaborate? how to use *-release?
4. is this a Cmake options? how to use it?

[Cheney-W]

1. If you are using a regular VS console project, you need to run vcpkg integrate install before opening VS. This will ensure that the vcpkg recognized by your project is the one where you executed the command. If you are using a CMake project, as Osyotr mentioned, you can directly use the vcpkg toolchain.
2. You can set the installed triplet to x64-windows-release.
3. This is a parameter for vcpkg install. You can use it like this: vcpkg install --clean-after-build.

[seigeweapon]

3. Sure I can use vcpkg install --clean-after-build, but this is classic mode, meaning I can't override the version of a specifc package, can I?

[Osyotr]

At this point you should start using search engines ;)
https://learn.microsoft.com/en-us/vcpkg/users/buildsystems/cmake-integration#vcpkg_install_options

[seigeweapon]

Thanks guys. I have one last question:
While I'm using a pretty old baseline, tag 2022.11.14. When building in classic mode, I noticed some packages' url is invalid, resulting in download error like this:

Downloading https://mirrors.sjtug.sjtu.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-a
ny.pkg.tar.zst
error: Failed to download from mirror set
error: https://repo.msys2.org/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst: failed
: status code 404
https://www2.futureware.at/~nickoe/msys2-mirror/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.p
kg.tar.zst: failed: status code 404
https://mirror.yandex.ru/mirrors/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zs
t: failed: status code 404
https://mirrors.tuna.tsinghua.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.ta
r.zst: failed: status code 404
https://mirrors.ustc.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst: fa
iled: status code 404
https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst: WinH
ttpSendRequest() failed: 12007
https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst: WinH
ttpSendRequest() failed: 12007
https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst: WinH
ttpSendRequest() failed: 12007
https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst: WinH
ttpSendRequest() failed: 12007
https://mirror.selfnet.de/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst: fail
ed: status code 404
https://mirrors.sjtug.sjtu.edu.cn/msys2/mingw/i686/mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.z
st: failed: status code 404

Then I tried manifest mode, using the same baseline, it works without this issue. How come this difference?

[dg0yt]

You seem to use an old checkout in classic mode, so old central scripts want to use old URLs which no longer work.
In contrast, manifest mode can use up-to-date central scripts with old port versions.