Cross compiling for windows: Cannot find -lpython3

[piegamesde]

I'm trying to cross compile my application which depends on pyo3 for Windows, but I get the following linker error:

error: linking with `/usr/bin/x86_64-w64-mingw32-gcc` failed: exit status: 1
  |
  = note: "/usr/bin/x86_64-w64-mingw32-gcc" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib/rsbegin.o" "/mnt/target/x86_64-pc-windows-gnu/release/deps/cli-291c7a6a47d029aa.cli.b057b1cc-cgu.6.rcgu.o" "-L" "/mnt/target/x86_64-pc-windows-gnu/release/deps" "-L" "/mnt/target/release/deps" "-L" "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/winapi-x86_64-pc-windows-gnu-0.4.0/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib" "-L" "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/windows_x86_64_gnu-0.34.0/lib" "-L" "/usr/x86_64-w64-mingw32/sys-root/mingw/lib/python3.9/" "-L" "/mnt/target/x86_64-pc-windows-gnu/release/build/bzip2-sys-2b2bedc7c32bb053/out/lib" "-L" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib" "-Wl,-Bstatic" "/tmp/rustcxRKxxg/libbzip2_sys-2346a47203437391.rlib" "-Wl,--start-group" "-Wl,--end-group" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib/libcompiler_builtins-3f91c8bbaf203c3c.rlib" "-Wl,-Bdynamic" "-ladvapi32" "-lwinapi_ntdll" "-lpoppler" "-lpoppler-glib" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lcairo" "-lwindows" "-lpython3" "-lbcrypt" "-ladwaita-1" "-lgtk-4" "-lpangowin32-1.0" "-lpangocairo-1.0" "-lpango-1.0" "-lharfbuzz" "-lgdk_pixbuf-2.0" "-lcairo-gobject" "-lcairo" "-lgraphene-1.0" "-lgio-2.0" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lgobject-2.0" "-lgtk-4" "-lpangowin32-1.0" "-lpangocairo-1.0" "-lpango-1.0" "-lharfbuzz" "-lgdk_pixbuf-2.0" "-lcairo-gobject" "-lcairo" "-lgraphene-1.0" "-lgio-2.0" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lgtk-4" "-lpangowin32-1.0" "-lpangocairo-1.0" "-lpango-1.0" "-lharfbuzz" "-lgdk_pixbuf-2.0" "-lcairo-gobject" "-lcairo" "-lgraphene-1.0" "-lgio-2.0" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lgraphene-1.0" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lgtk-4" "-lpangowin32-1.0" "-lpangocairo-1.0" "-lpango-1.0" "-lharfbuzz" "-lgdk_pixbuf-2.0" "-lcairo-gobject" "-lcairo" "-lgraphene-1.0" "-lgio-2.0" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lgdk_pixbuf-2.0" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lpango-1.0" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lharfbuzz" "-lcairo-gobject" "-lcairo" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lgio-2.0" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lwinapi_advapi32" "-lwinapi_bcrypt" "-lwinapi_cfgmgr32" "-lwinapi_credui" "-lwinapi_crypt32" "-lwinapi_cryptnet" "-lwinapi_fwpuclnt" "-lwinapi_gdi32" "-lwinapi_kernel32" "-lwinapi_msimg32" "-lwinapi_mswsock" "-lwinapi_ncrypt" "-lwinapi_ntdll" "-lwinapi_ole32" "-lwinapi_opengl32" "-lwinapi_secur32" "-lwinapi_shell32" "-lwinapi_synchronization" "-lwinapi_user32" "-lwinapi_winspool" "-lwinapi_ws2_32" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lgobject-2.0" "-lglib-2.0" "-lintl" "-lkernel32" "-lws2_32" "-lbcrypt" "-ladvapi32" "-luserenv" "-lkernel32" "-lgcc_eh" "-l:libpthread.a" "-lmsvcrt" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-luser32" "-lkernel32" "-Wl,--nxcompat" "-L" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib" "-o" "/mnt/target/x86_64-pc-windows-gnu/release/deps/cli-291c7a6a47d029aa.exe" "-Wl,--gc-sections" "-no-pie" "-Wl,-O1" "-nodefaultlibs" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib/rsend.o"
  = note: /usr/lib/gcc/x86_64-w64-mingw32/10.3.1/../../../../x86_64-w64-mingw32/bin/ld: cannot find -lpython3
          collect2: error: ld returned 1 exit status

Some more about my setup and environment:

• In my Cargo.toml: pyo3 = { version = "0.16.4", features = [ "auto-initialize", "anyhow", "abi3-py37" ] }
• System environment: Fedora 34 in Podman container
• Target: x86_64-pc-windows-gnu
• PKG_CONFIG_ALLOW_CROSS=1, PKG_CONFIG_PATH=/usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig/

I checked /usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig/ and it definitely contains a python3.pc (this is the one it looks for, right?).

• Installed mingw64-python3, mingw32-python3 and python3 packages, didn't help

[davidhewitt]

I checked /usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig/ and it definitely contains a python3.pc (this is the one it looks for, right?).

I actually am not sure if we need to integrate pkgconfig support into our build scripts for that to work? Or should the linker be checking pkgconfig stuff itself?

You might want to try the generate-abi3-import-lib feature, new in PyO3 0.16.4

[piegamesde]

Many thanks, it works now

[adamreichold]

I actually am not sure if we need to integrate pkgconfig support into our build scripts for that to work? Or should the linker be checking pkgconfig stuff itself?

No, the linker does not consider the various search paths provided pkgconfig modules. If we want to automatically discover a Python installation based on pkg-config data, we would need to use the pkg-config crate in our build script AFAIU.