diff options
| author | SulfurNitride <SulfurNitride@users.noreply.github.com> | 2026-03-13 06:26:18 -0500 |
|---|---|---|
| committer | SulfurNitride <SulfurNitride@users.noreply.github.com> | 2026-03-13 06:26:18 -0500 |
| commit | de4db79e5158017c002cc91c14fa9e58420e7774 (patch) | |
| tree | 74f8781aef49afa4e1cc3ba94c5a08876cadab0f /docker | |
| parent | 84a43a2e6afbdfab92c610d47aa75e3876ffacd6 (diff) | |
Bundle PBS Python 3.12 + PyQt6, restore Python plugins as .py files
- Replace portable Python approach with python-build-standalone 3.12.13
(20260310 release) bundled directly in the distribution. Python is now
always available without user setup — removes the Python settings tab.
- PyQt6 staged to plugins/libs/PyQt6/ with bundled Qt stripped out;
patchelf'd to use our existing bundled Qt in lib/ instead of duplicating it.
- Restored native-converted plugins back to .py files:
Form43Checker, ScriptExtenderPluginChecker, DDSPreview, basic_games,
rootbuilder, installer_omod. Removed all *_native CMake targets.
- pythonrunner.cpp: removed venv/MO2_PYTHON_DIR lookup; always uses
bundled Python at <exe_dir>/python, system Python as last-resort fallback.
- plugincontainer.cpp: removed fluorine/python_enabled gate (Python always
loads), demoted proxy loading log messages from warn to debug, silenced
[plugin-diag] stderr spam.
- Dockerfile: switched from uv venv to PBS install_only tarball; PBS Python
used for both build-time pybind11 compilation and runtime distribution.
- build-inner.sh: aggressive Python staging (strip test/tkinter/ensurepip/
distutils/lib2to3/idlelib; wipe build-time site-packages, restore psutil+vdf).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Diffstat (limited to 'docker')
| -rw-r--r-- | docker/Dockerfile | 25 | ||||
| -rwxr-xr-x | docker/build-inner.sh | 103 |
2 files changed, 108 insertions, 20 deletions
diff --git a/docker/Dockerfile b/docker/Dockerfile index 5ab99dc..ec5653b 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -50,16 +50,23 @@ ENV CMAKE_PREFIX_PATH="/opt/qt6/6.10.2/gcc_64:${CMAKE_PREFIX_PATH}" ENV PATH="/opt/qt6/6.10.2/gcc_64/bin:${PATH}" ENV LD_LIBRARY_PATH="/opt/qt6/6.10.2/gcc_64/lib:${LD_LIBRARY_PATH}" -# ── uv (Python package manager) ── -RUN curl -LsSf https://astral.sh/uv/install.sh | sh -ENV PATH="/root/.local/bin:${PATH}" +# ── python-build-standalone 3.12 (build-time + bundled runtime) ── +# Single Python for both compiling pybind11 modules and shipping in the tarball. +# The install_only_stripped tarball is ~28MB; we strip it further in build-inner.sh. +ARG PBS_VERSION=3.12.13 +ARG PBS_DATE=20260310 +RUN curl -fL --retry 3 -o /tmp/pbs-python.tar.gz \ + "https://github.com/astral-sh/python-build-standalone/releases/download/${PBS_DATE}/cpython-${PBS_VERSION}%2B${PBS_DATE}-x86_64-unknown-linux-gnu-install_only.tar.gz" && \ + tar xzf /tmp/pbs-python.tar.gz -C /opt/ && \ + mv /opt/python /opt/python-bundled && \ + rm /tmp/pbs-python.tar.gz +ENV BUILD_PYTHON=/opt/python-bundled/bin/python3 +ENV PATH="/opt/python-bundled/bin:${PATH}" -# ── Build-time Python venv (pybind11, sip, etc.) ── -RUN uv venv /opt/build-python --python 3.13 --seed && \ - uv pip install --python /opt/build-python/bin/python \ - pybind11==2.13.6 sip psutil vdf -ENV BUILD_PYTHON=/opt/build-python/bin/python -ENV PATH="/opt/build-python/bin:${PATH}" +# ── Build-time Python packages ── +# pybind11/sip/etc. for compiling mobase.so; PyQt6 is staged into the distribution. +RUN /opt/python-bundled/bin/pip3 install --no-cache-dir \ + pybind11==2.13.6 sip psutil vdf PyQt6 # ── Rust toolchain ── RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | \ diff --git a/docker/build-inner.sh b/docker/build-inner.sh index 0c0fbc7..3290069 100755 --- a/docker/build-inner.sh +++ b/docker/build-inner.sh @@ -20,9 +20,12 @@ if [ -n "${CMAKE_CXX_COMPILER_LAUNCHER:-}" ]; then CMAKE_EXTRA_ARGS+=("-DCMAKE_CXX_COMPILER_LAUNCHER=${CMAKE_CXX_COMPILER_LAUNCHER}") fi +PYTHON_ROOT="$(dirname "$(dirname "${BUILD_PY}")")" + cmake -S . -B build -G Ninja \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DPython_EXECUTABLE="${BUILD_PY}" \ + -DPython_ROOT_DIR="${PYTHON_ROOT}" \ ${PYBIND11_DIR:+-Dpybind11_DIR="${PYBIND11_DIR}"} \ -DBUILD_PLUGIN_PYTHON=ON \ "${CMAKE_EXTRA_ARGS[@]}" @@ -65,13 +68,7 @@ find build/libs -type f \( \ -name "libinibakery.so" -o \ -name "libbsa_extractor.so" -o \ -name "libbsa_packer.so" -o \ - -name "libproxy.so" -o \ - -name "libform43_checker_native.so" -o \ - -name "libscript_extender_checker_native.so" -o \ - -name "libpreview_dds_native.so" -o \ - -name "libbasic_games_native.so" -o \ - -name "librootbuilder_native.so" -o \ - -name "libinstaller_omod_native.so" \ + -name "libproxy.so" \ \) -exec cp -f {} "${OUT_DIR}/plugins/" \; # Python plugin loader (small — kept for optional Python support). @@ -81,10 +78,31 @@ if [ -d "build/src/src/plugins/libs" ]; then mkdir -p "${OUT_DIR}/plugins/libs" cp -f build/src/src/plugins/libs/mobase*.so "${OUT_DIR}/plugins/libs/" 2>/dev/null || true fi -# Python helper shims (needed by Python plugins when Python is enabled). +# Python helper shims for f in lzokay.py winreg.py pyCfg.py; do [ -f "build/src/src/plugins/${f}" ] && cp -f "build/src/src/plugins/${f}" "${OUT_DIR}/plugins/" done + +# Python plugins (simple single-file) +for pyfile in \ + "libs/form43_checker/src/Form43Checker.py" \ + "libs/script_extender_plugin_checker/src/ScriptExtenderPluginChecker.py" \ + "libs/preview_dds/src/DDSPreview.py" \ + "src/plugins/rootbuilder.py" \ + "src/plugins/installer_omod.py"; do + [ -f "${pyfile}" ] && cp -f "${pyfile}" "${OUT_DIR}/plugins/" +done + +# basic_games Python module (directory package) — copy the whole tree, .py files only +if [ -d "libs/basic_games" ]; then + cp -a "libs/basic_games" "${OUT_DIR}/plugins/basic_games" + # Remove non-Python clutter (metadata, lock files, vcpkg, etc.) + find "${OUT_DIR}/plugins/basic_games" \ + \( -name "*.toml" -o -name "*.lock" -o -name "*.json" \ + -o -name "*.txt" -o -name "*.md" -o -name "LICENSE" \ + -o -name "CMakeLists.txt" -o -name "CMakePresets.json" \) \ + -delete 2>/dev/null || true +fi # data/ dir (DDS headers etc., used by native plugins too). [ -d "build/src/src/plugins/data" ] && cp -a "build/src/src/plugins/data" "${OUT_DIR}/plugins/" @@ -204,9 +222,72 @@ if [ -f /usr/local/lib/libloot.so.0 ]; then ln -sf libloot.so.0 "${OUT_DIR}/lib/libloot.so" fi -# ── No portable Python runtime ── -# Python plugins are optional and use the system Python + venv when enabled. -# Native C++ plugins replace DDSPreview, Form43Checker, ScriptExtenderChecker, basic_games. +# ── Bundle PBS Python 3.12 runtime ── +# PYTHONHOME only needs lib/python3.12/ (the stdlib). We do NOT copy the +# binary, headers, static lib, or .py sources — only stripped .pyc + .so. +PBS_SRC="/opt/python-bundled" +PYTHON_OUT="${OUT_DIR}/python" +mkdir -p "${PYTHON_OUT}/lib" + +# Copy only the stdlib directory +cp -a "${PBS_SRC}/lib/python3.12" "${PYTHON_OUT}/lib/" + +# Remove only what is safe — test suites, GUI toolkits, dev tools. +# Do NOT strip network/stdlib modules; basic_games uses email, http, xml, urllib, etc. +find "${PYTHON_OUT}" -type d \( -name "test" -o -name "tests" \) \ + -exec rm -rf {} + 2>/dev/null || true +rm -rf "${PYTHON_OUT}/lib/python3.12/tkinter" +rm -rf "${PYTHON_OUT}/lib/python3.12/ensurepip" +rm -rf "${PYTHON_OUT}/lib/python3.12/distutils" +rm -rf "${PYTHON_OUT}/lib/python3.12/lib2to3" +rm -rf "${PYTHON_OUT}/lib/python3.12/idlelib" +rm -rf "${PYTHON_OUT}/lib/python3.12/turtledemo" +rm -f "${PYTHON_OUT}/lib/python3.12/turtle.py" +# Wipe site-packages entirely — build-time packages (pybind11, PyQt6, sip, etc.) +# are not needed at runtime. PyQt6 is staged separately to plugins/libs/PyQt6/. +rm -rf "${PYTHON_OUT}/lib/python3.12/site-packages" +mkdir -p "${PYTHON_OUT}/lib/python3.12/site-packages" +# Copy runtime-required packages back in +for pkg in psutil vdf; do + pkg_dir="$("${PBS_SRC}/bin/python3" -c "import importlib.util; s=importlib.util.find_spec('${pkg}'); print(s.submodule_search_locations[0] if s and s.submodule_search_locations else (s.origin if s else ''))" 2>/dev/null || true)" + if [ -d "${pkg_dir}" ]; then + cp -a "${pkg_dir}" "${PYTHON_OUT}/lib/python3.12/site-packages/" + elif [ -f "${pkg_dir}" ]; then + cp -f "${pkg_dir}" "${PYTHON_OUT}/lib/python3.12/site-packages/" + fi +done + +# Pre-compile .py → .pyc (PBS ships .py + .pyc; this ensures cache is fresh). +# We keep the .py source files — Python's SourceFileLoader requires them to +# find the corresponding __pycache__/*.pyc files. Deleting them breaks imports. +"${PBS_SRC}/bin/python3" -m compileall -q "${PYTHON_OUT}/lib/python3.12/" 2>/dev/null || true + +# Strip debug info from extension modules +find "${PYTHON_OUT}/lib/python3.12" -name "*.so" \ + -exec strip --strip-unneeded {} \; 2>/dev/null || true + +# libpython shared library goes in our lib/ (dlopen'd by librunner.so via $ORIGIN RPATH) +# Not placed inside python/ — PYTHONHOME doesn't need the shared lib alongside the stdlib. +cp -Lf "${PBS_SRC}/lib/libpython3.12.so.1.0" "${OUT_DIR}/lib/" +strip --strip-unneeded "${OUT_DIR}/lib/libpython3.12.so.1.0" 2>/dev/null || true +ln -sf libpython3.12.so.1.0 "${OUT_DIR}/lib/libpython3.12.so" +echo "Bundled PBS Python 3.12: $(du -sh "${PYTHON_OUT}" | cut -f1)" + +# ── Bundle PyQt6 (bindings only — reuse our bundled Qt, no duplicate Qt .so) ── +# PyQt6 pip wheel bundles Qt under PyQt6/Qt6/lib/ which we strip out. +# The binding .so files are patchelf'd to find our Qt in lib/. +PYQT6_SRC="$("${PBS_SRC}/bin/python3" -c 'import PyQt6, os; print(os.path.dirname(PyQt6.__file__))')" +PYQT6_OUT="${OUT_DIR}/plugins/libs/PyQt6" +mkdir -p "${PYQT6_OUT}" +cp -a "${PYQT6_SRC}/." "${PYQT6_OUT}/" +# Remove PyQt6's bundled Qt — we already have Qt in lib/ +rm -rf "${PYQT6_OUT}/Qt6" +# Patchelf all PyQt6 binding .so files to reach our lib/ via RPATH +# Path: plugins/libs/PyQt6/*.so → ../../.. = staging root → lib/ +find "${PYQT6_OUT}" -name "*.so" -exec \ + patchelf --force-rpath --set-rpath '$ORIGIN/../../../lib' {} \; 2>/dev/null || true +strip --strip-unneeded "${PYQT6_OUT}"/*.so 2>/dev/null || true +echo "Bundled PyQt6 (no Qt dupe): $(du -sh "${PYQT6_OUT}" | cut -f1)" # ── Strip all MO2 binaries ── echo "Stripping MO2 binaries..." |
