Developer tooling: one-command setup, docs from main, and the fork URLs - #15
Merged
tomvanmele merged 28 commits intoAug 27, 2026
Merged
Conversation
Three things, all found by actually running the tasks in a fresh environment. invoke release called compas_invocations2's clean with its defaults, and that removes build/ - which in this repository is the IPOPT tree build_ipopt.sh spends fifteen minutes staging, and what CMakeLists.txt defaults IPOPT_PREFIX to. Every release threw it away and left the next editable install failing with 'No IPOPT build at IPOPT_PREFIX=...'. Pass builds=False. Neither extra could run invoke docs on its own: [dev] has no mkdocs, [docs] has no invoke. So the install line the instructions gave produced an environment the documented task did not run in. [dev] includes [docs] now, which makes 'uv pip install -e ".[dev]"' the only install step. pymdown-extensions is declared while we are here - mkdocs.yml configures pymdownx.* directly and it was arriving only as a dependency of mkdocs-material. CI had its own copy of the mkdocs command line rather than calling invoke docs. The flags happened to match; nothing kept them matching. The docs job already installs [dev,docs], so invoke is there. The installation instructions now set up a uv environment, list the tasks, and say which of them need the compiled solver - only test and release, because the documentation is generated from src/ rather than from an imported package, so invoke docs and invoke lint work without ever building the extension.
invoke setup builds the solver and installs the package in one command on Windows, macOS and Linux. It installs the toolchain where that works without root, stages IPOPT unless it is staged already, and installs editable with the right environment - on Windows that means build_ipopt.sh inside the MSYS2 UCRT64 shell with CMAKE_ARGS and EXTRA_LINK_DIRS aimed at its toolchain, which was four exports to get right by hand. Idempotent, and JOBS=1 by default because MUMPS races under a parallel build. requirements-docs.txt installs the docs and lint toolchain in one command without building the package. An extra cannot do that: installing compas_cra[docs] builds compas_cra, which compiles the solver, and the documentation never needs it. The docs deploy step was gated on inputs.publish, which only release.yml sets, so merging a documentation fix built the site and threw it away. It runs on every push to main now. No extra CI: build.yml already runs this pipeline on every push to main, so the wheel is built either way. Every URL named the petrasvestartas fork - site_url, repo_url, [project.urls], the README badges, CITATION.cff, the tutorial and contribution links - while Pages serves the site from blockresearchgroup.github.io/compas_cra. Two README links were separately broken: the banner pointed into docs/_images, gone since the mkdocs migration, and the examples link at a latest/examples.html path on github.com.
…voke setup requirements-docs.txt becomes requirements.txt: it is the only requirements file, so it gets the plain name. Runtime dependencies stay in pyproject.toml, which is where the build backend reads them - the file says so in its header. The IPOPT FATAL_ERROR now suggests invoke setup first, since that is the one command that resolves it on any platform; build_ipopt.sh stays mentioned for whoever wants the manual route.
The development instructions are now bash blocks with one line of prose where a line earns it, in four short sections: Development (invoke setup), Documentation only (requirements.txt), Tasks, Manual build. The README and the contribution guide show the same invoke setup flow instead of the manual one, the README's installation link points at installation.md rather than the .rst file that has not existed since the sphinx days, and the contribution guide no longer tells people to add themselves to AUTHORS.md - deleted years ago - or to use virtualenv/conda.
First real run of invoke setup: the toolchain install worked, then the build died with 'JOBS' is not recognized as an internal or external command. That is cmd.exe talking - invoke runs commands through it, and cmd splits on && outside double quotes, single quotes protecting nothing. The bash -lc invocation lost everything after the first &&. The command now contains no cd, no && and no inline assignment; all three come through the environment instead. CHERE_INVOKING keeps the login shell in the invoking directory (setup already chdirs to base_folder), MSYSTEM=UCRT64 puts the toolchain on PATH, and build_ipopt.sh reads JOBS. The cygpath round-trip goes with the cd that needed it. Verified through invoke's actual cmd.exe shell on the machine that hit it: correct cwd, JOBS=1, gfortran and gcc resolving at /ucrt64/bin.
petrasvestartas
changed the title
…compile
Second finding from the first real invoke setup run. The IPOPT build completed,
then CMake declared g++ broken: exit 1, no output, on a trivial test file. The
compiler driver spawns cc1plus.exe from ucrt64\lib, and cc1plus finds its DLLs -
gmp, mpfr, isl, zstd - in ucrt64\bin, which only an MSYS2 shell puts on PATH.
Run from anywhere else it dies on load, silently. Reproduced outside CMake, and
confirmed the compile succeeds with ucrt64\bin prepended; _native_build_env now
carries PATH alongside CMAKE_ARGS and EXTRA_LINK_DIRS.
And a fifteen-minute build with no output is indistinguishable from a hung one:
build_ipopt.sh now passes --verbosity=${VERBOSITY:-1} to coinbrew, and invoke
setup defaults VERBOSITY to 2, so an interactive build streams the compile lines
while CI - which calls the script directly - keeps the quiet default.
…in [dev] Third and fourth findings from the first real end-to-end run on Windows. The editable-install extension links MSYS2's shared runtimes - libgfortran, libopenblas, libstdc++ - out of ucrt64\bin, and Python does not search PATH for extension DLLs, so importing compas_cra._native died with 'DLL load failed'. Platform wheels never see this: delvewheel grafts the DLLs in. The loader now adds the MSYS2 directory to the DLL search path when the first import fails, and only then; a repaired wheel imports on the first try and never reaches it. And invoke test was unusable from the documented install: [dev] had no pytest. CI installs pytest separately in the test job, which is how it went unnoticed. With these two, the whole chain runs clean on Windows from a bare uv venv: invoke setup (pacman + IPOPT + editable install), then invoke test - 10 passed in 36.66s against the locally built solver, IPOPT 3.14.19 confirmed in-process.
The DLL-search-path retry in compas_cra._native was the wrong layer: a Python package that reaches out to C:\msys64 at import time is not properly packaged - the environment should contain what the extension needs, the way a repaired wheel does. Reverted. Instead the CMake install copies the runtime DLL closure - libgcc_s_seh-1, libgfortran-5, libgomp-1, libopenblas, libquadmath-0, libstdc++-6, libwinpthread-1, computed with objdump - next to the module, behind a BUNDLE_RUNTIME_DLLS option that invoke setup switches on. CPython loads an extension's dependencies from the extension's own directory, so no runtime path manipulation is involved anywhere. The option defaults to OFF: the CI wheels get delvewheel's repair and must not carry a second, unmangled copy. nanobind's CMake does not cover this: nanobind_add_module builds the module and nothing else; runtime redistribution is delvewheel's job for wheels and the build system's job for local installs. Verified: import succeeds from PowerShell with no MSYS2 anywhere on PATH, invoke test passes 10/10 in 21.45s.
…g manual Build once, then a plain editable install is enough - the design the project should have had: CMakeLists.txt finds the MinGW toolchain itself on Windows and refuses MSVC with an explanation. Letting CMake default to Visual Studio compiled all of nanobind with cl.exe and died at link with LNK1181: the GCC-built libipopt.a and libcoinmumps.a cannot be linked by MSVC, so autodetection is not convenience, it is the only correct configuration. An explicitly configured compiler (CI passes D:/msys64 paths) still wins. The two things the build genuinely needs from the environment - ucrt64\bin on PATH for cc1plus's DLLs, and the Ninja generator - are written into the venv's own activation scripts by invoke setup, once, idempotently, Windows only. Venv-scoped: no shell profile, no system environment, nothing global. BUNDLE_RUNTIME_DLLS defaults to ON on Windows, so a plain reinstall keeps the environment self-contained; the CI wheel builds pass OFF explicitly and keep delvewheel as the only DLL provider. Verified from a clean shell with only the venv activated: uv pip install -e . rebuilds and reinstalls, the import succeeds with PATH reduced to C:\WINDOWS\system32 alone, and invoke test passes 10/10. Python-only edits need no reinstall - the editable install serves src/ directly.
CMakeLists.txt finds the MinGW toolchain itself now; pointing CMAKE_ARGS at the compilers by hand is no longer a step anyone has to take.
At 2 the make output of ASL and MUMPS - which is almost the entire fifteen minutes - still went to /dev/null, because coinbrew runs ThirdParty projects at invoke_make $((verbosity-1)) and level 1 is silent. Only Ipopt itself, the main project, streamed. Observed on the second real run: ASL building with no output despite the previous fix. At 3 the ThirdParty make lines stream too; their short autoconf configures stay quiet, which is the right amount of noise.
…write The banner Material prints on every build says MkDocs 2.0 removes the plugin system and rewrites theming with no migration path. Our spec was mkdocs>=1.6 with no ceiling, so the day 2.0 reaches PyPI a fresh environment would pull it and the docs build would break. Capped in the docs extra and requirements.txt.
mkdocs links pages by directory (use_directory_urls), which only a web server resolves to the index.html inside; opened from file:// every navigation click shows a folder listing. The instructions said to open dist/docs/index.html, which walks straight into that.
The docs/docs-serve split had the default backwards: the built site is not browsable from disk (mkdocs links pages by directory, which only a web server resolves - every click from file:// shows a folder listing), so the command a person reaches for first produced something they could not read, and the command they actually wanted was the other one. invoke docs now serves at localhost:8000 with live reload, --strict included. invoke docs --no-serve builds into dist/docs, which is what CI deploys - the docs job in pipeline.yml passes it. docs-serve is gone. Verified both ways: --no-serve builds in 1.75s, and the served site answers 200 for the root and the examples section (the dev server mounts under /compas_cra/ because site_url carries that path; / redirects there).
161 lines down to 116, prose cut to what earns its place. Install is the pip line, the viz extra, the verify one-liner and the Rhino header. Development is the uv + invoke setup block, the two-line after-story (Python edits free, reinstall is plain pip install -e .), and three subsections: Documentation only, Tasks, Manual build, plus the cibuildwheel route.
invoke setup covers the manual sequence, and building IPOPT by hand is documented in packaging/README.md - one pointer instead of a paragraph that compressed four platforms into six unreadable lines.
Found by actually running all 18 example scripts headless - the viewer scene fully constructed, only the blocking show() stubbed. cra_view passed plain Python lists to viewer.scene.add, which compas_viewer 2.x rejects with SceneObjectNotRegisteredError - the first crash anyone hit running an example with a viewer. Every list is wrapped in Collection now, and two stale size=/color= kwargs became pointsize=/pointcolor=. Three examples failed in the solver. The wedge and the three curved blocks converge to ~1e-8 violation and then cannot reach the hard tolerances; at 1e-8 acceptance they died a hair above the line, so acceptable-level moves to 1e-6 - the hard tolerances stay, acceptance only adds a stopping opportunity. The penalty solver gets the adaptive barrier update the CRA solver already had. And 06_arch_penalty passes d_bnd=1e-2: the default 1e-3 exhausts a 9000- iteration cap under both barrier strategies, 1e-2 converges in 65 iterations, and on the standard arch the penalty resultants then agree with cra_solve to 1e-3 - validated cross-formulation rather than against a remembered baseline. The arch example's Restoration_Failed-then-accepted exit is by design and unchanged: the point is feasible to 2.9e-12 and the backend judges it on its merits; the run-to-run difference against the test is threaded-BLAS rounding on a documented-degenerate problem. Two tests so this cannot regress silently: a headless viewer smoke test (skipped without the viz extra) and a penalty-arch regression test pinning both example variants. 12 tests pass; all 18 examples run to completion.
The reference screenshots come from the compas_view2 era, and its Arrow was a mesh: cylinder shaft, conical head, widths scaling with the arrow length. The compas_viewer port replaced it with VectorObject - a fixed pixel-width GL line with a four-sided pyramid head that degenerates to a sliver for axis-parallel directions - and passed the old world-unit body width (0.02) as a pixel linewidth, so the weight arrows rendered as disembodied heads. That is the 'only arrow heads' report, verbatim. Arrow is a mesh again, with the v0.3.0 parameters recovered from git history: head_portion=0.2, head_width=0.07, body_width=0.02, widths relative to length so a small force is a thin pin and a large one a bold arrow. Arrows go into the scene at opacity 0.999 deliberately - the transparency pass draws after the semi-transparent block faces, so an arrow inside a block stays bold, as compas_view2's painter order made it. Also settled, with numbers, whether the solver misportions the interface forces on the curved examples: it does not. The fully converged optimum (tol=1e-8, status optimal, 356 iterations) agrees with the fast acceptable solve to 1.7e-6 across all 72 sub-interface resultants, their sum equals the supported weight, and the corner concentration is present in both - it is the formulation's optimum. The old screenshots merely drew it with hairline pins. Renders of 00_simple_cube and 14_cube-curve-short compared against the documentation screenshots side by side; 18/18 examples run, 12/12 tests pass.
…verified physics Every fix here was cross-checked by running the historical code (commit 15e5edc, 2022-08-17 - the day the screenshots were captured - in an era-exact environment: compas 1.17.5, pyomo 6.4.1, numpy 1.23.5, Python 3.10, driving the AMPL ipopt.exe from this repo's own IPOPT build). Tension coloring: red only for net interface tension (sum_n < 0), the screenshot-era rule. The later per-vertex -1e-5 check flagged whole interfaces red over complementarity-relaxation noise - the shelf's red arrows over a green screenshot. Genuine tension keeps both its displays. Edges: uniform 1.5 width, dark gray on blocks, salmon on supports. 09_bridge: the active solver had been swapped to cra_penalty_solve; the screenshot was made with cra_solve. Restored, and verified identical to the 2022 solution to five decimals (sum 25.75815, max 2.57215, zero tension). View flags follow the screenshot (forcesdirect), which was made with uncommitted flags. 13_curve-3-blocks: the screenshot's density=0.1/scale=5 hits the iteration cap in the 2022 code and today alike - the author's own density=1 change days later was the workaround, not a regression. scale=0.5 restores the drawn arrow lengths of the published image. Rhino: the three curved examples ported to Rhino 8 scripts with a shared drawing module mirroring the desktop visualization, mesh arrows included. 18/18 examples run headless; 12/12 tests pass; lint clean.
… paper did The CRA system is square, and IPOPT's handling of square problems is the hidden variable behind every 'the forces used to look different' report. Until 3.14.11, IPOPT's convergence check on square problems ignored dual feasibility and bound complementarity, so it returned the first primal- feasible point of the barrier path - an interior point, every contact face loaded. Every figure in Kao et al. 2022 (doi:10.1016/j.cad.2022.103216) and in the docs is such a point. IPOPT 3.14.12 removed the special case (changelog: 'Changed handling of dual solution for square problems'), and the same problems then converge to a degenerate vertex: the curved-interface examples put the whole load on 8 of 72 sub-faces and zero the rest. Root-caused by building 3.14.9 and 3.14.14 from source against the same MUMPS 5.9.0: the flip is between them, in IPOPT - not MUMPS, and not this repository. Every code generation from the screenshot-day commit (15e5edc) to today produces both solutions depending only on the binary. The paper-era environment files corroborate: env_win.yml pinned ipopt==3.11.1, and conda-forge served at most 3.14.9-class builds when the figures were made. The fix is mu_target=2e-6 with matching tolerances: the barrier stops at the interior point on purpose. Calibrated against the 3.14.9 solutions on both curved examples (72/72 faces loaded; max resultant 0.00057 vs 0.00066). Equilibrium is untouched - constr_viol_tol holds the force balance at 1e-8; only the distribution on statically indeterminate contacts is selected. The arch reaches the same resultants (1.9570/0.8430) in 247 iterations, down from 668. And 13_curve-3-blocks runs its original published parameters again (density=0.1, scale=5): they only solve in this regime, which is why the author had switched to density=1 days after making the screenshot. 18/18 examples run; 12/12 tests pass; lint clean.
3.14.11, built from source against the same MUMPS 5.9, produces the spread solution with numbers identical to 3.14.9 (72/72 faces, max resultant 0.00066, optimal); 3.14.14 concentrates. The empirical boundary lands on the release whose own changelog documents the square-problem change.
…s the era At 2e-6 the tall cube spread across all faces but with the wrong shape - a few arrows an order of magnitude over the median (max/median 14x vs the era's 3x), which still read as concentration. 5e-6 reproduces the 3.14.9 distribution on both curved examples: short 0.00057/0.00050 vs the era's 0.00066/0.00041, tall 0.00191/0.00050 vs 0.00179/0.00058 (max/median). The degenerate examples are knife-edged in both mu_target and tol - curve-3-blocks solves at (5e-6, 1e-5) and exhausts even a 9000-iteration cap at (5e-6, 5e-5) or (3e-6, 1e-5) - so the option block now says to change them together or not at all, and the iteration cap rises to 9000, which the degenerate examples legitimately need in this regime. The arch keeps its resultants (1.9570/0.8428) with headroom intact at 810 iterations. 18/18 examples run; 12/12 tests pass.
The 5e-6 setting matched a from-source era binary; the figures disagree with that binary. The old square-problem shortcut stopped wherever its convergence check happened to trip, which made the stopping level platform luck: from- source 3.14.9/3.14.11 on this machine stair-step the curved-interface forces 4.5x, while the author's 2022 conda/macOS build - the one the documentation images actually came from - stopped higher, at near-uniform arrows. Exact-as- published can only mean the figures, so the barrier target calibrates to them: mu_target=1e-5 gives max/median 1.11 on cube-curve-tall and 1.20 on cube-curve-short, all 72 faces loaded, which is the picture. The interior regime legitimately ends solves as acceptable-level or feasible- restoration points with contact-complementarity products at the mu level, so acceptable_constr_viol_tol moves to 1e-6 - the arch's and curve-3-blocks' end points sit at 1.3e-7 and were being rejected by 30%. The arch keeps its resultants (1.9571/0.8437 vs 1.9570 under strict optimization) at 270 iterations. The option block documents the knife-edges: change mu_target and the tolerances together or not at all. Also closes the crash investigation: pre-3.14.12 static archives segfaulted in-process because they had been built by the MSYS gcc (a missing MSYSTEM=UCRT64 in the experiment harness, not an IPOPT defect) - rebuilt correctly, 3.14.11 links and runs cleanly. Pinning it was evaluated and rejected: its stopping level is the same platform luck that made the published results irreproducible in the first place. 18/18 examples run; 12/12 tests pass.
… fix The macOS path now carries MACOSX_DEPLOYMENT_TARGET=11.0 through both the IPOPT build and the editable install - the value CI pins for Apple Silicon and Intel alike (pipeline.yml) - so the static libraries and the extension agree on a target instead of the link failing against the Python build's older default. Xcode Command Line Tools are checked up front with a clear message: build_ipopt.sh needs xcrun for the Accelerate stub, and without the check that surfaced fifteen minutes into the build. Linux needs no environment at all, which is now stated rather than implied. A real bug surfaced while testing: the venv activation patch put ucrt64\bin ahead of the venv's Scripts, and ucrt64 carries a python.exe of its own - a bare python in an activated shell ran MSYS2's interpreter. The patch now re-prepends the venv directory after the toolchain. The macOS and Linux branches still cannot execute here, so their environment construction is pinned by per-OS mocked tests (tests/test_tasks.py) - pure logic, exercised everywhere the dev tooling is installed. invoke setup re-verified end-to-end on Windows: idempotent, all green. 17/17 tests pass; lint clean.
Pulled the mac session's setup evolution (Homebrew bootstrap, space-in-path IPOPT relocation) and re-verified the whole chain on Windows: invoke setup idempotent (skips the staged tree at its new resolved location), all examples run, lint clean. One fix: _brew() built the Homebrew candidate paths with os.path.join, which on a Windows machine running the test suite produces /usr/local\bin\brew and made the fallback test path-separator-dependent. Homebrew paths are macOS paths by definition - joined with a literal slash now, and the suite passes identically on both platforms. 36/36 tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything outstanding from the tooling review, in one PR. Each item was found by
setting up a fresh environment and running the tasks.
invoke setup— one command, three platformsThe Windows build was four exports and an MSYS2 shell to get right by hand. It is a task
now, and it works the same on Windows, macOS and Linux:
It installs the toolchain where that is possible without root (MSYS2 packages via
pacman; Homebrew formulae on macOS, including the
gfortransymlink thegccformuladoes not create), stages IPOPT unless
build/ipopt/stagealready holds it, and installseditable with the right environment — on Windows,
build_ipopt.shinside the MSYS2UCRT64 shell with
CMAKE_ARGSandEXTRA_LINK_DIRSaimed at its toolchain. On Linux thepackages need root, so it prints the
apt/dnfline and stops.Idempotent, and
JOBS=1by default because MUMPS races and dies under a parallel build.requirements.txtWorking on docs meant typing out a dozen packages. Now:
This has to be a requirements file rather than an extra: installing
compas_cra[docs]builds
compas_cra, which compiles the solver — and the documentation never needs it,because mkdocstrings reads
src/rather than an imported package.Docs deployed from main
The deploy step was gated on
inputs.publish, which onlyrelease.ymlsets. Merging adocumentation fix built the site and threw it away; the published site tracked the last
tag. It deploys on every push to
mainnow. No extra CI cost —build.ymlalready runsthis pipeline on every push to main, so the wheel exists either way.
Fork URLs
site_url,repo_url,[project.urls], the README badges,CITATION.cffand the linksin the tutorial and contribution pages all named
petrasvestartas/compas_cra, while Pagesserves the site from
blockresearchgroup.github.io/compas_cra. All 24 now point at theBRG repo. Two README links were separately broken: the banner pointed into
docs/_images,gone since the mkdocs migration moved it to
docs/assets/images, and the examples link ata
latest/examples.htmlpath on github.com.invoke releasedeleted the staged IPOPT treebuild.clean(ctx)doesshutil.rmtree("build/"), andCMakeLists.txt:20defaultsIPOPT_PREFIXtobuild/ipopt/stage. Every release threw away fifteen minutes of build.invoke docswas unreachable from the documented setup[dev]had no mkdocs and[docs]no invoke, so neither extra alone ran the task — butthe instructions said
pip install -e ".[dev]".[dev]includes[docs]now.pymdown-extensionsis declared too:mkdocs.ymlconfigurespymdownx.*directly and itwas arriving only via
mkdocs-material.CI parity
pipeline.ymlhad its own copy of themkdocs buildline. It callsinvoke docsnow.Verified
From a clean
uv venv --python 3.12built by the documented commands:uv pip install -r requirements.txt→invoke docsinvoke lint/format/checkinvoke setupregistration,--help, MSYS2 detection, env construction_build_ipoptskip-when-stagedbuild.clean(builds=False)Not verified, and worth a second pair of eyes: the parts of
invoke setupthat need atoolchain — the pacman/brew installs, the IPOPT build itself, and the editable install.
There is no Fortran toolchain on the machine this was written on, so those paths are
written from
pipeline.ymlanddocs/installation.mdrather than executed. CI does notcover them either, since it builds through cibuildwheel.
Still open, outside a PR:
invoke releaseneeds push access to this repository, and PyPIneeds a trusted publisher for
BlockResearchGroup/compas_crawithrelease.ymland thepypienvironment.