Documentation on freesas vs atsas

AGENTS.md

@@ -0,0 +1,86 @@
+# AGENTS.md
+
+This file provides guidance to coding agents working with the code in this repository.
+
+## Project
+
+FreeSAS: Python (+ Cython) tools for small-angle X-ray scattering analysis (SAXS/BioSAXS), developed at the ESRF. Reference: [J. Synchrotron Rad. (2022). 29, 1318-1328](https://scripts.iucr.org/cgi-bin/paper?ju5045). MIT licence.
+
+The package uses a **`src/` layout** (`src/freesas/`) built with **meson-python** — it **cannot** be imported from the sources: `src/freesas/__init__.py` raises `RuntimeError` if `freesas.version` has not been generated by the build.
+
+## Commands
+
+```bash
+# Local build + run a script without installing (patches PYTHONPATH on the fly)
+./bootstrap.py free_rg file.dat
+./bootstrap.py ipython
+
+# Unit tests: builds the project with meson first, then runs freesas.test.suite
+python run_tests.py
+python run_tests.py --installed            # test the installed version instead of building
+python run_tests.py -c                     # coverage report (requires coverage, lxml)
+python run_tests.py -v                     # verbose (repeat for more detail)
+
+# A single test / module (full import path, resolved by unittest.loadTestsFromNames)
+python run_tests.py freesas.test.test_dnn
+python run_tests.py freesas.test.test_autorg.TestAutoRg.test_autorg
+
+# End-to-end tests: they run the installed executables as subprocesses => require `pip install .`
+pip install .
+cd e2etest && python e2etest.py
+
+# Build / installation
+pip install .                              # meson-python, compiles the Cython extensions
+meson setup build && meson install -C build --destdir .
+
+# Sphinx documentation (build/sphinx)
+./build-doc.py
+
+# Lint (CI)
+pylint $(git ls-files '*.py')
+flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+```
+
+Both `run_tests.py` and `build-doc.py` call `bootstrap.build_project()`, which runs `meson setup build` then `meson install --destdir .` and returns `build/lib/python3.X/site-packages`.
+
+The test data are **not in the repository**: `src/freesas/test/utilstest.py` downloads them from `http://www.silx.org/pub/freesas/testdata` through `get_datafile(name)` (cached under `build/`, overridable with the `FREESAS_TESTDATA` environment variable). The list lives in `all_testdata.json`.
+
+## Architecture
+
+### Layers
+
+1. **Cython core** (`src/freesas/ext/*.pyx`) — the hot code: `_autorg.pyx` (Guinier fit, `autoRg`, the `InsufficientDataError` / `NoGuinierRegionError` exceptions), `_bift.pyx` (the `BIFT` class), `_cormap.pyx` (`measure_longest`), `_distance.pyx` (NSD between models). Every `.pyx` is declared as a `py.extension_module` in `src/freesas/ext/meson.build`; **adding a `.pyx` means editing that meson.build**.
+2. **Scientific modules** (`src/freesas/*.py`) — wrappers around the extensions: `autorg.py` (`auto_gpa`, `auto_guinier`, re-exports `autoRg`), `bift.py` (`auto_bift`), `cormap.py` (`gof`), `invariants.py` (Porod, Vc, Rambo-Tainer), `model.py` / `align.py` / `average.py` (3D PDB models, supcomb-style alignment, averaging), `transformations.py` (homogeneous transformation library, bulky external code), `plot.py` (matplotlib figures), `dnn.py` (pure NumPy inference of a dense Keras network).
+3. **CLI applications** (`src/freesas/app/*.py`) — each module exposes `build_parser()` + `main()`, referenced from `[project.scripts]` in `pyproject.toml`. Adding an application = a new module plus an entry in `pyproject.toml` **and** in `src/freesas/app/meson.build`.
+
+### Structural conventions
+
+- **`containers.py`** defines every result type as a `namedtuple` (`RG_RESULT`, `FIT_RESULT`, `RT_RESULT`, `EvidenceResult`, `StatsResult`…), with `__repr__` monkey-patched afterwards to produce the "native" output of the CLIs. Changing that `__repr__` changes what users see and breaks the e2e tests.
+- **`sas_argparser.py`**: `SASParser` wraps `argparse.ArgumentParser`, adding `-v/--verbose` and `-V/--version`; `GuinierParser` composes it with the standard arguments of the Guinier applications (files, `-o/--output`, `-f/--format` among native/csv/ssf, `-u/--unit` nm or Å). The CLIs never create an `ArgumentParser` directly.
+- **`fitting.py`**: `run_guinier_fit(fit_function, parser, logger)` is the pipeline shared by `free_rg`, `free_gpa` and `free_guinier`. The three applications differ only by the fit function passed in (`autoRg`, `auto_gpa`, `auto_guinier`) and the help text. Any new Guinier application should follow that pattern, see `src/freesas/app/auto_gpa.py`.
+- **`sasio.py`**: the single entry point for reading data (`load_scattering_data`, `parse_ascii_data`, `convert_inverse_angstrom_to_nanometer`). The internal unit is **nm⁻¹**; conversion from Å happens at the pipeline entrance.
+- **`resources/`**: data files (including `keras_models/Rg+Dmax.keras`) are reached **only** through `resource_filename()` (which copes with zip / frozen / distro packaging), never by a relative path. A new resource file must be added to the matching `meson.build`.
+
+### Tests
+
+`src/freesas/test/test_all.py` aggregates the `suite()` of every test module by hand; `e2etest/e2etest.py` does the same for the e2e tests. **A new test file must be added explicitly to that aggregator** (and to `src/freesas/test/meson.build`), otherwise it never runs. Every unit test module exposes a hand-built `suite()` function — the style is unittest, not pytest.
+
+### Versioning
+
+`version.py` (at the root, installed inside the package as `freesas/version.py`) is the single source of truth: `MAJOR/MINOR/MICRO/RELEV/SERIAL`. `meson.build` retrieves the version through `run_command(['version.py', '--wheel'])`. Do not edit the version anywhere else.
+
+### Compatibility
+
+`requires-python = '>=3.10'` (`pyproject.toml`). The GitHub Actions CI tests **3.10 → 3.14** on Ubuntu (`python-package-ubuntu.yml`) and on macOS (`python-package-mac.yml`); `pylint.yml` and `release.yml` run on 3.12.
+
+Python 3.10 syntax is therefore available: `match`, `X | Y` in runtime annotations, unions in `isinstance`. Do not go beyond 3.10 (no `type` statement from 3.12, no `class C[T]` generics), or the 3.10 jobs break.
+
+The `ci/requirement_*.txt`, `.travis.yml` and `ci/appveyor.yml` configurations are legacy and no longer match the active CI; only `e2etest/requirements_e2e.txt` is still consumed by the GitHub workflows.
+
+## Dense neural network
+
+`dnn.py` reimplements in NumPy the inference of a dense Keras model, read straight from the `.keras` archive (`config.json` + `model.weights.h5`) with no TensorFlow dependency, to predict Rg and Dmax. `preprocess()` normalises I by its maximum and interpolates onto 1024 regular q points over [0, 4] nm⁻¹ — **the input grid is hard-coded**, so the module expects `q` in nm⁻¹ and its useful domain is bounded by that grid. The model lives in `resources/keras_models/Rg+Dmax.keras` and is reached through `resource_filename()`.
+
+## FreeSAS compared to ATSAS
+
+`doc/source/freesas_vs_atsas.rst` documents a comparison of the two suites over the whole BM29 archive. Worth knowing before touching `bift.py` or the Guinier applications: the two suites agree on `Rg` to about 1 % but differ systematically by some 20 % on `Dmax`, and BIFT's `Dmax` depends on the error bars it is given (which GNOM ignores). The evaluation scripts live in `shannon/`, outside the installable package.

CLAUDE.md

@@ -0,0 +1,7 @@
+# CLAUDE.md
+
+See [AGENTS.md](AGENTS.md).
+
+The guidance for coding agents working in this repository lives in `AGENTS.md`,
+under a name that is not tied to one particular tool. Keep that file as the
+single source of truth and do not duplicate its content here.

ci/requirement_appveyor_2017.txt

@@ -7,7 +7,7 @@ numpy
 cython
 matplotlib
 scipy
-PyPDF2
+pypdf
 h5py
 silx
 pyFAI

ci/requirement_appveyor_2019.txt

@@ -7,7 +7,7 @@ numpy
 cython
 matplotlib
 scipy
-PyPDF2
+pypdf
 h5py
 silx
 pyFAI

ci/requirement_circle.txt

@@ -6,7 +6,7 @@ numpy
 cython
 matplotlib
 scipy
-PyPDF2
+pypdf
 h5py
 silx
 pyFAI

ci/requirement_travis.txt

@@ -8,7 +8,7 @@ numpy
 cython
 matplotlib
 scipy
-PyPDF2
+pypdf
 h5py
 silx
 pyFAI

doc/source/FreeSAS.rst

@@ -35,6 +35,7 @@ Usage
    bift
    dummy_atom_model
    quick_analysis
+   freesas_vs_atsas
 
 Project
 -------

doc/source/bift.rst

@@ -16,3 +16,8 @@ J. Appl. Cryst. (2000). 33, 1415-1421
 and on the work of Jesse Hopkins in BioXTAS-RAW.
 
 The result of `bift` is comparable but surely different from what `gnom` provides since the later is using Tikhonov regularization instead of bayesian inference.
+
+How different has been measured on the complete BM29 archive: the two agree on
+``Rg`` to about 1% but differ systematically by some 20% on ``Dmax``, and they
+respond very differently to the error bars on the intensities. See
+:doc:`freesas_vs_atsas`.