Skip to content

Survey area of interest (AOI) and geoDistanceToAOIBoundary expression function - #4588

Open
SteRiccio wants to merge 33 commits into
masterfrom
survey-area-of-interest
Open

SteRiccio wants to merge 33 commits into
masterfrom
survey-area-of-interest

Conversation

@SteRiccio

@SteRiccio SteRiccio commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Description

Requested by ASSEDEV Alerte Environnement: citizens report illegal waste dumping with Arena Mobile. Each report's location must be checked against a pilot area (e.g. the municipalities involved), also offline. The check must never block the submission, and it must tell a location outside the area apart from one too close to the boundary for its GPS accuracy. The result is stored with each report.

Spec: docs/superpowers/specs/2026-10-29-survey-multiple-areas-of-interest-design.md; plans: docs/superpowers/plans/2026-10-28-survey-area-of-interest.md, docs/superpowers/plans/2026-10-29-survey-multiple-areas-of-interest.md.

  • arena-core 2.16.0 (Multiple areas of interest: descriptors, provider, geoAOIContaining arena-core#494)

    • Survey prop areasOfInterest: list of descriptors (name, labels, fileUuid, fileName, size, polygons, vertices); the list order is the priority of overlapping areas.
    • AreaOfInterestProvider in the expression context (like categoryItemProvider): the geometries are never embedded in the survey, they are fetched on demand.
    • geoDistanceToAOIBoundary(coordinate, 'area_name'): signed distance in meters to the boundary of the area, negative inside, positive outside.
    • geoAOIContaining(coordinate): name of the first area containing the coordinate, empty if none.
    • Expression validation: expression.areaOfInterestNotFound for an unknown area name.
  • Areas of interest stored as survey files

    • Each area is a survey file of type areaOfInterest; the survey info prop areasOfInterest only holds the descriptors, so the geometries are not loaded with the survey (survey list, header, every survey fetch, the Arena Mobile survey JSON).
    • Server: AreaOfInterestProviderDefault reads the geometry from the survey file and caches it by file uuid (bounded, 50 areas; a file is never modified, a new upload gets a new uuid). It is passed next to the category item / taxon providers wherever records are evaluated: data entry, record creation from sampling point data, records merge, records validation, record check, data imports (flat data, Arena Mobile, Collect, ODK), chain results persistence.
    • Included in the survey export / import / clone and in the Arena Mobile survey package (surveyfiles/<uuid>.bin, Arena Mobile: survey package download (zip with the survey files, Arena export layout) #4589). The area files no longer referenced are deleted on publish.
  • Survey info → Map

    • New "Areas of interest" table (name, label, file, polygons, vertices; add, edit, download, delete), visible to every survey user. Add / edit dialog: name (unique identifier, used in the expressions), labels, GeoJSON file (max 10 MB, like the preloaded map layers; a geometry, a Feature, or a FeatureCollection whose polygons are merged).
    • The GeoJSON is validated and normalized in the browser (only the polygons are kept), then uploaded as a temporary survey file. The server rejects an upload whose content is not a valid area of interest.
    • Only the sample-based image interpretation settings still require Analysis.
  • Survey compare / fingerprint

    • The areas are compared as survey files, matched by area name and compared by labels / size / polygons / vertices, and applied as new files. The fingerprint includes the descriptors without the file uuid, which is different in every survey.
  • Expression editor

    • geoDistanceToAOIBoundary editor: coordinate attribute and area of interest; geoAOIContaining editor: coordinate attribute; warning when the survey has no areas. Function names, examples and descriptions in all the languages.
  • Coordinate attribute (fix)

    • Accuracy, altitude and altitude accuracy entered in the web form are stored as numbers (they were strings): -location.accuracy returned null, because the unary minus accepts only numbers.
  • Changelog and What's new

    • Entry for 2.11.0, crediting ASSEDEV Alerte Environnement; What's new item with screenshot, translated into all the languages.

Arena Mobile follow-up (after openforis/arena-mobile#289): an AreaOfInterestProvider reading the area files from the survey package folder.

Intended usage by survey designers:

aoi_distance (read-only decimal):  geoDistanceToAOIBoundary(location, 'pilot_area')
aoi_status (read-only code: inside / outside / uncertain):
  'inside'     apply if  aoi_distance < -location.accuracy
  'outside'    apply if  aoi_distance > location.accuracy
  'uncertain'  apply if  Math.abs(aoi_distance) <= location.accuracy
warnings on aoi_status:  aoi_status != 'outside',  aoi_status != 'uncertain'
municipality (read-only text):  geoAOIContaining(location)

Types of changes

  • New feature (non-breaking change which adds functionality)

How has this been tested

  • Unit tests: area of interest descriptors and validator; areaOfInterestParser; survey compare (areas added / changed / removed, apply plan); fingerprint; mobile package. yarn test:unit: 1320 tests.
  • Integration tests: 058surveyAreaOfInterestTest (two areas: geoDistanceToAOIBoundary and geoAOIContaining evaluated through the server provider, only descriptors in the survey, provider cache, invalid upload rejected); 013surveyFilesExportImportTest (two area files after export / import and clone). yarn test:integration: 176 / 177 (the failing one is the scheduler cluster lock test, flaky and unrelated).
  • yarn typecheck: no new errors (the 32 existing errors are also on master). eslint and prettier are clean.
  • E2E spec test/e2e/specs/areaOfInterest.spec.ts: add / save / reload / delete an area in Survey info → Map (invalid name rejected); data entry of a "citizen report" survey with locations inside, outside and near the boundary (status inside / outside / uncertain, warnings).
  • Manual check with Playwright: geoDistanceToAOIBoundary and geoAOIContaining editors in the expression editor, validation error for an unknown area name.

Do you consider this PR needs further testing?

  • No

🤖 Generated with Claude Code

…ession function

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SteRiccio SteRiccio self-assigned this Oct 28, 2026
SteRiccio and others added 7 commits October 28, 2026 13:44
… function

- arena-core 2.13.0: geoDistanceToAOIBoundary, survey areaOfInterest prop
- Survey info > Map: AOI editor (GeoJSON upload, summary, download, delete), Map tab visible without Analysis
- AOI validation, survey fingerprint and survey compare (shown as summary)
- Expression editor: function example/description, CallGeoDistanceToAOIBoundaryEditor
- Translations in all languages

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ey info props)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…as numbers

Entered as strings in the web form, they broke expressions like -location.accuracy (the unary minus accepts only numbers).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WHATS_NEW_SCREENSHOTS_DIR saves the screenshots used in the What's new dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SteRiccio
SteRiccio requested a balanced review from Copilot October 28, 2026 13:32
@SteRiccio
SteRiccio marked this pull request as ready for review October 28, 2026 13:32
@mergify

mergify Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The review surfaced no concrete, objective defects, as each examined concern area resolved cleanly without any outstanding blocking issues.

No open issues 🎉

What changed in this PR

This PR adds a survey Area of Interest (AOI) feature and a new geoDistanceToAOIBoundary(coordinate) expression function, requested by ASSEDEV for offline "citizen report" location checks in Arena Mobile. Survey designers upload a GeoJSON polygon in Survey info → Map; the new function returns the signed distance (meters) from a coordinate to the AOI boundary (negative inside, positive outside), enabling inside/outside/uncertain classification via default values and warnings. It builds on arena-core 2.13.1 (which adds SurveyProps.areaOfInterest and the evaluator function) and wires the prop through the survey info form, validation, survey compare/fingerprint, the expression editor, changelog, and the "What's new" panel.

Changes:

  • New AOI editor (upload/summary/download/delete) in Survey info → Map, with the Map tab now visible to all survey users (only sample-based interpretation still requires Analysis); shared parseAreaOfInterest/getAreaOfInterestSummary parser with unit tests and server-side validation.
  • New geoDistanceToAOIBoundary expression function support: function name, example, descriptions, and a dedicated CallGeoDistanceToAOIBoundaryEditor (with a no-AOI warning); CallSingleParameterEditor gains an optional label.
  • Coordinate attribute fix (accuracy/altitude stored as numbers), arena-core 2.13.1 bump, survey-compare AOI summary, changelog + "What's new" item (2.10.8) translated into all languages.
File Description
core/​geo/​areaOfInterestParser.ts New parser: Polygon/MultiPolygon/Feature/FeatureCollection → AOI, plus summary helper
test/​unit/​tests/​areaOfInterestParser.test.ts Unit tests for the parser and summary
core/​survey/​_survey/​surveyInfo.ts, survey.js New areaOfInterest info key and getAreaOfInterest getter
core/​survey/​_surveyValidator/​surveyInfoValidator.js Server-side validation of the AOI prop
core/​validation/​_validator/​validatorErrorKeys.ts New areaOfInterestInvalid error key
core/​survey/​surveyCompare/​surveyFingerprint.ts, surveyCompareSurveyInfo.ts Include AOI in fingerprint/compare
webapp/​.../​SurveyCompare/​surveyCompareValueFormatter.ts Format AOI as a summary in survey compare
webapp/​.../​SurveyInfo/​AreaOfInterestEditor.tsx New AOI editor component (upload/summary/download/delete)
webapp/​.../​SurveyInfo/​SurveyInfo.js, SurveyInfoMap.js, store/​useSurveyInfoForm.js, SurveyInfo.scss Map tab always shown; AOI value/setter wiring and styles
webapp/​components/​survey/​.../​types/​nodeDefCoordinate.js Store accuracy/altitude as numbers (not strings)
webapp/​components/​expression/​... (functions.js, functionExamples.js, callSingleParameterEditor.js, callGeoDistanceToAOIBoundaryEditor.tsx) Expression editor support for the new function
core/​expressionParser/​helpers/​functions.ts Register geoDistanceToAOIBoundary function name
webapp/​components/​WhatsNew/​WhatsNewSlide.tsx Open description links in a new tab
core/​whatsNew/​whatsNewItems.ts, core/​i18n/​resources/​*/​* "What's new" item (2.10.8) and translations (common/homeView/nodeDefEdit/validationErrors/whatsNew)
CHANGELOG.md 2.10.8 entry (plus minor whitespace cleanups)
package.json, yarn.lock Bump @openforis/arena-core to ^2.13.1 and turf deps
test/​e2e/​specs/​areaOfInterest.spec.ts, fixtures/​seed/​areaOfInterestSurvey.ts, resources/​area_of_interest.geojson E2E coverage for AOI upload and data-entry status
docs/​superpowers/​plans/​2026-10-28-survey-area-of-interest.md Implementation plan document

Review effort: Balanced

SteRiccio and others added 5 commits October 28, 2026 19:56
…rest

# Conflicts:
#	package.json
#	yarn.lock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The GeoJSON is no longer stored in the survey info props (loaded everywhere with the survey):
- survey file of type areaOfInterest, referenced by the info prop areaOfInterestFile
  (file uuid, name, size and polygons/vertices summary)
- the editor validates and normalizes the GeoJSON in the browser and uploads it as temporary
  survey file; the server rejects an invalid content
- the server reads the geometry from the file only where records are evaluated
  (includeAreaOfInterest), in memory, for the arena-core geoDistanceToAOIBoundary function
- included in the survey export/import/clone, in the Arena Mobile survey package and in the
  survey comparison (as survey file); unused files deleted on publish

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SteRiccio and others added 9 commits October 29, 2026 10:12
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- AreaOfInterestProviderDefault reads the geometry of an area of interest from its survey file
  and caches it by file uuid (files are never modified, a new upload gets a new uuid)
- passed next to the category item / taxon providers wherever records are evaluated
- removed the includeAreaOfInterest survey fetch option (geometry no longer set in the survey)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Table of the areas of interest (name, label, file, polygons, vertices) with an add / edit dialog:
the GeoJSON is validated and normalized in the browser and uploaded as temporary survey file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- geoDistanceToAOIBoundary editor: coordinate attribute and area of interest name
- new geoAOIContaining function editor (coordinate attribute)
- function descriptions, examples and expression.areaOfInterestNotFound message in all the languages

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n the expression)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SteRiccio and others added 2 commits October 29, 2026 10:39
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A moderate unresolved issue remains because the areaOfInterestProvider isn't threaded through the Collect import and chain results persistence paths, causing geo evaluations to silently yield null contrary to the PR description.

1 open issue

Review effort: Balanced

SteRiccio and others added 5 commits October 29, 2026 11:00
…a import multiple attribute clearing

Both update the dependent attributes (default values, applicability, validation) without
going through RecordManager.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… aligned

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants