[Collect One 3/60] Protos: forms (P1) by gino-m · Pull Request #2631 · google/ground-platform

gino-m · 2026-10-19T17:09:01Z

Collect One review series: P1 Protos: forms (3/60)

Tracking issue: #2629 · Stacked on: #2630 (C01) · Reviewer: @rfontanarosa
FYI: @gino-m @shobhitagarwal1612

Context

Collect One is the Kotlin Multiplatform successor to Ground: a single shared codebase for the web console, the mobile app (Android, iOS, browser), and a Spanner-backed user-data service. Forms use ProtoForms, a lossless proto3 encoding of ODK XForms.

This PR is one slice of the frozen wip/baobab tree (collect-one-review-base), landing through 60 stacked PRs into c1 (see #2629). It only adds the files listed below, and it's stacked on C01 (#2630), so the diff shows only this slice.

The proto PRs (P1–P3) import each other, e.g. survey_def.proto imports forms/form_def.proto and data/audit_record.proto. Nothing compiles them until shared/core lands in K1, after all three merge, so please review the schemas as a contract. CI only checks buf format here.

What's in this PR

shared/protos/README.md: the mental model behind all the schemas (data tables / map layers = persistent EntityRecords; forms = immutable encounter logs), the package map, and conventions.
forms/form_def.proto (groundplatform.v2.forms): FormDef and the XForms model. The model (ModelDef, RecordSchema, PrimaryInstance/SecondaryInstance, FieldBinding with relevance/constraint/calculate/required/readonly, PreloadType). The body (ControlDef, ControlType, GroupDef, RepeatDef, ItemsetDef, ChoiceItem, RangeConfig, GeoConfig, IntentConfig). Events and actions (EventType, ActionDef, e.g. setvalue/setgeopoint). Translations (TranslationCatalog, LocalizedString, MediaRef). ODK Entities (EntityDeclaration, EntityPropertyMapping, EntitySyncMetadata). Submission config. Optional view layout (ViewDef).
forms/types.proto: DataType, TypedValue, GeoPoint/GeoTrace/GeoShape.
forms/record_instance.proto: reflection-free RecordInstance tree (RecordNode, FieldValue, repeats, RecordMetadata aligned with OpenRosa <orx:meta>).
forms/audit_log.proto: ODK client audit log config and events.
forms/encrypted_submission.proto: envelope-encryption manifest for encrypted submissions.

Where it fits

ProtoForms is the canonical form representation everywhere. shared/core (K1–K5) converts it to and from XForms XML, TextProto and Proto3 JSON, evaluates it with an XPath 1.0 engine, and runs it in the form engine. Survey definitions (P2) embed FormDef. Submissions (P3) carry RecordInstances.

Not in this PR

Survey container, ACLs, organizations and their services → P2. Records, services, library, sync state → P3.
The formal model spec (one page per XForms area) → D2.
Serializers and round-trip tests that prove XML → proto → XML fidelity → K2/K3.

Focus areas

XForms compatibility boundary: everything here should be expressible in the ODK XForms spec (including jr:/odk: extensions and Entities) and round-trip through XLSForm → ODK Central → Collect/Enketo. Anything Ground-specific should live in P2/P3 instead.
Field numbering and oneof choices that would be painful to change later.
Naming and comments: do they cite the spec sections they model?

Safe to skim

Long enum comment blocks that restate ODK appearance and preload names.

Size

Source	Tests	Data	Files
1,409	0	0	6

Testing

bash scripts/buf-format.sh --check: PASS. CI: check (ktfmt, buf). Compilation and round-trip tests arrive with K1/K2.

gemini-code-assist

Code Review

This pull request introduces the canonical Protocol Buffer schemas and gRPC service definitions for the Collect One platform, establishing a structured data model for forms, surveys, and offline-first data synchronization. The feedback highlights several opportunities to improve schema correctness, type safety, and documentation. Key recommendations include renaming misleading 'base64_' prefixed fields that store raw binary bytes, using 'google.type.Date' instead of 'google.protobuf.Timestamp' for calendar dates to avoid timezone shifting bugs, and leveraging 'optional' fields to enable presence tracking for defaultable values like delimiters and geospatial coordinates. Additionally, it is advised to group mutually exclusive fields into a 'oneof' block and document OS-level privacy restrictions on sensitive hardware identifiers.

…key and encrypted_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

…ed_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

Rename encrypted_submission bytes fields to encrypted_key and encrypted_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

gino-m · 2026-10-20T01:35:12Z

Heads-up: encrypted form submissions are being removed from Collect One (no code used them, and the server needs to read submissions). The change for this slice is in #2635, which merges into this PR's branch. Please review it alongside this PR.

…ed_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

Rename encrypted_submission bytes fields to encrypted_key and encrypted_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

…631 (comment))

…e.proto (#2631)

…adata.today callers (#2631)

…s in oneof (#2631 #2631 (comment))

…n GeoPoint (#2631)

…ed_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

Rename encrypted_submission bytes fields to encrypted_key and encrypted_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

…d ODK spec links (#2631 #2631 (comment))

…ed_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

Rename encrypted_submission bytes fields to encrypted_key and encrypted_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

ProtoForms (`groundplatform.v2.forms`): XForms data model in proto3. Sliced from collect-one-review-base (wip/baobab @ b7fd591). Tracking: #2629.

…ed_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

….proto

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…sources in oneof

…nd stable item IDs / sort_order

Rename encrypted_submission bytes fields to encrypted_key and encrypted_signature Both fields are proto bytes, so they hold raw bytes, not base64 text; the base64_ prefix was misleading. Field numbers are unchanged (wire-compatible). Review: #2631 (comment) and #discussion_r4316641039.

gino-m added the collect-one Collect One review series (#2629) label Oct 19, 2026

gino-m requested a review from andreia-ferreira October 19, 2026 17:09

gino-m mentioned this pull request Oct 19, 2026

Review plan: land Collect One (wip/baobab) through 60 stacked PRs #2629

Open

4 of 72 tasks

gemini-code-assist Bot reviewed Oct 19, 2026

View reviewed changes

gino-m mentioned this pull request Oct 19, 2026

[Collect One 8/60] Docs: forms data model (D2) #2634

Closed

gino-m requested a review from rfontanarosa October 19, 2026 17:59

gino-m assigned andreia-ferreira and rfontanarosa Oct 19, 2026

gino-m mentioned this pull request Oct 20, 2026

Remove encrypted form submissions (P1) #2635

Merged

gino-m commented Oct 20, 2026

View reviewed changes

Comment thread shared/protos/forms/encrypted_submission.proto Outdated

gino-m force-pushed the review/c1/p1-protos-forms branch from f50601c to bceb8fe Compare October 20, 2026 08:32

gino-m added a commit that referenced this pull request Oct 20, 2026

carry-forward: note OS restrictions on hardware identifiers (#2631 #2…

282cfbe

…631 (comment))

gino-m added a commit that referenced this pull request Oct 20, 2026

carry-forward: use google.type.Date for today field in record_instanc…

abf3e99

…e.proto (#2631)

gino-m added a commit that referenced this pull request Oct 20, 2026

carry-forward: add google/type/date.proto import and update RecordMet…

c24b599

…adata.today callers (#2631)

gino-m added a commit that referenced this pull request Oct 20, 2026

carry-forward: clarify sms_delimiter and group ActionDef value source…

7af1328

…s in oneof (#2631 #2631 (comment))

gino-m added a commit that referenced this pull request Oct 20, 2026

carry-forward: support optional altitude_meters and accuracy_meters i…

0ad3819

…n GeoPoint (#2631)

gino-m force-pushed the review/c1/p1-protos-forms branch from 731b8c6 to 438c98a Compare October 21, 2026 07:22

gino-m unassigned andreia-ferreira Oct 21, 2026

gino-m removed the request for review from andreia-ferreira October 21, 2026 19:46

gino-m commented Oct 22, 2026

View reviewed changes

Comment thread shared/protos/forms/audit_log.proto

gino-m commented Oct 22, 2026

View reviewed changes

Comment thread shared/protos/README.md Outdated

gino-m added a commit that referenced this pull request Oct 22, 2026

carry-forward: update proto package to openforis.collectone.v1 and ad…

28b67cb

…d ODK spec links (#2631 #2631 (comment))

gino-m force-pushed the review/c1/p1-protos-forms branch from 14992ca to a1e183e Compare October 22, 2026 19:12

Base automatically changed from review/c1/c01-repo-scaffolding to c1 October 23, 2026 09:13

gino-m and others added 11 commits October 23, 2026 09:13

[Collect One 3/60] Protos: forms (P1)

390d958

ProtoForms (`groundplatform.v2.forms`): XForms data model in proto3. Sliced from collect-one-review-base (wip/baobab @ b7fd591). Tracking: #2629.

Apply suggestion from @gemini-code-assist[bot]

c278f60

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

docs: note OS restrictions on hardware identifiers in record_instance…

8e8ac80

….proto

fix(protos): import google/type/date.proto in record_instance.proto

18f6a65

Apply suggestion from @gemini-code-assist[bot]

4fe79ce

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Remove encrypted form submissions (P1) (#2635)

ff26e37

fix(protos): clarify sms_delimiter comment and group ActionDef value …

d1fe4ae

…sources in oneof

Replace *Patch messages with sparse domain protos (upsert + delete) a…

8346fab

…nd stable item IDs / sort_order

Order delete before upsert in sparse mutations (P1)

b485823

Update proto package to openforis.collectone.v1 and add ODK spec links

2a91e77

gino-m force-pushed the review/c1/p1-protos-forms branch from a1e183e to 2a91e77 Compare October 23, 2026 09:13

gino-m closed this Oct 23, 2026

Navigation Menu

Repository navigation

[Collect One 3/60] Protos: forms (P1) - #2631

[Collect One 3/60] Protos: forms (P1)#2631
gino-m wants to merge 11 commits into
c1from
review/c1/p1-protos-forms

gino-m commented Oct 19, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gino-m commented Oct 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

gino-m commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Collect One review series: P1 Protos: forms (3/60)

Context

What's in this PR

Where it fits

Not in this PR

Suggested reading order

Focus areas

Safe to skim

Size

Testing

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gino-m commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gino-m commented Oct 19, 2026 •

edited

Loading

gino-m commented Oct 20, 2026 •

edited

Loading