Skip to content

feat(data): ingest gittensor's master repository registry and its git history as versioned parameter records #8618

Description

@JSONbored

Problem

The registry file and its git history contain the full parameter record (emission_share, maintainer_cut, issue_discovery_share, label multipliers, scoring overrides) but in a form only developers can read.

Scope

  • Ingestion lane: fetch the registry file from the upstream repo on a schedule; diff against the last stored version; persist versioned records keyed by repo + parameter with effective timestamps and source commit SHAs
  • Backfill: walk the file's git history once to reconstruct the complete parameter timeline from first appearance
  • Handle repo additions/removals (registration/deregistration events) as first-class records
  • Provenance on every record: source commit SHA + upstream file path, so every value is checkable against the upstream repo

Forward source note

Upstream is moving repo hyperparameters toward an API (entrius/gittensor#1511, open). The registry file + git history remain canonical for the timeline (commit provenance is the point); when the upstream API lands, add it as the live-values source and reconcile against the latest file commit — a divergence between the two is itself a signal worth surfacing.

Implementation pointers (v1)

  • Script: scripts/sync-sn74-registry.ts (new), modeled on scripts/sync-subnets.ts / scripts/snapshot-adapters.ts
  • History: GitHub commits API filtered by file path on entrius/gittensor master_repositories.json; parse each version, diff params per repo, emit dated change records with source SHAs — one backfill pass, then incremental on schedule
  • Artifact: adapter lane, slug sn74-registry, validated against schemas/components/09-schemas-adapters-r2.schema.json conventions; registry/adapters + R2 snapshot path
  • Workflow: copy .github/workflows/gittensor-impact.yml skeleton (scheduled SN74 job precedent) — daily cron + workflow_dispatch, scoped write like that file

Acceptance

Every parameter value ever present in the registry file exists as a dated, commit-linked record; a spot-check of any record against upstream git history matches.

Metadata

Metadata

Assignees

Labels

backendmaintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions