A Kedro pipeline that competes in the Numerai tournament end-to-end: it downloads the real tournament data, tunes and trains a LightGBM model, predicts on the live round, and submits — all from one command.
./scripts/numerai/compete.shNumerai is a data-science tournament run by a hedge fund. Every day they publish an obfuscated dataset of stock-market features. You train a model, predict on the current live round, and upload your predictions. Weeks later, your predictions are scored against what actually happened in the market (correlation, "CORR"). Good predictions earn reputation and — only if you choose to stake the NMR cryptocurrency — money. Submitting is completely free and there is no obligation to stake.
Key vocabulary you'll see in this repo:
| Term | Meaning |
|---|---|
| era | One time period (~a week) of data. Rows in the same era are correlated, so all scoring is done per era. |
| feature | An anonymized stock attribute, binned into 5 values (0, 0.25, 0.5, 0.75, 1). |
| target | What you predict — a measure of future stock returns, also binned into 5 values. |
| live round | The current week's stocks, features only, no target. This is what you submit predictions for. |
| CORR | Per-era correlation between your predictions and the true target. The main score. |
| Sharpe | Mean CORR ÷ standard deviation of CORR across eras. Rewards consistency, not just lucky eras. |
flowchart LR
subgraph dp["data_processing"]
A[/"Numerai API<br/>(numerapi)"/] -->|download| B["load_tournament_data"]
B --> C[("train_data<br/>02_intermediate")]
B --> D[("val_data<br/>02_intermediate")]
end
subgraph mod["modeling"]
C --> E["tune_hyperparameters<br/>(Optuna, per-era CORR)"]
D --> E
E --> F[("best_params<br/>06_models")]
C --> G["train_model<br/>(LightGBM)"]
F --> G
G --> H[("model<br/>06_models")]
H --> I["predict"]
D --> I
I --> J[("val_predictions<br/>07_model_output")]
J --> K["evaluate<br/>(CORR mean / std / Sharpe)"]
K --> L[("metrics<br/>08_reporting")]
end
subgraph sub["submission"]
A -->|live round| M["fetch_live_data"]
M --> N[("live_data<br/>05_model_input")]
H --> O["predict_live<br/>(rank to 0–1)"]
N --> O
O --> P[("live_predictions<br/>07_model_output")]
P --> Q["submit_predictions"]
Q --> R[("submission_receipt<br/>08_reporting")]
Q -.->|upload| S[/"numer.ai"/]
end
Every box is a Kedro node (a Python function in
src/numeria/pipelines/*/nodes.py); every cylinder is a dataset declared
in conf/base/catalog.yml and saved under data/. Metrics and parameters are
also logged to MLflow automatically on every run.
sequenceDiagram
autonumber
participant You as You (or cron / Claude)
participant P as Kedro pipeline
participant N as Numerai API
You->>P: ./scripts/numerai/compete.sh
P->>N: download features.json, train, validation (cached after 1st run)
P->>P: Optuna search → maximize mean per-era CORR
P->>P: train final LightGBM on all training eras
P->>P: evaluate on validation → metrics.json + MLflow
P->>N: download live round features
P->>P: predict live, rank into (0, 1]
P->>N: upload_predictions (needs NUMERAI_PUBLIC_ID/SECRET_KEY)
N-->>P: submission id
P-->>You: submission_receipt.json
Note over N: ~20 days later: round resolves,<br/>your CORR score appears on numer.ai
./scripts/uv/setup.sh # uv sync: builds .venv, installs everything- Create a free account at numer.ai — your account comes with a model slot automatically.
- Go to Account → Your API keys → Add and create a key with "Upload submissions" permission.
- Copy the example env file and paste the two values in:
cp .env.example .env # then edit .env:
# NUMERAI_PUBLIC_ID=...
# NUMERAI_SECRET_KEY=....env is gitignored — your keys never leave your machine.
./scripts/numerai/compete.sh # full run: data → train → predict → submit
./scripts/numerai/compete.sh --dry-run # everything except the upload (safe to try first)
./scripts/numerai/compete.sh --submit-only # reuse trained model: fetch live → predict → submitWithout API keys the pipeline still runs fully — it just skips the upload and
says so in data/08_reporting/submission_receipt.json.
| Where | What | How |
|---|---|---|
data/08_reporting/metrics.json |
validation CORR mean/std/Sharpe | cat it |
data/08_reporting/submission_receipt.json |
did the upload happen, round, submission id | cat it |
| MLflow UI | every run's params + metrics over time | ./scripts/mlflow/ui.sh → port 5000 |
| Kedro Viz | interactive pipeline graph | ./scripts/kedro/viz.sh → port 4141 |
| numer.ai/models | official scores once rounds resolve | browser |
uv run kedro run # everything (= compete.sh)
uv run kedro run --pipeline training # download + tune + train + evaluate, no upload
uv run kedro run --pipeline submission # fetch live + predict + upload (model must exist)
uv run kedro run --pipeline data_processing # just refresh the dataAll in conf/base/parameters.yml:
data:
source: numerai # flip to "synthetic" for a fast offline demo (no network)
numerai:
feature_set: small # small (~42 features) → medium (~700) → all (~2300)
era_subsample: 4 # train on every 4th era; set 1 to use all (more RAM/time)
model:
optuna:
n_trials: 15 # more trials = better hyperparameters = slower
timeout: 90 # seconds budget for the search
submit:
dry_run: false # true = never upload
model_name: null # null = first model on your accountOverride anything per-run without editing files:
uv run kedro run --params "data.numerai.feature_set=medium,model.optuna.n_trials=50"The path to a stronger model (in rough order of payoff): more Optuna
trials → feature_set: medium → era_subsample: 1 → ensembling several
models / multiple targets. Iterate, watch validation Sharpe in MLflow, and
only ever stake what you can afford to lose.
New rounds open daily around 13:00 UTC with a short submission window (check numer.ai for the current schedule). Two easy options:
cron — retrain weekly, submit daily:
# m h dom mon dow command
30 13 * * 2 cd /workspaces/numeria && ./scripts/numerai/compete.sh >> info.log 2>&1
30 13 * * 3-6 cd /workspaces/numeria && ./scripts/numerai/compete.sh --submit-only >> info.log 2>&1Claude Code — from a claude session in this repo, ask:
schedule a daily routine at 13:30 UTC that runs ./scripts/numerai/compete.sh --submit-only, and a weekly one on Tuesdays that runs ./scripts/numerai/compete.sh, then check the receipt and tell me if the submission failed
Claude can also read metrics.json / MLflow between runs and propose
parameter changes — that's the "automatically run and improve" loop.
conf/base/ parameters.yml (knobs), catalog.yml (where data lives), mlflow.yml
conf/local/ credentials & local overrides (gitignored)
data/01_raw/ downloaded Numerai parquet files (cached)
data/02_intermediate ... 08_reporting/ each pipeline stage's outputs (see data/README.md)
src/numeria/pipelines/
data_processing/ download real data (or generate synthetic)
modeling/ Optuna tune → LightGBM train → predict → evaluate
submission/ fetch live → predict → upload
src/numeria/scoring.py per-era Numerai correlation helpers (numerai-tools)
scripts/ numerai/compete.sh, uv/setup.sh, dev/check.sh, kedro/viz.sh ... (see scripts/README.md)
tests/ offline unit tests (pytest; no network, no keys needed)
Numerai scores models over many weeks; no single submission wins anything.
What compounds is consistent, modest positive correlation (a good Sharpe).
This repo gives you the full loop — data, tuning, training, evaluation,
submission, tracking — so every experiment is reproducible and comparable in
MLflow. The model itself (LightGBM on the small feature set) is a solid,
standard baseline: expect it to be mid-pack, then iterate. And to repeat the
only financial advice in this file: submitting is free; staking NMR is
optional and you can lose it.
./scripts/dev/check.sh # ruff lint + format + pytest — run before committing
uv run pytest # tests only (all offline; synthetic data)