Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitleaksignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# A refused-value test fixture shaped like a key header, replaced by the next commit; no key was committed.
3c0aa4546785099d5bf058c8d5dbf909d94096e0:internal/config/aikey_test.go:private-key:59
2 changes: 2 additions & 0 deletions docs/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,8 @@ environment -> Secret and never enters `agentlab.yaml`, `state/` (the
rendered values carry the Secret's *name*), a log line or a process's argv.
Re-running with a new token rotates the Secret.

**A skills GitHub App** is the way a lab reaches GitHub without any token: agent-manager's chart takes a read-only App (`skills.github.app.secretName`) and resolves skills with its installation tokens, at the App's rate limit. `agentlab configure --github-app-id <id> --github-app-installation-id <id> --github-app-private-key-source op://<vault>/<item>/<field>` records it as `githubApp` in `agentlab.yaml` (the ids are public; the key is a reference). Every `up` and `platform` apply the ids to the Secret `agentlab-github-app` in `agent-platform` and hand the key to `beekeeper secret copy --to-secret` (key `private-key`, never read by agentlab; a failed placement fails the run), and the values name the Secret for agent-manager. The chart refuses the App together with `tokenSecret`, so `githubApp` and `githubToken.source` are not recorded together; a `$GITHUB_TOKEN` still reaches the portal and the migrate Job, which take a token only.

Without either the lab is as before: the values name no Secret, the
consumers call GitHub unauthenticated, and a Secret an earlier run created
stays — unreferenced — until `agentlab down` (a run that merely lacks the
Expand Down
1 change: 1 addition & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ The flags pin a value regardless of the discovery, with or without
| `--workspaces-github-app-id`, `--workspaces-github-client-id` | The public ids of the `github` instance's App (`platform.workspaces.github`); its private key and client secret never pass through agentlab. |
| `--ai-key-source <ref>` | Where the Anthropic key of the agents' default ModelConfig lives: a reference `beekeeper secret copy` resolves (`op://<vault>/<item>/<field>`, or `<file>#<path>` of a SOPS file), never a value. Every `up` and `platform` place it into the Secret `kagent/kagent-anthropic` through beekeeper (on PATH, or `configure` refuses), so a recreated lab carries the key without a manual step; `--ai-key-source ""` clears it (the key then comes from `$ANTHROPIC_API_KEY`, else a placeholder). See [The model](agents.md#the-model). |
| `--github-token-source <ref>` | Where the GitHub token of the portal's skill discovery and agent-manager's skill resolution lives: a reference `beekeeper secret copy` resolves, as for `--ai-key-source`, never a value. Every `up` and `platform` place it into the Secret `agentlab-github-token` in `agent-platform` and `kagent` through beekeeper, so a lab started without `$GITHUB_TOKEN` (an agent's) calls GitHub authenticated; `--github-token-source ""` clears it (the token then comes from `$GITHUB_TOKEN`, else GitHub is called unauthenticated). See [Agents](agents.md#the-github-token). |
| `--github-app-id <id>`, `--github-app-installation-id <id>`, `--github-app-private-key-source <ref>` | The skills GitHub App agent-manager resolves skills with (`githubApp`): its public ids and a reference to its private key that `beekeeper secret copy` resolves, never a value. Every `up` and `platform` write the Secret `agentlab-github-app` in `agent-platform` (the key placed by beekeeper) and wire `agent-manager.skills.github.app`. Not together with `--github-token-source`; `""` on all three clears it. |

## Environment variables

Expand Down
25 changes: 25 additions & 0 deletions internal/config/aikey_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,28 @@ func TestGitHubTokenSourceIsAReference(t *testing.T) {
t.Errorf("a pasted value must be refused naming githubToken.source, got %v", err)
}
}

// TestGitHubAppValidate: a complete App with numeric ids and a key reference
// is taken; a partial one, a non-numeric id, a pasted key and an App next to
// githubToken.source are refused naming githubApp.
func TestGitHubAppValidate(t *testing.T) {
good := GitHubApp{AppID: "123456", InstallationID: "7890", PrivateKeySource: "op://lab/skills-app/private-key"}
cfg := Default()
cfg.GitHubApp = good
if err := cfg.Validate(); err != nil {
t.Errorf("a complete App refused: %v", err)
}
for name, mutate := range map[string]func(*Config){
"partial": func(c *Config) { c.GitHubApp.InstallationID = "" },
"non-numeric": func(c *Config) { c.GitHubApp.AppID = "skills-app" },
"pasted key": func(c *Config) { c.GitHubApp.PrivateKeySource = "pasted value" },
"with token": func(c *Config) { c.GitHubToken.Source = "op://lab/github-token/credential" },
} {
cfg := Default()
cfg.GitHubApp = good
mutate(cfg)
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "githubApp") {
t.Errorf("%s: want a refusal naming githubApp, got %v", name, err)
}
}
}
8 changes: 8 additions & 0 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -1040,6 +1040,8 @@ type Config struct {
AIKey AIKey `yaml:"aiKey,omitempty"`
// GitHubToken is where the token of the lab's GitHub calls comes from.
GitHubToken GitHubToken `yaml:"githubToken,omitempty"`
// GitHubApp is the skills GitHub App agent-manager resolves skills with.
GitHubApp GitHubApp `yaml:"githubApp,omitempty"`

Users []User `yaml:"users"`
Platform Platform `yaml:"platform"`
Expand Down Expand Up @@ -1451,6 +1453,12 @@ func (c *Config) Validate() error {
if err := c.GitHubToken.Validate(); err != nil {
return fmt.Errorf("githubToken.source %q: %w", c.GitHubToken.Source, err)
}
if err := c.GitHubApp.Validate(); err != nil {
return fmt.Errorf("githubApp: %w", err)
}
if c.GitHubApp.Configured() && c.GitHubToken.Source != "" {
return errors.New("githubApp and githubToken.source both set: agent-manager takes one GitHub credential, its chart refuses both — clear one (`agentlab configure --github-token-source \"\"`)")
}
if c.SubstrateNodes < 0 || c.SubstrateNodes > MaxSubstrateNodes {
return fmt.Errorf("substrateNodes: %d, want 0 (the single-node lab) to %d", c.SubstrateNodes, MaxSubstrateNodes)
}
Expand Down
62 changes: 62 additions & 0 deletions internal/config/githubapp.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
package config

import (
"errors"
"fmt"
)

// GitHubApp is the read-only skills GitHub App agent-manager resolves skills
// with: installation tokens at the App's rate limit instead of a static
// token or GitHub's anonymous window. The App id and installation id are
// public; the private key is never this file: PrivateKeySource names where
// the operator's secret tooling finds it, and `agentlab up` and `platform`
// hand the reference to that tooling, which writes the key straight into the
// lab's Secret agentlab-github-app (internal/lab/githubapp.go).
type GitHubApp struct {
// AppID is the App's numeric id.
AppID string `yaml:"appId,omitempty"`
// InstallationID is the numeric id of the App's installation whose
// repositories the skills come from.
InstallationID string `yaml:"installationId,omitempty"`
// PrivateKeySource is a reference `beekeeper secret copy` resolves to the
// App's private key (PEM), in the form AIKey.Source takes.
PrivateKeySource string `yaml:"privateKeySource,omitempty"`
}

// Configured reports whether any part of the App is recorded; Validate
// refuses a partial one.
func (a GitHubApp) Configured() bool {
return a.AppID != "" || a.InstallationID != "" || a.PrivateKeySource != ""
}

// Validate refuses a partial App, an id that is not a number and a private
// key that is not a reference.
func (a GitHubApp) Validate() error {
if !a.Configured() {
return nil
}
if a.AppID == "" || a.InstallationID == "" || a.PrivateKeySource == "" {
return errors.New("appId, installationId and privateKeySource are set together or not at all")
}
for name, id := range map[string]string{"appId": a.AppID, "installationId": a.InstallationID} {
if !isDigits(id) {
return fmt.Errorf("%s %q is not a numeric GitHub id", name, id)
}
}
if err := ValidateSecretSource(a.PrivateKeySource); err != nil {
return fmt.Errorf("privateKeySource %q: %w", a.PrivateKeySource, err)
}
return nil
}

func isDigits(s string) bool {
if s == "" {
return false
}
for _, r := range s {
if r < '0' || r > '9' {
return false
}
}
return true
}
2 changes: 1 addition & 1 deletion internal/lab/agentstest.go
Original file line number Diff line number Diff line change
Expand Up @@ -877,7 +877,7 @@ func proveAgentsTestRefreshSkills(session *musterSession, fixture SkillsFixture,
if !authenticated {
step("%supdate_agent refreshSkills: skipped, agent-manager calls GitHub without a token", agentManagerToolPrefix)
note("deployment %s/%s carries no GitHub credential (%s): refreshSkills would resolve %s's head on the anonymous window this machine shares", platformNamespace, agentManagerMCPServer, strings.Join(agentManagerGitHubEnv, ", "), fixture.Repo)
return fmt.Sprintf("SKIP: update_agent{refreshSkills} — agent-manager calls GitHub unauthenticated, on GitHub's anonymous rate limit (60 requests an hour, shared by this machine); record githubToken.source (`agentlab configure --github-token-source <ref>`) or export $%s, then `agentlab platform`, to prove it", GitHubTokenEnv), nil
return fmt.Sprintf("SKIP: update_agent{refreshSkills} — agent-manager calls GitHub unauthenticated, on GitHub's anonymous rate limit (60 requests an hour, shared by this machine); record the skills GitHub App (`agentlab configure --github-app-id <id> --github-app-installation-id <id> --github-app-private-key-source <ref>`), githubToken.source or $%s, then `agentlab platform`, to prove it", GitHubTokenEnv), nil
}
step("%supdate_agent refreshSkills — the skill re-pins to %s's head %.12s", agentManagerToolPrefix, fixture.Repo, head)
refreshed, err := agentManagerUpdate(session, map[string]any{nameKey: agentsTestAgent, refreshSkillsKey: true})
Expand Down
2 changes: 1 addition & 1 deletion internal/lab/backstagetest_edit.go
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ func proveRefreshSkills(primary *portalSession, spec agentSpec) (string, error)
if !authenticated {
step("E4/E5 Update skills: skipped, agent-manager calls GitHub without a token")
note("deployment %s/%s carries no GitHub credential (%s): refreshSkills would resolve %s's head on the anonymous window this machine shares", platformNamespace, agentManagerMCPServer, strings.Join(agentManagerGitHubEnv, ", "), skillsTestRepo)
return fmt.Sprintf("SKIP: E4/E5 validate_agent{update, refreshSkills} and update_agent{refreshSkills} — agent-manager calls GitHub unauthenticated, on GitHub's anonymous rate limit (60 requests an hour, shared by this machine); record githubToken.source (`agentlab configure --github-token-source <ref>`) or export $%s, then `agentlab platform`, to prove them", GitHubTokenEnv), nil
return fmt.Sprintf("SKIP: E4/E5 validate_agent{update, refreshSkills} and update_agent{refreshSkills} — agent-manager calls GitHub unauthenticated, on GitHub's anonymous rate limit (60 requests an hour, shared by this machine); record the skills GitHub App (`agentlab configure --github-app-id <id> --github-app-installation-id <id> --github-app-private-key-source <ref>`), githubToken.source or $%s, then `agentlab platform`, to prove them", GitHubTokenEnv), nil
}
skill := spec.Skills[0]

Expand Down
25 changes: 18 additions & 7 deletions internal/lab/discover.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,13 @@ type Discovery struct {
// is set on the host.
GitHubTokenSource string
GitHubToken bool
ClusterExists bool
ClusterPorts map[int]bool
KindGateway string
Servers []HostServer
FLM *FLMServer
// GitHubApp is the skills GitHub App recorded (githubApp), zero when none.
GitHubApp config.GitHubApp
ClusterExists bool
ClusterPorts map[int]bool
KindGateway string
Servers []HostServer
FLM *FLMServer
// KVMMissing names the KVM devices this machine lacks (vmmanager.go):
// empty when the platform's VM provisioner can run as a pod of the node.
KVMMissing []string
Expand Down Expand Up @@ -110,9 +112,9 @@ const flmOwner = "FastFlowLM"
// Discover probes this machine. Nothing here needs the cluster; every probe
// is loopback or a local CLI and degrades to "not found".
func Discover(cfg *config.Config) *Discovery {
d := &Discovery{AnthropicKey: os.Getenv(AnthropicKeyEnv) != "", KeySource: cfg.AIKey.Source, GitHubTokenSource: cfg.GitHubToken.Source, GitHubToken: os.Getenv(GitHubTokenEnv) != ""}
d := &Discovery{AnthropicKey: os.Getenv(AnthropicKeyEnv) != "", KeySource: cfg.AIKey.Source, GitHubTokenSource: cfg.GitHubToken.Source, GitHubToken: os.Getenv(GitHubTokenEnv) != "", GitHubApp: cfg.GitHubApp}
d.Tools = toolVersions(dockerVersion())
if d.KeySource != "" || d.GitHubTokenSource != "" {
if d.KeySource != "" || d.GitHubTokenSource != "" || d.GitHubApp.Configured() {
d.SecretTool = secretToolVersion()
}
d.ClusterExists, d.ClusterPorts = kindNodePublishedPorts(cfg.ControlPlaneNode())
Expand Down Expand Up @@ -309,6 +311,10 @@ func (d *Discovery) Preflight() error {
problems = append(problems, fmt.Sprintf("%s is not on PATH (or does not answer) — githubToken.source %s is placed into the Secret %s/%s by `%s secret copy --to-secret` at every up and platform\n install: %s (or clear the source: agentlab configure --github-token-source \"\")",
secretTool, d.GitHubTokenSource, platformNamespace, gitHubTokenSecret, secretTool, secretToolInstall))
}
if d.GitHubApp.Configured() && d.SecretTool == "" {
problems = append(problems, fmt.Sprintf("%s is not on PATH (or does not answer) — githubApp.privateKeySource %s is placed into the Secret %s/%s by `%s secret copy --to-secret` at every up and platform\n install: %s (or clear the App: agentlab configure --github-app-id \"\" --github-app-installation-id \"\" --github-app-private-key-source \"\")",
secretTool, d.GitHubApp.PrivateKeySource, platformNamespace, gitHubAppSecret, secretTool, secretToolInstall))
}
if len(problems) == 0 {
return nil
}
Expand Down Expand Up @@ -417,11 +423,16 @@ func (d *Discovery) Report(cfg *config.Config) string {
line("Anthropic key", "no aiKey.source and $%s is not set — the default ModelConfig gets a placeholder (it resolves, agent turns fail at Anthropic) until the key is placed: `%s secret copy <ref> --to-secret kind-%s/%s/%s/%s`, or the source recorded (`agentlab configure --ai-key-source <ref>`), or the variable exported and `agentlab platform` re-run",
AnthropicKeyEnv, secretTool, cfg.ClusterName, kagentNamespace, anthropicSecret, anthropicSecretKey)
}
if d.GitHubApp.Configured() {
line("GitHub App", "githubApp %s (installation %s), private key from %s — %s places it into the Secret %s/%s at every up and platform: agent-manager resolves skills as the App, at its rate limit; agentlab never reads the key", d.GitHubApp.AppID, d.GitHubApp.InstallationID, d.GitHubApp.PrivateKeySource, secretTool, platformNamespace, gitHubAppSecret)
}
switch {
case d.GitHubTokenSource != "":
line("GitHub token", "githubToken.source %s — %s places it into the Secret %s/%s at every up and platform: the portal's skill discovery and agent-manager's skill resolution call GitHub authenticated (5000 requests an hour); agentlab never reads the value", d.GitHubTokenSource, secretTool, platformNamespace, gitHubTokenSecret)
case d.GitHubToken:
line("GitHub token", "$%s is set — the portal's skill discovery and agent-manager's skill resolution call GitHub authenticated (5000 requests an hour) from deploy time", GitHubTokenEnv)
case d.GitHubApp.Configured():
line("GitHub token", "no githubToken.source and $%s is not set — the portal's skill discovery shares this machine's unauthenticated GitHub window (60 requests an hour); agent-manager uses the App", GitHubTokenEnv)
default:
line("GitHub token", "no githubToken.source and $%s is not set — skill discovery and resolution share this machine's unauthenticated GitHub window (60 requests an hour) until the source is recorded (`agentlab configure --github-token-source <ref>`) or the variable exported, and `agentlab platform` re-runs", GitHubTokenEnv)
}
Expand Down
54 changes: 54 additions & 0 deletions internal/lab/githubapp.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
package lab

import (
"context"
"fmt"
"strings"

corev1 "k8s.io/api/core/v1"

"github.com/giantswarm/agentlab/internal/config"
)

// gitHubAppSecret is the Secret of the skills GitHub App in agent-platform/,
// in the shape agent-manager's chart reads (skills.github.app.secretName):
// the public ids, which agentlab applies, and the private key, which the
// secret tooling places from githubApp.privateKeySource so agentlab never
// reads it. Server-side apply owns only the id keys, so the key the tooling
// patched survives every later apply.
const (
gitHubAppSecret = "agentlab-github-app" // #nosec G101 -- Secret NAME, not a credential
gitHubAppSecretAppID = "app-id"
gitHubAppSecretInstallationID = "installation-id"
gitHubAppSecretPrivateKey = "private-key"
)

// gitHubAppWired is the render's answer: the App is recorded and the chart
// line takes the key (the 4.x agent-manager chart's skills.github.app).
func gitHubAppWired(cfg *config.Config) bool {
return cfg.GitHubApp.Configured() && !cfg.LegacyChart()
}

// ensureGitHubAppSecret is the deploy-time half, run before the install: the
// values name the Secret whenever the App is recorded, so a placement that
// fails fails the run and a lab never deploys agent-manager without it.
// Without an App nothing is written.
func ensureGitHubAppSecret(_ context.Context, cfg *config.Config) error {
if !gitHubAppWired(cfg) {
return nil
}
app := cfg.GitHubApp
if err := ensureSecret(platformNamespace, gitHubAppSecret, corev1.SecretTypeOpaque, map[string][]byte{
gitHubAppSecretAppID: []byte(app.AppID),
gitHubAppSecretInstallationID: []byte(app.InstallationID),
}); err != nil {
return err
}
answer, err := runSecretTool(secretCopyArgs(app.PrivateKeySource, secretTarget(cfg, platformNamespace, gitHubAppSecret, gitHubAppSecretPrivateKey))...)
if err != nil {
return fmt.Errorf("secret %s/%s key %s from githubApp.privateKeySource %s: %w\n agent-manager references it; when %s answers, re-run `agentlab platform`",
platformNamespace, gitHubAppSecret, gitHubAppSecretPrivateKey, app.PrivateKeySource, err, secretTool)
}
note("secret %s/%s: App %s, installation %s, private key placed from %s by %s (%s); agentlab never read it — agent-manager resolves skills as the App", platformNamespace, gitHubAppSecret, app.AppID, app.InstallationID, app.PrivateKeySource, secretTool, strings.TrimSpace(answer))
return nil
}
Loading
Loading