Skip to content

Commit 4df0115

Browse files
arefbehboudirdehuyss
arefbehboudi
and
rdehuyss
authored
Let onboarding configure a custom base URL (#89)
The credentials step could only set a model and an API key, so reaching a proxy, gateway or self-hosted server meant hand-editing the config file afterwards. ProviderConfig already carried baseUrl and three of the four factories applied it, so onboarding was the only missing piece. Add an optional Base URL field to the credentials step, carried through the session and written to agent.llm.providers.default.base-url only when it is filled in. The value is trimmed, pre-filled from existing configuration, and cleared alongside the model and API key when the provider changes. GoogleGenAiChatModelFactory was the one factory ignoring baseUrl: its SDK takes the endpoint as HttpOptions on the client rather than on the chat options, so it needed wiring separately. --------- Co-authored-by: rdehuyss <ronald.dehuysser@gmail.com>
1 parent f63975b commit 4df0115

22 files changed

Lines changed: 821 additions & 694 deletions

File tree

‎AGENTS.md‎

Lines changed: 35 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -77,37 +77,39 @@ User/Agent → TaskManager.create()
7777
- **`TaskNotFoundException`** (`base/.../tasks/TaskNotFoundException.java`): Thrown when a task file cannot be found or read by its ID.
7878

7979
### Workspace
80-
- `workspace/AGENT.md`: System instructions + user-specific information (editable during onboarding).
80+
- `workspace/AGENT.private.md`: System instructions + user-specific information (edited during onboarding step 4; falls back to `AGENT.md`).
81+
- `workspace/AGENT.md`: Fallback system-instructions file when `AGENT.private.md` is absent.
8182
- `workspace/AGENT-ORIGINAL.md`: Template / backup of original AGENT.md.
8283
- `workspace/INFO.md`: Environment context auto-injected into every prompt.
8384
- `workspace/context/`: Agent memory and context storage (e.g. `jobrunr.md`).
8485
- `workspace/skills/`: Extensible skill files (`SKILL.md` per skill, loaded dynamically by `SkillsTool`).
8586
- `workspace/tasks/`: Date-bucketed task files + `recurring/` sub-folder.
87+
- `workspace/agents/`: Subagent definition files (`<name>.md`) — see "Agents (Subagents)" below.
8688

8789
---
8890

8991
## Agent System
9092

9193
**`DefaultAgent`** (`base/src/main/java/ai/javaclaw/agent/DefaultAgent.java`) wraps Spring AI's `ChatClient`.
9294

93-
**Prompt construction** (`JavaClawConfiguration.java`):
94-
- System prompt = `workspace/AGENT.md` + `workspace/INFO.md`
95-
- **Advisors**: `SimpleLoggerAdvisor`, `ToolCallAdvisor`, `MessageChatMemoryAdvisor` (chat history)
95+
**Prompt construction** (`MainChatClientProvider.java`):
96+
- System prompt = `workspace/AGENT.private.md` (falls back to `workspace/AGENT.md`) + `workspace/INFO.md`
97+
- **Advisors**: `SimpleLoggerAdvisor`, `ToolSearchToolCallingAdvisor` (falls back to plain `ToolCallingAdvisor`), `MessageChatMemoryAdvisor` (chat history)
9698

9799
**Default Tools** always injected:
98100
| Tool | Purpose |
99101
|---|---|
100-
| `TaskTool` | `createTask`, `scheduleTask`, `scheduleRecurringTask` |
102+
| `JavaClawTaskTool` | `createTask`, `scheduleTask`, `scheduleRecurringTask` |
101103
| `CheckListTool` | Multi-step structured tracking (one `in_progress` at a time) |
102-
| `ShellTools` | Bash execution |
103104
| `FileSystemTools` | Read/write/edit files |
104105
| `SmartWebFetchTool` | Intelligent web scraping |
105106
| `SkillsTool` | Loads `SKILL.md` files from `workspace/skills/` |
106107
| `McpTool` | Runtime MCP server management |
107-
| `MCP Tools` | `SyncMcpToolCallbackProvider` |
108-
| `BraveWebSearchTool` | 15 results (only if Brave API key configured) |
108+
| MCP tools | `SyncMcpToolCallbackProvider` callbacks |
109+
| subagent `TaskTool` | Dispatch to subagents in `workspace/agents/` |
110+
| auto-discovered tools | e.g. `BraveWebSearchTool` (only if Brave API key configured) |
109111

110-
**Supported LLM Providers** — each lives in its own `providers/<name>/` module and contributes its `AgentOnboardingProvider` through an `@AutoConfiguration` class registered in the module's `AutoConfiguration.imports`. The active model is selected centrally by `spring.ai.model.chat` (Spring AI), not per-provider; the default `unknown` disables all of them until onboarding sets it:
112+
**Supported LLM Providers** — each lives in its own `providers/<name>/` module and contributes its `AgentOnboardingProvider` through an `@AutoConfiguration` class registered in the module's `AutoConfiguration.imports`. Each provider in `agent.llm.providers.*` is built through a `ChatModelFactory` SPI implementation and registered in `DefaultChatClientRegistry`; the `default` entry is the main agent's provider:
111113

112114
| Provider | Module | Default Model | API Key |
113115
|---|---|---|---|
@@ -137,17 +139,36 @@ Incoming message → ChannelMessageReceivedEvent (channel name, message text)
137139

138140
## Configuration Management
139141

140-
- **`ConfigurationManager`** (`base/src/main/java/ai/javaclaw/configuration/ConfigurationManager.java`): Updates nested YAML key-value paths in `application.yaml` via SnakeYAML. Publishes `ConfigurationChangedEvent` on update.
141-
- Runtime config file: `app/src/main/resources/application.yaml` (read at startup; mutated by onboarding).
142+
- **`ConfigurationManager`** (`base/src/main/java/ai/javaclaw/configuration/ConfigurationManager.java`): Updates nested YAML key-value paths in the runtime config file via SnakeYAML. Publishes `ConfigurationChangedEvent` on update, which triggers a full application restart.
143+
- Runtime config file: `app/src/main/resources/application.private.yaml` (gitignored; imported into `application.yaml` via `spring.config.import`; read at startup and mutated by onboarding and the agents UI). `application.yaml` holds static defaults only.
142144
- Key config paths:
143145
- `agent.onboarding.completed` — set to `true` after onboarding
144146
- `agent.workspace` — path to workspace root (`file:./workspace/`)
145-
- `spring.ai.model.chat` — overridden to selected provider/model during onboarding
147+
- `agent.llm.providers.<name>.*` — `provider`, `base-url`, `api-key`, `model` per named provider (the `default` entry drives the main agent)
146148
- `jobrunr.background-job-server.worker-count: 1`
147149
- `jobrunr.dashboard.port: 8081`
148150

149151
---
150152

153+
## Agents (Subagents)
154+
155+
The main assistant can delegate to subagents. Each subagent is defined by two pieces:
156+
157+
- **Instructions**: a Markdown file `workspace/agents/<name>.md` with `name`, `description`, and `model` frontmatter plus a Markdown body (the subagent's instructions).
158+
- **Provider config**: a named `agent.llm.providers.<name>` entry in `application.private.yaml` (`provider`, `base-url`, `api-key`, `model`). The subagent's `model:` frontmatter is the routing value — its own provider-entry name.
159+
160+
**`SubagentStore`** (`base/.../llm/SubagentStore.java`) reads/writes the Markdown files. `MainChatClientProvider` wires a subagent-dispatch `TaskTool` backed by a per-provider `ChatClient.Builder` map keyed by provider name.
161+
162+
The management UI (`GET /settings/agents`, template `settings/agents.html.peb`) is driven entirely by htmx fragments instead of a JSON API:
163+
164+
- **`AgentPageController`** (`app/src/main/java/ai/javaclaw/agents/AgentPageController.java`, an `@Controller`) serves HTML fragments:
165+
- `GET /settings/agents/fragments/list` → `settings/agents/list.html.peb`
166+
- `GET /settings/agents/new` and `GET /settings/agents/{name}/edit` → `settings/agents/drawer.html.peb`
167+
- `POST /settings/agents`, `PUT /settings/agents/{name}`, `DELETE /settings/agents/{name}` (form-encoded; validation errors return `422` + `HX-Retarget: #agent-drawer`)
168+
- The page shell loads the list via `hx-trigger="load"`, opens the drawer via `hx-get` + `hx-on:htmx:after-swap`, saves via `hx-post`/`hx-put`, and deletes via `hx-delete` + `hx-confirm`. The only remaining JavaScript is drawer open/close, provider→default-model cascade, and enabling 422-response swaps.
169+
170+
---
171+
151172
## Tools and Capabilities
152173
- **File System:** Read, write, and edit files.
153174
- **Shell:** Execute bash commands.
@@ -193,8 +214,8 @@ Templates live under `templates/onboarding/`, with plugin steps contributed from
193214

194215
## Tests
195216

196-
- `base/src/test/` — `TaskManagerTest`: task creation, file naming, JobRunr integration (in-memory storage + background server).
217+
- `base/src/test/` — `TaskManagerTest`, `TaskHandlerTest`, `FileSystemTaskRepositoryTest`: task lifecycle + file persistence; `DefaultChatClientRegistryTest` and `SubagentStoreTest`: model registry + subagent storage; `ConfigurationManagerTest`: YAML path updates.
197218
- `plugins/discord/src/test/` — `DiscordChannelTest`, `DiscordOnboardingProviderTest`: authorized Discord flow + onboarding config handling.
198219
- `plugins/telegram/src/test/` — `TelegramChannelTest`: unauthorized user rejection, authorized message flow (mocked).
199220
- `providers/anthropic/src/test/` — `AnthropicClaudeCodeBackendTest`: Claude Code OAuth token extraction.
200-
- `app/src/test/` — `OnboardingControllerTest`: session-based workflow; `JavaClawApplicationTests`: full Spring context load with Testcontainers.
221+
- `app/src/test/` — `OnboardingControllerTest`: session-based workflow; `AgentPageControllerTest`: agents page fragment + validation behavior; `JavaClawApplicationTests`: full Spring context load with Testcontainers.

app/src/main/java/ai/javaclaw/api/IndexController.java renamed to app/src/main/java/ai/javaclaw/IndexController.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
package ai.javaclaw.api;
1+
package ai.javaclaw;
22

33
import org.springframework.core.env.Environment;
44
import org.springframework.stereotype.Controller;

‎app/src/main/java/ai/javaclaw/agents/AgentPageController.java‎

Lines changed: 278 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,278 @@
1+
package ai.javaclaw.agents;
2+
3+
import ai.javaclaw.configuration.ConfigurationManager;
4+
import ai.javaclaw.llm.LlmProviderProperties;
5+
import ai.javaclaw.llm.LlmProviderProperties.ProviderConfig;
6+
import ai.javaclaw.llm.SubagentStore;
7+
import ai.javaclaw.llm.SubagentStore.Subagent;
8+
import ai.javaclaw.onboarding.AgentOnboardingProvider;
9+
import ai.javaclaw.onboarding.AgentOnboardingProviders;
10+
import jakarta.servlet.http.HttpServletResponse;
11+
import org.springframework.http.HttpStatus;
12+
import org.springframework.stereotype.Controller;
13+
import org.springframework.ui.Model;
14+
import org.springframework.web.bind.annotation.DeleteMapping;
15+
import org.springframework.web.bind.annotation.ExceptionHandler;
16+
import org.springframework.web.bind.annotation.GetMapping;
17+
import org.springframework.web.bind.annotation.PathVariable;
18+
import org.springframework.web.bind.annotation.PostMapping;
19+
import org.springframework.web.bind.annotation.PutMapping;
20+
import org.springframework.web.bind.annotation.RequestMapping;
21+
import org.springframework.web.bind.annotation.RequestParam;
22+
import org.springframework.web.bind.annotation.ResponseStatus;
23+
24+
import java.io.IOException;
25+
import java.util.ArrayList;
26+
import java.util.LinkedHashMap;
27+
import java.util.List;
28+
import java.util.Map;
29+
import java.util.regex.Pattern;
30+
31+
@Controller
32+
@RequestMapping("/settings/agents")
33+
public class AgentPageController {
34+
35+
private static final Pattern NAME_PATTERN = Pattern.compile("[a-z0-9-]+");
36+
private static final String PROVIDERS_PREFIX = "agent.llm.providers";
37+
38+
static class ValidationException extends RuntimeException {
39+
ValidationException(String message) {
40+
super(message);
41+
}
42+
}
43+
44+
static class NotFoundException extends RuntimeException {
45+
}
46+
47+
private final SubagentStore store;
48+
private final AgentOnboardingProviders providers;
49+
private final LlmProviderProperties providerProperties;
50+
private final ConfigurationManager configurationManager;
51+
52+
public AgentPageController(SubagentStore store,
53+
AgentOnboardingProviders providers,
54+
LlmProviderProperties providerProperties,
55+
ConfigurationManager configurationManager) {
56+
this.store = store;
57+
this.providers = providers;
58+
this.providerProperties = providerProperties;
59+
this.configurationManager = configurationManager;
60+
}
61+
62+
@GetMapping("/fragments/list")
63+
public String listFragment(Model model) {
64+
return populateList(model);
65+
}
66+
67+
@GetMapping("/new")
68+
public String newForm(Model model) {
69+
populateDrawer(model, null, false);
70+
return "settings/agents/drawer";
71+
}
72+
73+
@GetMapping("/{name}/edit")
74+
public String editForm(@PathVariable String name, Model model) {
75+
if (name == null || !NAME_PATTERN.matcher(name).matches()) {
76+
throw new NotFoundException();
77+
}
78+
populateDrawer(model, toDetail(name), true);
79+
return "settings/agents/drawer";
80+
}
81+
82+
@PostMapping
83+
public String create(@RequestParam Map<String, String> form, Model model, HttpServletResponse response) {
84+
String name;
85+
try {
86+
name = validateNewName(form.get("name"));
87+
validateProvider(form.get("provider"));
88+
} catch (ValidationException e) {
89+
return drawerError(model, response, form, e.getMessage(), false);
90+
}
91+
save(name, form);
92+
return populateList(model);
93+
}
94+
95+
@PutMapping("/{name}")
96+
public String update(@PathVariable String name, @RequestParam Map<String, String> form,
97+
Model model, HttpServletResponse response) {
98+
requireExisting(name);
99+
try {
100+
validateProvider(form.get("provider"));
101+
} catch (ValidationException e) {
102+
return drawerError(model, response, form, e.getMessage(), true);
103+
}
104+
save(name, form);
105+
return populateList(model);
106+
}
107+
108+
@DeleteMapping("/{name}")
109+
public String delete(@PathVariable String name, Model model) {
110+
if (name == null || !NAME_PATTERN.matcher(name).matches()) {
111+
throw new NotFoundException();
112+
}
113+
try {
114+
boolean removedFile = store.delete(name);
115+
boolean hadConfig = providerProperties.getProviders().containsKey(name);
116+
if (!removedFile && !hadConfig) {
117+
throw new NotFoundException();
118+
}
119+
configurationManager.removeProperty(PROVIDERS_PREFIX + "." + name);
120+
} catch (IOException e) {
121+
throw new RuntimeException("Failed to delete agent " + name, e);
122+
}
123+
return populateList(model);
124+
}
125+
126+
@ExceptionHandler(NotFoundException.class)
127+
@ResponseStatus(HttpStatus.NOT_FOUND)
128+
public void handleNotFound() {
129+
}
130+
131+
private String drawerError(Model model, HttpServletResponse response,
132+
Map<String, String> form, String error, boolean isEdit) {
133+
response.setStatus(HttpStatus.UNPROCESSABLE_ENTITY.value());
134+
response.setHeader("HX-Retarget", "#agent-drawer");
135+
populateDrawer(model, form, isEdit);
136+
model.addAttribute("error", error);
137+
return "settings/agents/drawer";
138+
}
139+
140+
private void save(String name, Map<String, String> form) {
141+
try {
142+
store.save(new Subagent(name, form.get("model"), form.get("description"), form.get("content")));
143+
} catch (IOException e) {
144+
throw new RuntimeException("Failed to save agent " + name, e);
145+
}
146+
147+
Map<String, Object> props = new LinkedHashMap<>();
148+
props.put(PROVIDERS_PREFIX + "." + name + ".provider", form.get("provider"));
149+
if (hasText(form.get("baseUrl"))) {
150+
props.put(PROVIDERS_PREFIX + "." + name + ".base-url", form.get("baseUrl").trim());
151+
}
152+
if (hasText(form.get("apiKey"))) {
153+
props.put(PROVIDERS_PREFIX + "." + name + ".api-key", form.get("apiKey").trim());
154+
}
155+
if (hasText(form.get("model"))) {
156+
props.put(PROVIDERS_PREFIX + "." + name + ".model", form.get("model").trim());
157+
}
158+
try {
159+
configurationManager.updateProperties(props);
160+
} catch (IOException e) {
161+
throw new RuntimeException("Failed to save agent " + name, e);
162+
}
163+
}
164+
165+
private String validateNewName(String raw) {
166+
String name = raw == null ? "" : raw.trim();
167+
if (name.isBlank() || !NAME_PATTERN.matcher(name).matches()) {
168+
throw new ValidationException("Agent name must match [a-z0-9-]+");
169+
}
170+
if (providerProperties.getProviders().containsKey(name) || store.exists(name)) {
171+
throw new ValidationException("An agent named '" + name + "' already exists");
172+
}
173+
return name;
174+
}
175+
176+
private void validateProvider(String provider) {
177+
if (provider == null || provider.isBlank()) {
178+
throw new ValidationException("A provider is required");
179+
}
180+
if (providers.findById(provider).isEmpty()) {
181+
throw new ValidationException("Unknown provider: " + provider);
182+
}
183+
}
184+
185+
private void requireExisting(String name) {
186+
if (name == null || !NAME_PATTERN.matcher(name).matches() || !store.exists(name)) {
187+
throw new NotFoundException();
188+
}
189+
}
190+
191+
private static boolean hasText(String s) {
192+
return s != null && !s.isBlank();
193+
}
194+
195+
private String populateList(Model model) {
196+
List<Map<String, String>> cards = new ArrayList<>();
197+
for (Subagent s : store.list()) {
198+
ProviderConfig config = providerProperties.getProviders().get(s.name());
199+
String provider = config != null ? config.getProvider() : null;
200+
String modelId = config != null ? config.getModel() : null;
201+
Map<String, String> card = new LinkedHashMap<>();
202+
card.put("name", s.name());
203+
card.put("sub", agentSub(provider, modelId, s.description()));
204+
cards.add(card);
205+
}
206+
model.addAttribute("agents", cards);
207+
return "settings/agents/list";
208+
}
209+
210+
private String agentSub(String provider, String modelId, String description) {
211+
String sub = labelFor(provider) + " · " + (modelId == null || modelId.isBlank() ? "—" : modelId);
212+
if (description != null && !description.isBlank()) {
213+
sub += " · " + description;
214+
}
215+
return sub;
216+
}
217+
218+
private void populateDrawer(Model model, Map<String, String> values, boolean isEdit) {
219+
String name = values == null ? "" : values.getOrDefault("name", "");
220+
model.addAttribute("isEdit", isEdit);
221+
model.addAttribute("providers", providerOptions());
222+
model.addAttribute("drawerTitle", isEdit ? "Edit " + name : "Add Agent");
223+
model.addAttribute("formAction", isEdit ? "/settings/agents/" + name : "/settings/agents");
224+
model.addAttribute("agentName", name);
225+
model.addAttribute("nameReadonly", isEdit);
226+
model.addAttribute("selectedProvider", values == null ? "" : values.getOrDefault("provider", ""));
227+
model.addAttribute("baseUrl", values == null ? "" : values.getOrDefault("baseUrl", ""));
228+
model.addAttribute("model", values == null ? "" : values.getOrDefault("model", ""));
229+
model.addAttribute("description", values == null ? "" : values.getOrDefault("description", ""));
230+
model.addAttribute("content", values == null ? "" : values.getOrDefault("content", ""));
231+
model.addAttribute("apiKeyMasked", values == null ? "" : values.getOrDefault("apiKeyMasked", ""));
232+
}
233+
234+
private Map<String, String> toDetail(String name) {
235+
ProviderConfig config = providerProperties.getProviders().get(name);
236+
Subagent agent = store.get(name).orElseThrow(NotFoundException::new);
237+
String provider = config != null ? config.getProvider() : "";
238+
Map<String, String> values = new LinkedHashMap<>();
239+
values.put("name", agent.name());
240+
values.put("provider", provider);
241+
values.put("baseUrl", config != null && config.getBaseUrl() != null ? config.getBaseUrl() : "");
242+
values.put("model", config != null && config.getModel() != null ? config.getModel() : "");
243+
values.put("description", agent.description() == null ? "" : agent.description());
244+
values.put("content", agent.content() == null ? "" : agent.content());
245+
values.put("apiKeyMasked", config != null ? maskApiKey(config.getApiKey()) : "");
246+
return values;
247+
}
248+
249+
private List<Map<String, String>> providerOptions() {
250+
List<Map<String, String>> options = new ArrayList<>();
251+
for (AgentOnboardingProvider p : providers.getAll()) {
252+
Map<String, String> m = new LinkedHashMap<>();
253+
m.put("id", p.getId());
254+
m.put("label", p.getLabel());
255+
m.put("defaultModel", p.defaultModel());
256+
options.add(m);
257+
}
258+
return options;
259+
}
260+
261+
private String labelFor(String providerId) {
262+
if (providerId == null || providerId.isBlank()) {
263+
return "";
264+
}
265+
return providers.findById(providerId).map(AgentOnboardingProvider::getLabel).orElse(providerId);
266+
}
267+
268+
private static String maskApiKey(String apiKey) {
269+
if (apiKey == null || apiKey.isBlank()) {
270+
return "";
271+
}
272+
String key = apiKey.trim();
273+
if (key.length() <= 7) {
274+
return "••••";
275+
}
276+
return key.substring(0, 3) + "..." + key.substring(key.length() - 4);
277+
}
278+
}

app/src/main/java/ai/javaclaw/chat/api/ChatController.java renamed to app/src/main/java/ai/javaclaw/chat/ChatController.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
package ai.javaclaw.chat.api;
1+
package ai.javaclaw.chat;
22

33
import org.springframework.beans.factory.annotation.Value;
44
import org.springframework.stereotype.Controller;

0 commit comments

Comments
 (0)