fix: address review findings on profiles: shared throttle, defaults, wait budget

Three problems a review found in how profiles interact with the rest of
the extension.

Throttle gates were replaced whenever the configured cap or interval
changed. Two profiles on one endpoint with different settings alternate
every tick, so each call got a fresh gate with a full set of permits and
the cap meant nothing. The gate is now reconfigured in place: a smaller
cap admits nothing until in-flight requests drain below it, a larger one
hands the new permits to the oldest waiters at once. release() repays a
shrunk-cap deficit before admitting anyone.

A profile config was built on ConfigStore.withDefaults(), which carries
model=gpt-4o-mini, so an Anthropic profile that named no model asked for
an OpenAI model. Profiles now load the file as the whole configuration,
the way llm:load-config does, and the provider's own default applies.

Every Await read the global timeout and retry budget. Calls now use the
calling agent's effective config, and an async call captures the budget
at launch so a later config change or rebinding cannot alter it.

Tests: throttle spec covers in-place reconfigure, shrink under load, and
grow with waiters; tests.txt covers a model-less Anthropic profile, a
profile's own one-second budget timing out while the observer succeeds,
and an async call keeping its budget after the global config changes.

Author: JNK234

demos/test-config-fast-timeout

@@ -0,0 +1,6 @@
+# Test fixture: global config with a one-second wait budget and no retry allowance
+provider=openai
+openai_api_key=test-key
+model=model-fast-timeout
+timeout_seconds=1
+retry_max_elapsed_seconds=0

demos/test-profile-anthropic

@@ -0,0 +1,3 @@
+# Test fixture: Anthropic profile with no model, so the provider default must apply
+provider=anthropic
+anthropic_api_key=test-key

demos/test-profile-slow

@@ -0,0 +1,6 @@
+# Test fixture: profile with a one-second wait budget and no retry allowance
+provider=openai
+openai_api_key=test-key
+model=model-slow
+timeout_seconds=1
+retry_max_elapsed_seconds=0

docs/API-REFERENCE.md

@@ -920,6 +920,12 @@ exists replaces it. A rejected reload leaves the existing profile untouched.
   setters act on the default configuration only. A profile is what its file
   says. To change a profile, edit the file and load it again.
 - Two profiles on the same provider and endpoint share one request throttle.
+  If their `max_concurrent_requests` or `min_request_interval_ms` differ, the
+  shared throttle takes whichever setting the most recent call carried, without
+  losing count of requests already in flight. Give profiles on one endpoint the
+  same throttling settings.
+- A profile's `timeout_seconds` and retry settings govern calls made through
+  it. An async call keeps the budget in force when it was launched.
 
 ### llm:use-profile
 

src/main/LLMExtension.scala

@@ -82,13 +82,16 @@ class LLMExtension extends DefaultClassManager {
    * Create an AwaitableReporter that wraps a Future to provide truly async behavior
    * The Future starts immediately but execution defers until runresult is called
    */
-  private def createAwaitableReporter(future: Future[String]): AnonymousReporter = {
+  private def createAwaitableReporter(future: Future[String], timeout: FiniteDuration): AnonymousReporter = {
     new AnonymousReporter {
       override def syntax: Syntax = Syntax.reporterSyntax(right = List(), ret = Syntax.StringType)
 
       override def report(context: Context, args: Array[AnyRef]): AnyRef = {
         try {
-          Await.result(future, getAwaitTimeout)
+          // The budget in force for the calling agent when the request was
+          // launched: a later config change or profile rebinding cannot alter
+          // how long an already-pending call may take.
+          Await.result(future, timeout)
         } catch {
           case e: Exception =>
             throw new ExtensionException(s"Async LLM operation failed: ${e.getMessage}")
@@ -271,8 +274,8 @@ class LLMExtension extends DefaultClassManager {
   /**
    * Get timeout from config, falling back to 30 seconds
    */
-  private def getTimeoutSeconds: Int =
-    configStore.get(ConfigStore.TIMEOUT_SECONDS).map { s =>
+  private def getTimeoutSeconds(config: ConfigStore): Int =
+    config.get(ConfigStore.TIMEOUT_SECONDS).map { s =>
       scala.util.Try(s.toInt).getOrElse {
         System.err.println(s"WARNING: Invalid timeout_seconds value '$s' (not a valid integer), using default 30")
         30
@@ -301,13 +304,13 @@ class LLMExtension extends DefaultClassManager {
    * bound is unchanged from before throttling existed, so an unthrottled model — the
    * default — behaves exactly as it did.
    */
-  private def getAwaitTimeout: FiniteDuration = {
-    val retryBudget = configStore.get(RetryPolicy.MAX_ELAPSED_SECONDS)
+  private def awaitTimeoutFor(config: ConfigStore): FiniteDuration = {
+    val retryBudget = config.get(RetryPolicy.MAX_ELAPSED_SECONDS)
       .flatMap(s => scala.util.Try(s.trim.toDouble).toOption)
       .filter(d => d >= 0.0 && d.isFinite)
       .map(d => (d * 1000.0).toLong.millis)
       .getOrElse(RetryPolicy.DefaultMaxElapsed)
-    getTimeoutSeconds.seconds + retryBudget
+    getTimeoutSeconds(config).seconds + retryBudget
   }
 
   /**
@@ -599,7 +602,7 @@ class LLMExtension extends DefaultClassManager {
 
         // Send chat request with user message included, but don't mutate history yet
         val responseFuture = provider.chatWithFullResponse(snapshotHistory(agent) :+ userMessage)
-        val response = Await.result(responseFuture, getAwaitTimeout)
+        val response = Await.result(responseFuture, awaitTimeoutFor(effectiveConfig(agent)))
         recordUsage(agent, response)
         val responseMessage = replyMessage(response)
 
@@ -640,7 +643,7 @@ class LLMExtension extends DefaultClassManager {
         }
 
         // Return AnonymousReporter that wraps the Future
-        createAwaitableReporter(responseFuture)
+        createAwaitableReporter(responseFuture, awaitTimeoutFor(effectiveConfig(agent)))
 
       } catch {
         case e: Exception =>
@@ -692,7 +695,7 @@ class LLMExtension extends DefaultClassManager {
 
         // Send chat request
         val responseFuture = provider.chatWithFullResponse(tempHistory.toSeq)
-        val response = Await.result(responseFuture, getAwaitTimeout)
+        val response = Await.result(responseFuture, awaitTimeoutFor(effectiveConfig(agent)))
         recordUsage(agent, response)
         val responseMessage = replyMessage(response)
 
@@ -754,7 +757,7 @@ class LLMExtension extends DefaultClassManager {
         // The prompt still spells out the options, so a provider that ignores
         // the constraint behaves exactly as it did before.
         val responseFuture = provider.chatWithFormat(tempHistory.toSeq, EnumFormat(choices))
-        val response = Await.result(responseFuture, getAwaitTimeout)
+        val response = Await.result(responseFuture, awaitTimeoutFor(effectiveConfig(agent)))
         recordUsage(agent, response)
 
         // Extract text: prefer content, fall back to thinking field
@@ -854,7 +857,7 @@ class LLMExtension extends DefaultClassManager {
         val userMessage = ChatMessage.user(inputText)
 
         val responseFuture = provider.chatWithFormat(snapshotHistory(agent) :+ userMessage, format)
-        val response = Await.result(responseFuture, getAwaitTimeout)
+        val response = Await.result(responseFuture, awaitTimeoutFor(effectiveConfig(agent)))
         recordUsage(agent, response)
 
         val content = response.firstContent.getOrElse("")
@@ -915,7 +918,7 @@ class LLMExtension extends DefaultClassManager {
         tempHistory += userMessage
 
         val responseFuture = provider.chatWithFormat(tempHistory.toSeq, JsonObjectFormat)
-        val response = Await.result(responseFuture, getAwaitTimeout)
+        val response = Await.result(responseFuture, awaitTimeoutFor(effectiveConfig(agent)))
         recordUsage(agent, response)
 
         val content = response.firstContent.getOrElse("")
@@ -998,7 +1001,7 @@ class LLMExtension extends DefaultClassManager {
 
         // Send with user message included, but don't mutate history yet
         val responseFuture = provider.chatWithFullResponse(snapshotHistory(agent) :+ userMessage)
-        val response = Await.result(responseFuture, getAwaitTimeout)
+        val response = Await.result(responseFuture, awaitTimeoutFor(effectiveConfig(agent)))
         recordUsage(agent, response)
 
         val answerText = response.firstContent.getOrElse("")
@@ -1335,11 +1338,14 @@ class LLMExtension extends DefaultClassManager {
         )
       }
 
-      val candidate = ConfigStore.withDefaults()
-      candidate.updateFromMap(config)
+      // Mirror llm:load-config: the file is the whole configuration, with no
+      // OpenAI-flavoured defaults layered underneath. A profile that names no
+      // model gets its own provider's default, not gpt-4o-mini.
+      val candidate = new ConfigStore()
+      candidate.loadFromMap(config)
       checkReadiness(desc, providerName, candidate)
 
-      try profiles.load(name, candidate.toMap)
+      try profiles.load(name, config)
       catch {
         case e: IllegalArgumentException =>
           throw new ExtensionException(s"llm:load-profile: ${e.getMessage}")

src/main/providers/RequestThrottle.scala

@@ -67,16 +67,53 @@ object SystemThrottleClock extends ThrottleClock {
  * @param clock          time source and delay scheduler
  */
 class RequestThrottle(
-  val maxConcurrent: Int,
-  val minIntervalMs: Long,
+  initialMaxConcurrent: Int,
+  initialMinIntervalMs: Long,
   clock: ThrottleClock = SystemThrottleClock
 ) {
-  require(maxConcurrent > 0, "maxConcurrent must be positive")
+  require(initialMaxConcurrent > 0, "maxConcurrent must be positive")
 
-  // Guards `available`, `waiters` and `lastStartMs` together. Held only for
-  // queue arithmetic — never across a user callback or an HTTP send.
+  // Guards `available`, `waiters`, `lastStartMs` and the limits together. Held
+  // only for queue arithmetic — never across a user callback or an HTTP send.
   private val lock = new Object
-  private var available: Int = maxConcurrent
+  @volatile private var currentMax: Int = initialMaxConcurrent
+  @volatile private var currentInterval: Long = initialMinIntervalMs
+  private var available: Int = initialMaxConcurrent
+
+  def maxConcurrent: Int = currentMax
+  def minIntervalMs: Long = currentInterval
+
+  /** Requests holding a permit right now. Can exceed the cap briefly after it shrinks. */
+  def inFlight: Int = lock.synchronized { currentMax - available }
+
+  /** Requests queued for a permit. */
+  def waiting: Int = lock.synchronized { waiters.size }
+
+  /**
+   * Change the limits without replacing the gate.
+   *
+   * Two profiles on one endpoint can carry different settings, and calls
+   * alternate between them every tick. Swapping in a fresh gate per change
+   * would hand every call a full set of permits and the cap would mean
+   * nothing. Adjusting in place keeps the in-flight count and the queue: a
+   * smaller cap admits nothing until requests drain below it, a larger one
+   * hands the new permits to the oldest waiters at once.
+   */
+  def reconfigure(newMaxConcurrent: Int, newMinIntervalMs: Long): Unit = {
+    require(newMaxConcurrent > 0, "maxConcurrent must be positive")
+    val admitted = lock.synchronized {
+      available += newMaxConcurrent - currentMax
+      currentMax = newMaxConcurrent
+      currentInterval = newMinIntervalMs
+      val handoffs = mutable.ListBuffer.empty[Promise[Unit]]
+      while (available > 0 && waiters.nonEmpty) {
+        available -= 1
+        handoffs += waiters.dequeue()
+      }
+      handoffs.toList
+    }
+    admitted.foreach(_.success(()))
+  }
 
   // FIFO, so a waiter cannot be overtaken indefinitely — with agents calling
   // every tick, barging would pass one over for the whole run. Only ever
@@ -155,7 +192,7 @@ class RequestThrottle(
    * exceed an RPM limit.
    */
   private def paceThenProceed()(implicit ec: ExecutionContext): Future[Unit] = {
-    if (minIntervalMs <= 0L) return Future.unit
+    if (currentInterval <= 0L) return Future.unit
 
     val waitMs = lock.synchronized {
       val now = clock.nowMs
@@ -164,7 +201,7 @@ class RequestThrottle(
       // timestamp and starting together.
       val earliest =
         if (lastStartMs == Long.MinValue) now
-        else math.max(now, lastStartMs + minIntervalMs)
+        else math.max(now, lastStartMs + currentInterval)
       lastStartMs = earliest
       earliest - now
     }
@@ -193,7 +230,11 @@ class RequestThrottle(
    */
   private def release(): Unit = {
     val handoff = lock.synchronized {
-      if (waiters.nonEmpty) Some(waiters.dequeue())
+      // A negative `available` means the cap shrank while more than the new
+      // limit were in flight. Those releases repay the deficit; nobody is
+      // admitted until the count is back under the cap.
+      if (available < 0) { available += 1; None }
+      else if (waiters.nonEmpty) Some(waiters.dequeue())
       else { available += 1; None }
     }
     handoff.foreach(_.success(()))
@@ -304,23 +345,27 @@ object RequestThrottle {
       // or race two replacements past each other. The remapping function does
       // no I/O — it runs while the map holds a bin lock, so the notice below is
       // emitted afterwards rather than inside it.
+      // One gate per endpoint for the life of the process. A changed setting
+      // reconfigures it in place rather than replacing it, so requests already
+      // in flight stay counted — two profiles with different caps alternating
+      // on one endpoint would otherwise each get a fresh, empty gate.
       val throttle = throttles.compute(key, (_, existing) => {
-        if (existing != null && existing.maxConcurrent == limit && existing.minIntervalMs == interval) {
-          existing
-        } else {
-          if (existing != null) replaced.set(true)
+        if (existing == null) {
           new RequestThrottle(limit, interval)
+        } else {
+          if (existing.maxConcurrent != limit || existing.minIntervalMs != interval) {
+            existing.reconfigure(limit, interval)
+            replaced.set(true)
+          }
+          existing
         }
       })
 
       if (replaced.get()) {
-        // Requests already running under the previous gate keep its permits, so
-        // in-flight work can briefly exceed the new cap. Reporting beats a stall
-        // or a silently changed limit.
         warnOnce(
           s"NOTE: request throttle for $providerName updated to " +
           s"$MAX_CONCURRENT_REQUESTS=$limit, $MIN_REQUEST_INTERVAL_MS=$interval. " +
-          "Requests already in flight finish under the previous limit."
+          "Requests already in flight count against the new limit."
         )
       }
       throttle