Skip to content

LK_UPSTREAM default is unreachable by construction in the :proxy image #14

Description

@harshaneel

Follow-up to #12, which added the :proxy image.

Problem

LK_UPSTREAM defaults to http://127.0.0.1:8080/v1 (cmd/localaik/main.go:23). That default is correct for the model-bundled images, where entrypoint.sh starts llama-server on 127.0.0.1:8080 in the same container, and it is correct for running the binary on a dev machine against a llama.cpp you started yourself.

It cannot ever be correct in :proxy. That image contains no inference engine, so its own loopback has nothing on port 8080 by construction.

An operator who forgets to set LK_UPSTREAM gets a container that starts cleanly, then answers 503 from /health and fails every request, while pointing at an address that could never have worked. The resolved upstream is printed at startup, so it is diagnosable, but the failure looks like "my model server is unreachable" rather than "I did not configure one."

Suggested fix

Options, roughly in order of preference:

  1. Have the :proxy image require LK_UPSTREAM and exit non-zero with a clear message when it is unset, rather than inheriting a default that is meaningless there. Needs a way for the binary to know which variant it is in, for example an ENV set only in the proxy stage.
  2. Keep the default but detect the specific case at startup: no credential configured, upstream is loopback, and no inference engine present. Warn loudly.
  3. Leave the code alone and document it harder in the README's :proxy section.

Option 1 is a real behaviour change for a published image, so it should land before :proxy has users rather than after.

Note that the flag and environment precedence must stay as it is: flag over environment over default. entrypoint.sh passes --upstream explicitly, so any change here must not disturb the bundled images.

Activity

  1. harshaneel commented on Aug 5, 2026

    @harshaneel
    OwnerAuthor

    Correction to the problem statement above: it claims "the resolved upstream is printed at startup, so it is diagnosable." That is wrong. main.go:47 logs only the port. Nothing logs the upstream, so a :proxy container with LK_UPSTREAM unset gives no signal at all about where it is pointing.

    That makes the failure worse than described. The operator sees localaik: listening on port 8090, then 503s from /health, with nothing anywhere connecting that to a missing variable.

    It also means logging the resolved upstream is worth doing regardless of which option is chosen, since it is a one-line diagnostic that helps every misconfiguration, not just this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions