Langsmith tracing

[jmsevin]

This pull request introduces LangSmith tracing for LangChain-based LLM calls, enabling improved observability and debugging of both agent and non-agent chat operations. The integration is configurable via environment variables, and trace context metadata is now propagated throughout the chat and LLM proxy layers. The most important changes are summarized below.

LangSmith/Tracing Integration:

• Added langsmith as a dependency and new LangSmith-related environment variables to .env.example and the settings model (pyproject.toml, .env.example, src/app/core/config.py). [1] [2] [3]
• Implemented configure_langsmith_tracing to set up LangSmith tracing on app startup (src/app/core/langsmith.py, src/app/core/lifespan.py). [1] [2] [3]

Trace Context Propagation:

• Built and propagated detailed trace_context metadata for agent and non-agent chat operations, including endpoint, session, thread, and query details (src/app/api/api_v1/endpoints/chat.py, src/app/api/api_v1/endpoints/chat_utils.py). [1] [2] [3] [4] [5] [6] [7] [8] [9]

Non-Agent and Agent Chat Improvements:

• Added _build_non_agent_trace_context helper and ensured all chat operations (e.g., formatting, language detection, rephrasing, chat messages) pass trace context to LLM calls (src/app/shared/infra/abst_chat.py). [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

LLM Proxy Enhancements:

• Updated LLMProxy to accept and log trace_context in all completion methods, and decorated key methods with @traceable for LangSmith compatibility (src/app/shared/infra/llm_proxy.py). [1] [2] [3]

These changes collectively enable end-to-end tracing and improved debugging for both agent and non-agent LLM-powered chat features.

[Copilot]

Pull request overview

This PR integrates LangSmith tracing across the LangChain-based chat flows to improve observability, by configuring tracing at app startup and propagating trace_context/tags/metadata through both agent and non-agent LLM calls.

Changes:

• Added LangSmith dependency + config/env plumbing and startup configuration.
• Propagated structured trace metadata through chat endpoints and chat/tutor services.
• Decorated LLM proxy completion methods for LangSmith compatibility and enriched agent/non-agent run config with tags/metadata.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/app/core/langsmith.py	Adds configure_langsmith_tracing() to set tracing-related env vars on startup.
src/app/core/lifespan.py	Calls tracing configuration during app lifespan startup.
src/app/core/config.py	Adds LangSmith-related settings fields.
src/app/shared/infra/tracing.py	Introduces shared tracing constants/enums for consistent run naming.
src/app/shared/infra/llm_proxy.py	Adds @traceable decorators and threads trace_context through completion APIs (plus new stream wrapper).
src/app/shared/infra/abst_chat.py	Builds and passes non-agent/agent trace metadata and tags into LLM + agent execution config.
src/app/api/api_v1/endpoints/chat.py	Builds agent trace context and propagates it into agent chat execution.
src/app/api/api_v1/endpoints/chat_utils.py	Propagates trace context through streaming agent helpers.
src/app/models/chat.py	Adds a TraceContext TypedDict to standardize agent trace metadata shape.
src/app/tutor/service/agents.py	Adds per-agent tags/metadata via RunnableConfig for tutor LangChain runs.
src/app/tutor/service/tutor.py	Adds trace metadata/tags to tutor syllabus agents and plumbs trace_context.
src/app/tutor/api/router.py	Constructs and forwards trace context for tutor syllabus endpoints.
pyproject.toml	Adds langsmith dependency.
poetry.lock	Locks langsmith (and transitive deps) versions.
.env.example	Documents new LangSmith tracing environment variables.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.