[suppathak]

Summary

• Fetch agent traces directly from MLflow experiments via eval-hub-sdk client (parameters.mlflow_traces_experiment_name)
• Re-inject mlflow.spanType/Inputs/Outputs into span attributes after SDK normalization so CLEAR's preprocessor correctly identifies LLM calls
• Add mlflow_workspace parameter to override sidecar-injected workspace (OpenShift AI)
• Switch to clear-eval[tool-calls]>=2.0.0 from PyPI (includes SPARC/tool-call analysis)
• Set parameters.separate_tools: true to enable SPARC mode

Changes

File	What
main.py	Span attribute re-injection, workspace/sidecar bypass logic
requirements.txt	PyPI-based clear-eval[tool-calls]>=2.0.0
Containerfile	Standard build (no local SDK copy)
README.md	Trace fetch, SPARC, workspace override docs
meta/job.json	Updated sample with MLflow trace fetch and SPARC params

Dependencies

• Requires eval-hub-sdk#152 for artifact path fix and traces.create()

Test plan

• Ruff passes on main.py
• End-to-end test: fetch traces from MLflow, run CLEAR eval (standard + SPARC), save results back to MLflow
• CI green

Made with Cursor

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: a5b95e14-a33e-4078-b95e-c962ca3e97d1

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[williamcaban]

@ruivieira can you take a look at this PR for any additional feedback or merge?

[ruivieira]

Thanks for the PR @suppathak .

I've left a comment on the MLFlow / multi-tenancy contract. Thank you!

[ruivieira]

One concern on the mlflow_workspace / mlflow_route_url bypass: allowing the adapter to override the sidecar-injected workspace and talk directly to MLflow breaks the multi-tenancy contract.

The sidecar exists precisely to ensure jobs can only access resources within their own tenant namespace. Would it be acceptable to restrict MLflow trace fetching to the same workspace as the job? That would keep the adapter within the platform's tenancy boundaries and avoid the need for the bypass entirely.

[suppathak]

Thanks @ruivieira .

Here is why we did this: when we deployed MLflow on Red Hat OpenShift AI, the EvalHub sidecar automatically injects MLFLOW_WORKSPACE set to the tenant namespace (in our case redhat-ods-applications). The problem is, MLflow doesn't have a workspace matching that namespace name. There are multiple workspaces available, and we uploaded our traces to the default workspace, but there's just no workspace called redhat-ods-applications.

So when the adapter runs and tries to fetch traces from MLflow, it's looking in a workspace that doesn't exist, and it comes back empty. The traces are there in default, but the sidecar is forcing all requests to go to redhat-ods-applications which has nothing.

The override was the workaround we found to make the end-to-end flow work, telling the adapter to use default instead and bypassing the sidecar's workspace injection.

Happy to hear if there's a better way to handle this on the platform side. for example, having the sidecar pick up the workspace from a job parameter rather than always defaulting to the namespace name.

[suppathak]

@ruivieira, let me know if you have any thought on this comment above. Thank you.