fix: accept the Transform Platform API URL as server_url (0.46.2)

[paulkarayan]

What & why

Problem: The Transform Platform's API Keys page hands you https://platform-api.transform.unstructured.io/api/v1, and the docs tell you to pass that value as server_url. Do it and every Platform call in this SDK fails with a 404: listing jobs, creating a workflow, checking a connector. The same URL works with curl, so the URL looks right and the SDK looks broken, and there is nothing in the error to point at the real cause. Anyone starting from the app's own copy button hits this on their first call.

Change: Treat hosts under unstructured.io as Unstructured API hosts, so a copied /api/v1 suffix is stripped from server_url the way it already was for unstructuredapp.io. Also clean the base URL for an operation-level server_url= override, which bypassed the cleaning hook entirely.

Linked ticket

none

Client-facing follow-up: reported while writing the Transform Python quickstart, where every SDK sample had to be written against a URL different from the one the app displays.

The bug

Every Platform operation in this SDK already carries its own path prefix. jobs.list_jobs requests /api/v1/jobs/, workflows.create_workflow requests /api/v1/workflows/, and so on. So the base URL must not carry /api/v1 of its own.

clean_server_url exists to strip exactly that kind of pasted-in path, but it only did so when the host contained unstructuredapp.io:

if "unstructuredapp.io" in parsed_url.netloc:
    ...
    clean_url = urlunparse(parsed_url._replace(path="", ...))
else:
    # For other domains, we want to keep the path
    clean_url = urlunparse(parsed_url._replace(params="", query="", fragment=""))

platform-api.transform.unstructured.io does not match, so the path was kept and the operation path was appended on top, giving /api/v1/api/v1/jobs/, which matches no route.

basesdk.py is generated, so the customization needs protecting: it is now in .genignore, the mechanism this repo already uses for general.py, users.py, retries.py and partition.py, with a guard test alongside the existing ones asserting that both the _get_url call and the .genignore entry survive. Without it a regeneration silently drops the fix and the doubled prefix returns. Freezing the file freezes the generated request, retry and hook plumbing too, so the entry carries the same un-freeze procedure general.py documents.

Three smaller problems came out of the same code while fixing it. The host test was a substring match, so unstructuredapp.io.example.com was treated as one of ours and had its path stripped and its scheme forced to HTTPS; it is now matched on domain boundaries and left alone. A server_url= passed to a single operation never reached the cleaning hook at all, because the hook runs at SDK init; that override is now cleaned in BaseSDK._get_url, the one point every operation's base URL passes through. And a fully qualified host carrying the terminal root dot (api.unstructuredapp.io.) has to be recognized explicitly, since the old substring test matched it by accident and the domain-boundary test does not; the path is stripped as before and the host keeps its dot, which changes the Host header and SNI and is the caller's choice to make.

What the patch changes, and what it does not

Every server_url shape the existing tests, the docs and the app use, run through clean_server_url on main and on this branch. Seven results change; sixteen are byte-identical.

server_url	main	this branch
https://platform-api.transform.unstructured.io/api/v1	https://platform-api.transform.unstructured.io/api/v1	https://platform-api.transform.unstructured.io	changed
http://platform-api.transform.unstructured.io/api/v1	http://platform-api.transform.unstructured.io/api/v1	https://platform-api.transform.unstructured.io	changed
platform-api.transform.unstructured.io/api/v1	http://platform-api.transform.unstructured.io/api/v1	https://platform-api.transform.unstructured.io	changed
platform-api.transform.unstructured.io	http://platform-api.transform.unstructured.io	https://platform-api.transform.unstructured.io	changed
https://platform-api.unstructured.io/api/v1	https://platform-api.unstructured.io/api/v1	https://platform-api.unstructured.io	changed
http://unstructuredapp.io.example.com/api/v1	https://unstructuredapp.io.example.com	http://unstructuredapp.io.example.com/api/v1	changed
http://myunstructuredapp.io/api/v1	https://myunstructuredapp.io	http://myunstructuredapp.io/api/v1	changed
https://platform-api.transform.unstructured.io	https://platform-api.transform.unstructured.io	same
https://platform.unstructuredapp.io/api/v1	https://platform.unstructuredapp.io	same
https://api.unstructuredapp.io/general/v0/general	https://api.unstructuredapp.io	same
unstructured-000mock.api.unstructuredapp.io/general/v0/general	https://unstructured-000mock.api.unstructuredapp.io	same
http://localhost:8000	http://localhost:8000	same
localhost:8000	http://localhost:8000	same
http://localhost:8000/my/endpoint/	http://localhost:8000/my/endpoint	same
localhost:8000/general/v0/general	http://localhost:8000/general/v0/general	same
https://unstructured.example.com/api/v1	https://unstructured.example.com/api/v1	same
http://not-unstructured.io/api/v1	http://not-unstructured.io/api/v1	same

The first five changed rows are the reported bug. The last two are the substring-match fix: those hosts are not ours, so they keep their path and their scheme.

Impact

Customers: Anyone using the Python SDK against the Transform Platform can now paste the API URL shown in the app, or set it from the documented UNSTRUCTURED_API_URL, and have jobs, workflows, sources, destinations and templates calls work. Today that exact value 404s on every call. Users who already worked around it by passing the bare host are unaffected; that keeps working. Users on unstructuredapp.io are unaffected; their URLs were already cleaned.

Internal (devs / ops / other teams): The docs can stop steering readers away from the URL the product displays. No service imports this code; it is a client library published to PyPI.

Wire contract / clients: No request or response shape changes. The only behavior change is which URL a request is sent to, and only for base URLs that were previously producing a doubled path. The one case where a user could notice a difference is a self-hosted deployment on a host under unstructuredapp.io or unstructured.io that genuinely serves the API beneath a subpath; that path is now stripped. Hosts outside those domains keep their path exactly as before, which the existing localhost subpath tests cover.

Deployment target considerations: This is a PyPI client library, not a deployed service, so SaaS / DI / in-VPC / on-prem / SND deploys are unaffected. Air-gapped users pointing the SDK at their own hostname keep the existing keep-the-path behavior, since their host is not under an Unstructured domain.

A note on the diff size

The last commit is ruff format over the files this change touches, plus seven noqa directives for pre-existing lint that cannot be auto-fixed without changing behaviour. It is formatting only and carries no behaviour change, so reading the first two commits on their own gives you the whole fix. Two of the noqas are worth knowing about: raise err in basesdk.py re-raises whatever an after-error hook returned, which is not always the active exception, so ruff's suggested bare raise would be a real bug.

Risk / rollback

Low. Small changes to URL normalization plus a .genignore entry, revert-safe, no migration and no flag.

How it was verified

Ran the unit suite on Python 3.11, 3.12 and 3.13 and the contract suite, plus pylint (10.00/10) and mypy, all green, matching what CI runs. uv.lock is unchanged, so the UV_LOCKED=1 install holds. Reproduced the bug and then the fix against the live Transform Platform API without an API key, which is enough to tell the two apart: a route that exists answers 401, a route that does not answers 404. Not exercised with a real API key end to end, and not exercised against a self-hosted deployment.

Proof

Repro, against the live API, before the fix:

$ curl -s -o /dev/null -w '%{http_code}\n' https://platform-api.transform.unstructured.io/api/v1/jobs/
401
$ curl -s -o /dev/null -w '%{http_code}\n' https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/
404

Through the SDK, before the fix:

server_url='https://platform-api.transform.unstructured.io'
  request sent: https://platform-api.transform.unstructured.io/api/v1/jobs/
  status:       401

server_url='https://platform-api.transform.unstructured.io/api/v1'
  request sent: https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/
  status:       404

Failing tests at HEAD before the fix, _test_unstructured_client/unit/test_server_urls.py::test_platform_request_url_has_a_single_api_prefix plus the hook tests:

FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[https://platform-api.transform.unstructured.io/api/v1]
FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[http://platform-api.transform.unstructured.io/api/v1]
FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[platform-api.transform.unstructured.io/api/v1]
FAILED test_custom_hooks.py::test_unit_clean_server_url_fixes_malformed_transform_platform_url[platform-api.transform.unstructured.io]
FAILED test_custom_hooks.py::test_unit_clean_server_url_leaves_lookalike_domains_alone[http://unstructuredapp.io.example.com/api/v1]

E  Failed: transform platform ... Expected https://platform-api.transform.unstructured.io, got https://platform-api.transform.unstructured.io/api/v1

After the fix, the same live check across both ways of passing the URL:

client-level, bare host
  sent:   https://platform-api.transform.unstructured.io/api/v1/jobs/
  status: 401

client-level, URL from the app (/api/v1)
  sent:   https://platform-api.transform.unstructured.io/api/v1/jobs/
  status: 401

operation-level, bare host
  sent:   https://platform-api.transform.unstructured.io/api/v1/jobs/
  status: 401

operation-level, URL from the app (/api/v1)
  sent:   https://platform-api.transform.unstructured.io/api/v1/jobs/
  status: 401

Every case now reaches the real route. Suites after the fix: unit and contract both pass, pylint 10.00/10, mypy clean.

Dependencies / merge order

none

Worked Example

from unstructured_client import UnstructuredClient

# The value the app's API Keys page gives you, pasted as-is.
client = UnstructuredClient(
    api_key_auth="YOUR_KEY",
    server_url="https://platform-api.transform.unstructured.io/api/v1",
)

client.jobs.list_jobs(request={})
# before: GET https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/ -> 404 {"detail":"Not Found"}
# after:  GET https://platform-api.transform.unstructured.io/api/v1/jobs/, the real route

Release

Bumped to 0.46.2 with CHANGELOG and RELEASES entries.

[paulkarayan]

Repro-first proof

Bugfix — server_url copied from the Transform Platform API Keys page (.../api/v1) doubled the path and 404'd every Platform SDK call

Reproduced the broken state

• Environment: api
• How: UnstructuredClient(server_url='https://platform-api.transform.unstructured.io/api/v1').jobs.list_jobs(request={}) against the live Transform Platform API, unauthenticated. A route that exists answers 401; a route that does not answers 404.
• Observed: SDK sent GET https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/ -> 404 {"detail":"Not Found"}. Same call with the bare host sent /api/v1/jobs/ -> 401 (route exists). Operation-level server_url= was broken the same way, on top of the client-level case.
• Evidence: Live: curl -o /dev/null -w '%{http_code}' https://platform-api.transform.unstructured.io/api/v1/jobs/ -> 401; .../api/v1/api/v1/jobs/ -> 404. Reported by Monica Renneke, https://unstructuredai.slack.com/archives/C03LXHVAX7H/p1787419074187999

Failing test (red)

• _test_unstructured_client/unit/test_server_urls.py::test_platform_request_url_has_a_single_api_prefix (unit) — committed

5 failed at HEAD: test_unit_clean_server_url_fixes_malformed_transform_platform_url[https://platform-api.transform.unstructured.io/api/v1] and 3 siblings, plus test_unit_clean_server_url_leaves_lookalike_domains_alone[http://unstructuredapp.io.example.com/api/v1]. The request-URL test asserted the doubled URL: got https://platform-api.transform.unstructured.io/api/v1/api/v1/jobs/

Fix

• Recognize unstructured.io hosts (matched on domain boundaries, not substring) in clean_server_url, and clean the base URL in BaseSDK._get_url so an operation-level server_url= override is normalized too.
• Files: src/unstructured_client/_hooks/custom/clean_server_url_hook.py, src/unstructured_client/basesdk.py

Proof it's resolved

• Test green: yes
• Evidence: Live re-run of the same four cases (client-level and operation-level, bare host and /api/v1) all send https://platform-api.transform.unstructured.io/api/v1/jobs/ and get 401 -- the route now exists for every one. Suites: 283 passed 1 xfailed (unit), 64 passed (contract), pylint 10.00/10, mypy clean, zero new ruff findings vs origin/main.

---

Auto-generated from this branch's .proof.toml (repro-first proof gate). Advisory.

[cubic-dev-ai]

All reported issues were addressed across 7 files

<sub>Shadow auto-approve: would not auto-approve because issues were found.

Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 4 files (changes from recent commits).

<sub>Shadow auto-approve: would not auto-approve. Auto-approval blocked by 1 unresolved issue from previous reviews.

Re-trigger cubic</sub>

[awalker4]

We can drop this comment. We're no longer autogenerating, so we can change anything we need in the code. In fact, this whole file may be able to go away.

[awalker4]

Per the last comment, I would just do a quick test and see if this one line transforms the url whether it was passed in the operation or the base client. If so, we don't even need the hook file anymore.

[awalker4]

Unused file, if you don't mind deleting it while you're in here.

[awalker4]

LGTM! All of my comments are just about cleanup now that we own the codebase in full, no need to step around auto-generated files. I don't want to block the fix, though. So feel free to merge with a fast follow.

[paulkarayan]

@awalker4
Agreed these are all cleanup, so i have addressed. easier than i expected. note the slight hook detail.