Skip to content

Commit e112db8

Browse files
authored
feat(relationships): fill related data from ?include= and make all related resources accessible (#188)
1 parent f046b02 commit e112db8

121 files changed

Lines changed: 1747 additions & 471 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎AGENTS.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@ A handful of conventions are pervasive enough that you'll regret breaking them.
5757

5858
2. **JSON:API attribute names go through `Field(alias="...")`.** The API sends `created-at`, Python uses `created_at`. Pair with `model_config = ConfigDict(populate_by_name=True, validate_by_name=True)` on the model. ([MODELS.md](docs/MODELS.md))
5959

60+
2b. **Top-level response/resource models inherit `TFEModel`, not `BaseModel`.** It's config-light (you keep your own `model_config`) and adds the lossless `.relationships`/`.included`/`.related()`/`.has_*` accessors. Wire the resource's parser to call `attach_jsonapi(model, data, included)` so they're populated. Options/sub-object/enum models stay on `BaseModel`. ([MODELS.md](docs/MODELS.md) — TFEModel vs BaseModel)
61+
6062
3. **`model_dump(by_alias=True, exclude_none=True)` for write payloads.** Without `by_alias=True` you'll send snake_case to the API and it will silently drop the fields. Add `mode="json"` if the options contain enums.
6163

6264
4. **For new public APIs, prefer typed `TFEError` subclasses.** The error hierarchy in `errors.py` is part of the public API, and downstream consumers often `except TFEError:` once. Existing methods still expose many `ValueError` paths; do not change those established exceptions unless the breaking-change impact is explicitly accepted.

‎CHANGELOG.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,24 @@
11
# Unreleased
22

3+
## Enhancements
4+
5+
### Relationships
6+
Related data is now a complete, first-class part of every response. Before, `?include=` often did not actually fill the related fields it returned, and anything the SDK did not model as a typed field was dropped on the floor. Now the full set of related resources the API hands back is always available to you: typed where pytfe models it, raw where it does not. The practical win is that **you are no longer limited to the relationships pytfe has added typed support for.** You can read any related resource in a response without dropping to manual HTTP or waiting for a new SDK release.
7+
8+
* `?include=` now fills in related data. When the SDK models a relation as a typed field (for example `workspace.outputs`, `policy_set.current_version`, `organization_membership.user`, `run_event.actor`), passing `?include=<relation>` fills that field with the real record instead of an id-only stub.
9+
* Relations the SDK does **not** model are no longer lost. Every top-level resource model now derives from a new `pytfe.models.TFEModel` base and gains read-only accessors for the raw JSON:API data the API returned: `model.relationships`, `model.included`, `model.related(name)`, `model.included_by(type, id)`, and the `model.has_relationships` and `model.has_included` flags. So when a relation has no typed field of its own (for example an organization's `subscription`, or a workspace `readme`), `?include=` still returns it and you reach it with `model.related("subscription")` or `model.included_by(type, id)`. These accessors are read-only extras that never appear in `model_dump()` or affect equality, so this is additive and non-breaking. List endpoints expose the relationship refs but do not yet fill `included`. See [docs/related-resources.md](docs/related-resources.md) for the per-resource table and a "typed field vs raw accessor" guide.
10+
* Added `?include=` support to three reads that previously had no include option, matching the HCP Terraform API:
11+
* `teams.read(team_id, TeamReadOptions(include=[...]))`: `users`, `organization-memberships`.
12+
* `task_stages.read(task_stage_id, TaskStageReadOptions(include=[...]))`: `run`, `run.workspace`, `task-results`, `policy-evaluations`.
13+
* `organizations.read(name, OrganizationReadOptions(include=[...]))`: `subscription`. The new `options` argument is optional, so existing calls are unchanged.
14+
15+
## Bug Fixes
16+
17+
### Relationships
18+
* Fixed `workspaces.read*(include=[WorkspaceIncludeOpt.OUTPUTS])` returning outputs with `None` name, value, and type. Workspace `outputs` is now filled from the `included` data. [#134](https://github.com/hashicorp/python-tfe/issues/134)
19+
* Fixed `policy_set.read*(include=[current_version | newest_version])` returning an id-only stub. `PolicySetVersion` is now exported from `pytfe.models` and fully resolved, so the version's `source`, `created_at`, and `status` are populated.
20+
* Fixed `variable_set.read` inventing placeholder values (such as `name="workspace-<id>"` or `key="var-<id>"`) for `workspaces`, `projects`, and `vars`. These are now id-only stubs by default and fill from `included` when requested.
21+
322
# Released
423
# v1.1.0
524

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,7 @@ and upstream HCP Terraform API docs.
164164
| Need | Start here |
165165
|---|---|
166166
| Configure the SDK | [Authentication](./docs/authentication.md), [Pagination](./docs/pagination.md), [Logging](./docs/LOGGING.md) |
167-
| API guides | [API index](./docs/api/index.md), [API coverage](./docs/api-coverage.md), [Workspaces](./docs/api/workspaces.md), [Runs/plans/applies](./docs/api/runs-plans-applies.md), [State versions](./docs/api/state-versions.md) |
167+
| API guides | [API index](./docs/api/index.md), [API coverage](./docs/api-coverage.md), [Related resources (`include`)](./docs/related-resources.md), [Workspaces](./docs/api/workspaces.md), [Runs/plans/applies](./docs/api/runs-plans-applies.md), [State versions](./docs/api/state-versions.md) |
168168
| Scenario guides | [API-driven run](./docs/scenarios/api-driven-run.md), [State management](./docs/scenarios/state-management.md), [Migrate workspaces and state](./docs/scenarios/migrate-workspaces-and-state.md), [Team access onboarding](./docs/scenarios/team-access-onboarding.md), [No-code provisioning](./docs/scenarios/no-code-provisioning.md), [TFE identity bootstrap](./docs/scenarios/tfe-identity-bootstrap.md), [TFE admin bootstrap](./docs/scenarios/tfe-admin-bootstrap.md), [OIDC dynamic credentials](./docs/scenarios/oidc-dynamic-credentials.md) |
169169
| Operations guides | [Troubleshooting](./docs/troubleshooting.md), [Errors](./docs/errors.md), [Terraform Enterprise](./docs/terraform-enterprise.md) |
170170
| Contribute to the SDK | [CONTRIBUTING](./docs/CONTRIBUTING.md), [ITERATORS](./docs/ITERATORS.md), [MODELS](./docs/MODELS.md), [RESOURCE](./docs/RESOURCE.md) |

‎docs/MODELS.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,42 @@ class Foo(BaseModel):
3737
...
3838
```
3939

40+
## Base class: `TFEModel` vs `BaseModel`
41+
42+
| Inherit from | For |
43+
|---|---|
44+
| **`TFEModel`** (`pytfe.models`, defined in `models/_base.py`) | **Top-level resource models**: anything returned from a `read`/`list`/`create`/`update` that corresponds to a JSON:API *resource object* (`Workspace`, `Run`, `Project`, `Policy`, `AdminRun`, …). |
45+
| **`BaseModel`** | Everything else: `*CreateOptions` / `*UpdateOptions` / `*ListOptions`, nested attribute sub-objects (`WorkspacePermissions`, `VCSRepo`, …), enums, and `*List` envelopes. |
46+
47+
`TFEModel` is **config-light**: it adds no `model_config`, so you still set your own (`extra="allow"`, etc.) exactly as above. What it adds is the lossless related-resource escape hatch: `.relationships`, `.included`, `.included_by(type, id)`, `.related(name)`, and the `.has_relationships` / `.has_included` presence flags. These are private attributes, so they never touch `model_dump()` and add no public fields, and `TFEModel` also overrides `__eq__` to ignore them, so equality stays identical to a plain `BaseModel`. Inheriting it is additive and non-breaking.
48+
49+
For the accessors to be *populated* (not just present-and-empty), the resource's parser must hand the raw JSON:API resource dict (and any document `included`) to `attach_jsonapi`:
50+
51+
```python
52+
from .._jsonapi import attach_jsonapi, parse_relationships # in a resources/*.py
53+
54+
def _foo_from(data, included=None):
55+
attr = dict(data.get("attributes") or {})
56+
attr["id"] = data.get("id")
57+
attr.update(parse_relationships(data.get("relationships"), _FOO_REL_MAP, included=included))
58+
return attach_jsonapi(Foo.model_validate(attr), data, included)
59+
```
60+
61+
`attach_jsonapi(obj, data, included)` is the one line that captures both raw blocks; pass `included=payload.get("included")` from any `read`/`list` that supports `?include=`. See [related-resources.md](related-resources.md) for the consumer-facing view.
62+
63+
### Why not put it on every model
64+
65+
`TFEModel` is scoped to resource objects on purpose. The escape hatch is only meaningful for things parsed from a JSON:API *resource object* (a thing with a `relationships` block, returned from `read`/`list`/`create`/`update`). Putting it on `*Options`, sub-objects, and `*List` envelopes would cost more than it gives:
66+
67+
* **It does nothing on its own.** The accessors stay empty until a parser calls `attach_jsonapi(...)`. A request/options model is never parsed from a response, so its accessors would be permanently empty and pointless.
68+
* **It adds a confusing, response-only surface to request objects.** A `WorkspaceReadOptions` is something you *build and send*. Exposing `.relationships`, `.included`, and `.related(...)` on it (always empty) is misleading in code, in docs, and in editor autocomplete.
69+
* **It reserves names.** `TFEModel` claims `relationships`, `included`, `related`, `included_by`, `has_relationships`, and `has_included`. If a model later needs a field with one of those names, Pydantic warns (`Field name "..." shadows an attribute in parent "TFEModel"`) and the field silently wins, quietly breaking the escape hatch on that model. Keeping non-resource models on `BaseModel` avoids reserving those names where they are not needed.
70+
* **It changes the public class hierarchy and equality** for many public, downstream-facing models. `TFEModel`'s `__eq__` is intentionally equivalent to `BaseModel`'s for public fields, but there is no reason to swap a custom `__eq__` onto the hundreds of options and sub-object models that behave like plain Pydantic today.
71+
72+
What it does **not** break: `frozen=True` models stay hashable, because Pydantic regenerates `__hash__` for a frozen subclass. The only real failure mode is the name collision above.
73+
74+
Rule of thumb: inherit `TFEModel` when the model is a JSON:API resource you parse *from* a response; use `BaseModel` for everything you *build* (options), *nest* (sub-objects), or *wrap* (`*List` envelopes).
75+
4076
## Field aliases: JSON:API hyphens → Python snake_case
4177

4278
HCP Terraform speaks JSON:API, which uses hyphenated attribute names (`created-at`, `auto-apply`, `state-versions`). Python uses snake_case. Bridge with `Field(alias=...)`:

‎docs/api-coverage.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,9 @@ partially covered are listed at the bottom of this page.
7575
| | Azure OIDC configurations | `client.azure_oidc_configurations` | ✅ |
7676
| | GCP OIDC configurations | `client.gcp_oidc_configurations` | ✅ |
7777
| | Vault OIDC configurations | `client.vault_oidc_configurations` | ✅ |
78-
| Admin | SAML / SCIM / SMTP / token-TTL settings | `client.admin` | ✅ |
78+
| Admin (TFE site-admin) | Organizations, users, runs, workspaces | `client.admin.organizations` / `.users` / `.runs` / `.workspaces` | ✅ |
79+
| | Terraform / OPA / Sentinel versions | `client.admin.terraform_versions` / `.opa_versions` / `.sentinel_versions` | ✅ |
80+
| | SAML / SCIM / SMTP settings + SCIM tokens | `client.admin.saml_settings` / `.scim_settings` / `.scim_tokens` / `.smtp_settings` | ✅ |
7981

8082
## Partial coverage
8183

@@ -117,8 +119,6 @@ Public HCP Terraform API resources that do not yet have a pytfe client namespace
117119
| User tokens | Personal (user) API tokens. |
118120
| VCS events | — |
119121

120-
### Terraform Enterprise only (separate admin API)
121-
122-
| Resource | Notes |
123-
|---|---|
124-
| Site-admin | TFE site-admin endpoints (admin organizations, users, runs, workspaces, Terraform / OPA / Sentinel versions). Not part of the public HCP Terraform API. |
122+
> Note: the TFE site-admin API (`/api/v2/admin/*`, TFE-only — not part of the
123+
> public HCP Terraform API) **is** implemented under `client.admin` (see the
124+
> Admin rows above).

0 commit comments

Comments
 (0)