The REST API of the datasets of a hub, with versioned schemas, fine-grained token scopes and pagination
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-11-02 15:14:19 +01:00
docs feat: API v1's transfers ask dataset_transfer, served only with it, answering one 202 whether or not the account exists, its recipient named to the sender as the sender named it, and 429 past the offers of a day (DL-17, CO-1) 2026-10-25 18:11:06 +01:00
src The API lists leave out what its owner hid, except for its owner 2026-11-02 15:10:54 +01:00
.gitignore chore: scaffold splent_feature_dataset_api (light) 2026-10-25 11:32:24 +01:00
COPYING chore: scaffold splent_feature_dataset_api (light) 2026-10-25 11:32:24 +01:00
COPYING.LESSER chore: scaffold splent_feature_dataset_api (light) 2026-10-25 11:32:24 +01:00
MANIFEST.in chore: scaffold splent_feature_dataset_api (light) 2026-10-25 11:32:24 +01:00
pyproject.toml chore: release splent_io/splent_feature_dataset_api to v0.5.2 2026-11-02 15:14:19 +01:00
README.md feat: API v1's transfers ask dataset_transfer, served only with it, answering one 202 whether or not the account exists, its recipient named to the sender as the sender named it, and 429 past the offers of a day (DL-17, CO-1) 2026-10-25 18:11:06 +01:00

splent_feature_dataset_api

The REST API of the datasets and their files, /api/v1/datasets*, /api/v1/me/datasets and /api/v1/files*, out of splent_feature_dataset (milestone H7 of uvlhub 3.0). The routes are the ones dataset served up to 0.10.0, at the same addresses, with the same methods and the same answers: a client of uvlhub 2.12, the marketplace's first, notices nothing. What is new comes beside them:

  • Scopes per kind of call (AP-4): datasets:read, datasets:write (upload), datasets:publish (publish and new-version), datasets:transfer and datasets:delete. uvlhub 2.12's read_dataset and write_dataset are aliases that stand for them: a key carried over with write_dataset keeps publishing.
  • Public reads. What is published is read without a token, a number of calls a minute per address (apikeys' APIKEYS_PUBLIC_READS_PER_MINUTE).
  • Tokens limited to some datasets. The token form and POST /api/v1/auth/tokens ("resources": {"dataset": [4]}) limit a token to some of its owner's lineages; it acts on those alone.
  • A resource layer (AP-1, resource.py): each route names the scope, what it loads and who may act on it, and every refusal is v1's own.
  • Versioned schemas (AP-2, schemas.py): every body is built from one, and the same schema is what the route tells apidocs it answers.
  • One way to page (AP-3, pagination.py): page, per_page (1 to 100), and next and prev.
  • A trail of every write (AP-10, trail.py), with the token that made it, through the framework's record_event (kept by splent_feature_audit).
  • A written versioning policy (AP-7): docs/versioning.md.
  • The upload of any file (H11): on a hub without the UVL layer, /api/v1/datasets/upload takes a file of any kind the hub takes, which hubfile's ingest takes or refuses (below).
  • Descriptions in Markdown (D21): the upload takes description_markdown besides 2.12's description, and the answers add description_markdown and description_format (below).
  • Transfers through dataset_transfer (DL-17): the five routes of the transfers ask splent_feature_dataset_transfer's TransferService, and are served only in a product that has it (below).

The scopes

Each kind of call asks for its own scope, registered with apikeys from scopes.py. Only reading is a default; the others, which change things, are registered with default=False, so a token gets them only when its creator asks.

Scope Lets a token Default
datasets:read Read datasets, their versions, files and transfers Yes
datasets:write Upload a dataset, as a draft No
datasets:publish Publish a draft or a new version, which mints its identifier No
datasets:transfer Offer a dataset to another account, and answer or take back an offer; registered only with dataset_transfer No
datasets:delete Delete a draft No
read_dataset uvlhub 2.12's, an alias of datasets:read Never offered
write_dataset uvlhub 2.12's, an alias of the four that write Never offered

A token limited to some of its owner's datasets acts on those alone, a lineage at a time, and reads the rest only where anyone may.

How a route is written

Each route says what it needs to the resource layer (resource.py), which resolves the token, loads what the route acts on, asks dataset's access policy and the token's reach, and refuses the way v1 always did:

@dataset_api_bp.route("/api/v1/datasets/<int:dataset_id>/publish", methods=["POST"])
@trailed("publish")
@api_resource(PUBLISH, DATASET, OWNER)
def api_publish_dataset(dataset):
    ...

Every body is built from a versioned schema (DatasetV1, LineageV1, TransferV1...), and the route's openapi attribute, an OpenAPI 3.1 operation as data (docs.py), hands the same schema to apidocs, so the document cannot drift from the answer. @trailed records every write once it is answered, whatever the answer, with the framework's record_event: the action api.<operation> (api.publish), so the trail's screen narrows to them with api., and the token's id and prefix, never the token.

v1 only grows: a new route, key or optional parameter. What would break a client goes to a v2 served beside it, and each route of v1 it replaces answers with Deprecation, Sunset once decided, and a Link to why (docs/versioning.md). GET /api/v1/files/<id>/metadata, which no feature can answer since uvl merged the UVL layer, is deprecated and kept while uvlhub's frozen manifest lists it.

The upload

On a hub with the UVL layer (uvl's FlamapyService, uploads.takes_uvl), the upload takes a UVL model, byte for byte as uvlhub's did: uvl_file or file, or its text as uvl_content (or content), UTF-8, .uvl added to a name without it, at most 1 MB and checked by uvl when a token sends it, and the answer reads feature_models_created.

On a hub without it, it takes one file of any kind: file or uvl_file, or its text as content named by filename (dataset's creation.file_draft). Its bytes and name are kept as they came, and hubfile's ingest decides, as for every file that comes in: its type by its name (HUBFILE_FILE_EXTENSIONS; ["*"] takes any), its size (a 413 past what its type takes) and the type's check. uvl_content there is a 400: such a hub takes no UVL model. Both answer files_created besides.

The description

A dataset's description is Markdown, or the HTML uvlhub 2.12 stored, kept as it is until its owner edits it (dataset's descriptions.py, D21). The upload takes either:

Key What it is Stored as
description_markdown Markdown, the new key markdown, as written
description HTML, as a client of 2.12 sends it html_legacy, cleaned by dataset's sanitiser

With both, description_markdown wins, so a client that sends both is read alike by a hub of 2.12, which knows only description, and by this one. Each is at most 16,000 characters.

The answers (DatasetV1, OwnDatasetV1) keep description as v1 always answered it, HTML: the description rendered and cleaned by dataset's sanitiser, whichever its format, which a 2.12 client draws as it did. After the keys v1 had they add description_markdown, the Markdown its author wrote (null for inherited HTML), and description_format, markdown or html_legacy. A client that shows the Markdown renders and cleans it as the hub does.

The transfers

/api/v1/datasets/<id>/transfer (POST), /api/v1/datasets/transfers (GET) and /api/v1/datasets/transfers/<id>/accept, /decline and /cancel (POST) are transfers.py's. The offers are dataset_transfer's since dataset 0.14.0, so they are added to the blueprint as each app registers it (serves_transfers, serve_transfers in __init__.py) when the product has dataset_transfer, with the endpoints they always had (dataset_api.api_transfer_dataset_ownership...). A product without it serves none of them, and registers no datasets:transfer scope; write_dataset still stands for it, so a key of 2.12 transfers wherever a hub does.

They keep 2.12's methods, statuses and keys (CO-1). DL-17 changes three things a client can see:

  • An offer to an email with no account, or with one that is switched off, and one to a user_id with no account behind it, is the 202 of any offer, where 2.12 answered 404 "Target account not found" or 400 "The target account is not active.": it is kept, reaches nobody and lapses.
  • The to_user_id of an offer to an email is null for its sender until the account answers (its recipient sees its own id, and an offer to a user_id names that id, as before), so no answer says whom an address belongs to.
  • An account makes DATASET_TRANSFER_MAX_OFFERS_PER_DAY offers a day; one more is 429 transfer_limit, with Retry-After. An offer lapses, unanswered, after DATASET_TRANSFER_EXPIRY_DAYS: cancelled from then on, and answering it is a 400.

Use it in a product

splent feature:add splent_feature_dataset_api

It requires apikeys 0.10.0 and dataset 0.13.0: 0.11.0 stopped serving the API itself when this feature is installed, 0.12.0 makes the upload of any file (creation.file_draft) and the neutral names this feature reads (landing_url(), deposition_id()), and 0.13.0 the description's formats (description_markdown, description_html). Without this feature dataset still serves v1, deprecated, in a product with apikeys, without its transfers since dataset 0.14.0. The transfers take dataset_transfer 0.1.0 or later.

The line's model gets the node and its constraints from docs/openhubs_spl.uvl.patch: dataset_api => apikeys, auth and dataset, and dataset no longer requires apikeys. The working copy of openhubs_spl.uvl has it, as an optional feature of repositorio; the published version 9 does not yet. A hub without an API leaves both dataset_api and apikeys out. It uses hubfile for the files when the product has it, and its writes reach the trail when the product has audit.

Settings

None of its own. apikeys' decide what a token is offered and how many calls a caller without one makes:

Setting Of
APIKEYS_SCOPES apikeys An alias names what it stands for: ["read_dataset", "write_dataset"] offers the five datasets:*
APIKEYS_DEFAULT_SCOPES apikeys ["read_dataset"] grants datasets:read
APIKEYS_PUBLIC_READS_PER_MINUTE apikeys 60 by default; 0 asks every caller for a token
APIKEYS_HEADER_ALIASES apikeys ["X-API-Key"] for the marketplace's client
DATASET_PUBLISH_WAIT dataset How long publish and new-version wait for the identifier

What moved, for a product's contract

Each rule keeps its methods; its endpoint goes from dataset.<view> to dataset_api.<view>, the view's name unchanged (dataset.api_publish_dataset is dataset_api.api_publish_dataset). uvl's CORS on /api/v1/datasets/upload for flamapyIDE is by path, so it follows.

Develop

splent feature:test dataset_api --with dataset --with apikeys
splent feature:contract dataset_api

Release

splent feature:release splent_feature_dataset_api --dry-run --bump minor
splent feature:release splent_feature_dataset_api

Release apikeys and dataset first: this feature's neutral app takes their newest releases.