No description
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-11-13 18:50:42 +00:00
src fix: a ZENODO_API_URL that stops at the host or at /api is read as the deposition API, a 404 or 405 creating a deposition is a configuration error that says so, and every refusal logs the method, the URL and what Zenodo answered 2026-11-13 19:50:06 +01:00
.gitignore feat: ✨ add splent_feature_zenodo (uvlhub port) 2026-10-18 21:42:05 +02:00
COPYING chore: license under LGPL-3.0-or-later, as the hub core is 2026-10-23 11:38:26 +02:00
COPYING.LESSER chore: license under LGPL-3.0-or-later, as the hub core is 2026-10-23 11:38:26 +02:00
MANIFEST.in feat: ✨ add splent_feature_zenodo (uvlhub port) 2026-10-18 21:42:05 +02:00
pyproject.toml chore: release splent-io/splent_feature_zenodo to v0.7.0 2026-11-13 18:50:42 +00:00
README.md refactor: ZenodoService reads its settings from app.config, where the framework resolves them from the schema; config.py reads no environment and a timeout that is not one stops the start 2026-10-31 17:51:34 +01:00

splent_feature_zenodo

The zenodo feature of a SPLENT product line.

A SPLENT feature is a Python package that a product composes with others: its models, routes, services, templates and hooks are registered when the product starts. What this feature provides and requires is listed in the [tool.splent.contract] section of pyproject.toml, which splent feature:contract --write keeps in sync with the source.

This one publishes a hub's datasets on Zenodo, each as a record with its DOI: it is a publication provider of the dataset feature (dataset's DepositionProvider, F5-2), next to local_publication. It owns no table and no page: dataset drives every publication through the provider, and never names this feature.

What it provides

  • ZenodoProvider (provider.py), registered with dataset's register_deposition_provider as zenodo, whose identifiers are DOIs. create opens a deposition, new_version a draft of a new version with the files of the one before deleted, upload sends the deposit's ZIP, publish mints the DOI and answers it with the concept DOI, fetch_record reads a deposition, delete_unpublished deletes a draft (PB-3), discard drops an edit left open, and update_metadata edits, updates and publishes again, discarding the edit when a step fails.
  • Its errors are dataset's provider errors (PB-4): ZenodoValidationError is a ProviderValidationError (422), ZenodoAuthError and ZenodoConfigurationError a ProviderAuthError (502), ZenodoUnavailableError and ZenodoTimeoutError a ProviderUnavailable (503), ZenodoRateLimitedError a ProviderRateLimited (503, with the Retry-After Zenodo asked for).
  • ZenodoService, the client, registered in the service locator: create_deposition, create_new_deposition, upload_zip, publish_deposition, get_deposition, get_doi, get_concept_doi, update_deposition, discard_deposition, delete_deposition, create_new_version_draft, delete_all_deposition_files, update_draft_metadata, metadata_for and build_metadata, plus record_url, is_configured() and health(). The provider uses the one the locator has, so a refinement of it is the one that talks to Zenodo.
  • GET /zenodo/test, for signed-in users, which answers {"success": true|false, "messages": [...]}. The retry dialog of "My datasets" asks it before offering to publish again. It calls health(): one read of the deposition list, a page of one, whose answer is reused for a minute. It never writes to the account.

The token

Zenodo takes a personal access token. There is no OAuth flow and no callback URL to register. Create the token under Account, Applications, Personal access tokens, with the deposit:write and deposit:actions scopes, in the account that will own the hub's records. Production (zenodo.org) and the sandbox (sandbox.zenodo.org) are separate sites, with separate accounts and separate tokens.

ZENODO_ACCESS_TOKEN is declared as <SET>, so product:deploy asks for it and the marketplace lists it as required. In a development .env the placeholder counts as no token: nothing is sent, is_configured() is False and the retry dialog says Zenodo is not set up.

The token travels in an Authorization: Bearer header, never in the URL, which ends up in exception messages, tracebacks and proxy logs. Nothing this feature logs contains it. It can edit every record the account ever published, so rotate it whenever it may have been seen, and when a hub moves to a new server.

Sandbox and production

With ZENODO_API_URL empty, the API is production Zenodo in production and the sandbox anywhere else. Production is what the framework calls production: SPLENT_ENV, then FLASK_ENV.

Production Zenodo is refused before anything is sent, with a ZenodoConfigurationError that says how to proceed, unless ZENODO_ALLOW_PRODUCTION is true, and that holds in production too. is_configured() and health() answer False in that case. A Zenodo DOI is permanent, and a test run once minted three, so reaching production Zenodo has to be a decision of the deployment that makes it, not a default or a URL. The profile cannot make it: product:build writes SPLENT_ENV=prod into every deploy, so a staging server runs as production as well.

  • The deployment that publishes the hub's real records sets ZENODO_ALLOW_PRODUCTION=true in its own .env. Not in the product's [tool.splent.config], which every build of the product reads, the staging one included.
  • A staging server sets ZENODO_API_URL to https://sandbox.zenodo.org/api/deposit/depositions, with a sandbox token. When it starts from a copy of the production .env, it also sets ZENODO_ALLOW_PRODUCTION back to false and drops the production token: the flag travels with the copy, and nothing in the code can tell.

The test suite closes the HTTP transport for every test that is not browser driven (tests/conftest.py), so no test can reach Zenodo, whatever it mocks.

Settings

Each one is declared, with its default, in [tool.splent.config] and typed in [tool.splent.config_schema]. The framework resolves them into app.config, where ZenodoService reads them. A product overrides only those it decides.

Setting Default What it does
ZENODO_ACCESS_TOKEN <SET> The token above. Without one nothing is sent.
ZENODO_API_URL empty The deposition API. Empty means production in production and the sandbox anywhere else.
ZENODO_ALLOW_PRODUCTION false Lets the hub reach production Zenodo, in production too. Only in the .env of the deployment that publishes real records.
ZENODO_CONNECT_TIMEOUT 5 Seconds to wait for Zenodo to accept a connection.
ZENODO_READ_TIMEOUT 30 Seconds to wait for an answer to any call but the upload.
ZENODO_UPLOAD_TIMEOUT 90 Seconds to wait for the answer once the ZIP is sent. Zenodo checks the file first, so it grows with the largest dataset.
ZENODO_MAX_RETRIES 2 Times a read, an update or a delete is sent again.
ZENODO_KEYWORDS none Keywords every deposit gets after the dataset's own tags.
ZENODO_COMMUNITY empty A Zenodo community each deposit asks to join. Its curators accept or decline each request.
ZENODO_ACCESS_RIGHT open open, or closed for records whose files only their owner sees on Zenodo.

Each timeout bounds one call, not a publication. See below for what that means for the worker and the proxy in front of the hub.

The generic layer names no product. A hub that wants its records found together, or under its own community, says so in its own [tool.splent.config]:

SITE_NAME = "uvlhub"
ZENODO_KEYWORDS = ["uvlhub"]

What a deposit says

What a deposit says comes from dataset's Deposit, which it builds from the same fields and rules as the DataCite record of the dataset's landing (fair_metadata.deposit_for, PB-6); test_provider.py checks the two say the same. This feature adds what is Zenodo's own:

  • upload_type is always dataset. The publication type a dataset form asks for describes the article the dataset goes with, which travels as a related identifier (isSupplementTo), and only when its DOI can be read.
  • creators are the dataset's authors, or a single "Anonymous" for an anonymous dataset. The anonymity is read from the dataset itself, so a caller that passes anonymous=False for an anonymous dataset still deposits it anonymously.
  • keywords are the dataset's tags and then ZENODO_KEYWORDS, trimmed, without blanks, each once regardless of case.
  • access_right comes from ZENODO_ACCESS_RIGHT, communities from ZENODO_COMMUNITY, and the license is the one every dataset of the hub is published under, dataset's DATASET_LICENSE (dataset 0.6.0 and later; CC-BY-4.0 with an older dataset).
  • A dataset published through the hub's API carries a note that says so, with the publishing account's id, since its author credit came from that client. The note names the hub by SITE_NAME, which the theme puts in app.config, or says "the hub" without one. An anonymous record gets no note.

Timeouts, retries and errors

Every call has a timeout. A GET, PUT or DELETE that could not connect, or got a 429, 502, 503 or 504, is sent again up to ZENODO_MAX_RETRIES times with a doubling pause, and a Retry-After of up to ten seconds is obeyed. A POST never is: a second create is a second deposition, and a second publish may be a second DOI. A read that timed out is not either, since it already waited as long as the hub will.

These bound each call, not a publication. Since dataset 0.7.0 a publication runs in the background (the framework's task executor), and the request waits for it at most dataset's DATASET_PUBLISH_WAIT seconds; the paragraph below is about the run, and about a product that still runs an older dataset. A first publication is a create, an upload, a publish and two reads, for the DOI and the concept DOI, so with the defaults it can wait about four minutes before any retry: four calls of 5 + 30 s and an upload of 5 + 90 s. With an older dataset that runs in the request, and a gunicorn or nginx timeout of 120 s cuts it short: a worker killed after the publish leaves a DOI minted on Zenodo that the hub never stored, so such a product raises the worker's and the proxy's timeouts for the routes that publish above that sum. With dataset 0.7.0 the request only waits, those timeouts only have to be above DATASET_PUBLISH_WAIT, and a run stopped half way (an RQ worker's timeout, a restart) is found by the next attempt and by dataset's publication-reconcile, which ask Zenodo first.

Every failure is a ZenodoError. Each carries user_message, a sentence a page or an API response can show, and http_status, the status a route of the hub answers with; str() is the detail for the log.

Error When http_status
ZenodoUnavailableError No answer, or a 5xx. Worth trying later. 503
ZenodoTimeoutError A timeout above (a kind of unavailable). 503
ZenodoRateLimitedError 429, with retry_after when Zenodo said (a kind of unavailable). 503
ZenodoConfigurationError No token, or production outside production. Only an operator can fix it. 502
ZenodoAuthError 401 or 403: a wrong or revoked token, or one without the scopes (a kind of configuration error). 502
ZenodoValidationError 400 or 422. user_message names the fields Zenodo refused. 422
ZenodoError Anything else, such as a 404 or an answer that is not JSON. 502

ZenodoService exposes them as attributes (ZenodoService.UnavailableError, ZenodoService.ValidationError and so on), so a caller that gets the service from the locator catches them without importing this feature.

delete_all_deposition_files fails when a file cannot be listed or deleted, so a new version is never published with the previous version's file next to the new one. A publish that Zenodo accepted (202) never fails on its answer, since the DOI is minted by then: get_doi asks for it.

Use it in a product

splent feature:add splent_feature_zenodo

Develop

splent feature:test splent_feature_zenodo
splent feature:contract splent_feature_zenodo

Release

splent feature:release splent_feature_zenodo --dry-run --bump patch
splent feature:release splent_feature_zenodo

The first command rehearses the release and changes nothing. The second publishes it to the git forge and to PyPI, and a PyPI version is permanent.