- Python 100%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
| docker | ||
| src | ||
| .gitignore | ||
| COPYING | ||
| COPYING.LESSER | ||
| MANIFEST.in | ||
| pyproject.toml | ||
| README.md | ||
splent_feature_elasticsearch
Reusable Elasticsearch infrastructure for SPLENT products (service archetype).
Ships a single-node Elasticsearch 8.x container and a generic, domain-agnostic
ElasticsearchService that any feature can consume through the service locator.
What it provides
ElasticsearchService(registered in the service locator):available(),index_name(name),ensure_index(name, mappings=None, settings=None),index_document(index, id, body),bulk_index(index, documents),delete_document(index, id),delete_by_query(index, query),refresh_index(name),search(index, query_body),delete_index(name),alias_targets(alias),point_alias(alias, index),health().- Docker:
docker/docker-compose.dev.yml(host port published, security off) anddocker/docker-compose.prod.yml(no port published, security on), both with a healthcheck and a namedelasticsearch_datavolume. - CLI:
splent feature:elasticsearch status.
Security
The production node runs with security on and publishes no port: the app
reaches it by service name on the product's network, and nothing outside the
host reaches it at all. The node takes ELASTICSEARCH_PASSWORD as the
password of its elastic superuser, and the app signs in as
ELASTICSEARCH_USERNAME with that same password. docker/.env.prod.example
ships it as <SET>, so product:deploy generates one and
docker compose up refuses to start until it is set. HTTP stays plain inside
the product's network, which is the only place it travels.
The node reads the password each time its container is created, so changing
it in .env and running docker compose up -d rotates it. That stops
holding once someone sets the elastic password through the security API,
which then wins over the variable. The healthcheck signs in with the same
password, which curl takes from its environment, so it never shows on a
command line.
Still open: the app signs in as elastic, the node's superuser, and the
product's web container gets the whole .env, that password included. A
leak from that container gives control of the whole node (security,
snapshots, settings), not only of the product's <prefix>_* indices. The
fix is a role limited to those indices and a user or API key of the app's
own, with the elastic password left to the operator; the feature does not
create them yet. A key made by hand goes in ELASTICSEARCH_API_KEY, and it
needs the monitor cluster privilege besides the index ones, because
available() and status ask the node who it is.
The development node runs without security, like the rest of the development
stack, and ELASTICSEARCH_PASSWORD stays empty there. The CI test job runs
the development setup too.
ELASTICSEARCH_API_KEY, when set, wins over the user and the password. That
is how a product talks to a cluster it does not run: set it together with
ELASTICSEARCH_URL. Credentials never appear in a message the service writes
or in what status prints, even when they are written into the URL.
splent feature:elasticsearch status tells a node that does not answer
(unreachable) from one that answers and refuses (refused), which after
turning security on almost always means the credentials:
URL: http://splent_feature_elasticsearch:9200
Auth: user elastic
Enabled: yes
Indices: uvlhub_splent_*
Status: refused
Reason: Elasticsearch at http://splent_feature_elasticsearch:9200 rejected the request: AuthenticationException(401, ...)
One node, many products
Each product runs its own node, with its own data volume, on its own network.
Index names are still a flat, node-wide namespace, though, and a product may
be pointed at a node somebody else runs: two products that both index pages
there would overwrite each other's documents and answer searches with each
other's material.
Every index name handed to the service is prefixed with
ELASTICSEARCH_INDEX_PREFIX, whose shipped value __PRODUCT__ is resolved to
the product name by product:env --merge and product:build. A feature keeps
using its own short name and never has to know.
es.ensure_index("pages") # creates innosoft_app_pages
es.index_name("pages") # "innosoft_app_pages"
splent feature:elasticsearch status prints the namespace in use, which is the
quickest way to check that a product is not sharing indices with its neighbour.
An index is a cache and never the authority on visibility. This service stores and returns whatever documents it is given; whether a particular reader may see a particular hit is decided at request time by the feature that owns the material.
Analyzers and mappings
ensure_index takes the index settings as well as the mappings. A mapping
that names a custom analyzer needs it in the settings at creation, because an
open index takes no new analyzers:
es.ensure_index(
"pages_v1",
settings={"analysis": {"analyzer": {"folded": {
"type": "custom", "tokenizer": "standard",
"filter": ["lowercase", "asciifolding"],
}}}},
mappings={"properties": {"title": {"type": "text", "analyzer": "folded"}}},
)
An index that already exists is left as it is, whatever either argument says.
Changing a mapping without emptying search
Deleting an index to build it again with a new mapping leaves search empty
until the reindex ends. Instead, build each version under a name of its own
and reach it through an alias, which point_alias moves in one atomic step:
es.ensure_index("pages_v2", mappings=MAPPINGS, settings=SETTINGS)
es.bulk_index("pages_v2", documents)
es.refresh_index("pages_v2")
for old in es.point_alias("pages", "pages_v2"):
es.delete_index(old)
Readers and writers keep using pages throughout. point_alias returns the
indices the alias pointed at before, and alias_targets("pages") says where it
points now.
An index created before versioning holds the alias's own name, and an alias
cannot share it. point_alias("pages", "pages_v1") refuses in that case;
replace_index=True deletes that index in the same atomic step, once the new
one is built.
Graceful degradation and typed errors
Elasticsearch is a soft dependency. available() never raises, and every other
operation raises ElasticsearchError, never a raw client exception. Two
subclasses say what happened and what to do about it.
available() is False both when the node does not answer and when it
answers and refuses, and a consumer falls back either way. The second is
almost always wrong credentials, so it is logged as a warning, at most one
every five minutes per node, while a node that is down is only logged at
debug.
| Exception | Meaning | Usual response |
|---|---|---|
ElasticsearchUnavailable |
Disabled, or the node could not be reached. | Fall back to a slower search. |
ElasticsearchRequestError |
The node answered and refused the request (malformed query, rejected mapping, missing permission or credentials). | Let it surface. Retrying will not help. |
from splent_framework.services.service_locator import service_proxy
from splent_io.splent_feature_elasticsearch.services import ElasticsearchUnavailable
es = service_proxy("ElasticsearchService")
try:
results = es.search("posts", {"query": {"match": {"title": query}}})
except ElasticsearchUnavailable:
results = fallback_sql_search(query)
Bulk indexing
A full reindex is thousands of documents, and one HTTP round trip each turns a
maintenance command into something nobody runs. bulk_index consumes any
iterable lazily, sends batches through the _bulk API and reports what
happened, so a single bad document does not cost you the other 9999.
report = es.bulk_index(
"pages",
({"id": page.id, "body": {"title": page.title, "body": page.text}} for page in pages),
)
es.refresh_index("pages")
# {"indexed": 9999, "failed": 1, "errors": [{"id": 42, "error": "..."}]}
Only a failure of the request itself raises. Per-document failures come back in the report, with the error list capped so a broken mapping cannot flood a log.
Deleting by query
delete_by_query(index, query) deletes every document the query clause
matches and returns how many went:
es.delete_by_query("pages", {"term": {"dataset_id": 7}}) # 12
query is the clause itself, not a whole search body. A document changed
while the deletion runs is skipped rather than failing the rest, the deletion
is visible to the next search unless refresh=False, and a missing index
deletes nothing without raising.
Configuration
| Env var | Default | Purpose |
|---|---|---|
ELASTICSEARCH_URL |
http://localhost:9200 |
Node URL. The Docker .env sets http://splent_feature_elasticsearch:9200 for container-to-container access. |
ELASTICSEARCH_ENABLED |
true |
Feature kill switch. When false all operations degrade immediately. |
ELASTICSEARCH_INDEX_PREFIX |
the product's name | Namespace for this product's indices. Declared as __PRODUCT__, resolved to the product name; with neither a prefix nor SPLENT_APP the service refuses to name an index. |
ELASTICSEARCH_USERNAME |
elastic |
User the app signs in as when a password is set. |
ELASTICSEARCH_PASSWORD |
empty | Password of that user, and of the production node's elastic user. Empty in development; <SET> in docker/.env.prod.example. |
ELASTICSEARCH_API_KEY |
empty | API key for a cluster this product does not run. Wins over the user and the password. |
ELASTICSEARCH_HOST_PORT |
9201 |
Host port published in development only. |
ELASTICSEARCH_JAVA_OPTS |
-Xms512m -Xmx512m |
Heap for the production node (docker/.env.prod.example). |
ELASTICSEARCH_ENABLED, ELASTICSEARCH_INDEX_PREFIX and
ELASTICSEARCH_USERNAME are declared in [tool.splent.config]. The URL and
the host port, which name the feature's container, live in
docker/.env.example; production reads docker/.env.prod.example instead,
which asks for the password and carries the heap.
Upgrading from 0.1
- A production deployment needs
ELASTICSEARCH_PASSWORD.product:deployasks for it and offers a random one; on a host without the CLI, set it indeploy/.envbeforedocker compose up. An existing data volume keeps its indices: the node takes the password on the next start. health()reportsauth,statusanderrorbesides what it did.available()calls the client'sinfo()instead ofping(). A test that mocks the client and setsping()has to setinfo()now._client()and_translated_errors()stay as they were, but they were never public:ensure_index(settings=...),delete_by_queryand the aliases cover what a consumer reached them for.
Tests
splent feature:test splent_feature_elasticsearch
Unit tests mock the client boundary, so no network access is required.