service-oa-records

OGC API Records service for SwissGeo, built on pygeoapi with an OpenSearch backend and multilingual record support.

Branch	Status	Coverage
develop
main

Note

This is still in POC phase

Overview

This service exposes Swiss geospatial catalogue data as an OGC API Records endpoint. pygeoapi handles the OGC API layer; records are stored in OpenSearch and queried via SwissGeoProvider, a custom provider that adds language-aware field selection and link patching on top of pygeoapi's built-in OpenSearchCatalogueProvider.

Client
  │  ?lang=de&f=json
  ▼
uvicorn (app.py)          ← patches call_api_threadsafe to inject lang/fmt
  │                          into the executor thread-local before each call
  ▼
pygeoapi Starlette app
  │
  ▼
SwissGeoProvider          ← extends OpenSearchCatalogueProvider
  │  query() / get()
  ├─ reads lang from thread-local (set by app.py)
  ├─ mask_prop()    – filters/sorts on <lang>.properties.<field>
  ├─ calls super().query() / super().get()
  ├─ picks the <lang> record out of each document
  └─ _patch_links() – appends ?lang=…&f=… to same-host links
  │
  ▼
OpenSearch

swissgeo_provider.py

Why app.py is needed

pygeoapi's Starlette integration runs provider calls in a thread pool. By the time the provider executes, the Starlette request context is no longer accessible. app.py monkey-patches call_api_threadsafe to call set_request_params(lang, fmt) just before dispatching each call, storing the values in a threading.local that SwissGeoProvider reads. Flask users are handled via a fallback flask.request.args read inside _get_lang_and_fmt().

Localized index layout

The indexes are written by manage.py oar_opensearch_export in service-control. Each document holds one complete record per language next to the shared id and type:

{"id": "ch.bafu.moose", "type": "Feature", "de": {record}, "fr": {record}, "it": {record}, "rm": {record}, "en": {record}}

Only the fields under <lang>.properties that are listed in the mapping are indexed. The request language decides which record is returned and which subtree filters (?concept=, ?protocol=, …) and sorting run against. Free-text search (?q=) matches the records of all languages, so ?q=wander&lang=fr finds records by their German title too. The available fields are read from the de subtree of the mapping; they are the same in every language.

Sorting is supported on text fields that have a sort keyword subfield in the mapping (title; name and acronym for organizations). Sorting on other text fields returns 400.

Link patching

_patch_links() appends ?lang=<lang>&f=<fmt> to any link whose href is relative or starts with PYGEOAPI_SERVER_URL. External links are left untouched.

Configuration

Provider registration in pygeoapi-config.yml:

providers:
  - type: record
    name: swissgeo_provider.SwissGeoProvider
    data: ${OPENSEARCH_URL:-http://localhost:9200}/swissgeo-catalog
    id_field: externalId
    time_field: recordCreated
    title_field: title
    languages:
      - en
      - de
      - fr
      - it

Key environment variables:

Variable	Default	Purpose
PYGEOAPI_SERVER_URL	/	Base URL used to identify same-host links for patching
OPENSEARCH_URL	http://localhost:9200	OpenSearch base URL
PYGEOAPI_CONFIG	/pygeoapi/pygeoapi-config.yml	pygeoapi config file path
THREADPOOL_MAX_WORKERS	16	Threads running the API calls; also the OpenSearch connection pool size
OPENSEARCH_TIMEOUT	30	OpenSearch request timeout in seconds
OPENSEARCH_MAX_RETRIES	3	OpenSearch retries, also on timeouts
CACHE_CONTROL_HEADER	public, max-age=300	Cache-Control header set on GET/HEAD responses, except 5xx and responses that already have one; empty disables it
UVICORN_LIMIT_CONCURRENCY	100 (image)	Open connections/requests before uvicorn answers 503
MALLOC_ARENA_MAX	2 (image)	Limits glibc malloc arenas to reduce memory fragmentation

pygeoapi creates a new provider instance for every request. SwissGeoProvider therefore creates the OpenSearch client and reads the index mapping only once per index, and shares both across instances. With aws4auth: true a single refreshable SigV4 signer is used.

Running locally

Important

This service does not start any backing services of its own. It expects the service-control stack to be running, which provides OpenSearch on localhost:9200 (with the catalogue indexes loaded) and the OTLP collector on localhost:4317. Start that stack first. Also run manage.py oar_opensearch_export at least once to create the OpenSearch indexes and write the data.

Setup

make setup

Creates the virtualenv with uv sync, creates .env from .env.default if it is missing, and drops you into a shell with those variables exported. .env is gitignored, so local overrides (e.g. a different OPENSEARCH_URL) survive. A .env.local, if present, takes precedence over .env.

Run on the host

make serve

Run the built image

make dockerrun

Builds the image and runs it with --net=host, so it reaches the same localhost:9200 / localhost:4317 as make serve. This is for verifying the deployment artifact; day-to-day development uses make serve.

Debugging

make serve-debug

Runs the app under debugpy listening on port 5678 and waiting for a client, so you can attach before the first request is handled.

Then attach your debugger (e.g. "Attach to Docker (swissgeo_provider)" in Zed) to localhost:5678.

To debug the container instead, build the image with --build-arg INSTALL_DEBUGPY=true (debugpy is not part of the production image) and run it with PYDEBUG=true.

Project structure

pygeoapi-swissgeo-extensions/
  app.py                  # Starlette entrypoint; patches call_api_threadsafe
  swissgeo_provider.py    # SwissGeoProvider: language selection + link patching
pygeoapi-config.yml       # pygeoapi server + collection configuration
static-s3/                # 1:1 catalog data from S3