flex-container-orchestrator
===========================

``flex-container-orchestrator`` provides the command-line orchestration layer for
the FLEXPART-IFS workflow used by the European Weather Cloud (EWC) blueprint.

The current EWC runtime is synchronous and command-line driven. It:

1. selects an IFS forecast cycle from ECMWF MARS, or uses an explicitly requested
   cycle;
2. retrieves and preprocesses the required IFS fields with ``flexprep``;
3. runs FLEXPART-IFS through Docker Compose; and
4. runs Pyflexplot to generate dispersion plots.

The installed command is ``flexpart-workflow``.

Runtime overview
----------------

::

    flexpart-workflow
          |
          +-- --latest
          |      `-- check recent IFS cycles in MARS and select the newest
          |          complete cycle for the requested forecast length
          |
          +-- --date YYYYMMDD --time HH
          |      `-- use an explicit IFS cycle
          |
          +-- --step HOURS
          +-- --location SITE
          |
          v
       flexprep
     MARS -> GRIB
          |
          v
    FLEXPART-IFS
    Docker Compose
          |
          v
      Pyflexplot
    Docker Compose

Requirements
------------

The EWC blueprint installs and configures the runtime dependencies. For a manual
or development installation, the orchestrator expects:

* Python >= 3.10 and < 3.13;
* a native ECMWF MARS client with valid user access;
* Docker with the Docker Compose plugin; and
* access to the workflow container images configured in ``docker-compose.yml``.

The Python dependency on ``flexprep`` is installed with the orchestrator.

Installation for development
----------------------------

Clone the repository and install it with Poetry:

.. code-block:: console

    $ cd flex-container-orchestrator
    $ poetry install --with dev

The package exposes the same CLI used on the EWC VM:

.. code-block:: console

    $ poetry run flexpart-workflow --help

Configuration
-------------

By default, ``flexpart-workflow`` reads:

::

    /etc/flexpart-ifs/workflow.yml

For local development, pass a configuration file explicitly:

.. code-block:: console

    $ poetry run flexpart-workflow \
        --config examples/workflow.yml \
        --date 20260812 \
        --time 06 \
        --step 4 \
        --location BEZ

A minimal configuration is provided in ``examples/workflow.yml``. It contains
static runtime settings such as paths, the flexprep domain/grid, and plotting
configuration. Forecast cycle, forecast duration, and release site are runtime
CLI arguments and are intentionally not fixed in the YAML configuration.

Example configuration
~~~~~~~~~~~~~~~~~~~~~

.. code-block:: yaml

    main:
      runtime:
        compose_file: ./docker-compose.yml
        run_dir: ./runtime/runs
        plot_dir: ./runtime/plots

      flexprep:
        output_dir: ./runtime/meteorology
        area: [65, -10, 35, 47]
        grid: [0.25, 0.25]
        max_level: 137
        mars_class: od
        expver: "1"

      workflow:
        plotting_preset: opr/ifs-hres-eu/all_pdf
        emission_duration_hours: 1

Usage
-----

Use the latest complete IFS cycle available in MARS:

.. code-block:: console

    $ flexpart-workflow --latest --step 24 --location BEZ

Use an explicit cycle for a reproducible run:

.. code-block:: console

    $ flexpart-workflow \
        --date 20260812 \
        --time 06 \
        --step 24 \
        --location BEZ

``--time`` must be one of ``00``, ``06``, ``12``, or ``18`` when ``--date`` is
used.

Workflow behaviour
------------------

Before FLEXPART is started, the orchestrator verifies that the required IFS
fields are available for the complete requested forecast range. Meteorology is
retrieved and preprocessed with ``flexprep`` and cached by forecast cycle.
FLEXPART is launched only after the expected preprocessed GRIB files are
present and non-empty.

The FLEXPART and Pyflexplot components are executed as short-lived Docker
Compose services. Runtime data remain on the host through bind-mounted
directories.

On an EWC blueprint deployment, the default locations are:

::

    /var/lib/flexpart-ifs/meteorology
    /var/lib/flexpart-ifs/runs
    /var/lib/flexpart-ifs/plots

EWC deployment
--------------

The recommended installation path for EWC users is the
``flexpart-ifs-ewc-blueprint`` repository. The blueprint installs this
orchestrator, the native MARS tooling, Docker, the workflow configuration, and
the required container images.

After deployment and user-specific MARS credential setup, a workflow can be
started with:

.. code-block:: console

    $ flexpart-workflow --latest --step 24 --location BEZ

Development and tests
---------------------

Run the unit tests with:

.. code-block:: console

    $ poetry run python -m pytest

Run static-analysis tools with:

.. code-block:: console

    $ poetry run pylint flex_container_orchestrator
    $ poetry run mypy flex_container_orchestrator

MeteoSwiss development environment
----------------------------------

The repository also contains MeteoSwiss-specific CI/development configuration.
When ``mchbuild`` is available, the existing local build/test commands can be
used:

.. code-block:: console

    $ pipx install mchbuild
    $ mchbuild local.build local.test

These helpers are not required by the EWC runtime.