Mtg-cloud-autometar-poc

Abstract

This project was started with the intent of creating a pipeline to easily download images from different EUMETSAT products for preliminary analysis. The ultimate goal is to assist AMAROC by automating the visual analysis of clouds over Swiss airports. So far, we have implemented two pipelines to download, crop, and process different EUMETSAT products. These pipelines can download selected EUMETSAT products, crop them to the region of interest, and overlay information useful for meteorologists to better describe conditions over Swiss airports.

Project Introduction

In this repository you will find a simple and convenient way to download, store, and process data from EUMETSAT products. The repository supports downloading EU:EUM:DAT:0665 and EU:EUM:DAT:0662, which belong to the FCI Level-1 products. FCI Level-1 products are optical imagery acquired by EUMETSAT satellites. The data consist of 16 spectral channels at different wavelengths, used to measure various aspects of clouds over Europe and Africa. The pipelines can also download EU:EUM:DAT:0684, which belongs to the FCI Level-2 products. FCI Level-2 products provide derived numerical information about clouds.

For more information on these products, consult the guides provided by EUMETSAT (see the Sources section). The repository provides functions to retrieve, store, and process these products based on user-specified coordinates and timestamps.

Project Overview and Features

Technologies Used

The whole project is python based, I utilise libraries like satpy to load and cut the files downloaded with eumdac. As a base einveronment to install all the libraries needed I used mini-forge and conda.

Key Features

The project offers a couple of very useful functions like the possibility to download a precise product with a precise timestamp for both FCI L1 and L2 products.

FCI L1 products

For FCI L1 products, the system offers the possibility to download and save a specific group of chunks based on given coordinates or on the acronym of the specified airport. Unfortunately if the interested area is between two chunks the pipeline is not able to put the two chunks togheter. Currently, the only two airports saved in memory are LSGG (Geneva Airport) and LSZH (Zürich Airport). If you are interested in more airports, all you need to do is add the coordinates and the acronym of the new airport in the airports file inside the constants folder. If you prefer, you can also directly specify the coordinates of interest when calling the pipeline command.

By default, the selected area is a square of 70 km in length around the specified center, but you can also define the size of the area of interest in the pipeline commands. To select a specific set of timestamps, you just need to specify the start and end date and time of the desired time window. For more information on the possible attributes, refer to the source code.

Te pipeline also saves the cut zarr files of the different channels so to occupy less space on the device but do not get rid of the "old", larger in volume, files.

To process this data, the pipeline offers a dedicated function that extracts the selected chunk and generates images for the various datasets and composites. The images are saved in folders divided by timestamp, but if you prefer them organized by channel, you can use the function reorganize_fci_img().

By default, the pipeline draws 3 circles with radii of 8, 16, and 30 km, as well as a series of skywatchers near the selected airports. To add, remove, or change these elements, you can edit the parameters in the corresponding file in the constants folder. All this information is added using the function draw_fci_l1_spatial_info().

At the end, the pipeline creates a GIF for each available channel to visualize all timestamps together, using the function create_gif().

Keep in mind that to process specific EU:EUM:DAT:0665 composites you also need EU:EUM:DAT:0662 datasets. By default, if you download EU:EUM:DAT:0665 products, the corresponding EU:EUM:DAT:0662 products are automatically downloaded as well.

Final data structure:

• fci_l1_data/
  • fci_l1_gifs/
    • FDHSL_gifs/
    • HRFL_gifs/
  • fci_l1_nc_files/
  • fci_l1_processed_img/

contained in the fci_data folder

FCI L2 products

The pipeline for the FCI L2 products offer less functionalities with respect to the FCI L1 pipeline

For FCI L2 products, the system offers the possibility to download and save a specific group of chunks based on given coordinates or on the acronym of the specified airport. Unfortunately if the interested area is between two chunks the pipeline is not able to put the two chunks togheter. Currently, the only two airports saved in memory are LSGG (Geneva Airport) and LSZH (Zürich Airport). If you are interested in more airports, all you need to do is add the coordinates and the acronym of the new airport in the airports file inside the constants folder. If you prefer, you can also directly specify the coordinates of interest when calling the pipeline command.

By default, the selected area is a square of 70 km in length around the specified center, but you can also define the size of the area of interest in the pipeline commands. To select a specific set of timestamps, you just need to specify the start and end date and time of the desired time window. For more information on the possible attributes, refer to the source code.

The final folder structure is the same retrieved in the FCI L1 pipeline, with the exception of not creating any tipe of gif.

By default, the pipeline draws 3 circles with radii of 8, 16, and 30 km, as well as a series of skywatchers near the selected airports. It also draws over the image the borders and the various rivers and lakes that you can find in the selected region. To add, remove, or apply changes to these elements, you can edit the parameters in the corresponding file in the constants folder. All this information is added using the function prcss_fci_l2_data().

Final data structure:

• fci_l2_data/
  • fci_l2_nc_files/
  • fci_l2_processed_img/

contained in the fci_data folder

Einvorment setup

conda env create -f environment.yml

Examples

To run the following examples, you need to have the repository cloned in your local machine, and you need to have the conda environment activated inside the reposirtory mtg-cloud-autometar-poc.

fci l1 example:

python get_fci_l1_data.py --airport LSGG --download_start 2025-08-02T14:30 --download_end 2025-08-02T14:32 --box_size 70

This command will download the FCI L1 data for the LSGG airport between 2025-08-02T14:30 and 2025-08-02T14:32, with a box size of 70 km. Then automatically it will process the data and create a GIF for each channel.

fci l2 example:

python get_fci_l2_data.py --airport LSGG --download_start 2025-08-02T14:30 --download_end 2025-08-02T14:32 --box_size 70

This command will download the FCI L1 data for the LSGG airport between 2025-08-02T14:30 and 2025-08-02T14:32, with a box size of 70 km. Then automatically it will process the data and create a GIF for each channel.