mirror of
https://github.com/MeteoSwiss/gawkenya.git
synced 2026-10-18 20:46:08 +00:00
Read-only mirror of https://github.com/MeteoSwiss/gawkenya — MeteoSwiss. Issues & pull requests at the source.
- Jupyter Notebook 93.5%
- Python 6.2%
- R 0.1%
- HTML 0.1%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
| .vscode | ||
| analyses | ||
| config | ||
| docs | ||
| housekeeping | ||
| logs | ||
| monitoring | ||
| obsolete | ||
| processing | ||
| results | ||
| sandbox | ||
| tests | ||
| thesis/tobias | ||
| toolbox | ||
| .gitignore | ||
| .python-version | ||
| ae31_analysis_sandbox.ipynb | ||
| aerosol_aurora_3000_check.ipynb | ||
| aerosol_process_level1_data.ipynb | ||
| aq_conference_2025_nairobi.ipynb | ||
| archive_data.ipynb | ||
| avo.ipynb | ||
| avo_plot_temporal_aggregates.ipynb | ||
| boxplots.ipynb | ||
| cdp2_aurora_mappings.csv | ||
| compile_level1_to_level2_files.py | ||
| compile_raw_data_files.ipynb | ||
| dashboard.py | ||
| eaton-ups.ipynb | ||
| ecc.ipynb | ||
| evaluate_avo_fidas_ae3x_hourly.py | ||
| ez_flag_data copy.ipynb | ||
| ez_flag_data.ipynb | ||
| g2401_process_level1_data.ipynb | ||
| kplc.ipynb | ||
| LICENSE | ||
| mch_dwh_jretrieve.ipynb | ||
| meteo_compile_data_for_submission.ipynb | ||
| meteo_process_level1_data.ipynb | ||
| meteo_recode_flagged_data_for_DWH.ipynb | ||
| mkn_aerosol_analysis.ipynb | ||
| nappy-2.0.2.tar.gz | ||
| ne300.ipynb | ||
| ne300_from_logger.parquet | ||
| organize_files_cli.py | ||
| preprocess-ne300.ipynb | ||
| README.md | ||
| requirements.txt | ||
| sandbox.ipynb | ||
| satellite.ipynb | ||
| satellite_vs_insitu.ipynb | ||
gawkenya
Support an end-to-end data chain from instrument to raw data to clean data to products for GAW Kenya.
setup
Install pyenv
- Follow directions under https://realpython.com/intro-to-pyenv/, basically $ curl https://pyenv.run | bash $ pyenv install -v 3.11
- Verify with
$ pyenv versions, which should show something like
- system (set by /home/zue/users/jkl/.pyenv/version) 3.11.7
- Test installation (NB: this will take some time) with $ pyenv global 3.11.7 $ python -m test
- Create virtual environment with $ pyenv virtualenv 3.11.7 gawkenya
- Activate the environment $ pyenv local gawkenya
- Install requirements (NB: path depends on your local git) $ pyenv exec pip install -r "/home/zue/users/jkl/Public/git/gawkenya/requirements.txt"
Install nappy (needed for AMES files only)
- pip install nappy
If that does not work, try: - pip install git+https://github.com/joergklausen/nappy.git
Repo structure
analyses: Analyses of level 2 data (from WDC), comparison with e.g. CAMS data. See separate README.input/: scripts to read the data (CAMS model data and measurement data from WDC and ) \output/: Save analyses figures (saved locally only, excluded from git) \plotting/: scripts for plotting:
plot_curve_fits.ipynb: Use the NOAA curve-fitting approach (saved inutils/ccg_filter) to run curve fits and remove trends from the data. Make plots with the fitted lines.
utils: utilities required for the analyses
datalevel1- monthly folders for monthly .parquet files (g2401)
- yearly .parquet data (ae33, aerosol: A11a, S11a, S11c, S11k, S11m, S11s)
level2level3: various gridded model and satellite datacams: all selected cams data at the Mt. Kenya grid (cams_best_Grid_merged_MKN.nc)
noaaupswdc: Mount Kenya measurement downloaded from the World Data centers (ebas and WDCGG)
ez_flag_data- TODO: explainations
housekeeping- TODO: explainations
monitoring- TODO: explainations
obsolete: contains the scripts by Sarina to make data plots for a poster.processing: contains one python script to process data from each instrument (e.g.ae33.py) or database (e.g.ebas.py)results: some outputssandboxteststoolbox.[...]_compile_[...].ipynb: jupyter notebooks to compile .... TODO[...]_process_[...].ipynb: jupyter notebooks to process .... TODO