opendatazurich/sdk
Watch
2
mirror of https://github.com/opendatazurich/sdk.git synced 2026-10-19 19:05:25 +00:00
Code Issues Releases Activity
Everyting about OGD and SDK (Städtischer DatenKatalog) | Read-only mirror of https://github.com/opendatazurich/sdk — Stadt Zürich. Issues & pull requests at the source.
62 commits 2 branches 0 tags 425 KiB
  • Python 100%
Find a file
Exact
Repository files (latest commit first)
Filename Latest commit message Latest commit date
alexanderguentert 05191d0d91
 Merge pull request #5 from opendatazurich/preparation_initialimport 
Preparation initialimport Welle 1
2026-09-07 15:06:23 +02:00
ckan2sdk add sheet for rechtsgrundladen 2026-06-08 11:37:25 +02:00
sdk2ckan fix: linter 2024-04-29 08:24:54 +02:00
.flake8 init: project setup 2024-04-23 08:22:24 +02:00
.gitignore clean up for initialdaten import 2025-06-18 11:48:54 +02:00
.python-version init: project setup 2024-04-23 08:22:24 +02:00
poetry.lock Update poetry env and minor changes 2024-06-19 09:46:10 +02:00
pyproject.toml Update poetry env and minor changes 2024-06-19 09:46:10 +02:00
README.md update readme 2026-09-07 14:55:57 +02:00

README.md

SDK

poetry code style code style

This project contains the mapper from CKAN to SDK (ckan2sdk)

Prerequisites

Before you begin, ensure you have met the following requirements:

  • You have installed the latest version of poetry

Project Setup

Make sure poetry in installed correctly -> poetry --version

Next, run

poetry install  # to install all dependencies
poetry shell    # to enter poetry environment
flake8 --append-config .flake8 # appends user defined formatting rules to flake8 default set

sdk2ckan

Not done here, but in OGD-Pipeline on CMP (managed by Banian).

ckan2sdk

The script takes metadata from OGD catalog and formats an Excel file that can be imported into SDK (dataspot). This is used for the Initialimport of OGD metadata in SDK.

Usage Execute main script mapping.py. The script:

  • creates Excel Exports by organization (Dienstabteilung/Organisationseinheit)
  • filters certain organizations for pioneer group
  • excludes unwanted datasets (geo data oder ssz data)
  • adds "Datenlieferant" in SDK format
  • change field mapping to match the names of the SDK

Parameters an helper scripts

  • mapping.py: Contains all mappings that are needed, like column and value mappings for CKAN-SDK, or columns to export
  • filter_org.py: Contains filter variables for organization (Dienstabteilung, Departement). Use names from grobstruktur.json
  • grobstruktur.json: Maps Dienstabteilung and Departement in correct spelling for SDK
  • ckan_api.py: Helper functions for CKAN-API
  • cleaner.py: Helper functions for data cleaning and fuzzy-matching
  • exports.py: Helper functions for excel exports

Existing scripts

Exporting CKAN to DKUEL

Export CKAN metadata to CSVs for DK-ÜL import

Reads out metadata and does some reformating of certain ckan fields (attributes, date)

Extractting CKAN attributes

Extract STZH CKAN attributes to csv

Mapping CKAN Organisation to ORGANIZATIONAL UNITS

Mapping Metadata of Datenlieferant / Dataowner

Contributors