Skip to content

Tree from a file

This tree is not written on the page. The fence below holds a single line, src: trees/ocean-obs.yaml, and the tree itself lives in snippets/trees/ocean-obs.yaml, next to the JSON Schema that editors use to check it. The same file could be shown on several pages.

ocean-obs/

ocean-obs / pyproject.toml

pyproject.toml

committed

Package metadata and tool settings

Dependencies and tool settings for the processing package.

[project]
name = "ocean-obs"
dependencies = ["xarray", "netCDF4"]

ocean-obs / data

data/

Measurements, from instrument output to gridded products

Contents

  • raw/Untouched instrument files, never edited
  • processed/Quality-controlled, gridded output

ocean-obs / data / raw

raw/

gitignored

Untouched instrument files, never edited

Written by
The ship's logging system
Size
About 2 GB per cruise

Raw files stay exactly as the instrument wrote them, so every processing step can be rerun from scratch. They are too large for git and are synced from the archive instead.

ocean-obs / data / processed

processed/

generated

Quality-controlled, gridded output

Contents

ocean-obs / data / processed / underway.nc

underway.nc

generated

One file per cruise

Written by
make process

ocean-obs / notebooks

notebooks/

Exploration and figures

Contents

ocean-obs / notebooks / overview.ipynb

overview.ipynb

committed

Cruise track and first look at the data

ocean-obs / process.py

process.py

committed

Turns raw files into processed ones

The processing entry point. Its description lives in snippets/process-py.md and is referenced from the tree file with body_file, exactly as from a fence.

import xarray as xr


def process(raw: str, out: str) -> None:
    ds = xr.open_dataset(raw)
    ds.where(ds.qc_flag == 0).to_netcdf(out)

Raw data stays out of git: see data/raw/.