Integrations

Metaflow

active

Platform production flows: one metadata service, YuniKorn pods, Airflow DAGs.

Metaflow is the production flow path for the federation. Signals pins weathership/oss-metaflow (rch/devenv) and runs one metadata service (:30180) and one S3 datastore on RustFS (s3://metaflow/metaflow/). Engines copy config/metaflow/platform.json and opt in; they do not stand up a second metadata service on a shared host.

Tasks are @kubernetes pods. YuniKorn admits them onto resource-class leaves. Production schedules are Airflow DAGs (airflow create); events flow through Knative Eventing (signals-events/default) rather than a second workflow SoR. Run records and artifacts are first-class data-product facts in the Signals warehouse.

Upstream: Outerbounds. Architecture: Platform Metaflow in the Signals tree.