Metaflow
activePlatform production flows: one metadata service, YuniKorn pods, Airflow DAGs.
Metaflow is the production flow path for
the federation. Signals pins
weathership/oss-metaflow
(rch/devenv) and runs one metadata service (:30180) and one
S3 datastore on RustFS (s3://metaflow/metaflow/). Engines copy
config/metaflow/platform.json and opt in; they do not stand up a
second metadata service on a shared host.
Tasks are @kubernetes pods. YuniKorn
admits them onto resource-class leaves. Production schedules are
Airflow DAGs (airflow create); events flow through Knative Eventing
(signals-events/default) rather than a second workflow SoR. Run
records and artifacts are first-class data-product facts in the
Signals warehouse.
Upstream: Outerbounds. Architecture: Platform Metaflow in the Signals tree.