# Data Flows and Factory

# Introduction

A Data Factory is a system for processing data using Data flows. Specifically:

  • Flow = a workflow or processing flow for transforming or analysing data.
  • Factory is the system for creating and orchestrating flows.

# Domain Model

  • Tasks
  • DAGs (Flows)
  • Factory

# CKAN v3

The Data Factory system is called AirCan and is built on top of AirFlow. Components:

Status: Beta. AirCan and ckanext-aircan are in active use in production. GUI is under development.

  • Research - list of tools and concepts with some analysis
  • History - some previous thinking and work (2016-2019)

# Appendix: Our Previous Work

See also History page.