Data Engineering #
All the less-obvious-but-necessary work necessary to keep data systems operational.
This is the part that turns data science from an academic exercise into something real and operational.
The (Essential) Plumbing of Data #
A lot of the language used in data engineering systems borrows from plumbing. “Pipes”. “Flows”. “Streams”. “Watermarks”.
It’s an apt analogy - data engineering is often concerned with the process of moving streams of data from one space to another, and storing the data in reservoirs to serve some purpose.