Data Engineering

Data Engineering #

All the less-obvious-but-necessary work necessary to keep data systems operational.

This is the part that turns data science from an academic exercise into something real and operational.

The (Essential) Plumbing of Data #

A lot of the language used in data engineering systems borrows from plumbing. “Pipes”. “Flows”. “Streams”. “Watermarks”.

It’s an apt analogy - data engineering is often concerned with the process of moving streams of data from one space to another, and storing the data in reservoirs to serve some purpose.

Let’s stay in touch #