Apache Beam #
A tool for handling both batch and stream data processing pipelines.
Resources #
A lot of the content here is pulled from the Tour of Beam.
Testing #
I like to test Beam out locally in a throwaway directory.
$ mkdir test-beam
$ cd test-beam
$ python3 -m venv venv
$ source venv/bin/activate
$ pip3 install apache-beam
Tada! that’s it.
Notebooks #
There are some handy notebooks to get started: