ExampleTemplate

Nightly backup: a database table to CSV in S3

Every night, exports a city database table (here: permits) to a dated CSV file in S3, so there's always a plain-text copy outside the database.

This flow

3 steps

StartsOn a schedule

SystemsPostgreSQL / PostGIS, Amazon S3 / S3-compatible

Branches0

Stored recordsNone: pass-through

§ The flow

Scroll sideways on small screens. Colors show the kind of step: start, read, change, decide, write.

StartReadFilesDatabaseChangeDecideWriteAct

§ Step by step

01 · Start · On a schedule
Nightly at 2:30 am
- Cron 30 2 * * * (workspace time zone)
Then: Whole permits table
02 · Database · Read database · PostgreSQL / PostGIS
Whole permits table
- Reads from PostgreSQL / PostGIS with a read-only query
- ```
SELECT permit_number, permit_type, status, address, applicant, issued_on, valuation, ST_Y(geom) AS lat, ST_X(geom) AS lng, updated_at
FROM permits
ORDER BY permit_number
```
- Row ID column: permit_number (used for replay and lineage)
Then: Dated CSV file
03 · Files · Write file · Amazon S3 / S3-compatible
Dated CSV file
- Writes CSV file permits-{{ $today }}.csv to citydata/{{ $today }}
- One file with every record

§ Systems

Swap any of them for another system with the same kind of data.

§ Next

We'll set up a workspace with your systems in a sandbox and build your first flow with you.