Automation for Analysts: Pipelines
Chain extract-transform steps with logging, retries, and human-readable failure modes.
Tuition is informational only. Enrollment staff confirm availability and policies by email.
Request a syllabus callOverview
Design small pipelines that behave politely when vendors change formats. You will wrap tasks in functions, add structured logging, and schedule runs with simple cron-friendly entry points.
What you build
- Task decomposition patterns
- Structured logging with correlation ids
- Retry policies that avoid duplicate writes
- Dry-run modes for destructive steps
- Packaging with pyproject basics
- Smoke tests using fixture files
- Handover notes for on-call rotation
Outcomes
- Deliver a two-step pipeline with logged failures
- Demonstrate a dry-run path for reviewers
- Document rollback expectations in plain language
Mentor on record
Haneul Park
Lead Python instructor focused on analyst-friendly tooling and reviewable notebooks.
FAQ
Which orchestration tools appear?
We stay with CLI entry points and simple schedulers. Airflow-style platforms are mentioned only as comparison reading.
Cloud accounts?
You can complete everything locally. Cloud uploads are optional and not graded.
Known limitation?
We do not cover Kubernetes or container networking.
Participant notes
Dry-run mode alone saved us from a bad vendor cutover email blast.