Recipe: Observe Your First Pipeline
You are here because you want to detect, resolve, and prevent data issues in a pipeline -- but how to get started?
This is the purpose of the below step-by-step recipe, which will guide the success of your data observability initiative:
- Identify a couple of issues you have faced in this project (or another you want to protect).
- Determine the impacts (time, costs, money, energy, mood, trust, etc.).
- Team up with the technical referents for the applications /steps in your pipeline.
- Determine the technologies used for the applications in your pipeline.
- Based on issues defined in 1., prioritize which application to enrich with data observability first, then second, and so forth.
- Use the visualization tool to select the observations to create rules to detect and anticipate issues defined in 1.