Import Matillion Circuit Breaker

agent / circuit breaker the circuit breaker feature in matillion allows you to control the execution of a pipeline's job based on predefined pre or post conditions this feature adds an extra layer of control and flexibility to your data processing workflows once the circuit breaker detects a failure, it immediately halts the execution of the job and marks it as failed this prevents further processing, ensuring that potentially problematic data is not propagated downstream by making the circuit breaker a critical condition for the pipeline's success, you can proactively identify and respond to issues, thereby maintaining the integrity and reliability of your data processing workflows install in matillion to install the circuit breaker feature in matillion and configure the necessary environment variables, follow these steps import the matillion job note this will be updated to follow the new ' shared job ' component log in to your matillion instance navigate to the "project" tab click on the "manage shared jobs" button click on import select the melt file located in this folder https //public usnek com/n/repository/kensu public/releases/matillion agent/kensu circuit breaker%20(rev 7) melt confirm by clicking on ok configure matillion environment variables in the matillion user interface, go to the "project" tab click on "manage environment variables" from the sidebar menu locate the following environment variables kensu hub user the username for your kensu hub kensu hub password the password for your kensu hub kensu hub url the url for your kensu hub instance, with /api/v1/ click on the "add new" button to create a new environment variable enter the name of the environment variable (e g , kensu hub user ) and its corresponding value repeat the above steps for the remaining environment variables ( kensu hub password and kensu hub url ), entering their respective values save the environment variable configurations use in matillion in matillion jobs to incorporate the circuit breaker functionality, you need to add a specific matillion activity within your pipeline this activity acts as a container for the logic that determines whether the job should proceed or be halted evaluating pre and post conditions the circuit breaker evaluates both pre and post conditions to determine the job's execution status pre conditions these conditions check for any issues or problems in the upstream application or data sources if any pre conditions fail, indicating a problem in the data supply chain, the circuit breaker will fail post conditions these conditions monitor the application and detect any reported issues or tickets if a post condition fails, indicating an issue in the application's state, the circuit breaker will fail