Sending records to error

• To send records to error

• Adding an Error Collector Node to a Data Axon

There are three types of errors that can occur:

• Systemic

• Logical

• Data

Systemic errors include errors such as a service failing or the instance failing. Logical errors include errors such as an axon run failing. Data errors include errors such as invalid credit card numbers, invalid date formats, and invalid zip codes.

Algoreus provides a set of over 50 functions to help you remove common errors from a dataset.

---

To send records to error

Follow these steps:

1. Click the drop-down button next to the column name.

2. Click 'Send to error', and then select the condition to send bad records to error.

Algoreus removes values that match the specified condition from the sample and adds the 'send to error' directive to the recipe. When you run the data axon, the transformation is applied to all values in the column.

---

Adding an Error Collector Node to a Data Axon

When you add an Algoreus transformation with a recipe that includes the 'send to error' directive to a data axon, you can choose to connect it to an Error Collector node. Error Collector nodes are typically connected to a downstream sink node, such as GCS.

When you run an axon with an Error Collector, the records flagged with 'send to error' flow from the Algoreus transformation to the Error Collector and then to a sink node. After the axon run completes, you can examine the bad records written to the sink to better understand the problems with the data.

If your recipe includes 'send to error' transformation steps, but the axon doesn’t include an Error Collector, the records flagged as 'send to error' are dropped during the axon run.

---