Configuring Axon in Algoreus
Last updated
Was this helpful?
Last updated
Was this helpful?
Once you have deployed an axon, the following aspects can be configured:
You can modify the compute profile used to execute this axon. For instance, you might want to run the axon against an existing cluster instead of the default cluster.
For each axon, the Instrumentation feature can be enabled or disabled. By default, Instrumentation is activated. If Instrumentation is enabled, running the axon will generate metrics for each node. The metrics for the source, transformation, and sink nodes slightly differ and include:
Records out
Records in
Total number of errors
Records out per second
Min process time (one record)
Max process time (one record)
Standard deviation
Average processing time
These metrics are displayed on the Metrics tab for each node in an axon. Unless the environment is resource-constrained, it is advised to always keep the Instrumentation setting on.
For streaming axons, the Batch interval for streaming data can also be set.
For each axon, Spark or MapReduce can be selected as the execution engine, with Spark being the default option. Additional engine configurations can be added through custom configurations, which typically apply to Spark more frequently than to MapReduce.
You can designate the memory and the number of CPUs for the driver and executor. The driver manages the Spark job while the executor handles data processing.
After the axon execution concludes, you can set up Algoreus to dispatch alerts and initiate post-processing tasks. While designing the axon, axon alerts can be created. Once the axon is deployed, these alerts can be viewed, but no additional alerts can be added or existing ones edited. You can select the action to be performed by Algoreus upon the completion of an axon run, such as:
Send email
Execute a database query
Create a marker file
Make an HTTP call
Run a MySQL query
You can choose to send an alert or trigger a post-processing task based on the run status. It is also possible to create custom nodes to perform specific tasks. Various open-source nodes can be added as an axon alert as well.