Home
Turium Algoreus
Turium Algoreus
  • Turium Algoreus Documentation
    • Turium Algoreus Overview
      • How to Guides
        • Axons (Pipeline) User Guide
          • Algoreus Genesis
          • Algoreus Node
          • Steps for a simple batch Axon in Algoreus
          • Configuring Axon in Algoreus
          • Deploying an Axon in Algoreus
          • Running an Axon in Algoreus
          • Viewing and downloading logs in the Genesis in Algoreus
          • Scheduling an Axon in Algoreus
          • Reusable Axons in Algoreus
          • Using Triggers in Algoreus
          • Working with multiple versions of the same node in Algoreus
          • Modifying a draft Axon in Algoreus
          • Editing a deployed Axon in Algoreus
          • Duplicating An Axon in Algoreus
          • Deleting an Axon in Algoreus
          • Deploying nodes from the Algoreus Hub
          • Using node templates in Algoreus
          • Exporting and importing Axons in Algoreus
          • Dynamic resource configuration in Algoreus
          • Working with namespaces in Algoreus
        • Soma (Transformation) User Guide
          • Algoreus Soma Overview
          • Algoreus Soma Concepts
          • Algoreus Soma UI components
          • Working with multiple datasets
          • Navigating between Soma and Algoreus Genesis
          • Editing a transformation created in the Soma
          • Soma data types
          • Working with connections in Soma
          • Parsing a CSV file
          • Strings Formatting
          • Sending records to error
          • Working with numbers in Soma
          • Working with Decimal types in Soma
          • Performing date transformations in Soma
          • Filtering records
          • Finding and replacing values in a column
          • Filling null or empty cells
          • Copying, deleting, and keeping columns
          • Renaming a column
          • Joining two columns
          • Swapping two column names
          • Extracting fields to retrieve values
          • Exploding fields
          • Masking data
          • Encoding records to store or transfer data
          • Decoding records to store or transfer data
          • Applying a Hashing algorithm to a column
          • Upgrading the Soma transformation node version
          • Viewing and downloading a schema in Soma
          • Viewing Soma Service logs
        • Cerebellum (Operations and Monitoring) User Guide
          • Logging and Monitoring
          • Metrics
          • Dashboard and Reports
          • Preferences and Runtime Arguments
          • Transaction Service Maintenance
        • Engram (Metadata) User Guide
          • System Metadata
          • Discovery and Lineage
          • Audit Logging
          • Metadata Management
          • Accessing Metadata Programmatically
          • Metadata Field-Level Lineage
        • Clone (Replication) User Guide
          • Cloning overview
          • Clone Concepts
          • Adding Transformations to a Cloning Job
          • Deleting a Cloning Job
          • Tutorial: Cloning data from Oracle Database to BigQuery
        • Algology (Visualisation) User Guide
          • Dashboards
            • Using Dashboards
            • Building Dashboards
            • Manage dashboards
            • Publishing Dashboard
            • Playlist
            • Create and manage reports
            • Share dashboards and panels
            • Access Dashboard Usage
            • Search Dashboards
          • Panel Editor
            • Configure Panel Options
            • Configure standard options
          • Visualisations
            • Alert List
            • Bar Chart
            • Bar Gauge
            • Candlestick Panel
            • Canvas
            • Dashboard List
            • Flame Graph
            • Gauge
            • Heatmap
            • Histogram
            • Logs
            • Node Graph
            • Traces Panel
            • Pie Chart
            • State Timeline
            • Stat Panel
            • Time series
            • Trend Panel
            • Text Panel
            • Table
            • GeoMap
            • Datagrid Panel
            • Status history
            • Annotations
          • Explore
            • Logs in Explore
            • Queries in Explore
            • Tracing in Explore
            • Inspector in Explore
    • Turium Algoreus Connectors
Powered by GitBook
On this page
  • Data Axon Modes
  • Data Axon Development Process Overview

Was this helpful?

  1. Turium Algoreus Documentation
  2. Turium Algoreus Overview
  3. How to Guides
  4. Axons (Pipeline) User Guide

Algoreus Genesis

PreviousAxons (Pipeline) User GuideNextAlgoreus Node

Last updated 1 year ago

Was this helpful?

Algoreus Genesis is your one-stop solution for data axons creation and management. It's a graphical tool that enables users - ranging from ETL developers and business data analysts to data scientists, engineers, and developers - to design and control data flows with the objective of data extraction, transformation, aggregation, and loading.

Whether you're interfacing with on-premise or cloud data sources, Algoreus Genesis simplifies data ingestion, integration, and migration processes. Building and managing data lakes have never been more streamlined.

  1. Creating Axons: Navigate to the top left of the Genesis to select the type of data axon you wish to create - batch or real-time. Below this, you'll find the node palette, which hosts all available nodes needed for axon construction.

  2. Validating Axons: Each stage of the axon has built-in validation logic to ensure successful execution of your transformations.

  3. ETL Development Tasks: Genesis also assists with other ETL tasks including axon debugging via Preview, axon deployment, axon saving, axon resource configuration, axon job scheduling, and import/export of axons.

  4. Nodes: The building blocks of a axon are called nodes, encompassing sources, transformations, and sinks. To add any node to an axon, simply click on it in the node section.

  5. Node Configuration: Each node has mandatory fields indicated with an asterisk. Utilize the 'Label' field under the configuration section of each node to provide a comprehensive description. Depending on the node, you might observe an Input Schema on the left, a Configuration section in the middle, and an Output Schema on the right.

  6. Schema Editing: You can tailor the output schema based on your requirements, permitting or disallowing nulls for each field, modifying the data type of any field, and pruning unnecessary fields from the output schema.


Data Axon Modes

Genesis facilitates two modes: Draft and Deployed.

  1. Draft Mode: Within the Genesis, you can save your ongoing work on an Axon as a draft at any time. This allows for the configuration to be preserved and editing to be resumed at your convenience. To create a draft, assign your Axon a unique name and click 'Save'.

  2. Deployed Mode: Upon completing axon design and debugging, and validating the data in Preview, the axon can be deployed. Once deployed, the axon transforms into an immutable, non-versioned entity. The axon can be run, deleted, or duplicated (which creates a draft version of the axon in the Genesis canvas).


Data Axon Development Process Overview

  1. Establish an axon - incorporate sources, transformations, and sinks.

  2. Preview the output - adjust the axon as needed to produce the desired output.

  3. Deploy the Axon.

  4. Configure Axon resources.

  5. Execute the Axon.

  6. Validate the output in the sinks.

Take advantage of Algoreus Genesis and transform your data management tasks with simplified, efficient processes.


Data Axon Modes
Data Axon Development Process Overview