Home
Turium Algoreus
Turium Algoreus
  • Turium Algoreus Documentation
    • Turium Algoreus Overview
      • How to Guides
        • Axons (Pipeline) User Guide
          • Algoreus Genesis
          • Algoreus Node
          • Steps for a simple batch Axon in Algoreus
          • Configuring Axon in Algoreus
          • Deploying an Axon in Algoreus
          • Running an Axon in Algoreus
          • Viewing and downloading logs in the Genesis in Algoreus
          • Scheduling an Axon in Algoreus
          • Reusable Axons in Algoreus
          • Using Triggers in Algoreus
          • Working with multiple versions of the same node in Algoreus
          • Modifying a draft Axon in Algoreus
          • Editing a deployed Axon in Algoreus
          • Duplicating An Axon in Algoreus
          • Deleting an Axon in Algoreus
          • Deploying nodes from the Algoreus Hub
          • Using node templates in Algoreus
          • Exporting and importing Axons in Algoreus
          • Dynamic resource configuration in Algoreus
          • Working with namespaces in Algoreus
        • Soma (Transformation) User Guide
          • Algoreus Soma Overview
          • Algoreus Soma Concepts
          • Algoreus Soma UI components
          • Working with multiple datasets
          • Navigating between Soma and Algoreus Genesis
          • Editing a transformation created in the Soma
          • Soma data types
          • Working with connections in Soma
          • Parsing a CSV file
          • Strings Formatting
          • Sending records to error
          • Working with numbers in Soma
          • Working with Decimal types in Soma
          • Performing date transformations in Soma
          • Filtering records
          • Finding and replacing values in a column
          • Filling null or empty cells
          • Copying, deleting, and keeping columns
          • Renaming a column
          • Joining two columns
          • Swapping two column names
          • Extracting fields to retrieve values
          • Exploding fields
          • Masking data
          • Encoding records to store or transfer data
          • Decoding records to store or transfer data
          • Applying a Hashing algorithm to a column
          • Upgrading the Soma transformation node version
          • Viewing and downloading a schema in Soma
          • Viewing Soma Service logs
        • Cerebellum (Operations and Monitoring) User Guide
          • Logging and Monitoring
          • Metrics
          • Dashboard and Reports
          • Preferences and Runtime Arguments
          • Transaction Service Maintenance
        • Engram (Metadata) User Guide
          • System Metadata
          • Discovery and Lineage
          • Audit Logging
          • Metadata Management
          • Accessing Metadata Programmatically
          • Metadata Field-Level Lineage
        • Clone (Replication) User Guide
          • Cloning overview
          • Clone Concepts
          • Adding Transformations to a Cloning Job
          • Deleting a Cloning Job
          • Tutorial: Cloning data from Oracle Database to BigQuery
        • Algology (Visualisation) User Guide
          • Dashboards
            • Using Dashboards
            • Building Dashboards
            • Manage dashboards
            • Publishing Dashboard
            • Playlist
            • Create and manage reports
            • Share dashboards and panels
            • Access Dashboard Usage
            • Search Dashboards
          • Panel Editor
            • Configure Panel Options
            • Configure standard options
          • Visualisations
            • Alert List
            • Bar Chart
            • Bar Gauge
            • Candlestick Panel
            • Canvas
            • Dashboard List
            • Flame Graph
            • Gauge
            • Heatmap
            • Histogram
            • Logs
            • Node Graph
            • Traces Panel
            • Pie Chart
            • State Timeline
            • Stat Panel
            • Time series
            • Trend Panel
            • Text Panel
            • Table
            • GeoMap
            • Datagrid Panel
            • Status history
            • Annotations
          • Explore
            • Logs in Explore
            • Queries in Explore
            • Tracing in Explore
            • Inspector in Explore
    • Turium Algoreus Connectors
Powered by GitBook
On this page
  • To extract values based on a pattern
  • Using delimiters
  • Using positions

Was this helpful?

  1. Turium Algoreus Documentation
  2. Turium Algoreus Overview
  3. How to Guides
  4. Soma (Transformation) User Guide

Extracting fields to retrieve values

PreviousSwapping two column namesNextExploding fields

Last updated 1 year ago

Was this helpful?

You may have valuable information stored in a single column, split by a delimiter. To execute transformations on this data, you need to segregate the values into different columns. In Algoreus, you can retrieve values from a column and generate a new column for the extracted values. This extraction can be based on:

  • Patterns

  • Delimiters

  • Positions

  • Utilizing patterns

You can retrieve values from fields in String columns using the following common patterns:

  • Credit cards

  • Date

  • Date time

  • Email

  • URLs from HTML anchors

  • IPv4 address

  • ISBN codes

  • Mac address

  • N digits number

  • SSN

  • Start/End pattern

  • Time


To extract values based on a pattern

Follow these steps:

  1. From the drop-down menu, click Extract data > Utilizing patterns.

  2. Select a pattern to extract the fields. Optionally, click Show pattern to view the regex for that pattern.

Algoreus extracts the fields based on the chosen pattern and adds the extract-regex-groups directive to the recipe. When you run the data axon, the transformation will be applied to all values in the column.


Using delimiters

You can break down a column into two or more columns based on a delimiter. The extraction of values can be based on the following delimiters:

  • Comma

  • Tab

  • Pipe

  • Whitespace

  • Custom separator

Note: If you select Custom separator, a regular expression (regex) is required to define the separator. Regular expressions allow for the use of intricate search patterns when splitting the data in the column. It supports standard Java regular expression constructs.

If the value does not contain the specified separator, then no additional rows are generated.

To extract values based on a delimiter:

  1. From the drop-down menu, click Extract data > Using delimiters.

  2. Select the delimiter to use to extract the fields.

Algoreus extracts the fields based on the chosen delimiter and adds the split-to-columns directive to the recipe. When you run the data axon, the transformation will be applied to all values in the column.


Using positions

You can extract a part of a string based on its position in the string.

To extract fields based on positions from a column:

  1. From the drop-down menu, click Extract data > Using positions.

  2. The column appears with a blue background, indicating that you are in Extract mode.

  3. Highlight the portion of one value you want to extract for all values in the column. The Extract dialog box appears showing the positions you selected to extract:

  4. Provide a name for the new column.

Algoreus extracts the fields based on the selected pattern and adds the cut-character directive to the recipe. When you run the data axon, the transformation will be applied to all values in the column.


To extract values based on a pattern
Using delimiters
Using positions