Cloning overview
Businesses traditionally maintain separate data stores for different types of workloads. Typically, Operational Data Stores (ODS) are used for production systems such as Point of Sales (PoS) systems, payroll systems, ERP systems, and so forth. Additionally, separate data warehouses are maintained for analytical purposes to ensure that analytics workloads do not negatively impact the performance of the ODS.
As a result, a need arises to replicate data between these different data stores. Factors such as increasing data volumes, growth in real-time use cases, and cloud migration have resulted in the need to consistently deliver data to various target subsystems and partners in real-time as soon as data changes in the source. To fulfill this need quickly and efficiently, it's important to clone only the delta (the data that has changed, often only a fraction of the data), rather than transferring entire batches in bulk all the time. A well-tuned data replication system is essential to unlock numerous business use cases such as migration and consolidation, as well as IT initiatives such as Business Intelligence and Data Warehousing, Data Science and Machine Learning, without adversely affecting critical production workloads.
For a long time, businesses have utilized Change Data Capture (CDC) to meet their real-time data replication requirements. Algoreus Clone empowers traditional enterprise ETL developers, data scientists, and business analysts to fulfill their data integration needs, with an intuitive graphical user interface. When combined with the existing data integration and ETL capabilities of Algoreus, this feature facilitates the following applications and use cases:
Access to up-to-date data for real-time analytics
Efficient data movement across modern, hybrid architectures
Faster data sharing and consistency
Unlocking traditional and modern processing paradigms
In essence, this capability enables users to:
Configure their source and target databases
Assess the impact of replication to detect incompatibilities and missing features before starting replication
Incrementally replicate data from source to target
Validate that data was successfully replicated
Monitor the data replication process
You can utilize Algoreus Clone to replicate changes at low-latency and in real-time from transactional and operational databases. With an easy-to-use wizard interface, users can set up replication jobs within minutes, requiring no coding.
Most companies store their most valuable data - transactional and operational data - on-prem in traditional relational databases. While outdated migrations or batch ETL uploads can move the data to their analytical data warehouse, these high-latency approaches cannot support continuous data axons and real-time operational decision-making. Users often struggle to reliably replicate data continuously from the transactional and operational database into their analytical data warehouse with minimal impact, and lacking the most up-to-date data for analytics in real-time hampers users' ability to make accurate decisions.
Algoreus offers a low-latency, real-time replication of data from transactional and operational databases. For the initial load of data, Algoreus Clone enables zero-downtime, zero-data-loss snapshot replication from databases. As an enterprise-grade solution, it has built-in, real-time monitoring to validate that the database transactions have loaded successfully, minimizing risk by ensuring data consistency. Algoreus Clone also features an inspection capability that helps identify schema, connectivity, configuration, and feature incompatibilities to spot issues and provide corrective actions for easier setup.
Not only does it load snapshots and continuously replicate changes, but it also provides in-flight processing such as filtering (tables, columns, operations, records), transformations into the desired schema, and data masking. This in-memory processing minimizes ETL workloads, improves performance, reduces complexity, and facilitates compliance. Algoreus Clone offers log-based replication from Oracle using Oracle XStream and SQLServer through SQL Server CDC. All of the CDC jobs can be accessed and configured (filtering, transformation, data masking) via Algoreus easy-to-use Wizards and drag-and-drop UI, accelerating the delivery of CDC to your analytical data warehouse. Algoreus also records end-to-end lineage including all filters and transformations integrated into its lineage tracking for compliance.
Last updated