Engram (Metadata) User Guide
Metadata is an important capability of Algoreus. It helps show how datasets and programs are related to each other and aids in understanding the impact of a change before it is implemented.
These features provide full visibility into the impact of changes while also providing an audit trail of access to datasets by node applications and systems. Together, they offer a clear view when identifying trusted data sources and enable tracking the flow of sensitive data.
Algoreus captures metadata from various sources, including user-defined specifications, on different entities and objects. .
Metadata in Algoreus, comprising properties (a list of key-value pairs) or tags (a list of keys), can be used to annotate artifacts, applications, node programs, datasets, views, and custom entities.
Using the Algoreus Engram, you can set, retrieve, and delete these metadata annotations.
Metadata keys, values, and tags must adhere to the supported characters in Algoreus and are limited to 50 characters in length. The entire metadata object associated with a single entity is limited to 10K bytes in size.
System Metadata
While Algoreus Engram allows users to tag entities with metadata properties and tags, it also automatically tags entities with system properties and tags (system metadata) by default.
Discovery and Lineage
Metadata can be utilized to tag various Algoreus components, making them easily discoverable, identifiable, and manageable. Lineage provides a view of all data access for a specified time range, along with details of the origin of that access.
Audit Logging
Algoreus offers chronological logging that provides evidence of operations or changes made to entities.
Algoreus Engram UI
This user interface allows users to visualize the flow of data into and out of datasets.
Accessing metadata programmatically
Metadata can be retrieved or updated programmatically from Algoreus programs and nodes, primarily for enabling metadata-based processing.
Last updated
Was this helpful?