Discovery and Lineage
Last updated
Was this helpful?
Last updated
Was this helpful?
Metadata can be applied to various Algoreus components to enhance their discoverability and management efficiency. This facilitates the process of locating specific components within Engram.
For instance, a dataset might be marked as experimental, or an application might be designated as a production unit. These tagged entities can be easily found by implementing search queries with the associated metadata.
With the search feature, it's possible to locate entities:
possessing a certain value for any key within their properties;
having a specific key paired with a particular value in their properties; or
containing a unique tag.
You could, for instance, locate a dataset that contains a "field with a certain name" or a "field with a designated name and a specific type".
To facilitate metadata searches, the Algoreus Engram can be utilized.
Lineage can be extracted for dataset entities. A lineage illustrates all data interactions related to the entity within a specified timeframe and provides information about the origin of these interactions.
In the case of datasets, lineage can denote whether a dataset access was for read, write, or both actions, provided the methods within the dataset carry the right annotations. If such annotations are lacking, the lineage merely indicates that a dataset access occurred without specifying if it was for reading or writing.