Discovery and Lineage

• Discovery

• Lineage

Discovery

Metadata can be applied to various Algoreus components to enhance their discoverability and management efficiency. This facilitates the process of locating specific components within Engram.

For instance, a dataset might be marked as experimental, or an application might be designated as a production unit. These tagged entities can be easily found by implementing search queries with the associated metadata.

With the search feature, it's possible to locate entities:

• possessing a certain value for any key within their properties;

• having a specific key paired with a particular value in their properties; or

• containing a unique tag.

You could, for instance, locate a dataset that contains a "field with a certain name" or a "field with a designated name and a specific type".

To facilitate metadata searches, the Algoreus Engram can be utilized.

---

Lineage

Lineage can be extracted for dataset entities. A lineage illustrates all data interactions related to the entity within a specified timeframe and provides information about the origin of these interactions.

In the case of datasets, lineage can denote whether a dataset access was for read, write, or both actions, provided the methods within the dataset carry the right annotations. If such annotations are lacking, the lineage merely indicates that a dataset access occurred without specifying if it was for reading or writing.

---