Preferences and Runtime Arguments
Preferences in Algoreus provide the ability to save configuration information at various levels of the system, including the Algoreus instance, namespace, application, and program levels. A configuration is represented by a map of string-string pairs. Preferences can be retrieved, saved, and deleted through the Preferences and through the Command Line Interface. When programs are started, all the preferences at the different levels are collapsed into a single map. Preferences are persisted across a restart of either programs or Algoreus.
Order of Preferences
The overriding order of preferences is (from lowest to highest precedence):
Algoreus Instance < Namespace < Application < Program < Runtime Arguments (passed in during start of the program)
Programs such as Spark programs, services, workflows, and workers will receive the resolved preferences and can be accessed through the getRuntimeArguments method of the context
For services and workers
Preferences are available to the initialize method in the context.
For MapReduce and Spark
Preferences are available to the initialize and destroy methods in the context. The initialize method can pass them to the mappers and reducers through the job configuration.
When a workflow receives preferences, it passes them to each MapReduce in the workflow.
Preferences and Runtime Arguments in the Algoreus Cerebellum
In the Algoreus Cerebellum, preferences can be set at the different levels (as described above), with runtime arguments being available at the lowest level of individual elements.
Preferences are retained, and are persisted across different runs and restarts of applications and Algoreus Cerebellum. Runtime arguments (available under the Arguments buttons in the Algoreus Cerebellum) are one-time only, for a particular invocation, and are not persisted. Use the Preferences button with the same key-value pairs for arguments you would like persisted.
System-populated Runtime Arguments
On each program run, Algoreus populates the runtime arguments with pre-defined values (currently, one):
logical.start.time: The start time of the run as a timestamp in milliseconds. If the run is started by a schedule, this will be equal to the trigger time for the schedule. For example, if the schedule is set to run at midnight on Jan 1, 2016 UTC, the logical start time would be 1451606400000.
If an argument is passed in to start a program run, it will override the system-populated value.
Scoped Runtime Arguments for Workflow
When a workflow is configured, you may want to pass specific runtime arguments to the different programs and datasets used inside the workflow. To achieve this, you can prefix the runtime arguments with a <scope>. Currently supported scopes are dataset, mapreduce, and spark.
Last updated