Class ComputationConfig

Root configuration object.

This should contain all of the input from all sources including user/pipeline input and the MDL Service.

Inheritance

Object

ComputationConfig

Inherited Members

Object.Equals(Object)

Object.Equals(Object, Object)

Object.GetHashCode()

Object.GetType()

Object.MemberwiseClone()

Object.ReferenceEquals(Object, Object)

Object.ToString()

Namespace: Mangrove.Coordinator.Configuration.Contract

Assembly: Coordinator.Configuration.Contract.dll

Syntax

public class ComputationConfig

Remarks

To add another property to this class, follow these steps:

1. Add a property with the desired name and a helpful docstring.
2. Add a constructor parameter setting that property. Note: make sure you add it before the AuditHistory parameter (that's the only thing which is not a ConfigElement<T>, and will potentially be a bunch of configs).
3. Add to the constructor of ExtendedConfig.
4. Add the property to the AppendOnto method of the MergeableComputationConfig class.
5. Create a unit test verifying that the new field (de)serializes as expected.

Constructors

ComputationConfig(ConfigElement<String>, ConfigElement<String>, ConfigElement<IReadOnlyDictionary<String, IReadOnlyList<String>>>, ConfigElement<IReadOnlyCollection<String>>, ConfigElement<IReadOnlyDictionary<String, DataSourceSetting>>, ConfigElement<IReadOnlyCollection<String>>, ConfigElement<IReadOnlyCollection<SegmentGroup>>, ConfigElement<MetricsPlanInfo>, ConfigElement<IReadOnlyDictionary<String, String>>, ConfigElement<String>, ConfigElement<String>, ConfigElement<String>, ConfigElement<IReadOnlyCollection<String>>, ConfigElement<IReadOnlyCollection<String>>, ConfigElement<String>, ConfigElement<String>, ConfigElement<Boolean>, ConfigElement<IReadOnlyCollection<String>>, ConfigElement<OverrideAggregationLevel>, IReadOnlyList<ConfigElement<ComputationConfig>>)

Standard constructor. For a description of the parameters, see docstrings for their corresponding properties.

Declaration

public ComputationConfig([PropertyMap("StartDateTime")] ConfigElement<string> startDateTime = null, [PropertyMap("EndDateTime")] ConfigElement<string> endDateTime = null, [PropertyMap("Flights")] ConfigElement<IReadOnlyDictionary<string, IReadOnlyList<string>>> flights = null, [PropertyMap("Filters")] ConfigElement<IReadOnlyCollection<string>> filters = null, [PropertyMap("DataSourceOverrides")] ConfigElement<IReadOnlyDictionary<string, DataSourceSetting>> dataSourceOverrides = null, [PropertyMap("RequiredMetrics")] ConfigElement<IReadOnlyCollection<string>> requiredMetrics = null, [PropertyMap("SegmentGroups")] ConfigElement<IReadOnlyCollection<SegmentGroup>> segmentGroups = null, [PropertyMap("MetricsPlanInfo")] ConfigElement<MetricsPlanInfo> metricsPlanInfo = null, [PropertyMap("TokenReplacements")] ConfigElement<IReadOnlyDictionary<string, string>> tokenReplacements = null, [PropertyMap("OutputPath")] ConfigElement<string> outputPath = null, [PropertyMap("TimestampColumn")] ConfigElement<string> timestampColumn = null, [PropertyMap("FlightColumn")] ConfigElement<string> flightColumn = null, [PropertyMap("Profiles")] ConfigElement<IReadOnlyCollection<string>> profiles = null, [PropertyMap("MetricCategories")] ConfigElement<IReadOnlyCollection<string>> metricCategories = null, [PropertyMap("RandomizationUnit")] ConfigElement<string> randomizationUnit = null, [PropertyMap("Extern")] ConfigElement<string> extern = null, [PropertyMap("Unpivot")] ConfigElement<bool> unpivot = null, [PropertyMap("Triggers")] ConfigElement<IReadOnlyCollection<string>> triggers = null, [PropertyMap("OverrideAggregationLevel")] ConfigElement<OverrideAggregationLevel> overrideAggregationLevel = null, [PropertyMap("AuditHistory")] IReadOnlyList<ConfigElement<ComputationConfig>> auditHistory = null)

Parameters

Type	Name	Description
ConfigElement<String>	startDateTime
ConfigElement<String>	endDateTime
ConfigElement<IReadOnlyDictionary<String, IReadOnlyList<String>>>	flights
ConfigElement<IReadOnlyCollection<String>>	filters
ConfigElement<IReadOnlyDictionary<String, DataSourceSetting>>	dataSourceOverrides
ConfigElement<IReadOnlyCollection<String>>	requiredMetrics
ConfigElement<IReadOnlyCollection<SegmentGroup>>	segmentGroups
ConfigElement<MetricsPlanInfo>	metricsPlanInfo
ConfigElement<IReadOnlyDictionary<String, String>>	tokenReplacements
ConfigElement<String>	outputPath
ConfigElement<String>	timestampColumn
ConfigElement<String>	flightColumn
ConfigElement<IReadOnlyCollection<String>>	profiles
ConfigElement<IReadOnlyCollection<String>>	metricCategories
ConfigElement<String>	randomizationUnit
ConfigElement<String>	extern
ConfigElement<Boolean>	unpivot
ConfigElement<IReadOnlyCollection<String>>	triggers
ConfigElement<OverrideAggregationLevel>	overrideAggregationLevel
IReadOnlyList<ConfigElement<ComputationConfig>>	auditHistory

Properties

AuditHistory

Sequence of all previous "states" of this config element.

Declaration

public IReadOnlyList<ConfigElement<ComputationConfig>> AuditHistory { get; }

Property Value

Type	Description
IReadOnlyList<ConfigElement<ComputationConfig>>

DataSourceOverrides

Overrides for DataSourceSetting values keyed on data source name.

Declaration

public ConfigElement<IReadOnlyDictionary<string, DataSourceSetting>> DataSourceOverrides { get; }

Property Value

Type	Description
ConfigElement<IReadOnlyDictionary<String, DataSourceSetting>>

EndDateTime

ISO 8601 string representation of the end time for a particular computation. This is inclusive / exclusive depending on how the data source treats start and end times.

Declaration

public ConfigElement<string> EndDateTime { get; }

Property Value

Type	Description
ConfigElement<String>

Extern

String representation of the "extern type" the generated script should target. This should match one of the values of Mangrove.MetricSetSyntaxTree.ExpressionNodes.ExternKind.

Declaration

public ConfigElement<string> Extern { get; }

Property Value

Type	Description
ConfigElement<String>

Filters

Any additional filters to apply after reading from the data source.

Declaration

public ConfigElement<IReadOnlyCollection<string>> Filters { get; }

Property Value

Type	Description
ConfigElement<IReadOnlyCollection<String>>

Remarks

For now these should be names of expressions (ColumnReference objects) which exist in the MetricsPlan.

TODO: once we have a parser / compiler, allow these to be expressions for custom filter conditions.

FlightColumn

The column in datasource which stores the unprocessed flight data.

Declaration

public ConfigElement<string> FlightColumn { get; }

Property Value

Type	Description
ConfigElement<String>

Flights

A mapping of FlightName to the list of flight specification parameters, e.g. FlightAllocationID, or a Numberline.

Declaration

public ConfigElement<IReadOnlyDictionary<string, IReadOnlyList<string>>> Flights { get; }

Property Value

Type	Description
ConfigElement<IReadOnlyDictionary<String, IReadOnlyList<String>>>

MetricCategories

List of names of metricCategories(s) from the "host" MetricsPlan to use in computation.

Declaration

public ConfigElement<IReadOnlyCollection<string>> MetricCategories { get; }

Property Value

Type	Description
ConfigElement<IReadOnlyCollection<String>>

Remarks

It may add more during "metricCategories dependency resolution" internally.

MetricsPlanInfo

The metadata needed to obtain a local or "production" copy of a MetricsPlan file.

Declaration

public ConfigElement<MetricsPlanInfo> MetricsPlanInfo { get; }

Property Value

Type	Description
ConfigElement<MetricsPlanInfo>

OutputPath

Path of the output of computation. The "type" of output is inferred from the path.

Declaration

public ConfigElement<string> OutputPath { get; }

Property Value

Type	Description
ConfigElement<String>

Remarks

For example, if the path is /foo/bar/baz.ss and the compute fabric is Cosmos, the output will be a structured stream. But if the path is /foo/bar/baz.parquet, the output will be a Parquet file.

OverrideAggregationLevel

String containing the trigger condition

Declaration

public ConfigElement<OverrideAggregationLevel> OverrideAggregationLevel { get; }

Property Value

Type	Description
ConfigElement<OverrideAggregationLevel>

Profiles

List of names of profile(s) from the "host" MetricsPlan to use in computation.

Declaration

public ConfigElement<IReadOnlyCollection<string>> Profiles { get; }

Property Value

Type	Description
ConfigElement<IReadOnlyCollection<String>>

Remarks

There should only ever be one profile when creating a config to pass to Blitz. It may add more during "profile dependency resolution" internally.

RandomizationUnit

String of the form TableName.ExpressionName, representing the randomization unit for an experiment. If this is null, no variance will be computed.

Declaration

public ConfigElement<string> RandomizationUnit { get; }

Property Value

Type	Description
ConfigElement<String>

Remarks

The user is not expected to set this in the initial config. The compiler will inject this information into a MetricSetConfiguration payload.

RequiredMetrics

Metrics that must appear in the output.

Declaration

public ConfigElement<IReadOnlyCollection<string>> RequiredMetrics { get; }

Property Value

Type	Description
ConfigElement<IReadOnlyCollection<String>>

SegmentGroups

Specification of all segmentation to carry out in a computation.

Declaration

public ConfigElement<IReadOnlyCollection<SegmentGroup>> SegmentGroups { get; }

Property Value

Type	Description
ConfigElement<IReadOnlyCollection<SegmentGroup>>

StartDateTime

ISO 8601 string representation of the start time for a particular computation. This is inclusive / exclusive depending on how the data source treats start and end times.

Declaration

public ConfigElement<string> StartDateTime { get; }

Property Value

Type	Description
ConfigElement<String>

TimestampColumn

The column in the input data source which is the timestamp for the record.

Declaration

public ConfigElement<string> TimestampColumn { get; }

Property Value

Type	Description
ConfigElement<String>

TokenReplacements

Token overrides to be replaced in all tables and expressions names, parameters, string literals.

Declaration

public ConfigElement<IReadOnlyDictionary<string, string>> TokenReplacements { get; }

Property Value

Type	Description
ConfigElement<IReadOnlyDictionary<String, String>>

Remarks

Key example is providing "wildcard" replacements for the values that are only known at run time. For example, in a metric set one can use #DATE# placeholder when defining metrics, which then needs to be replaced with an actual date value supplied in the config.

Triggers

String containing the trigger condition

Declaration

public ConfigElement<IReadOnlyCollection<string>> Triggers { get; }

Property Value

Type	Description
ConfigElement<IReadOnlyCollection<String>>

Unpivot

Whether or not to "unpivot" (melt) the results at the end of computation. See RFC #1 for more details on this.

Declaration

public ConfigElement<bool> Unpivot { get; }

Property Value

Type	Description
ConfigElement<Boolean>

Methods

OverwriteHistory(IReadOnlyList<ConfigElement<ComputationConfig>>)

Overwrite history by creating a new ComputationConfig

Declaration

public ComputationConfig OverwriteHistory(IReadOnlyList<ConfigElement<ComputationConfig>> auditHistory)

Parameters

Type	Name	Description
IReadOnlyList<ConfigElement<ComputationConfig>>	auditHistory

Returns

Type	Description
ComputationConfig

OverwriteRequiredMetrics(ConfigElement<IReadOnlyCollection<String>>)

Overwrite requiredMetrics by creating a new ComputationConfig

Declaration

public ComputationConfig OverwriteRequiredMetrics(ConfigElement<IReadOnlyCollection<string>> requiredMetrics)

Parameters

Type	Name	Description
ConfigElement<IReadOnlyCollection<String>>	requiredMetrics

Returns

Type	Description
ComputationConfig

Extension Methods

ObjectExtensions.EventId(Object)