Blitz

You may reach this page via: aka.ms/mangrove/blitz.

Installation

Blitz is a command-line executable to interact with serialized MetricsPlan objects. You can install it by downloading and extracting the latest drop from Mangrove CI. To find the drop, click on the latest build, then in the "Rolling build triggered" tab click "published" under "Related", and download the drop from there. The Blitz.exe file is located in drop\ClosedSource\Blitz\bin\Release\net472\Blitz.exe but you need to download the whole drop. Make sure to add the directory containing Blitz.exe to your system path.

Use

This section is a very simple overview of the "flow" of using Blitz for script generation. It assumes you have already obtaibed Blitz.exe following the instructions above.

  1. To start using Blitz,

    a. Make sure you have need the Azure CLI installed.

    b. Obtain the value of the secret key akv-mangrove-testing from the Azure Key Vault mangrove-kv

    c. To authenticate, run the following command in your PowerShell

    az login --service-principal --username '4000d40b-4231-42a4-9583-356a4e502236' --password '<secret you obtained above>' --tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' --allow-no-subscriptions

    Alternatively, you can use the secret akv-mangrove-msv3 directly each time you run Blitz, see the instructions below.

  2. Write a computation configuration for Blitz. Here is an example configuration file:

# This is used in MasterCoordinator/Scope_CodeGeneration
#   and in Blitz_Tests/IntegrationTestAsync
# This is also the only example of an AllUp profile being tested.
# Reasonably complete Blitz configuration file with examples of the kind of
# information you might want to specify.
StartDateTime: 4/3/2019 2:00:00 AM
EndDateTime: 4/4/2019 12:00:00 AM
# Map flight name -> specification list, namely
# [numberline, flight name, allocation ID]. Some of them can be empty.
Flights:
  Oulie3121cf: ['', Oulie3121cf, 30009499]
  Oulie3121: ['', Oulie3121, 30009498]
# Blitz doesn't demand that the "flight" segment be called FLIGHT, but the
# legacy MDL compiler does require that.
SegmentGroups:
- Segments:
  - FLIGHT
- Level: 1
  AggregateAll: true
  Segments:
  - MKT
  - DeviceManufacturer
MetricsPlanInfo:
  Name: OutlookMobile
  # Fix version because this change to MDL Core (as expected) breaks MetricsPlan generation
  # due to OutlookMobile using placeholders that will not be supported:
  # https://experimentation.visualstudio.com/DefaultCollection/Analysis%20and%20Experimentation/_git/AnE.ExP.Mdl/pullrequest/135511
  Version: 11637738875117101448
# Output path of the result of the computation described by the generated
# script. Not to be confused to the path of the generated script itself.
OutputPath: /local/projects/users/lsrep/Foray/2019/04/03/AnaExp_Others_Job0_Run0__lsrep_AutoJob_20190403_170256_005n649_OutlookMobile.OutlookAndroidShort.AB.Standard_2019-04-02T1900_2019-04-03T1700_BlitzConfig.txt
# If this is not specified, Blitz will attempt to compute ALL metrics.
Profiles:
- OutlookMobile.OutlookAndroidShort.AB.Standard
# This tells Blitz to compute variance for all metrics.
RandomizationUnit: true
# We currently support Scope, Kusto, Hypercube, and SparkSql.
Extern: Scope
# Transform data from (short, wide) to (long, narrow) to prepare for loading
# into xCard. This is only supported for Scope.
Unpivot: true

For more examples look in the integration tests.

  1. To generate a script, run:
Blitz.exe emit --config '<path to your config file>' --output '<where to put the generated script>'

If you did not go through the authentication process described above, you can submit the secret directly by running

Blitz.exe emit --config '<path to your config file>' --output '<where to put the generated script>' --secret '<secret you obtained from the key vault>'

Containerized Blitz and jar generator

A&E publishes a Docker container containing Blitz and jar generator to the Azure Container Registry named aneexp. You can download and run this image following the code snippets below, which follow Push your first image to a private Docker container registry using the Docker CLI, and need the Azure CLI installed.

# Log in into Azure (this will cause a browser window to open). You can skip
# after running the command once.
az login

# Set subscription to the specific account this registry lives in.
az account set -s 80d2c6c6-fa64-4ab1-8aa5-4e118c6b16ce

# Log the Docker CLI in to that registry.
az acr login --name aneexp

# Pull the latest build of the Blitz image.
docker pull aneexp.azurecr.io/mangrove/blitz

# Ask Blitz what the possible commands are.
docker run aneexp.azurecr.io/mangrove/blitz help

# Actually run Blitz to generate a script.
# The last 3 arguments are only for Hypercube, and they are ignored for other pipelines.
# <maven username> and <maven password> are used to access private maven repository 'https://experimentation.pkgs.visualstudio.com/_packaging/Public/maven/v1'
docker run --volume "$($PWD):/foo" --workdir /foo aneexp.azurecr.io/mangrove/blitz /src/blitz.sh '<Linux-friendly path to config>' '<output dir>' '<jar output dir>' '<maven username>' '<maven password>'