Automate The World

One of the important aspects of Durable Functions is the fact, that you're able to access each orchestration history, inpect and analyze it when you need to do so. This is a great addition to the whole framework since you're able to both:

• leverage current features like Application Insights integration
• write a custom solution, which will extend current capabilities and easily integrate with your systems

However what are the options, which we're given? Let's take a look!

Application Insights

There's an extensive guide available here, which explains in details how one can access and use Application Insights integration and query the data. There's one important information listed there:

By default, Application Insights telemetry is sampled by the Azure Functions runtime to avoid emitting data too frequently. 
This can cause tracking information to be lost when many lifecycle events occur in a short period of time. 

So it's important to be aware of this flaw and always design your solution so it's don't rely on possibly lost information.

Another important thing is the configuration:

/
{
    "logger": {
        "categoryFilter": {
            "categoryLevels": {
                "Host.Triggers.DurableTask": "Information"
            }
        }
    }
}

Sometimes it's imporant to adjust logging levels since in Functions there's many events that are being constantly logged - if you're using Azure subscription with limits(like MSDN subscriptions) you can easily run out of free quota, what will prevent your functions from logging.

Storage

This is nothing surprising that Durable Functions also use Storage Account to store data. You can get the same results as accessing AI by going to the Storage Account attached to the Function App instance and accessing history table. Its name is listed in host.json:

/
{
  "durableTask": {
    "HubName": "SampleHubVS"
  }
}

So when you access your Storage Account, it should be there:

In the previous post I was working on very basic concepts of Durable Functions. Since they are connected to a simple in-cloud game engine, I'll go a bit further and show you how sub-orchestrations help in shaping a solution, so all concepts are decoupled and isolated.

Why sub-orchestration?

While it's perfectly fine to build your orchestrations using multiple activities called one-by-one(or parallelized - it's still a valid solution), sometimes you'd like to isolate different concepts emerging from a one project. Let's consider our example - we'd like to create a galaxy with N planets inside it. There're two approaches possible:

• perform all operations inside one orchestration so each action is an activity
• decouple those actions so one orchestration could call another(and we can call them separately)

Let's consider following example:

/
[FunctionName("ProvisionNewDevices")]
public static async Task ProvisionNewDevices(
    [OrchestrationTrigger] DurableOrchestrationContext ctx)
{
    string[] deviceIds = await ctx.CallActivityAsync<string[]>("GetNewDeviceIds");

    // Run multiple device provisioning flows in parallel
    var provisioningTasks = new List<Task>();
    foreach (string deviceId in deviceIds)
    {
        Task provisionTask = ctx.CallSubOrchestratorAsync("DeviceProvisioningOrchestration", deviceId);
        provisioningTasks.Add(provisionTask);
    }

    await Task.WhenAll(provisioningTasks);

    // ...
}

Here you can see one of the biggest advantages of such approach - you can run multiple flows simultaneously and just await until each is finished. With several activities it's still doable, however I'd rather consider it an antipattern.

Making a working solution

My current orchestration looks like this:

/
[FunctionName("Galaxy_Create_Start")]
public static async Task<HttpResponseMessage> StartOrchestration(
	[HttpTrigger(AuthorizationLevel.Function, "post", Route = "orchestration/start")] HttpRequestMessage req,
	[OrchestrationClient] DurableOrchestrationClient starter,
	TraceWriter log)
{
	// Function input comes from the request content.
	var instanceId = await starter.StartNewAsync("Galaxy_Create", null);

	var payload = await req.Content.ReadAsStringAsync();
	log.Info($"Started orchestration with ID = '{instanceId}'.");
	log.Info($"The payload is: {payload}");

	return starter.CreateCheckStatusResponse(req, instanceId);
}

[FunctionName("Galaxy_Create")]
public static async Task<string> RunImpl([OrchestrationTrigger] DurableOrchestrationContext context)
{
	var result = await Task.WhenAll(context.CallActivityAsync<string>("Utility_Coords"),
		context.CallActivityAsync<string>("Utility_Galaxy_Name"));
	var galaxyContext = new CreateGalaxyContext(result[1], result[0]);

	await context.CallActivityAsync("Galaxy_Create_Impl", galaxyContext);
	await context.CallSubOrchestratorAsync("Planet_Create", galaxyContext);

	return "Galaxy created!";
}

[FunctionName("Galaxy_Create_Impl")]
public static async Task CreateGalaxy(
	[ActivityTrigger] CreateGalaxyContext context,
	[Table("galaxies")] IAsyncCollector<GalaxyDataEntity> galaxies)
{
		await galaxies.AddAsync(new GalaxyDataEntity(context.Name, context.Coords));                
}

As you can see, there's a special call, which schedules another orchestration within this one:

/
await context.CallSubOrchestratorAsync("Planet_Create", galaxyContext);

Let's look at this new orchestration:

/
[FunctionName("Planet_Create")]
public static async Task<string> RunImpl([OrchestrationTrigger] DurableOrchestrationContext context)
{
	var activities = new List<Task>();
	var number = await context.CallActivityAsync<int>("Utility_Number");
	var galaxyContext = JsonConvert.DeserializeObject<JArray>(context.GetInputAsJson().ToString()).First;

	for (var i = 0; i < number; i++)
	{
		var result = await Task.WhenAll(context.CallActivityAsync<string>("Utility_Coords"),
			context.CallActivityAsync<string>("Utility_Planet_Name"),
			context.CallActivityAsync<string>("Utility_Planet_Type"));

		var planetContext = new CreatePlanetContext(galaxyContext.ToObject<CreateGalaxyContext>(), result[0],
			result[1], (PlanetType) Enum.Parse(typeof(PlanetType), result[2]));
		activities.Add(context.CallActivityAsync<int>("Planet_Create_Impl", planetContext));
	}

	await Task.WhenAll(activities);

	return "Planet created!";
}

[FunctionName("Planet_Create_Impl")]
public static async Task CreatePlanet(
	[ActivityTrigger] CreatePlanetContext context,
	[Table("planet")] IAsyncCollector<PlanetDataEntity> planets)
{
	await planets.AddAsync(new PlanetDataEntity(context.GalaxyContext.Name, context.Name, context.Coords, context.Type));
}

The great thing is that we can pass a full context to the sub-orchestration, so it can use data, which was obtained by the previous activities.

Summary

Sub-orchestrations are a great addition to the Durable Functions SDK, especially that they're such a simple concept. I strongly encourage you to try it all by yourself, so you can feel how powerful the concept is.