Is Event Grid faster than Azure Functions? #2

In the previous post I presented you the result of a basic smoke test using Blob Trigger in Azure Functions and the same functionality in Event Grid. The outcome was not surprising - Event Grid seems to be faster and more reliable way of notifying other services about new blobs. What if we perform a stress test? Is anything going to change? Let's check this!

Publisher

For the current episode I used following producer:

/
private static void Main()
{

	MainAsync().GetAwaiter().GetResult();
}

private static async Task MainAsync()
{
	while (true)
	{
		var storageAccount = CloudStorageAccount.Parse("");
		var blobClient = storageAccount.CreateCloudBlobClient();
		var container = blobClient.GetContainerReference("functionsgrid");
		container.CreateIfNotExists();
		var blockBlob = container.GetBlockBlobReference(Guid.NewGuid().ToString());

		blockBlob.UploadText(JsonConvert.SerializeObject(new Blob()));
		Console.WriteLine($"[{DateTime.Now}] Blob uploaded!");

		await Task.Delay(10);
	}
}

public class Blob
{
	public Blob()
	{
		Id = Guid.NewGuid();
		Created = DateTime.Now;
	}

	public Guid Id { get; set; }

	public DateTime Created { get; set; }

	public string Text { get; set; }
}

What is more I run 5 producers at the very same time. 

Results

Here are the results:

How to interpret this chart? On the y-axis we have total execution time(in miliseconds). You can clearly see the difference between Functions and Event Grid(in fact the maximum execution time for Function was greater than 30 minutes!). What is more, the median for Function lies between 50 and 100 seconds.

Conclusions

It seems that there's a clear improvement when it comes to processing times when switching from Azure Functions to Event Grid. You may ask why one would like to use Azure Functions when the difference in processing time is so obvious? As always - you have to ask yourself what are your requirements and what are the current recommendations. The rule of a thumb would be, that if I need predictable delivery time, I'd go for Event Grid. If I don't mind whether a function is called after 1 second or 1 minute, Azure Function is still a viable option.

Appending data to Azure Storage Blob concurrently

Append Blob is a fairly common feature of Azure Storage, which makes all kinds of logs or data aggregations a piece of cake. While the whole concept is super-easy(at least from the SDK client point of view), using it in a real scenario could give you headaches. Why you may ask? Well, the nature of appends is not so obvious and at the first glance our perceptions could be deceived.

A quick look at the documentation reveals how limited are our options to append data to a blob concurrently

Four Horsemen of the Parallelypse

There're 5 methods in total available on CloudAppendBlob type:

  • AppendBlob
  • AppendFromStream
  • AppendText
  • AppendFromByteArray
  • AppendFromFile

As you can see, I grouped them a little so we have two categories:

  • methods for multiple writers scenario
  • methods for a single writer scenario

Now the question is - how do we know, that one method is designed for this specific scenario? Well, the easiest option is to read the documentation. This is a description of AppendText taken from the API reference:

Appends a string of text to an append blob.
This API should be used strictly in a single writer scenario because the API internally uses the append-offset conditional header to avoid duplicate blocks which does not work in a multiple writer scenario.

So what happens if you try to use such method in a e.g. Azure Function?

The remote server returned an error: (412) The append position condition specified was not met 

This is not the best description, isn't it?

Gimme a code snippet!

The easiest way to fix this issue is to transform a string to a stream:

/
using (var ms = new MemoryStream())
using(var sw = new StreamWriter(ms))
{
	await sw.WriteLineAsync("Serialized_data");
	await sw.FlushAsync();

	ms.Position = 0;

	await blob.AppendBlockAsync(ms);
}

Summary

Personally I found this issue is not so obvious - I involuntarily used AppendText method, which looked as the best match to my code and after some time I noticed those 412 error codes. The one thing you have to remember when using AppendBlock is the fact, that each block cannot exceed 4 MB size each. This - along with the limit of 50k append operations - allows for building a blob of max size equal to 195 GBs, which should be fine for the most of projects.