Digging deeper again - querying Application Insights data

In the one of the previous post I presented you the basics of using Analytics in Application Insights. Just to remind you - it's a tool, which allows you to get much more detailed information regarding how your application perform and behaves. In this post I'll try to encourage you to use this tool on daily basis, especially when something is clearly wrong and you don't want to click through all those tabs in Azure Portal.

Quickly finding an exception

Let's consider following example - you're informed by your client, that he or she got an error saying Object reference not set to an instance of an object. You know the time and assume, that we're facing NullReferenceException here. What could you do to quickly find more details regarding this exception? Let's consider following query:

/
exceptions
    | where timestamp >= ago(24h) 
        and type == "System.NullReferenceException" 
        and outerAssembly 
            contains "MyProject" 
    | project method, details, timestamp 

This query will project a simple result containing a method in which an exception was thrown, details(like inner stack) and the exact time when it occurred. You can modify it and use different timestamp to get results, which will satisfy you the most.

Finding pages which are loading too slow

This is another example - you have a SPA and you'd like to know, which pages are loading longer, that a fixed threshold. Fortunately we can use a really simple query, which can help us here:

/
pageViews
    | where duration > 500

Yes, this is really it. Of course you can extend this query and use other metrics to further filter the data like:

/
pageViews
    | where duration > 50
        and operation_Name == "some_operation"
        and timestamp > ago(1h)  

Or even visualize your data to be aware which slower requests are really the slowest ones:

/
pageViews
    | where duration >= 250
    | render piechart  

Custom metrics

You can use predefined metrics to gather some information about your application, but where this tool really shines is the possibility to query custom metric. This allows you to easily build your own reports and really focus on what you're interested in.

For example I'd like to know how many times given method is called so I can be aware of all potential hot-paths and focus my optimization efforts on it. To do this I need two things:

  • a tool, which will decorate methods calls with a custom metrics logging(AOP/assembly weaving will help here)
  • custom query in Application Insights Analytics

Let's say, that this decorator will log one thing - a method name. Now in Analytics we could create a query, which would use this custom metric to show us which method is called the most.

Limiting data being logged using Application Insights in Azure Functions

As you may know, Azure Functions have a preview of Application Insights integration enabled. This is another great addition to our serverless architecture since we don't have to add this dependency on our own - it's just there. However, there're some problems when it comes to handling the amount of data, which is being collected, especially when your're on an MSDN subscription.

Problem

When you enable Application Insights for your Function App, each function will start collecting different metrics(traces, errors, requests) at different scale. When you go to Azure Portal and access Data volume management tab in the Application Insights blade, you'll see, that there's one metric, which really exceeds our expectations(at least when it comes to the volume of the data traced):

As you can see, Message data takes 75% of the total amount of data collected

When you click on any bar, you'll access Data point volume tab - now we can understand, what kind of 'message' is really being logged:

Although chart says Message, data type related to this particular type of message is Trace

Configuring AI integration

Logging traces is perfectly fine, however we don't always want to do so(especially if you're on an MSDN subscription and don't want to be blocked). If you go to this page, you'll see a detailed information regarding both enabling and working with Application Insights. The most interesting part for us is the configuration section:

/
{
  "logger": {
    "categoryFilter": {
      "defaultLevel": "Information",
      "categoryLevels": {
        "Host.Results": "Error",
        "Function": "Error",
        "Host.Aggregator": "Information"
      }
    },
    "aggregator": {
      "batchSize": 1000,
      "flushTimeout": "00:00:30"
    }
  },
  "applicationInsights": {
    "sampling": {
      "isEnabled": true,
      "maxTelemetryItemsPerSecond" : 5
    }
  }
}

As you can see, we're able to set different levels for each category of data being logged. According to comments in this issue on GitHub, the easiest way to actually limit the data being logged is to set your configuration to the following:

/
{
  "logger": {
    "categoryFilter": {
      "defaultLevel": "Error",
      "categoryLevels": {
        "Host.Aggregator": "Information"
      }
    }
}

This way you should be able to avoid logging to much data or hitting your daily cap for Application Insights.

I strongly recommend you to play with AI integration in Azure Functions and provide feedback regarding possible features or enhancements. It's a great way to collaborate with a team responsible for a product and a chance to make it even better.