Azure Functions offers another hosting model called Azure Functions Premium Plan. With premium plans, instead of paying per function execution, you pay for the underlying compute instances that are hosting your functions. This is often more expensive, but it also ensures there are always a pre-set number of warmed instances ready to execute your function.

That's great, but what if I only really need those pre-warmed instances for a short period of time when I'm expecting a lot of incoming traffic. The rest of the time, I would rather use a Consumption Plan to save on hosting costs.

I thought the choice of hosting plan was something you needed to make up front but it turns out that you can actually move an Azure Function App from a consumption plan to a premium plan (and back again).

Thanks to Simon Timms for starting this discussion on Twitter. We got very helpful responses from folks on the Azure Functions team:

Jeff Hollan has a great sample using an Azure Durable Function to scale an Azure Function App to a premium plan for a specified amount of time, then automatically scale back down to a consumption plan.

https://t.co/6C9l3PQDoZ

— Jeff Hollan (@jeffhollan) April 2, 2020

This is a super cool sample. It uses the Azure Resource Manager REST API to make changes to the target function app resources. For my project however, I didn't really want to spin up another Azure Function to manage my Azure Functions. I just wanted an easy way to scale my 12 function apps up to premium plans for a couple hours, then scale them back down to a consumption plan.

I decided to try using the AZ CLI for this and it turned out really well. I was able to write a simple script to scale up and down.

Setting up the AZ CLI

First up, install the az cli.

Once installed, you'll need to login to your Azure Subscription.

1

az login

A browser window will popup, prompting you to log in to your Azure account. Once you've logged in, the browser window will close and the az cli will display a list of subscriptions available in your account. If you have more than one subscription, make sure you select the one you want to use.

1

az account set --subscription YourSubscriptionId

Create a Resource Group

You will need a resource group for your Storage and CDN resources. If you don't already have one, create it here.

1

az group create --name DavesFunctionApps --location WestUS2

Most commands will require you to pass in a --resource-group and --location parameters. These parameters are -g and -l for short, but you can save yourself even more keystrokes by setting defaults for az.

1
2

az configure -d group=DavesFunctionApps
az configure -d location=WestUS2

Creating a (temporary) Premium Hosting Plan

There is a strange requirement with Azure Functions / App Service. As per Jeff Hollan's sample:

The Azure Functions Premium plan is only available in a sub-set of infrastructure in each region. Internally we call these "webspaces" or "stamps." You will only be able to move your function between plans if the webspace supports both consumption and premium. To make sure your consumption and premium functions land in an enabled webspace you should create a premium plan in a new resource group. Then create a consumption plan in the same resource group. You can then remove the premium plan. This will ensure the consumption function is in a premium-enabled webspace.

First, add an Azure Functions Premium plan to the resource group.

1

az functionapp plan create -n dave_temp_premium_plan --sku EP1 --min-instances 1

You can delete this premium plan using the command below after you've deployed a function app to this resource group . Don't forget to delete the premium plan. These cost $$$

1

az functionapp plan delete -n dave_temp_premium_plan

Creating a Function App

There are many options for creating a new function app. I really like the func command line tool which I installed using npm. Check out the Azure Functions Core Tools GitHub Repo for details on other options for installing the func tooling.

1

npm i -g azure-functions-core-tools@3 --unsafe-perm true

The focus of this blog post is around scaling a function app. If you don't already have an app built, you can follow along with this walkthrough to create a function app.

A function app requires a Storage Account resource. An Application Insights resource is also highly recommended as this really simplifies monitoring your function app after it has been deployed. Let's go ahead and create those 2 resources.

1
2
3
4

az storage account create -n davefuncappstorage
az extension add -n application-insights
az monitor app-insights component create --app davefuncappinsights

Now we can create our Azure Function App resource with a consumption plan, passing in the name of the storage account and app insights resources that we just created. In my case, I'm specifying the dotnet runtime on a Windows host.

1

az functionapp create --consumption-plan-location WestUS2 --name davefuncapp123 --os-type Windows --runtime dotnet --storage-account davefuncappstorage --app-insights davefuncappinsights --functions-version 3

Remember to delete that temporary Premium Hosting Plan now!

1

az functionapp plan delete -n dave_temp_premium_plan

Deploying your Function App using the az cli

This is a bit outside the scope of this blog post but I like using the az cli to deploy my function apps because it's easy to incorporate that into my CI/CD pipelines. Since my app is using the dotnet runtime, I use the dotnet publish command to build the app.

1

dotnet publish -c release

Then, zip the contents of the publish folder (bin\release\netcoreapp3.1\publish\).

In PowerShell:

1

Compress-Archive -Path .\bin\release\netcoreapp3.1\publish\* -DestinationPath .\bin\release\netcoreapp3.1\package.zip

or in Bash

1

zip -r ./bin/release/netcoreapp3.1/package.zip ./bin/release/netcoreapp3.1/publish/

Finally, use the az functionapp deployment command to deploy the function app.

1

az functionapp deployment source config-zip  -n davefuncapp123 --src ./bin/release/netcoreapp3.1/package.zip

Scale up to a premium plan

Okay, now that we have a functioning (pun intended) app deployed and running on a consumption plan, let's see what it takes to scale this thing up to a premium plan.

First, create a new Premium Hosting Plan with the parameters that make sense for the load you are expecting. The --sku parameter refers to the size of the compute instance: EP1 is the smallest. The --min-instancs parameter is the number of pre-warmed instances that will always be running for this hosting plan. The --max-burst parameter is the upper bounds on the number of instances that the premium plan can elastically scale out if more instances are needed to handle load.

1

az functionapp plan create -n davefuncapp123_premium_plan --sku EP1 --min-instances 4 --max-burst 12

Next, move the function app to that premium hosting plan.

1

az functionapp update --plan davefuncapp123_premium_plan -n davefuncapp123

That's it! All it took was those 2 command and your function app is now running on a premium plan!

Scale back down to a consumption plan

Of course, that premium plan isn't cheap. You might only want your function app running on the premium plan for a short period of time. Scaling back down is equally easy.

First, move the function app back to the consumption based plan. In my case, the name of the consumption plan is WestUS2Plan. You should see a consumption plan in your resource group.

1

az functionapp update --plan WestUS2Plan -n davefuncapp123

Next, delete the premium hosting plan.

1

az functionapp plan delete -n davefuncapp123_premium_plan 

Wrapping it up

In this post, we saw how easy it is to move a function app between Premium and Consumption plans. A couple very simple az commands can help you get the performance and features of the Premium plan only when you need it while taking advantages of the simplicity and cost savings of a Consumption plan the rest of the time.

Setting up the AZ CLI

First up, install the az cli.

Once installed, you'll need to login to your Azure Subscription.

1

az login

A browser window will popup, prompting you to log in to your Azure account. Once you've logged in, the browser window will close and the az cli will display a list of subscriptions available in your account. If you have more than one subscription, make sure you select the one you want to use.

1

az account set --subscription YourSubscriptionId

Create a Resource Group

You will need a resource group for your Storage and CDN resources. If you don't already have one, create it here.

1

az group create --name DavesFancyApp --location SouthCentralUs

Most commands will require you to pass in a --resource-group and --location parameters. These parameters are -g and -l for short, but you can save yourself even more keystrokes by setting defaults for az.

1
2

az configure -d group=DavesFancyApp
az configure -d location=SouthCentralUs

Create a Storage Account for Static Hosting

First, create a storage account:

1

az storage account create --name davefancyapp123

Then, enable static site hosting for this account.

1

az storage blob service-properties update --account-name davefancyapp123 --static-website --404-document 404.html --index-document index.html

Your storage account will now have a blob container named $web. That contents of that container will be available on the URL accountname.z21.web.core.windows.net/. For example, https://davefancyapp123.z21.web.core.windows.net/.

Deploying your app

To deploy your app to the site, all you need to do is copy your app's static files to the $web container in the storage account you created above. For my react app, that means running npm run build and copying the build output to the $web container.

1

az storage blob upload-batch --account-name davefancyapp123 -s ./build -d '$web'

Now your site should be available via the static hosting URL above. That was easy!

Create a CDN Profile and Endpoint

Next up, we are going to put a Content Delivery Network (CDN) endpoint in front of the blob storage account. We want to use a CDN for a couple reasons. First, it's going to provide much better performance overall. CDNs are optimized for delivering web content to user's devices and we should take advantage of that as much as possible. The second reason is that a CDN will allow us to configure SSL on a custom domain name.

First, we will need to create a CDN Profile. There are a few different of CDNs offerings available in Azure. You can read about them here. In this example, we will us the Standard Microsoft CDN.

1

az cdn profile create -n davefancyapp123cdn --sku Standard_Microsoft

Next, we will create the CDN endpoint. Here we need to set the origin to the static hosting URL from the previous step. Note that we don't include the protocol portion of the URL.

1

az cdn endpoint create -n davefancyapp123cdnendpoint --profile-name davefancyapp123cdn --origin davefancyapp123.z21.web.core.windows.net --origin-host-header davefancyapp123.z21.web.core.windows.net --enable-compression

Note: See the az cli docs for more information on the options available when creating a CDN endpoint.

Now your site should be available from endpointname.azureedge.net. In my case https://davefancyapp123cdnendpoint.azureedge.net/. Note that the endpoint is created quickly but it can take some time for the actual content to propagate through the CDN. You might initially get a 404 when you visit the URL.

Create CDN Endpoint Rules

These 2 steps are optional. The first one is highly recommended. The second is optional depending on the type of app your deploying.

First, create URL Redirect rule to redirect any HTTP requests to HTTPS.

1

az cdn endpoint rule add -n davefancyapp123cdnendpoint --profile-name davefancyapp123cdn --rule-name enforcehttps --order 1 --action-name "UrlRedirect"  --redirect-type Found --redirect-protocol HTTPS --match-variable RequestScheme --operator Equal --match-value HTTP

Next, if you're deploying a Single Page Application (SPA) built in your favourite JavaScript framework (e.g. Vue, React, Angular), you will want a URL Rewrite rule that returns the app's root index.html file for any request to a path that isn't an actual file. There are many variations on how to write this rule. I found this to be the simplest one that worked for me. Basically if the request path is not for a specific file with a file extension, rewrite to index.html. This allows users to directly navigate to a route in my SPA and still have the CDN serve the index.html that bootstraps the application.

1

az cdn endpoint rule add -n davefancyapp123cdnendpoint --profile-name davefancyapp123cdn --rule-name sparewrite --order 2 --action-name "UrlRewrite" --source-pattern '/' --destination /index.html --preserve-unmatched-path false --match-variable UrlFileExtension --operator LessThan --match-value 1

Configuring a domain with an Azure Managed Certificate

The final step in configuring the CDN Endpoint is to configure a custom domain and enable HTTPS on that custom domain.

You will need access to update DNS records for the custom domain. Add a CNAME record for your subdomain that points to the CDN endpoint URL. For example, I created a CNAME record on my davepaquette.com domain:

1

CNAME    fancyapp   davefancyapp123cdnendpoint.azureedge.net

Once the CNAME record has been created, create a custom domain for your endpoint.

1

az cdn custom-domain create --endpoint-name davefancyapp123cdnendpoint --profile-name davefancyapp123cdn -n fancyapp-domain --hostname fancyapp.davepaquette.com

And finally, enable HTTPs. Unfortunately, this step fails due to a bug in the AZ CLI. There's a fix on it's way for this but it hasn't been merged into the CLI tool yet.

1

az cdn custom-domain enable-https --endpoint-name davefancyapp123cdnendpoint --profile-name davefancyapp123cdn --name fancyapp-domain

Due to the bug, this command returns InvalidResource - The resource format is invalid. For now, you can do this step manually in the Azure Portal. When using CDN Managed Certificates, the process is full automated. Azure will verify your domain using the CNAME record above, provision a certificate and configure the CDN endpoint to use that certificate. Certificates are fully managed by Azure. That includes generating new certificates so you don't need to worry about your certificate expiring.

CDN Managed Certificates for Root Domain

My biggest frustration with Azure CDN Endpoints is that CDN managed certificates are not supported for the apex/root domain. You can still use HTTPS but you need to bring your own certificate.

The same limitation exists for managed certificates on App Service. If you share my frustration, please upvote here.

Deploying updates to your application

The CDN will cache your files. That's great for performance but can be a royal pain when trying to deploy updates to your application. For SPA apps, I have found that simply telling the CDN to purge index.html is enough to ensure updates are available very shortly after deploying a new version. This works because most JavaScript frameworks today use WebPack which does a good job of cache-busting your JavaScript and CSS assets. You just need to make sure the browser is able to get the latest version of index.html and updates flow through nicely.

When you upload your latest files to blob storage, follow it with a purge command for index.html on the CDN endpoint.

1
2

az storage blob upload-batch --account-name davefancyapp123 -s ./build -d '$web'
az cdn endpoint purge -n davefancyapp123cdnendpoint --profile-name davefancyapp123cdn --no-wait --content-paths '/' '/index.html'

The purge command can take a while to complete. We pass the --no-wait option so the command returns immediately.

My thoughts on az

Aside from the bug I ran in to with enabling HTTPS on the CDN endpoint, I've really enjoyed my experience with the az cli. I was able to fully automate resource creation and deployments using the GitHub Actions az cli action. I can see az becoming my preferred method of managing Azure resources.

Request Telemetry

For an ASP.NET Core process, the Application Insights SDK will automatically collect data about every request that the server process receives. This specific type of telemetry is called Request telemetry and it contains a ton of very useful data including: the request path, the HTTP verb, the response status code, the duration, the timestamp when the request was received.

The default data is great, but I often find myself wanting more information. For example, in a multi-tenant application, it would be very useful to track the tenant id as part of the request telemetry. This would allow us to filter data more effectively in the Application Insights portal and craft some very useful log analytics queries.

Adding custom data to Request Telemetry

All types of telemetry in Application Insights provide an option to store custom properties. In the previous post, we saw how to create an ITelemetryInitializer to set properties on a particular telemetry instance. We could easily add custom properties to our Request telemetry using a telemetry initializer.

1
2
3
4
5
6
7

public class CustomPropertyTelemetryInitializer : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
      requestTelemetry.Properties["MyCustomProperty"] = "Some Useful Value";
    }
}

Any custom properties you add will be listed under Custom Properties in the Application Insights portal.

But telemetry initializers are singletons and often don't have access to the useful data that we want to add to request telemetry. Typically the data we want is related in some way to the current request and that data wouldn't be available in a singleton service. Fortunately, there is another easy way to get an instance of the request telemetry for the current request.

1
2

var requestTelemetry = HttpContext.Features.Get<RequestTelemetry>();
requestTelemetry.Properties["TenantId"] = "ACME_CORP"; 

You can do it anywhere you have access to an HTTP Context. Some examples I have seen include: Middleware, ActionFilters, Controller action methods, OnActionExecuting in a base Controller class and PageModel classes in Razor Pages.

Filtering by Custom Properties in the Portal

Once you've added custom properties to Request Telemetry, you can use those custom properties to filter data in the Application Insights portal. For example, you might want to investigate failures that are occurring for a specific tenant or investigate performance for a particular tenant.

This type of filtering can be applied almost anywhere in the portal and can help narrow things down when investigating problems.

Writing Useful Log Analytics Queries

Now this is where things get really interesting for me. What if we had one particular tenant complaining about performance. Wouldn't it be interesting to plot out the average request duration for all tenants? We can easily accomplish this using a log analytics query.

1
2
3

requests
| summarize avg(duration) by tostring(customDimensions.TenantId), bin(timestamp, 15m)
| render timechart

This simple query will produce the following chart:

Small variations on this query can be extremely useful in comparing response times, failure rates, usage and pretty much anything else you can think of.

Wrapping it up

TenantId is just an example of a custom property. The custom properties that are useful for a particular application tend to emerge naturally as you're investigating issues and sifting through telemetry in Application Insights. You will eventually find yourself saying "I wish I knew what xxx was for this request`. When that happens, stop and add that as a custom property to the request telemetry. You'll thank yourself later.

In any application that involves more than a single server process/service, the concept of Cloud Role becomes really important in Application Insights. A Cloud Role roughly represents a process that runs somewhere on a server or possibly on a number of servers. A cloud role made up of 2 things: a cloud role name and a cloud role instance.

Cloud Role Name

The cloud role name is a logical name for a particular process. For example, I might have a cloud role name of "Front End" for my front end web server and a name of "Weather Service" for a service that is responsible for providing weather data.

When a cloud role name is set, it will appear as a node in the Application Map. Here is an example showing a Front End role and a Weather Service role.

However, when Cloud Role Name is not set, we end up with a misleading visual representation of how our services communicate.

By default, the application insights SDK attempts to set the cloud role name for you. For example, when you're running in Azure App Service, the name of the web app is used. However, when you are running in an on-premise VM, the cloud role name is often blank.

Cloud Role Instance

The cloud role instance tells us which specific server the cloud role is running on. This is important when scaling out your application. For example, if my Front End web server was running 2 instances behind a load balancer, I might have a cloud role instance of "frontend_prod_1" and another instance of "frontend_prod_2".

The application insights SDK sets the cloud role instance to the name of the server hosting the service. For example, the name of the VM or the name of the underlying compute instance hosting the app in App Service. In my experience, the SDK does a good job here and I don't usually need to override the cloud role instance.

Setting Cloud Role Name using a Telemetry Initializer

Telemetry Initializers are a powerful mechanism for customizing the telemetry that is collected by the Application Insights SDK. By creating and registering a telemetry initializer, you can overwrite or extend the properties of any piece of telemetry collected by Application Insights.

To set the Cloud Role Name, create a class that implements ITelemetryInitializer and in the Initialize method set the telemetry.Context.Cloud.RoleName to the cloud role name for the current application.

1
2
3
4
5
6
7
8

public class CloudRoleNameTelemetryInitializer : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
      // set custom role name here
      telemetry.Context.Cloud.RoleName = "Custom RoleName";
    }
}

Next, in the Startup.ConfigureServices method, register that telemetry initializer as a singleton.

1

services.AddSingleton<ITelemetryInitializer, CloudRoleNameTelemetryInitializer>();

For those who learn by watching, I have recorded a video talking about using telemetry initializers to customize application insights.

Using a Nuget Package

Creating a custom telemetry initializer to set the cloud role name is a simple enough, but it's something I've done so many times that I decided to publish a Nuget package to simplify it even further.

First, add the AspNetMonsters.ApplicationInsights.AspNetCore Nuget package:

1

dotnet add package AspNetMonsters.ApplicationInsights.AspNetCore

Next, in call AddCloudRoleNameInitializer in your application's Startup.ConfigureServices method:

1

services.AddCloudRoleNameInitializer("WeatherService");

Filtering by Cloud Role

Setting the Cloud Role Name / Instance is about a lot more than seeing your services laid out properly in the Application Map. It's also really important when you starting digging in to the performance and failures tabs in the Application Insights portal. In fact, on most of the sections of the portal, you'll see this Roles filter.

The default setting is all. When you click on it, you have the option to select any combination of your application's role names / instances. For example, maybe I'm only interested in the FrontEnd service and WeatherService that were running on the dave_yoga920 instance.

These filters are extremely useful when investigating performance or errors on a specific server or within a specific service. The more services your application is made up of, the more useful and essential this filtering become. These filters really help focus in on specific areas of an application within the Application Insights portal.

Next Steps

In this post, we saw how to customize telemetry data using telemetry initializers. Setting the cloud role name is a simple customization that can help you navigate the massive amount of telemetry that application insights collects. In the next post, we will explore a more in complex example of using telemetry initializers.

Application Insights has built-in support for .NET, Java, Node.js, Python, and Client-side JavaScript based applications. This blog post is specifically about .NET applications. If you're application is built in another language, head over to the docs to learn more.

Codeless Monitoring vs Code-based Monitoring

With codeless monitoring, you can configure a monitoring tool to run on the server (or service) that is hosting your application. The monitoring tool will monitor running processes and collect whatever information is available for that particular platform. There is built in support for Azure VM and scale sets, Azure App Service, Azure Cloud Services, Azure Functions, Kubernetes applications and On-Premises VMs. Codeless monitoring is a good option if you want to collect information for applications that have already been built and deployed, but you are generally going to get more information using Code-based monitoring.

With code-based monitoring, you add the Application Insights SDK. The steps for adding the SDK are well document for ASP.NET Core, ASP.NET, and .NET Console applications so I don't need to re-hash that here.

If you prefer, I have recorded a video showing how to add Application Insights to an existing ASP.NET Core application.

Telemetry

Once you've added the Application Insights SDK to your application, it will start collecting telemetry data at runtime and sending it to Application Insights. That telemetry data is what feeds the UI in the Application Insights portal. The SDK will automatically collection information about your dependencies calls to SQL Server, HTTP calls and calls to many popular Azure Services. It's the dependencies that often are the most insightful. In a complex system it's difficult to know exactly what dependencies your application calls in order to process an incoming request. With App Insights, you can see exactly what dependencies are called by drilling in to the End-to-End Transaction view.

In addition to dependencies, the SDK will also collect requests, exceptions, traces, customEvents, and performanceCounters. If your application has a web front-end and you add the JavaScript client SDK, you'll also find pageViews and browserTimings.

Separate your Environments

The SDK decides which Application Insights instance to send the collected telemetry based on the configured Instrumentation Key.

In the ASP.NET Core SDK, this is done through app settings:

1
2
3
4
5

{
  "ApplicationInsights": {
    "InstrumentationKey": "ccbe3f84-0f5b-44e5-b40e-48f58df563e1"
  }
}

When you're diagnosing an issue in production or investigating performance in your production systems, you don't want any noise from your development or staging environments. I always recommend creating an Application Insights resource per environment. In the Azure Portal, you'll find the instrumentation key in the top section of the Overview page for your Application Insights resource. Just grab that instrumentation key and add it to your environment specific configuration.

Use a single instance for all your production services

Consider a micro-services type architecture where your application is composed of a number of services, each hosted within it's own process. It might be tempting to have each service point to a separate instance of Application Insights.

Contrary to the guidance of separating your environments, you'll actually get the most value from Application Insights if you point all your related production services to a single Application Insights instance. The reason for this is that Application Insights automatically correlates telemetry so you can track a particular request across a series of separate services. That might sound a little like magic but it's not actually as complicated as it sounds.

It's this correlation that allows the Application Map in App Insights to show exactly how all your services interact with each other.

It also enables the end-to-end transaction view to show a timeline of all the calls between your services when you are drilling in to a specific request.

This is all contingent on all your services sending telemetry to the same Application Insights instance. The Application Insights UI in the Azure Portal has no ability to display this visualizations across multiple Application Insights instances.

You don't need to be on Azure

I've often heard developers say "I can't use Application Insights because we're not on Azure". Well, you don't need to host your application on Azure to use Application Insights. Yes, you will need an Azure subscription for the Application Insights resource, but your application can be hosted anywhere. That includes your own on-premise services, AWS or any other public/private cloud.

Next Steps

Out of the box, Application Insights provides a tremendous amount of value but I always find myself having to customize a few things to really get the most out of the telemetry. Fortunately, the SDK provides some useful extension points. My plan is to follow up this post with a few more posts that go over those customizations in detail. I also have started to create a NuGet package to simplify those customizations so stay tuned!

*Update

Other posts in this series:Setting Cloud Role NameEnhancing Application Insights Request Telemetry

After my recent misadventures attempting to use Noda Time with Entity Framework Core, I decided to see what it would take to use Dapper in a the same scenario.

A quick recap

In my app, I needed to model an Event that occurs on a particular date. It might be initially tempting to store the date of the event as a DateTime in UTC, but that's not necessarily accurate unless the event happens to be held at the Royal Observatory Greenwich. I don't want to deal with time at all, I'm only interested in the date the event is being held.

NodaTime provides a LocalDate type that is perfect for this scenario so I declared a LocalDate property named Date on my Event class.

1
2
3
4
5
6
7

public class Event
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
    public LocalDate Date {get; set;}
}

Querying using Dapper

I modified my app to query for the Event entities using Dapper:

1
2
3
4
5
6
7

var queryDate = new LocalDate(2019, 3, 26);
using (var connection = new SqlConnection(myConnectionString))
{
    await connection.OpenAsync();
    Events = await connection.QueryAsync<Event>(@"SELECT [e].[Id], [e].[Date], [e].[Description], [e].[Name]
FROM [Events] AS[e]");
}

The app started up just fine, but gave me an error when I tried to query for events.

System.Data.DataException: Error parsing column 1 (Date=3/26/19 12:00:00 AM - DateTime) ---> System.InvalidCastException: Invalid cast from 'System.DateTime' to 'NodaTime.LocalDate'.

Likewise, if I attempted to query for events using a LocalDate parameter, I got another error:

1
2
3
4
5
6
7
8
9

var queryDate = new LocalDate(2019, 3, 26);
using (var connection = new SqlConnection("myConnectionString"))
{
    await connection.OpenAsync();

    Events = await connection.QueryAsync<Event>(@"SELECT [e].[Id], [e].[Date], [e].[Description], [e].[Name]
FROM [Events] AS[e]
WHERE [e].[Date] = @Date", new { Date = queryDate });
}

NotSupportedException: The member Date of type NodaTime.LocalDate cannot be used as a parameter value

Fortunately, both these problems can be solved by implementing a simple TypeHandler.

Implementing a Custom Type Handler

Out of the box, Dapper already knows how to map to the standard .NET types like Int32, Int64, string and DateTime. The problem we are running into here is that Dapper doesn't know anything about the LocalDate type. If you want to map to a type that Dapper doesn't know about, you can implement a custom type handler. To implement a type handler, create a class that inherits from TypeHandler<T>, where T is the type that you want to map to. In your type handler class, implement the Parse and SetValue methods. These methods will be used by Dapper when mapping to and from properties that are of type T.

Here is an example of a type handler for LocalDate.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

public class LocalDateTypeHandler : TypeHandler<LocalDate>
{
    public override LocalDate Parse(object value)
    {
        if (value is DateTime)
        {
            return LocalDate.FromDateTime((DateTime)value);
        }

        throw new DataException($"Unable to convert {value} to LocalDate");
    }

    public override void SetValue(IDbDataParameter parameter, LocalDate value)
    {
        parameter.Value = value.ToDateTimeUnspecified();
    }
}

Finally, you need to tell Dapper about your new custom type handler. To do that, register the type handler somewhere in your application's startup class by calling Dapper.SqlMapper.AddTypeHandler.

1

Dapper.SqlMapper.AddTypeHandler(new LocalDateTypeHandler());

There's a NuGet for that

As it turns out, someone has already created a helpful NuGet package containing TypeHandlers for many of the NodaTime types so you probably don't need to write these yourself. Use the Dapper.NodaTime package instead.

Wrapping it up

TypeHandlers are a simple extension point that allows for Dapper to handle types that are not already handled by Dapper. You can write your own type handlers but you might also want to check if someone has already published a NuGet package that handles your types.

Noda Time is an alternative date and time API for .NET. It helps you to think about your data more clearly, and express operations on that data more precisely.

An example using NodaTime

In my app, I needed to model an Event that occurs on a particular date. It might be initially tempting to store the date of the event as a DateTime in UTC, but that's not necessarily accurate unless the event happens to be held at the Royal Observatory Greenwich. I don't want to deal with time at all, I'm only interested in the date the event is being held.

NodaTime provides a LocalDate type that is perfect for this scenario so I declared a LocalDate property named Date on my Event class.

1
2
3
4
5
6
7

public class Event
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
    public LocalDate Date {get; set;}
}

Using Entity Framework

This app was using Entity Framework Core and there was a DbSet for the Event class.

1
2
3
4
5
6
7
8

public class EventContext : DbContext
{
    public EventContext(DbContextOptions<EventContext> options) : base(options)
    {
    }

    public DbSet<Event> Events { get; set; }
}

This is where I ran into my first problem. Attempting to run the app, I was greeted with a friendly InvalidOperationException:

InvalidOperationException: The property 'Event.Date' could not be mapped, because it is of type 'LocalDate' which is not a supported primitive type or a valid entity type. Either explicitly map this property, or ignore it using the '[NotMapped]' attribute or by using 'EntityTypeBuilder.Ignore' in 'OnModelCreating'.

This first problem was actually easy enough to solve using a ValueConverter. By adding the following OnModelCreating code to my EventContext, I was able to tell Entity Framework Core to store the Date property as a DateTime with the Kind set to DateTimeKind.Unspecified. This has the effect of avoiding any unwanted shifts in the date time based on the local time of the running process.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

public class EventContext : DbContext
{
    public EventContext(DbContextOptions<EventContext> options) : base(options)
    {
    }

    public DbSet<Event> Events { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        base.OnModelCreating(modelBuilder);
        var localDateConverter = 
            new ValueConverter<LocalDate, DateTime>(v =>  
                v.ToDateTimeUnspecified(), 
                v => LocalDate.FromDateTime(v));

        modelBuilder.Entity<Event>()
            .Property(e => e.Date)
            .HasConversion(localDateConverter);
    }
}

With that small change, my application now worked as expected. The value conversions all happen behind the scenes so I can just use the Event entity and deal strictly with the LocalDate type.

But what about queries

I actually had this application running in a test environment for a week before I noticed a serious problem in the log files.

In my app, I was executing a simple query to retrieve the list of events for a particular date.

1
2

var queryDate = new LocalDate(2019, 3, 25);
Events = await context.Events.Where(e => e.Date == queryDate).ToListAsync();

In the app's log file, I noticed the following warning:

Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'where ([e].Date == __queryDate_0)' could not be translated and will be evaluated locally.

Uh oh, that sounds bad. I did a little more investigation and confirmed that the query was in fact executing SQL without a WHERE clause.

1
2

SELECT [e].[Id], [e].[Date], [e].[Description], [e].[Name]
FROM [Events] AS [e]

So my app was retrieving EVERY ROW from Events table, then applying the where filter in the .NET process. That's really not what I intended to do and would most certainly cause me some performance troubles when I get to production.

So, the first thing I did was modified my EF Core configuration to throw an error when a client side evaluation like this occurs. I don't want this kind of thing accidently creeping in to this app again. Over in Startup.ConfigureServices, I added the following option to ConfigureWarnings.

1
2
3
4

services.AddDbContext<EventContext>(options =>
        options.UseSqlServer(myConnectionString)
        .ConfigureWarnings(warnings => 
            warnings.Throw(RelationalEventId.QueryClientEvaluationWarning)));

Throwing an error by default is the correct behavior here and this is actually something that will be fixed in Entity Framework Core 3.0. The default behavior in EF Core 3 will be to throw an error any time a LINQ expression results in client side evaluation. You will then have the option to allow those client side evaluations.

Fixing the query

Now that I had the app throwing an error for this query, I needed to find a way for EF Core to properly translate my simple e.Date == queryDate expression to SQL. After carefully re-reading the EF Core documentation related for value converters, I noticed a bullet point under Limitations:

Use of value conversions may impact the ability of EF Core to translate expressions to SQL. A warning will be logged for such cases. Removal of these limitations is being considered for a future release.

Well that just plain sucks. It turns out that when you use a value converter for a property, Entity Framework Core just gives up trying to convert any LINQ expression that references that property. The only solution I found was to query for my entities using SQL.

1
2
3
4
5

var queryDate = new LocalDate(2019, 3, 25);
Events = await context.Events.
    FromSql(@"SELECT [e].[Id], [e].[Date], [e].[Description], [e].[Name]
FROM[Events] AS[e]
WHERE [e].[Date] = {0}", queryDate.ToDateTimeUnspecified()).ToListAsync();

Wrapping it up

NodaTime is a fantastic date and time library for .NET and you should definitely consider using it in your app. Unfortunately, Entity Framework Core has some serious limitations when it comes to using value converters so you will need to be careful. I almost got myself into some problems with it. While there are work-arounds, writing custom SQL for any query that references a NodaTime type is less than ideal. Hopefully those will be addressed in Entity Framework Core 3.

In today's post, we explore a pattern to prevent multiple users (or processes) from accidentally overwriting each other's change. Given our current implementation for updating the Aircraft record, there is potential for data loss if there are multiple active sessions are attempting to update the same Aircraft record at the same time. In the example shown below, Bob accidentally overwrites Jane's changes without even knowing that Jane made changes to the same Aircraft record

The pattern we will use here is Optimistic Offline Lock, which is often also referred to as Optimistic Concurrency Control.

Modifying the Database and Entities

To implement this approach, we will use a rowversion column in SQL Server. Essentially, this is a column that automatically version stamps a row in a table. Any time a row is modified, the rowversion column will is automatically incremented for that row. We will start by adding the column to our Aircraft table.

1

ALTER TABLE Aircraft ADD RowVer rowversion

Next, we add a RowVer property to the Aircraft table. The property is a byte array. When we read the RowVer column from the database, we will get an array of 8 bytes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

public class Aircraft 
{
    public int Id { get; set; }
    public string Manufacturer {get; set;}
    public string Model {get; set;}
    public string RegistrationNumber {get; set;}
    public int FirstClassCapacity {get; set;}
    public int RegularClassCapacity {get; set;}
    public int CrewCapacity {get; set;}
    public DateTime ManufactureDate {get; set; }
    public int NumberOfEngines {get; set;}
    public int EmptyWeight {get; set;}
    public int MaxTakeoffWeight {get; set;}
    public byte[] RowVer { get; set; }
}   

Finally, we will modify the query used to load Aircraft entities so it returns the RowVer column. We don't need to change any of the Dapper code here.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

public async Task<Aircraft> Get(int id)
{ 
    Aircraft aircraft;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
SELECT 
Id
,Manufacturer
,Model
,RegistrationNumber
,FirstClassCapacity
,RegularClassCapacity
,CrewCapacity
,ManufactureDate
,NumberOfEngines
,EmptyWeight
,MaxTakeoffWeight
,RowVer
FROM Aircraft WHERE Id = @Id";
        aircraft = await connection.QuerySingleAsync<Aircraft>(query, new {Id = id});
    }
    return aircraft;
}

Adding the Concurrency Checks

Now that we have the row version loaded in to our model, we need to add the checks to ensure that one user doesn't accidentally overwrite another users changes. To do this, we simply need to add the RowVer to the WHERE clause on the UPDATE statement. By adding this constraint to the WHERE clause, we we ensure that the updates will only be applied if the RowVer has not changed since this user originally loaded the Aircraft entity.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

public async Task<IActionResult> Put(int id, [FromBody] Aircraft model)
{
    if (id != model.Id) 
    {
        return BadRequest();
    }

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
UPDATE Aircraft 
SET  Manufacturer = @Manufacturer
  ,Model = @Model
  ,RegistrationNumber = @RegistrationNumber 
  ,FirstClassCapacity = @FirstClassCapacity
  ,RegularClassCapacity = @RegularClassCapacity
  ,CrewCapacity = @CrewCapacity
  ,ManufactureDate = @ManufactureDate
  ,NumberOfEngines = @NumberOfEngines
  ,EmptyWeight = @EmptyWeight
  ,MaxTakeoffWeight = @MaxTakeoffWeight
WHERE Id = @Id
      AND RowVer = @RowVer";
      
      await connection.ExecuteAsync(query, model);
    }

    return Ok();
}

So, the WHERE clause stops the update from happening, but how do we know if the update was applied successfully? We need to let the user know that the update was not applied due to a concurrency conflict. To do that, we add OUTPUT inserted.RowVer to the UPDATE statement. The effect of this is that the query will return the new value for the RowVer column if the update was applied. If not, it will return null.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

public async Task<IActionResult> Put(int id, [FromBody] Aircraft model)
{
    byte[] rowVersion;
    if (id != model.Id) 
    {
        return BadRequest();
    }

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
UPDATE Aircraft 
SET  Manufacturer = @Manufacturer
  ,Model = @Model
  ,RegistrationNumber = @RegistrationNumber 
  ,FirstClassCapacity = @FirstClassCapacity
  ,RegularClassCapacity = @RegularClassCapacity
  ,CrewCapacity = @CrewCapacity
  ,ManufactureDate = @ManufactureDate
  ,NumberOfEngines = @NumberOfEngines
  ,EmptyWeight = @EmptyWeight
  ,MaxTakeoffWeight = @MaxTakeoffWeight
  OUTPUT inserted.RowVer
WHERE Id = @Id
      AND RowVer = @RowVer";
        rowVersion = await connection.ExecuteScalarAsync<byte[]>(query, model);
    }

    if (rowVersion == null) {
        throw new DBConcurrencyException("The entity you were trying to edit has changed. Reload the entity and try again."); 
    }
    return Ok(rowVersion);
}

Instead of calling ExecuteAsync, we call ExecuteScalarAsync<byte[]>. Then we can check if the returned value is null and raise a DBConcurrencyException if it is null. If it is not null, we can return the new RowVer value.

Wrapping it up

Using SQL Server's rowversion column type makes it easy to implement optimistic concurrency checks in a .NET app that uses Dapper.

If you are building as REST api, you should really use the ETag header to represent the current RowVer for your entity. You can read more about this pattern here.

In today's post, we explore a more complex scenario that involves executing multiple write operations. In order to ensure consistency at the database level, these operations should all succeed / fail together as a single transaction. In this example, we will be inserting a new ScheduledFlight entity along with an associated set of Flight entities.

As a quick reminder, a Flight represents a particular occurrence of a ScheduledFlight on a particular day. That is, it has a reference to the ScheduledFlight along with some properties indicating the scheduled arrival and departure times.

1
2
3
4
5
6
7
8
9

public class Flight 
{
    public int Id {get; set;}
    public int ScheduledFlightId {get; set;}
    public ScheduledFlight ScheduledFlight { get; set;}
    public DateTime Day {get; set;}
    public DateTime ScheduledDeparture {get; set;}
    public DateTime ScheduledArrival {get; set;}
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

public class ScheduledFlight 
{
    public int Id {get; set;}
    public string FlightNumber {get; set;}

    public int DepartureAirportId {get; set;}
    public Airport DepartureAirport {get; set;}
    public int DepartureHour {get; set;}
    public int DepartureMinute {get; set;}

    public int ArrivalAirportId {get; set;}
    public Airport ArrivalAirport {get; set;}        
    public int ArrivalHour {get; set;}
    public int ArrivalMinute {get; set;}

    public bool IsSundayFlight {get; set;}
    public bool IsMondayFlight {get; set;}
    // Some other properties
}

Inserting the ScheduledFlight

Inserting the ScheduledFlight and retrieving the database generated id is easy enough. We can use the same approach we used in the previous blog post.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

// POST api/scheduledflight
[HttpPost()]
public async Task<IActionResult> Post([FromBody] ScheduledFlight model)
{
    int newScheduledFlightId;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var insertScheduledFlightSql = @"
INSERT INTO [dbo].[ScheduledFlight]
    ([FlightNumber]
    ,[DepartureAirportId]
    ,[DepartureHour]
    ,[DepartureMinute]
    ,[ArrivalAirportId]
    ,[ArrivalHour]
    ,[ArrivalMinute]
    ,[IsSundayFlight]
    ,[IsMondayFlight]
    ,[IsTuesdayFlight]
    ,[IsWednesdayFlight]
    ,[IsThursdayFlight]
    ,[IsFridayFlight]
    ,[IsSaturdayFlight])
VALUES
    (@FlightNumber
    ,@DepartureAirportId
    ,@DepartureHour
    ,@DepartureMinute
    ,@ArrivalAirportId
    ,@ArrivalHour
    ,@ArrivalMinute
    ,@IsSundayFlight
    ,@IsMondayFlight
    ,@IsTuesdayFlight
    ,@IsWednesdayFlight
    ,@IsThursdayFlight
    ,@IsFridayFlight
    ,@IsSaturdayFlight);
SELECT CAST(SCOPE_IDENTITY() as int)";
        newScheduledFlightId = await connection.ExecuteScalarAsync<int>(insertScheduledFlightSql, model);
    }
    return Ok(newScheduledFlightId);
}

According to the bosses at Air Paquette, whenever we create a new ScheduledFlight entity, we also want to generate the Flight entities for the next 12 months of that ScheduledFlight. We can add a method to the ScheduledFlight class to generate the flight entities.

NOTE: Let's just ignore the obvious bugs related to timezones and to flights that take off and land on a different day.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

public IEnumerable<Flight> GenerateFlights(DateTime startDate, DateTime endDate)
{
    var flights = new List<Flight>();
    var currentDate = startDate;

    while (currentDate <= endDate)
    {
        if (IsOnDayOfWeek(currentDate.DayOfWeek))
        {
            var departureTime = new DateTime(currentDate.Year, currentDate.Month, currentDate.Day, DepartureHour, DepartureMinute, 0);
            var arrivalTime = new DateTime(currentDate.Year, currentDate.Month, currentDate.Day, ArrivalHour, ArrivalMinute, 0);
            var flight = new Flight
            {
                ScheduledFlightId = Id,
                ScheduledDeparture = departureTime,
                ScheduledArrival = arrivalTime,
                Day = currentDate.Date
            };
            flights.Add(flight);
        }
        currentDate = currentDate.AddDays(1);
    }
    return flights;
}
public bool IsOnDayOfWeek(DayOfWeek dayOfWeek)
{
    return     (dayOfWeek == DayOfWeek.Sunday && IsSundayFlight)
            || (dayOfWeek == DayOfWeek.Monday && IsMondayFlight)
            || (dayOfWeek == DayOfWeek.Tuesday && IsTuesdayFlight)
            || (dayOfWeek == DayOfWeek.Wednesday && IsWednesdayFlight)
            || (dayOfWeek == DayOfWeek.Thursday && IsThursdayFlight)
            || (dayOfWeek == DayOfWeek.Friday && IsFridayFlight)
            || (dayOfWeek == DayOfWeek.Saturday && IsSaturdayFlight);
}

Now in the controller, we can add some logic to call the GenerateFlight method and then insert those Flight entities using Dapper.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

// POST api/scheduledflight
[HttpPost()]
public async Task<IActionResult> Post([FromBody] ScheduledFlight model)
{
    int newScheduledFlightId;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var insertScheduledFlightSql = @"
INSERT INTO [dbo].[ScheduledFlight]
    ([FlightNumber]
    ,[DepartureAirportId]
    ,[DepartureHour]
    ,[DepartureMinute]
    ,[ArrivalAirportId]
    ,[ArrivalHour]
    ,[ArrivalMinute]
    ,[IsSundayFlight]
    ,[IsMondayFlight]
    ,[IsTuesdayFlight]
    ,[IsWednesdayFlight]
    ,[IsThursdayFlight]
    ,[IsFridayFlight]
    ,[IsSaturdayFlight])
VALUES
    (@FlightNumber
    ,@DepartureAirportId
    ,@DepartureHour
    ,@DepartureMinute
    ,@ArrivalAirportId
    ,@ArrivalHour
    ,@ArrivalMinute
    ,@IsSundayFlight
    ,@IsMondayFlight
    ,@IsTuesdayFlight
    ,@IsWednesdayFlight
    ,@IsThursdayFlight
    ,@IsFridayFlight
    ,@IsSaturdayFlight);
SELECT CAST(SCOPE_IDENTITY() as int)";
        newScheduledFlightId = await connection.ExecuteScalarAsync<int>(insertScheduledFlightSql, model);

        model.Id = newScheduledFlightId;
        var flights = model.GenerateFlights(DateTime.Now, DateTime.Now.AddMonths(12));

    var insertFlightsSql = @"INSERT INTO [dbo].[Flight]
    ([ScheduledFlightId]
    ,[Day]
    ,[ScheduledDeparture]
    ,[ActualDeparture]
    ,[ScheduledArrival]
    ,[ActualArrival])
VALUES
    (@ScheduledFlightId
    ,@Day
    ,@ScheduledDeparture
    ,@ActualDeparture
    ,@ScheduledArrival
    ,@ActualArrival)";

        await connection.ExecuteAsync(insertFlightsSql, flights);

    }
    return Ok(newScheduledFlightId);
}

Note that we passed in an IEnumerable<Flight> as the second argument to the ExecuteAsync method. This is a handy shortcut in Dapper for executing a query multiple times. Instead of writing a loop and calling ExecuteAsync for each flight entity, we can pass in a list of flights and Dapper will execute the query once for each item in the list.

Explicitly managing a transaction

So far, we have code that first inserts a ScheduledFlight, next generates a set of Flight entities and finally inserting all of those Flight entities. That's the happy path, but what happens if something goes wrong along the way. Typically when we execute a set of related write operations (inserts, updates and deletes), we want those operations to all succeed or fail together. In the database world, we have transactions to help us with this.

The nice thing about using Dapper is that it uses standard .NET database connections and transactions. There is no need to re-invent the wheel here, we can simply use the transaction patterns that have been around in .NET since for nearly 2 decades now.

After opening the connection, we call connection.BeginTransaction() to start a new transaction. Whenever we call ExecuteAsync (or any other Dapper extension method), we need to pass in that transaction. At the end of all that work, we call transaction.Commit(). Finally, we wrap the logic in a try / catch block. If any exception is raised, we call transaction.Rollback() to ensure that none of those write operations are committed to the database.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

[HttpPost()]
public async Task<IActionResult> Post([FromBody] ScheduledFlight model)
{
    int? newScheduledFlightId = null;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var transaction = connection.BeginTransaction();

        try
        {
            var insertScheduledFlightSql = @"
INSERT INTO [dbo].[ScheduledFlight]
    ([FlightNumber]
    ,[DepartureAirportId]
    ,[DepartureHour]
    ,[DepartureMinute]
    ,[ArrivalAirportId]
    ,[ArrivalHour]
    ,[ArrivalMinute]
    ,[IsSundayFlight]
    ,[IsMondayFlight]
    ,[IsTuesdayFlight]
    ,[IsWednesdayFlight]
    ,[IsThursdayFlight]
    ,[IsFridayFlight]
    ,[IsSaturdayFlight])
VALUES
    (@FlightNumber
    ,@DepartureAirportId
    ,@DepartureHour
    ,@DepartureMinute
    ,@ArrivalAirportId
    ,@ArrivalHour
    ,@ArrivalMinute
    ,@IsSundayFlight
    ,@IsMondayFlight
    ,@IsTuesdayFlight
    ,@IsWednesdayFlight
    ,@IsThursdayFlight
    ,@IsFridayFlight
    ,@IsSaturdayFlight);
SELECT CAST(SCOPE_IDENTITY() as int)";
            newScheduledFlightId = await connection.ExecuteScalarAsync<int>(insertScheduledFlightSql, model, transaction);

            model.Id = newScheduledFlightId.Value;
            var flights = model.GenerateFlights(DateTime.Now, DateTime.Now.AddMonths(12));

            var insertFlightsSql = @"INSERT INTO [dbo].[Flight]
    ([ScheduledFlightId]
    ,[Day]
    ,[ScheduledDeparture]
    ,[ActualDeparture]
    ,[ScheduledArrival]
    ,[ActualArrival])
VALUES
    (@ScheduledFlightId
    ,@Day
    ,@ScheduledDeparture
    ,@ActualDeparture
    ,@ScheduledArrival
    ,@ActualArrival)";

            await connection.ExecuteAsync(insertFlightsSql, flights, transaction);
            transaction.Commit();
        }
        catch (Exception ex)
        { 
            //Log the exception (ex)
            try
            {
                transaction.Rollback();
            }
            catch (Exception ex2)
            {
                // Handle any errors that may have occurred
                // on the server that would cause the rollback to fail, such as
                // a closed connection.
                // Log the exception ex2
            }
            return StatusCode(500);
        }
    }
    return Ok(newScheduledFlightId);
}

Managing database transactions in .NET is a deep but well understood topic. We covered the basic pattern above and showed how Dapper can easily participate in a transaction. To learn more about managing database transactions in .NET, check out these docs:

• SqlConnection.BeginTransaction
• Transactions in ADO.NET

Wrapping it up

Using transactions with Dapper is fairly straight forward process. We just need to tell Dapper what transaction to use when executing queries. Now that we know how to use transactions, we can look at some more advanced scenarios like adding concurrency checks to update operations to ensure users aren't overwriting each other's changes.

In today's post, we explore how easy it is to perform basic Insert, Update and Delete operations using the same Aircraft entity that we used in the first post in this series. Basically, instead of using Dapper's QueryAsync extension method that we used to retrieve data, we will use the ExecuteAsync method.

As a quick reminder, here is the Aircraft class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

public class Aircraft 
{
    public int Id { get; set; }

    public string Manufacturer {get; set;}

    public string Model {get; set;}

    public string RegistrationNumber {get; set;}

    public int FirstClassCapacity {get; set;}

    public int RegularClassCapacity {get; set;}

    public int CrewCapacity {get; set;}

    public DateTime ManufactureDate {get; set; }

    public int NumberOfEngines {get; set;}
}   

NOTE: In these examples, I am ignoring some important aspects like validation. I want to focus specifically on the Dapper bits here but validation is really important. In a real-world scenario, you should be validating any data that is passed in to the server. I recommend using Fluent Validation.

Insert

Inserting a single new record is really easy. All we need to do is write an INSERT statement with parameters for each column that we want to set.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

[HttpPost()]
public async Task<IActionResult> Post([FromBody] Aircraft model)
{
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var sqlStatement = @"
INSERT INTO Aircraft 
(Manufacturer
,Model
,RegistrationNumber
,FirstClassCapacity
,RegularClassCapacity
,CrewCapacity
,ManufactureDate
,NumberOfEngines
,EmptyWeight
,MaxTakeoffWeight)
VALUES (@Manufacturer
,@Model
,@RegistrationNumber
,@FirstClassCapacity
,@RegularClassCapacity
,@CrewCapacity
,@ManufactureDate
,@NumberOfEngines
,@EmptyWeight
,@MaxTakeoffWeight)";
        await connection.ExecuteAsync(sqlStatement, model);
    }
    return Ok();
}

The version of the ExecuteAsync method we used here accepts two parameters: a string containing the SQL statement to execute and an object containing the parameter values to bind to the statement. In this case, it is an instance of the Aircraft class which has properties with names matching the parameters defined in the INSERT statement.

Our Aircraft table's Id column is an auto-incremented identity column. That means the primary key is generated by the database when the row is inserted. We will likely need to pass that value back to whoever called the API so they know how to retrieve the newly inserted Aircraft.

An easy way to get the generated Id is to add SELECT CAST(SCOPE_IDENTITY() as int) after the INSERT statement. The SCOPE_IDENTITY() function returns the last identity value that was generated in any table in the current session and current scope.

Now, since the SQL statement we are executing will be returning a single value (the generated id), we need to call ExecuteScalarAsync<int>. The ExecuteScalarAsync method executes a SQL statement that returns a single value whereas the ExecuteAsync method executes a SQL statement that does not return a value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

[HttpPost()]
public async Task<IActionResult> Post([FromBody] Aircraft model)
{
    int newAircraftId;
    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var sqlStatement = @"
INSERT INTO Aircraft 
(Manufacturer
,Model
,RegistrationNumber
,FirstClassCapacity
,RegularClassCapacity
,CrewCapacity
,ManufactureDate
,NumberOfEngines
,EmptyWeight
,MaxTakeoffWeight)
VALUES (@Manufacturer
,@Model
,@RegistrationNumber
,@FirstClassCapacity
,@RegularClassCapacity
,@CrewCapacity
,@ManufactureDate
,@NumberOfEngines
,@EmptyWeight
,@MaxTakeoffWeight);

SELECT CAST(SCOPE_IDENTITY() as int)";
        newAircraftId = await connection.ExecuteScalarAsync<int>(sqlStatement, model);
    }
    return Ok(newAircraftId);
}

Update

Updating an existing entity is similar to inserting. All we need is a SQL statement containing an UPDATE statement that sets the appropriate columns. We also want to make sure we include a WHERE clause limiting the update only to the row with the specified Id.

Again, the parameters in the SQL statement match the names of the properties in our Aircraft class. All we need to do is call the ExecuteAsync method passing in the SQL statement and the Aircraft entity.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

// PUT api/aircraft/id
[HttpPut("{id}")]
public async Task<IActionResult> Put(int id, [FromBody] Aircraft model)
{
    if (id != model.Id) 
    {
        return BadRequest();
    }

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var sqlStatement = @"
UPDATE Aircraft 
SET  Manufacturer = @Manufacturer
,Model = @Model
,RegistrationNumber = @RegistrationNumber 
,FirstClassCapacity = @FirstClassCapacity
,RegularClassCapacity = @RegularClassCapacity
,CrewCapacity = @CrewCapacity
,ManufactureDate = @ManufactureDate
,NumberOfEngines = @NumberOfEngines
,EmptyWeight = @EmptyWeight
,MaxTakeoffWeight = @MaxTakeoffWeight
WHERE Id = @Id";
        await connection.ExecuteAsync(sqlStatement, model);
    }
    return Ok();
}

Delete

Deleting an entity is the easiest of the three operations since it only requires a single parameter: the unique Id to identify the entity being deleted. The SQL statement is a simple DELETE with a WHERE clause on the Id column. To execute the delete, call the ExecuteAsync method passing in the SQL statement and an anonymous object containing the Id to delete.

1
2
3
4
5
6
7
8
9
10
11
12
13

// DELETE api/aircraft/id
[HttpDelete("{id}")]
public async Task<IActionResult> Delete(int id)
{

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var sqlStatement = "DELETE Aircraft WHERE Id = @Id";
        await connection.ExecuteAsync(sqlStatement, new {Id = id});
    }
    return Ok();
}

I really appreciate how simple delete is using Dapper. When using Entity Framework, delete requires you to first fetch the existing entity, then delete it. That requires 2 round trips to the database while the approach we used here only requires a single round trip.

Wrapping it up

Basic insert, update and delete operations are easy to implement using Dapper. Real world scenarios are often a little more complex and we will dig into some of those scenarios in future posts:

• Bulk inserts, updates and deletes
• Managing transactions
• Optimistic concurrency checks

In today's post, we explore paging through large result sets. Paging is a common technique that is used when dealing with large results sets. Typically, it is not useful for an application to request millions of records at a time because there is no efficient way to deal with all those records in memory all at once. This is especially true when rendering data on a grid in a user interface. The screen can only display a limited number of records at a time so it is generally a bad use of system resources to hold everything in memory when only a small subset of those records can be displayed at any given time.

Source: AppStack Bootstrap Template

Modern versions of SQL Server support the OFFSET / FETCH clause to implement query paging.

In continuing with our airline theme, consider a Flight entity. A Flight represents a particular occurrence of a ScheduledFlight on a particular day. That is, it has a reference to the ScheduledFlight along with some properties indicating the scheduled arrival and departure times.

1
2
3
4
5
6
7
8
9

public class Flight 
{
    public int Id {get; set;}
    public int ScheduledFlightId {get; set;}
    public ScheduledFlight ScheduledFlight { get; set;}
    public DateTime Day {get; set;}
    public DateTime ScheduledDeparture {get; set;}
    public DateTime ScheduledArrival {get; set;}
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

public class ScheduledFlight 
{
    public int Id {get; set;}
    public string FlightNumber {get; set;}

    public int DepartureAirportId {get; set;}
    public Airport DepartureAirport {get; set;}
    public int DepartureHour {get; set;}
    public int DepartureMinute {get; set;}

    public int ArrivalAirportId {get; set;}
    public Airport ArrivalAirport {get; set;}        
    public int ArrivalHour {get; set;}
    public int ArrivalMinute {get; set;}

    public bool IsSundayFlight {get; set;}
    public bool IsMondayFlight {get; set;}
    // Some other properties
}

Writing the query

As we learned in a previous post, we can load the Flight entity along with it's related ScheduledFlight entity using a technique called multi-mapping.

In this case, loading all the flights to or from a particular airport, we would use the following query.

1
2
3
4
5
6

SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode

But this query could yield more results than we want to deal with at any given time. Using OFFSET/FETCH, we can ask for only a block of results at a time.

1
2
3
4
5
6
7
8
9

SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
ORDER BY f.Day, sf.FlightNumber
OFFSET @Offset ROWS
FETCH NEXT @PageSize ROWS ONLY

Note that an ORDER BY clause is required when using OFFSET/FETCH.

Executing the Query

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

//GET api/flights
[HttpGet]
public async Task<IEnumerable<Flight>> Get(string airportCode, int page=1, int pageSize=10)
{
    IEnumerable<Flight> results;

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
ORDER BY f.Day, sf.FlightNumber
OFFSET @Offset ROWS
FETCH NEXT @PageSize ROWS ONLY;
";

        results = await connection.QueryAsync<Flight, ScheduledFlight, Flight>(query,
          (f, sf) =>
              {
                  f.ScheduledFlight = sf;
                  return f;
              },
            new { AirportCode = airportCode,
                              Offset = (page - 1) * pageSize,
                              PageSize = pageSize }
            );        
    }

    return results;
}

Here we calculate the offset by based on the page and pageSize arguments that were passed in. This allows the caller of the API to request a particular number of rows and the starting point.

One step further

When dealing with paged result sets, it can be useful for the caller of the API to also know the total number of records. Without the total number of records, it would be difficult to know how many records are remaining which in turn makes it difficult to render a paging control, a progress bar or a scroll bar (depending on the use case).

A technique I like to use here is to have my API return a PagedResults<T> class that contains the list of items for the current page along with the total count.

1
2
3
4
5

public class PagedResults<T>
{
    public IEnumerable<T> Items { get; set; }
    public int TotalCount { get; set; }
}

To populate this using Dapper, we can add a second result set to the query. That second result set will simply be a count of all the records. Note that the same WHERE clause is used in both queries.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
ORDER BY f.Day, sf.FlightNumber
OFFSET @Offset ROWS
FETCH NEXT @PageSize ROWS ONLY;

SELECT COUNT(*)
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode

Now in our code that executes the query, we will the QueryMultipleAsync method to execute both SQL statements in a single round trip.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

//GET api/flights
[HttpGet]
public async Task<PagedResults<Flight>> Get(string airportCode, int page=1, int pageSize=10)
{
    var results = new PagedResults<Flight>();

    using (var connection = new SqlConnection(_connectionString))
    {
        await connection.OpenAsync();
        var query = @"
SELECT f.*, sf.*
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
ORDER BY f.Day, sf.FlightNumber
OFFSET @Offset ROWS
FETCH NEXT @PageSize ROWS ONLY;

SELECT COUNT(*)
FROM Flight f
INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
INNER JOIN Airport a ON sf.ArrivalAirportId = a.Id
INNER JOIN Airport d ON sf.DepartureAirportId = d.Id
WHERE a.Code = @AirportCode OR d.Code = @AirportCode
";

        using (var multi = await connection.QueryMultipleAsync(query,
                    new { AirportCode = airportCode,
                          Offset = (page - 1) * pageSize,
                          PageSize = pageSize }))
        {
            results.Items = multi.Read<Flight, ScheduledFlight, Flight>((f, sf) =>
                {
                    f.ScheduledFlight = sf;
                    return f;
                }).ToList();

            results.TotalCount = multi.ReadFirst<int>();
        }
    }

    return results;
}
    

Wrapping it up

Paged result sets is an important technique when dealing with large amounts of data. When using a full ORM like Entity Framework, this is implemented easily using LINQ's Skip and Take methods. It's so easy in fact that it can look a little like magic. In reality, it is actually very simple to write your own queries to support paged result sets and execute those queries using Dapper.

The default MVC project templates already include jQuery, so you might use that. You'll probably end up writing a lot of code if you go down that path. Chances are, you will want to use a JavaScript framework that offers two-way data binding between the elements in the DOM to a your model data.

It seems that for many people, Knockout.js is the default library to use in these scenarios. I won't get into the specifics but I think that Knockout is a little dated and that there are better options these days. If you want to dig into some of the issues with Knockout, you can read Simon Timms' rant on the subject.

Vue.js

My fellow ASP.NET Monster James Chambers recently strongly recommended I take a look at Vue.js. I had been meaning to give Vue a try for some time now and I finally had a chance to use it on a recent project. Let me tell you...I love it.

I love it for a whole bunch of reasons. The documentation is great! It is super easy to get drop in to your existing project and it doesn't get in the way. For what I needed to do, it just did the job and allowed me to get on with my day. It is also designed to be "incrementally adoptable", which means you can start out with just using the core view layer, then start pulling in other things like routing and state management if/when you need them.

A simple example

I won't go into great detail about how to use Vue. If you want a full tutorial, head on over to the Vue docs. What I want to show here is just how simple it is to drop Vue into an existing ASP.NET MVC project and add a bit of client side functionality.

The simplest example I can think of is a set of cascading dropdowns. Let's consider a form where a user is asked to enter their Country / Province. When the Country is selected, we would expect the Province dropdown to only display the valid Provinces/States for the selected Country. That probably involves a call to an HTTP endpoint that will return the list of valid values.

1
2
3
4
5
6
7
8

public class ProvinceLookupController : Controller
{
    public ActionResult Index(string countryCode)
    {
        var provinces = ProvinceLookupService.GetProvinces(countryCode);
        return Json(provinces, JsonRequestBehavior.AllowGet);
    }
}

Including Vue in your Razor (cshtml) view

The easiest way to include Vue on a particular Razor view is to link to the vue.js file from a CDN. You can add the following Script tag to your scripts section.

1
2
3

@section scripts  {
    <script src="https://cdn.jsdelivr.net/npm/vue@2.5.22/dist/vue.js"></script>
}

Be sure to check the docs to make sure you are referencing the latest version of Vue.

Binding data to the our View

Now that you have included the core Vue library, you can start using Vue to bind DOM elements to model data.

Start by defining a Vue object in JavaScript. You can add this in a new <script> tag in your scripts section.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

@section scripts  {
    <script src="https://cdn.jsdelivr.net/npm/vue@2.5.22/dist/vue.js"></script>

    <script type="text/javascript">
        var app = new Vue({
            el: '#vueApp',
            data: {
                selectedCountryCode: null,
                countries: [
                    { code: 'ca', name: 'Canada' },
                    { code: 'us', name: 'United States' }
                ]            
            }
        });
    </script>
}

This Vue object targets the DOM element with id vueApp and contains some simple data. The currently selected country code and the list of countries.

Now, back in the HTML part of your csthml, wrap the form in a div that has an id="vueApp".

1
2
3

<div id="vueApp">
    <!-- your form -->
</div>

Next, bind the <select> element to the data in your Vue object. In Vue, data binding is done using a combination of custom attributes that start with v- and the double curly bracket (aka. Mustache) syntax for text.

1
2
3
4
5
6
7
8
9
10
11

<div class="form-group">
    @Html.LabelFor(model => model.CountryCode, new { @class = "control-label col-md-2" })
    <div class="col-md-10">
        <select id="CountryCode" name="CountryCode" class="form-control" 
                v-model="selectedCountryCode">
            <option v-for="country in countries" v-bind:value="country.code">
                
            </option>
        </select>
    </div>
</div>

Now, when you run the app, you should see a dropdown containing Canada and United States.

Adding functionality

Next, you will want to add some client side logic to get the list of valid provinces from the server whenever the selected country changes.

First, add an empty provinces array and a selectedProvinceCode property to the Vue object's data.

Next, add a method called countryChanged to the Vue object. This method will call the ProvinceLookup action method on the server, passing in the selectedCountryCode as a parameter. Assign the response data to the provinces array.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

var app = new Vue({
    el: '#vueApp',
    data: {
        selectedCountryCode: null,
        countries: [
            { code: 'ca', name: 'Canada' },
            { code: 'us', name: 'United States' }
        ],
        selectedProvinceCode: null,
        provinces: []
    },
    methods: {
        countryChanged: function () {
            $.getJSON('@Url.Action("Index", "ProvinceLookup")?countryCode=' + this.selectedCountryCode, function (data) {
                this.provinces = data;
            }.bind(this));
        }
    }
});

Here I used jQuery to make the call to the server. In the Vue community, Axios is a popular library for making HTTP requests.

Back in the HTML, bind the change event from the country select element to the countryChanged method using the v-on:change attribute.

1
2
3
4
5
6

<select id="CountryCode" name="CountryCode" class="form-control" 
        v-model="selectedCountryCode" v-on:change="countryChanged">
    <option v-for="country in countries" v-bind:value="country.code">
        
    </option>
</select>

Now you can add a select element for the provinces.

1
2
3
4
5
6
7
8
9
10
11

<div class="form-group">
    @Html.LabelFor(model => model.ProvinceCode, new { @class = "control-label col-md-2" })
    <div class="col-md-10">
        <select id="ProvinceCode" name="ProvinceCode" class="form-control"
                v-model="selectedProvinceCode">
            <option v-for="province in provinces" v-bind:value="province.Code">
                
            </option>
        </select>
    </div>
</div>

Voila! You now have a working set of cascading dropdowns.

One last thing

You might want to disable the provinces dropdown whenever a request is being made to get the list of provinces for the selected country. You can do this by adding an isProvincesLoading property to the Vue object's data, then setting that property in the countryChanged method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

var app = new Vue({
    el: '#vueApp',
    data: {
        selectedCountryCode: null,
        countries: [
            { code: 'ca', name: 'Canada' },
            { code: 'us', name: 'United States' }
        ],
        selectedProvinceCode: null,
        provinces: [],
        isProvincesLoading: false
    },
    methods: {
        countryChanged: function () {
            this.isProvincesLoading = true;
            $.getJSON('@Url.Action("Index", "ProvinceLookup")?countryCode=' + this.selectedCountryCode, function (data) {
                this.provinces = data;
                this.isProvincesLoading = false;
            }.bind(this));
        }
    }
});

In your HTML, bind the disabled attribute to the isProvincesLoading property.

1
2
3
4
5
6
7
8

<div class="form-group">
<select id="ProvinceCode" name="ProvinceCode" class="form-control"
        v-model="selectedProvinceCode"
        v-bind:disabled="isProvincesLoading">
    <option v-for="province in provinces" v-bind:value="province.Code">
        
    </option>
</select>

Putting it all together

Here is the entire cshtml file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

@{
    ViewBag.Title = "Location Settings";
}
@model Mvc5VueJsExample.Models.LocationSettingsModel

<h2>@ViewBag.Title.</h2>
<h3>@ViewBag.Message</h3>

<div id="vueApp">
    @using (Html.BeginForm("LocationSettings", "Home", FormMethod.Post, new { @class = "form" }))
    {
        <div class="form-group">
            @Html.LabelFor(model => model.CountryCode, new { @class = "control-label col-md-2" })
            <div class="col-md-10">
                <select id="CountryCode" name="CountryCode" class="form-control" 
                        v-model="selectedCountryCode" v-on:change="countryChanged">
                    <option v-for="country in countries" v-bind:value="country.code">
                        
                    </option>
                </select>
            </div>
        </div>

        <div class="form-group">
            @Html.LabelFor(model => model.ProvinceCode, new { @class = "control-label col-md-2" })
            <div class="col-md-10">
                <select id="ProvinceCode" name="ProvinceCode" class="form-control"
                        v-model="selectedProvinceCode"
                        v-bind:disabled="isProvincesLoading">
                    <option v-for="province in provinces" v-bind:value="province.Code">
                        
                    </option>
                </select>
            </div>
        </div>
        <button class="btn btn-primary" type="submit">Save</button>
    }

</div>
@section scripts  {
    <script src="https://cdn.jsdelivr.net/npm/vue@2.5.22/dist/vue.js"></script>

    <script type="text/javascript">
        var app = new Vue({
            el: '#vueApp',
            data: {
                selectedCountryCode: null,
                countries: [
                    { code: 'ca', name: 'Canada' },
                    { code: 'us', name: 'United States' }
                ],
                selectedProvinceCode: null,
                provinces: [],
                isProvincesLoading: false
            },
            methods: {
                countryChanged: function () {
                    this.isProvincesLoading = true;
                    $.getJSON('@Url.Action("Index", "ProvinceLookup")?countryCode=' + this.selectedCountryCode, function (data) {
                        this.provinces = data;
                        this.isProvincesLoading = false;
                    }.bind(this));
                }
            }
        });
    </script>
}

Wrapping it up

I hope this gives you a taste for how easy it is to work with Vue. My current thinking is that Vue should be the default choice for client side frameworks in existing ASP.NET MVC apps.

NOTE: This example uses ASP.NET MVC 5 to illustrate that Vue can be used with brownfield applications. It would be just as easy, if not easier, to use Vue in an ASP.NET Core project.

I wanted to install the site extension that enables Application Insights Monitoring of a live website. After digging into existing arm templates, I found the name of that extension is Microsoft.ApplicationInsights.AzureWebSites.

After searching for way too long, I eventually found PowerShell command I needed on a forum somewhere. I can't find it again so I'm posting this here in hopes that it will be easier for others to find in the future.

Installing a site extension to an existing App Service Web App

1

New-AzureRmResource -ResourceType "Microsoft.Web/sites/siteextensions" -ResourceGroupName MyResourceGroup -Name "MyWebApp/SiteExtensionName" -ApiVersion "2018-02-01" -Force

For example, given a resource group named Test, a web app named testsite and a site extension named Microsoft.ApplicationInsights.AzureWebSites.

1

New-AzureRmResource -ResourceType "Microsoft.Web/sites/siteextensions" -ResourceGroupName "Test" -Name "testsite/Microsoft.ApplicationInsights.AzureWebSites" -ApiVersion "2018-02-01" -Force

Installing a site extension to a Web App Deployment Slot

The scenario I ran into was actually attempting to add this site extension to a deployment slot. When you create a deployment slot, it doesn't copy over any existing site extensions, which is a problem because when you swap your new slot to production, your new production slot ends up losing the site extensions that were in the old production slot.

1

New-AzureRmResource -ResourceType "Microsoft.Web/sites/slots/siteextensions" -ResourceGroupName MyResourceGroup -Name "MyWebApp/SlotName/SiteExtensionName" -ApiVersion "2018-02-01" -Force

Using the same example as above and a slot named Staging:

1

New-AzureRmResource -ResourceType "Microsoft.Web/sites/slots/siteextensions" -ResourceGroupName "Test" -Name "testsite/Staging/Microsoft.ApplicationInsights.AzureWebSites" -ApiVersion "2018-02-01" -Force

Update: April 16, 2018 Something really cool happened in the comments. The amazing Phil Bolduc very kindly pointed out that the query I wrote was not optimal and as a result, my benchmarks were not showing the best results. He didn't stop there, he also submitted a pull request to the sample repo so I could rerun my benchmarks. Great job Phil and thanks a ton for being constructive in the comments section! I have updated the post to include Phil's superior query.

In today's post, we look at another option for how to load Many-to-One relationships. In the last post, we used a technique called Multi-Mapping to load related Many-to-One entities. In that post, I had a theory that maybe this approach was not the most efficient method for loading related entities because it duplicated a lot of data.

To recap, we would like to load a list of ScheduledFlight entities. A ScheduleFlight has a departure Airport and an arrival Airport.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

public class ScheduledFlight 
{
    public int Id {get; set;}
    public string FlightNumber {get; set;}

    public Airport DepartureAirport {get; set;}
    public int DepartureHour {get; set;}
    public int DepartureMinute {get; set;}

    public Airport ArrivalAirport {get; set;}        
    public int ArrivalHour {get; set;}
    public int ArrivalMinute {get; set;}

   //Other properties omitted for brevity 
}

public class Airport 
{
    public int Id {get; set;}
    public string Code {get; set;}
    public string City {get; set;}
    public string ProvinceState {get; set;}
    public string Country {get; set;}
}

Using Multiple Result Sets

In the previous post, we loaded the ScheduledFlight entities and all related Airport entities in a single query. In this example we will use 2 separate queries: One for the ScheduledFlight entities, one for the related arrival and departure Airport entities. These 2 queries will all be executed as a single sql command that returns multiple result sets.

1
2
3
4
5
6
7
8
9
10
11
12
13

SELECT s.Id, s.FlightNumber, s.DepartureHour, s.DepartureMinute, s.ArrivalHour, s.ArrivalMinute, s.IsSundayFlight, s.IsMondayFlight, s.IsTuesdayFlight, s.IsWednesdayFlight, s.IsThursdayFlight, s.IsFridayFlight, s.IsSaturdayFlight,
s.DepartureAirportId, s.ArrivalAirportId
FROM ScheduledFlight s
INNER JOIN Airport a1
ON s.DepartureAirportId = a1.Id
    WHERE a1.Code = @FromCode
    
SELECT Airport.Id, Airport.Code, Airport.City, Airport.ProvinceState, Airport.Country
FROM Airport
  WHERE Airport.Id = @DepartureAirportId
    OR Airport.Id IN (SELECT s.ArrivalAirportId
    FROM ScheduledFlight s
WHERE s.DepartureAirportId = @DepartureAirportId)

Using Dapper's QueryMultipleAsync method, we pass in 2 arguments: the query and the parameters for the query.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

public async Task<IEnumerable<ScheduledFlight>> GetAlt(string from)
{
  IEnumerable<ScheduledFlight> scheduledFlights;
  using (var connection = new SqlConnection(_connectionString))
  {
  await connection.OpenAsync();
  var query = @"
SELECT s.Id, s.FlightNumber, s.DepartureHour, s.DepartureMinute, s.ArrivalHour, s.ArrivalMinute, s.IsSundayFlight, s.IsMondayFlight, s.IsTuesdayFlight, s.IsWednesdayFlight, s.IsThursdayFlight, s.IsFridayFlight, s.IsSaturdayFlight,
s.DepartureAirportId, s.ArrivalAirportId
FROM ScheduledFlight s
INNER JOIN Airport a1
ON s.DepartureAirportId = a1.Id
    WHERE a1.Code = @FromCode
    
SELECT Airport.Id, Airport.Code, Airport.City, Airport.ProvinceState, Airport.Country
FROM Airport
WHERE Airport.Id = @DepartureAirportId
  OR Airport.Id IN (SELECT s.ArrivalAirportId
       FROM ScheduledFlight s
   WHERE s.DepartureAirportId = @DepartureAirportId)";

    using (var multi = await connection.QueryMultipleAsync(query, new{FromCode = from} ))
    {
        scheduledFlights = multi.Read<ScheduledFlight>();
        var airports = multi.Read<Airport>().ToDictionary(a => a.Id);
        foreach(var flight in scheduledFlights)
        {
            flight.ArrivalAirport = airports[flight.ArrivalAirportId];
            flight.DepartureAirport = airports[flight.DepartureAirportId];
        }

    }
  }
  return scheduledFlights;
}

The QueryMultipleAsync method returns a GridReader. The GridReader makes it very easy to map mutliple result sets to different objects using the Read<T> method. When you call the Read<T> method, it will read all the results from the next result set that was returned by the query. In our case, we call Read<ScheduledFlight> to read the first result set and map the results into a collection of ScheduledFlight entities. Next, we call Read<Airport> to read the second result set. We then call ToDictionary(a => a.Id) to populate those Airport entities into a dictionary. This is to make it easier to read the results when setting the ArrivalAirport and DepartureAirport properties for each ScheduledFlight.

Finally, we iterate through the scheduled flights and set the ArrivalAirport and DepartureAirport properties to the correct Airport entity.

The big difference between this approach and the previous approach is that we no longer have duplicate instances for Airport entities. For example, if the query returned 100 scheduled flights departing from Calgary (YYC), there would be a single instance of the Airport entity representing YYC, whereas the previous approach would have resulted in 100 separate instances of the Airport entity.

There is also less raw data returned by the query itself since the columns from the Airport table are not repeated in each row from the ScheduleFlight table.

Comparing Performance

I had a theory that the multi-mapping approach outlined in the previous blog post would be less efficient than the multiple result set approach outlined in this blog post, at least from a memory usage perspective. However, a theory is just theory until it is tested. I was curious and also wanted to make sure I wasn't misleading anyone so I decided to test things out using Benchmark.NET. Using Benchmark.NET, I compared both methods using different sizes of data sets.

I won't get into the details of Benchmark.NET. If you want to dig into it in more detail, visit the official site and read through the docs. For the purposes of this blog post, the following legend should suffice:

1
2
3
4
5
6
7

Mean      : Arithmetic mean of all measurements
Error     : Half of 99.9% confidence interval
StdDev    : Standard deviation of all measurements
Gen 0     : GC Generation 0 collects per 1k Operations
Gen 1     : GC Generation 1 collects per 1k Operations
Gen 2     : GC Generation 2 collects per 1k Operations
Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)

10 ScheduledFlight records

As I suspected, the difference is minimal when dealing with small result sets. The results here are in microseconds so in both cases, executing the query and mapping the results takes less 1/2 a millisecond. The mutliple result sets approach takes a little longer, which I kind of expected because of the overhead of creating a dictionary and doing lookups into that dictionary when setting the ArrivalAirport and DepartureAirport properties. The difference is minimal and in a most real world scenarios, this won't be noticable. What is interesting is that even with this small amount of data, we can see that there is ~1 more Gen 0 garbage collection happening per 1,000 operations. I suspect we will see this creep up as the amount of data increases.

100 ScheduledFlight records

When mapping 100 results, the multiple result sets query is already almost 25% faster. Keep in mind though that both cases are still completing in less than 1ms so this is very much still a micro optimization (pun intented). Either way, less than a millsecond to map 100 records is crazy fast.

1000 ScheduledFlight records

Here we go! Now the multiple result sets approach finally wins out, and you can see why. There are way more Gen 0 and Gen 1 garbage collections happening per 1,000 operations when using the multi-mapping approach. As a result, the multiple result sets approach is nearly twice as fast as the multi mapping approach.

10,000 ScheduledFlight records

One last test with 10,000 records shows a more substantial difference. The multiple result sets approach is a full 22ms faster!

Wrapping it up

I think that in most realistic scenarios, there is no discernable difference between the 2 approaches to loading many-to-one related entities. If you loading larger amounts of records into memory in a single query, then the multiple result sets approach will likely give you better performance. If you are dealing with < 100 records per query, then you likely won't notice a difference. Keep in mind also that your results will vary depending on the specific data you are loading.

In today's post, we will start our journey into more complex query scenarios by exploring how to load related entities. There are a few different scenarios to cover here. In this post we will be covering the Many-to-One scenario.

Continuing with our sample domain for the ever expanding Air Paquette airline, we will now look at loading a list of ScheduledFlight entities. A ScheduleFlight has a departure Airport and an arrival Airport.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

public class ScheduledFlight 
{
    public int Id {get; set;}
    public string FlightNumber {get; set;}

    public Airport DepartureAirport {get; set;}
    public int DepartureHour {get; set;}
    public int DepartureMinute {get; set;}

    public Airport ArrivalAirport {get; set;}        
    public int ArrivalHour {get; set;}
    public int ArrivalMinute {get; set;}

   //Other properties omitted for brevity 
}

public class Airport 
{
    public int Id {get; set;}
    public string Code {get; set;}
    public string City {get; set;}
    public string ProvinceState {get; set;}
    public string Country {get; set;}
}

Side Note: Let's ignore my poor representation of the arrival and departure times of the scheduled flights. In a future most we might look using Noda Time to properly represent these values.

Loading everything in a single query

Using Dapper, we can easily load a list of ScheduledFlight using a single query. First, we need to craft a query that returns all the columns for a ScheduledFlight, the departure Airport and the arrival Airport in a single row.

1
2
3
4
5
6
7
8

SELECT s.Id, s.FlightNumber, s.DepartureHour, s.DepartureMinute, s.ArrivalHour, s.ArrivalMinute, s.IsSundayFlight, s.IsMondayFlight, s.IsTuesdayFlight, s.IsWednesdayFlight, s.IsThursdayFlight, s.IsFridayFlight, s.IsSaturdayFlight,
       a1.Id, a1.Code, a1.City, a1.ProvinceState, a1.Country,
   a2.Id, a2.Code, a2.City, a2.ProvinceState, a2.Country
FROM ScheduledFlight s
INNER JOIN Airport a1
ON s.DepartureAirportId = a1.Id
INNER JOIN Airport a2
ON s.ArrivalAirportId = a2.Id

We use the QueryAsync method to load a list of ScheduledFlight entities along with their related DepartureAirport and ArrivalAirport entities. The parameters we pass in are a little different from what we saw in our previous posts.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32