The cloud make this so much easier. We could spin up a server in minutes and with platform as a service we didn't have to worry about underlying compute or updates to operating systems, or patching. It was a dream. I'd tell clients "You're an insurance company leave being an IT company to Microsoft/Amazon/Google".

Recently though I've been looking at the cost of hosting an application in Azure. Even for our small applications it is surprisngly expensive. We had a tendency to vastly overbuild the hosting when for applications that have only hundreds or low thousands of users. What really gets us is the cost of the database.

This isn't a huge or highly performant database, but it is still costing us almost $400/month. For a small application with only a few hundred users this is a huge cost. Add on dev and test environments and the cost is even higher. Painful.

I wanted to understand if I was still recommending the right thing to people so I started looking at alternatives. My first stop was looking at a VPS provider. I wanted something in Canada becasue this is where I live and I wanted to support local businesses. I found a provider called OVH that had a VPS with 8 vCores, 24 Gig of memory and a 200 Gig SSD for $32 a month off contract.

Now obviously vCores aren't a standard measurement unit so it's hard to compares that with something from another offering like Azure but on the surface this feels similar to a D12 from Azure which is $380/month and that's in USD so closer to $500/month CAD. The OVH offering is about 1/15th the cost of Azure.

Yikes!

So I figured I'd sign up. Maybe the added cost here would be becasue it was really arduous to get signed up. I'll admit going through the process wasn't as polished as a cloud provier. Little things like this

Is that a last name field or a field for people who only have a single name? Is Madonna buying hosting a frequently? The 2FA setup, though, was probably the best I've ever done. It had the really nice feature of letting you record the name of the app you were using for 2FA. I have multiple apps for 2FA and this is a really nice feature.

I did spot this warning, though

Okay so there might be some latency in getting this server provisioned. That sucks, but perhaps I've gotten too used to the instant gratificaiton of demanding compute and getting it 5 minutes later. The email I got said that the server would be provisioned within the next 7 days. That's a long time to wait but whilte we do let's talk about why this VPS might not be a great idea. What are we losing by not using cloud hosting?

Well first off it's clear that scaling up and down is going to be so much harder. I can't scale the machine up instantly when there is a spike in traffic. Equally, if that spike subsites I can't scale down quickly. So I'm going to have to be comfortable knowing that I've got to pay for excess capacity. A lot of the line of business applications I work with have almost no traffic on the weekend and evenings but then predictable stable load during the day. I'm going to have to over provision to handle the maximum load. Maybe I can mine bitcoin or something on the weekends. Is that SETI@home project still a thing?

Next I'm losing the platform as a service benefits. I'm going to have to patch the operating system and database myself. I get to do backups, restores all that myself. So I'm going to trade my time for reduced hosting costs. Let's imagine that I'm setting up a database server and a web server which is kind of typical for a line of business application. This is going to run to $700/month in Azure(I'm going to use Candian dollars from here out). So that gives me $650 a month to play with for manual patching and updating and all that jazz. If I pay myself $200/hour (woo, massive pay raise) then I can spend 3 hours a month on that work and still come out ahead. I don't know if that scales well to dozens of servers but for a small application it might be worth it.

I'm likely going to have an increased cost for the initial set up of the server. I've got to figure out which web server to use, probably some default settings for PostgreSQL and tie it into a build pipeline. I suppose we'll be deploying via SFTP or something like that.

Finally, I'm losing the reliability and security of the cloud. I don't have the same level of redundancy and failover that I would get with a cloud provider. If my server goes down, it's on me to get it back up. If there is a security breach, it's on me to fix it. This is a big risk for any business, but especially for small businesses that may not have the resources to deal with these issues.

The other piece of the puzzle here is that the knowledge and expertise required to manage a VPS is higher than that required to use a cloud provider. I've been doing this computer stuff for year and I'm confident in my ability to teasily manage this but then I'm also still a bit sad about how Open Solaris failed to take off. I think that a lot of this gap can be closed using AI agents these days.

It's a calculated risk.

So, dear readers, I pulled the trigger and signed up for the VPS. Once it is provisioned then we'll return here and talk through setting up and deploying to it.

• ClearCase
• Perforce
• Subversion
• CVS
• Visual Source Safe
• Mercurial
• Git
• CCC/Harvest

I'm using Jujutsu at my day job now because it just layers transparently on top of git so I don't need to go seeking permission. I'm only a few days into using it and I'm not thoroughly convinced yet that it is better than git but I'm willing to keep trying.

Here are some of the commands I'm using so far:

Get the latest version of the code from a central repository locally

1

jj git fetch

Start new work from the latest mainline

1

jj new main@origin -m "Whatever I'm going to work on"

Bookmark the work with a name I'm going to use as a branch in git

1

jj bookmark my-feature-branch

Push my work up to Github

1

jj git push --allow-new

Create a new commit before my current one that I can squash into

1

jj new -B @ -m "Some description of the work"

Push individual files into the parent change

1

jj squash path/to/file1 path/to/file2

I'll keep expanding this document with new commands as I uncover them.

The first thing to do is to install the OpenAPI generator. It's written in Java which is obviously a decision I'm solidly against. As I run a Mac most of the time I prefer to use brew since it's easier that trying to figure out how to build stuff with Maven.

1

brew install openapi-generator

Now comes the fun part: figuring out the options to use. There are bunch of different generators for different languages and on top of that each generator has options. The C# generator is called csharp and the options are described in some detail here https://openapi-generator.tech/docs/generators/csharp. In my case I was looking to generator a client for a US Government API that they seem to be really snippy about people getting their hands on documentation about it without going through a heap of hoops so we'll just call it USGovAPI because this administration is not one I want to be on the wrong side of.

In addition I wanted to generate one for .NET 8 because the project is still running on the long term support version of .NET. So the command ended up being

1

openapi-generator generate -i swagger.json -g csharp -o out/usgoveapi --additional-properties=packageName=USGovAPI,targetFramework=net8.0

And with that we get a nice little C# client that we can use to call the USGovAPI. It even includes some unit tests to go along with it.

Fixing this is easy enough. I needed to add a couple of lines to my dependabot file to limit the sorts of updates it did to just minor and patch updates. Notice the update-types section.

1
2
3
4
5
6
7
8
9
10
11
12
13

updates:
  - package-ecosystem: "nuget"
    directory: "/."
    groups:
      all_packages:
        update-types:
          - "minor"
          - "patch"
        patterns:
          - "*"
    open-pull-requests-limit: 50
    schedule:
      interval: "weekly"

With this in place dependabot is only proposing minor and patch updates to my .NET packages. It does mean that if we see other major version updates to non-Microsoft packages we'll have to manually update them.

I tried overriding the entry point for the container and attaching to it to see what was going on but I couldn't see anything wrong. I was able to write to the directory without issue. Eventually I stumbled on a note in the RavenDB documentation which mentioned a change in the 6.x version of RavenDB which meant that Raven no longer ran as root inside the container.

K8S has the ability to change the ownership of the volume to the user that the container is running as. This is done by setting the fsGroup property in the pod spec. In this case Raven runs as UID 999. So I updated my tanka spec to include the fsGroup property and the problem was solved.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

...
deployment: deployment.new($._config.containers.ravendb.name) {
      metadata+: {
        namespace: 'wigglepiggle-' + $._config.environment,
        labels: {
          app: $._config.containers.ravendb.name,
        },
      },
      spec+: {
        replicas: 1,
        selector: {
          matchLabels: $._config.labels,
        },
        template: {
          metadata+: {
            labels: $._config.labels,
          },
          spec+: {
            securityContext: {
              fsGroup: 999,
              fsGroupChangePolicy: 'OnRootMismatch',
            },
            containers: [
              {
                name: $._config.containers.ravendb.name,
                image: $._config.containers.ravendb.image,
                ports: [{ containerPort: 8080 }, { containerPort: 38888 }],
                volumeMounts: [
                  {
                    name: 'data',
                    mountPath: '/var/lib/ravendb/data',
                  },
                ],
                env: [
                  {
                    name: 'RAVEN_Setup_Mode',
                    value: 'None',
                  },
                  {
                    name: 'RAVEN_License_Eula_Accepted',
                    value: 'true',
                  },
                  {
                    name: 'RAVEN_ARGS',
                    value: '--log-to-console',
                  },
                  {
                    name: 'RAVEN_Security_UnsecuredAccessAllowed',
                    value: 'PrivateNetwork',
                  },
                ],
              },
            ],
            volumes: [
              {
                name: 'data',
                persistentVolumeClaim: {
                  claimName: $._config.containers.ravendb.name,
                },
              },
            ],
          },
        },
      },
    },
...

This generated yml like

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: service-control-ravendb
  name: service-control-ravendb
  namespace: wigglepiggle-dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: service-control
      environment: dev
  template:
    metadata:
      labels:
        app: service-control
        environment: dev
    spec:
      containers:
      - env:
        - name: RAVEN_Setup_Mode
          value: None
        - name: RAVEN_License_Eula_Accepted
          value: "true"
        - name: RAVEN_ARGS
          value: --log-to-console
        - name: RAVEN_Security_UnsecuredAccessAllowed
          value: PrivateNetwork
        image: ravendb/ravendb:6.0-latest
        name: service-control-ravendb
        ports:
        - containerPort: 8080
        - containerPort: 38888
        volumeMounts:
        - mountPath: /var/lib/ravendb/data
          name: data
      securityContext:
        fsGroup: 999
        fsGroupChangePolicy: OnRootMismatch
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: service-control-ravendb

1

docker logs -f --tail 10 5efd3ee0074a

There in the logs was the culprit No space left on device. Uh oh, what's going on here? Sure enough the disk was full.

1

df -h

1
2
3
4
5
6
7

Filesystem                         Size  Used Avail Use% Mounted on
tmpfs                              1.6G  1.9M  1.6G   1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv   96G   93G     0 100% /
tmpfs                              7.8G     0  7.8G   0% /dev/shm
tmpfs                              5.0M     0  5.0M   0% /run/lock
/dev/sda2                          2.0G  182M  1.7G  10% /boot
tmpfs                              1.6G   12K  1.6G   1% /run/user/1001

This stuff is always annoying because you get to a point where you can't run any commands because there is no space left. I started with cleaning up some small parts of docker

docker system prune -a

Then cleaned up docker logs

sudo truncate -s 0 /var/lib/docker/containers/**/*-json.log

This then gave me enough space to run docker system df and see where the space was being used. Containers were the culprit. So next was to run

docker ps --size

Which showed me the web scraper container had gone off the rails and was using over a 100GiB of space.

1

7cf14084c56a   webscraper:latest       "wsce start -v -y"       7 weeks ago   Up 20 minutes               0.0.0.0:7002->9924/tcp, [::]:7002->9924/tcp                                webscraper       123GB (virtual 125GB)

This thing is supposed to be stateless so I just killed an removed it.

1
2
3

docker kill 7cf14084c56a
docker rm 7cf14084c56a
docker compose up -d webscraper

After a few minutes these completed and all was good again. So we'll keep an eye on that service and perhaps reboot it every few months to keep it in check.

1
2
3
4
5
6
7
8
9

nodeLinker: node-modules

npmScopes:
  stimms:
    npmRegistryServer: "https://npm.pkg.github.com"
    
npmRegistries:
  "https://npm.pkg.github.com":
    npmAlwaysAuth: true

Now in the build I needed to add a step in to populate the GITHUB_TOKEN which can be used for authentication. I found quite a bit of documentation which suggested that the .yarnrc.yml file would be able to read the environment variable but I had no luck with that approach. Instead I added a step to the build to populate the GITHUB_TOKEN in the .npmrc file.

1
2
3
4
5
6
7
8
9

- name: Configure GitHub Packages Auth
    run: echo "//npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}" > ~/.npmrc
    env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Build
    run: |
    yarn install
    yarn lint
    yarn build

The final thing to remember is that by default the GITHUB_TOKEN here doesn't have read permission over your packages. You'll need to go into the package settings and add the repository to the list of repositories which can use the package. You just need read access. If you don't do this step you're going to see an error like error Error: https://npm.pkg.github.com/@stimms%2fuicomponents: authentication token not provided

Install it with

1

dotnet tool install --global dotnet-outdated

Then you can run it in the root of your project to list the packages which will be updated with

1

dotnet outdated

Then, if you're happy, run it again with

1

dotnet outdated -u

to actually get everything updated.

• Kata 1 - Sending a message
• Kata 2 - Publishing a message
• Kata 3 - Switching transports
• Kata 4 - Long running processes
• Kata 5 - Timeouts
• Kata 6 - When things go wrong

NServiceBus is built to be reliable. It is designed to be able to handle errors and to be able to recover from them. The most common reason for an error to show up in NServiceBus is that a message fails to process. This could be because the message is malformed, a resource which is needed to process the message is unavailable or because the handler throws an exception. Throwing exceptions in handlers is actually the preferred way to handle errors in NServiceBus.

When a message fails to process NServiceBus will attempt to reprocess the message. NServiceBus defines two different retry mechanisms: immediate and delayed retry. Immediate retries will reprocess at once and by default happen 5 times. Delayed retries fire at 10, 20 and 30 second after the failed retries and they too will fire immediate retries. So by the time a message has fully failed it's been attempted 20 times. If all these attempts fail then the message is shuffled off to an error queue. This error queue is not typically watched by NService Bus and recovering a message form it is a manual process.

In NServiceBus installations I've worked on we monitor the error queues fairly closely and work to make the application resilient to errors such that the queues are only populated by messages which have failed due to resources being unavailable. Don't just ignore these messages as they are likely important user interactions or business processes which will cost you money if you don't address them.

The Kata

Let's make one of our messages fail to process and watch what happens. In this scenario let's fail to properly sanitize our inputs to the messages and have them take on a value which makes no sense. We have a message EatCake which has a property NumberOfCakes. Let's make it so that the NumberOfCakes is set to -1 and that we have a validation in place in our message handler to throw when we see this problem. Let's also log a message so we can see the retries happening.

My Solution

1. Modify the EatCake message hander to have a validation to make sure cakes are not eaten in the negative.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

public async Task Handle(EatCake message, IMessageHandlerContext context)
    {
        if (message.NumberOfCakes < 0)
        {
            Console.WriteLine("Negative cake? Really?");
            throw new Exception("Negative cake? Really?");
        }
        if (message.NumberOfCakes == 0)
        {
            Console.WriteLine("No cake to eat");
            throw new Exception("No cake to eat");
        }
        
        Console.WriteLine($"Cake eaten, NumberOfCakes = {message.NumberOfCakes}; Flavour = {message.Flavour}");
        await context.Publish(new CakeEaten
        {
            EatingFinishedAt = DateTime.Now
        });
    }

2. Modify the sender to send a message with a negative number of cakes

1
2
3
4
5
6
7
8
9
10

switch (line)
{
    case "send":
        await endpointInstance.Send("NServiceBusKataReceiver", new EatCake
        {
            Flavour = "Coconut",
            NumberOfCakes = -1
        });
        break;
        ...

3. You may need to configure the error queue in the receiver endpoint.

1

endpointConfiguration.SendFailedMessagesTo("NServiceBusKataReceiverError");

And build the error queue using the handy dandy rabbitmq-transport tool

1

rabbitmq-transport endpoint create NServiceBusKataAnotherReceiver -c "host=localhost" --errorQueue NServiceBusKataAnotherReceiverError

Things to try now

1. Run the sender and produce some messages with negative numbers of cakes
2. Observe the logs for the receiver and see the retries happening
3. Wait a bit for the retries to be exhausted and see the message end up in the error queue
4. Open up the rabbitmq management console and look at the error queue. Pay special attention to the headers of the message. You'll see the number of retries and the time the message was first sent as well as the exception message.

Things to think about

Looking at messages in the RabbitMQ management console is great for a single message but in the aggregate there is often a need for better tooling. Particular provide some tooling in the form of ServiceInsight which can be used to monitor the flow of messages through the system. This can be used to see where messages are failing and to help diagnose the problem.

• Kata 1 - Sending a message
• Kata 2 - Publishing a message
• Kata 3 - Switching transports
• Kata 4 - Long running processes
• Kata 5 - Timeouts
• Kata 6 - When things go wrong

I think we're pretty used to the idea that a timeout is something we want to avoid - it usually means that a server isn't available or that we've run a query that is taking too long. But there are lots of places in business processes where timeout are just part of the process we're modeling and aren't an error at all. As an example consider a shopping cart: if a user adds an item to the cart and then doesn't check out within a certain period of time we might want to send them a reminder and even offer them a discount on their purchases. If we've modeled the checkout process as a saga then this reminder can be set up as a timeout.

The Kata

We just implemented a cake order saga in the last kata. Let's extend that saga to include a timeout. If the cake isn't shipped within 2 minutes of the order being placed then we should send a message to the bakery to ask them to check on the order. Obviously 2 minutes is a pretty quick turn around on a cake but I don't imagine you want to hang around for hours writing this kata.

My Solution

1. Add a new message to the messages project

1
2
3
4
5
6
7
8

namespace messages;

using NServiceBus;

public class CakeOrderStalled : IMessage
{
    public Guid OrderId { get; set; }
}

2. Modify the saga to start a timeout when an order is placed. The CakeOrderPlaced handler becomes

1
2
3
4
5
6

public Task Handle(CakeOrderPlaced message, IMessageHandlerContext context)
{
    Console.WriteLine($"Order {message.OrderId} placed");
    
    return RequestTimeout<CakeOrderStalled>(context, TimeSpan.FromMinutes(2), new CakeOrderStalled{ OrderId = message.OrderId });
}

3. Add a handler for the timeout message

1
2
3
4
5

public Task Timeout(CakeOrderStalled state, IMessageHandlerContext context)
{
    Console.WriteLine($"Order {Data.OrderId} stalled!");
    return Task.CompletedTask;
}

4. Have the saga implement

1

IHandleTimeouts<CakeOrderStalled>

4. We missed it previously but we need to have canceling the order or completing the order mark the saga as complete. To do that add MarkAsComplete to the handlers for those messages

1
2
3
4
5
6

public Task Handle(CakeOrderCanceled message, IMessageHandlerContext context)
{
    Console.WriteLine($"Order {message.OrderId} canceled");
    MarkAsComplete();
    return Task.CompletedTask;
}

Things to try now

1. Run the applications and in the sender app try starting the saga with o to start an order. Wait a couple of minutes and see that the timeout fires.
2. Try starting the saga and then canceling it with o followed by c and see what happens when the timeout fires.

Things to think about

Once a timeout is registered there is no way to cancel it. How come? What would you do if you needed to cancel a timeout? If messages are delayed is it possible that timeouts could fire when an expected action has actually happened? How would you mitigate that risk?

• Kata 1 - Sending a message
• Kata 2 - Publishing a message
• Kata 3 - Switching transports
• Kata 4 - Long running processes
• Kata 5 - Timeouts
• Kata 6 - When things go wrong

Consider the bakery which is creating the cake for me to eat: when it starts making a new cake because I ate the last one it has a bunch of things it needs to coordinate. They need to preheat ovens, gather ingredients, mix ingredients, grease pans, fill pans, put pans in the oven, remember to take the cake out, cool it, ice it... the list goes on and on - no wonder cake is so expensive. Coordinating all these activities is complex in a distributed system, really in any system. There are a lot of corner cases that we usually fail to consider in a non-distributed system which become much more apparent when building out a process manager. What if we preheat the oven but then discover that we're all out of flour? In that monolithic system we might throw an exception and hope that somebody is monitoring for it in a log file somewhere. Realistically that's never going to happen. In the meantime nobody has shut the oven off and the bakery burns down.

A saga allows us to store the state of a process, to react to messages as they come in and send new messages. We use this to coordinate the activities of the bakery. In our example above we can probably call the process "BakeCakeSaga". When messages come in relating to the order then we need to be able to find a way to look up the state and make modifications to it. NServiceBus implements this through a method called ConfigureHowToFindSaga. This function will provide a mapping from every message that interacts with the saga to find the saga data. For our example we'd probably use something like an order id.

Let's build out a very simple saga which responds to just a few messages in our system so we can see how it works. Saga can get pretty complex but they are quite testable so that's nice.

The Kata

Create a saga which handles the messages CakeOrderPlaced, CakeOrderCanceled, CakeOrderShipped. Each of these messages will contain an OrderId, a GUID, which will be used to identify the saga as well as whatever information might be associated with those messages. For now just write out to the console when each of these messages is received - unless you want to bake me a cake which I will accept.

The Solution

1. Add some mechanism to handle the persistence of saga data. For now we'll just use the in learning persistence. In the various program.cs files add

1

var persistence = endpointConfiguration.UsePersistence<LearningPersistence>();

2. Create messages classes in the messages project

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

namespace messages;

using NServiceBus;

public class CakeOrderPlaced : IEvent
{
    public Guid OrderId { get; set; }
    public Guid CustomerId { get; set; }
    public Date OrderDate { get; set; }
}

public class CakeOrderCanceled : IEvent
{
    public Guid OrderId { get; set; }
    public string Reason { get; set; }
}

public class CakeOrderShipped : IEvent
{
    public Guid OrderId { get; set; }
    public string ShippingReferenceNumber { get; set; }
}

3. Create a saga class in the receiver project

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

using NServiceBus;
using messages;
public class CakeOrderSaga : Saga<CakeOrderSagaData>,
    IAmStartedByMessages<CakeOrderCanceled>,
    IAmStartedByMessages<CakeOrderPlaced>,
    IAmStartedByMessages<CakeOrderShipped>
{

    protected override void ConfigureHowToFindSaga(SagaPropertyMapper<CakeOrderSagaData> mapper)
    {
        mapper.MapSaga(sagaData => sagaData.OrderId)
            .ToMessage<CakeOrderCanceled>(x => x.OrderId)
            .ToMessage<CakeOrderPlaced>(x => x.OrderId)
            .ToMessage<CakeOrderShipped>(x => x.OrderId);
    }

    public Task Handle(CakeOrderPlaced message, IMessageHandlerContext context)
    {
        Console.WriteLine($"Order {message.OrderId} placed");
        return Task.CompletedTask;
    }

    public Task Handle(CakeOrderCanceled message, IMessageHandlerContext context)
    {
        Console.WriteLine($"Order {message.OrderId} canceled");
        Data.OrderCanceled = true;
        return Task.CompletedTask;
    }

    public Task Handle(CakeOrderShipped message, IMessageHandlerContext context)
    {
        Console.WriteLine($"Order {message.OrderId} shipped");
        Data.OrderShipped = true;
        return Task.CompletedTask;
    }

}

4. Add a saga data class to the receiver project

1
2
3
4
5
6
7
8

using NServiceBus;

public class CakeOrderSagaData : ContainSagaData
{
    public Guid OrderId { get; set; }
    public bool OrderCanceled { get; set; }
    public bool OrderShipped { get; set; }
}

5. Modify the sender project's program.cs to send the messages adding a loop which sends all the different messages involved in the saga.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

bool continueMessages = true;
Guid orderId = Guid.NewGuid();

while (continueMessages)
{
    var line = Console.ReadLine();
    switch (line)
    {
        case "p":
            await endpointInstance.Publish(new CakeOrderPlaced { OrderId = orderId });
            break;
        case "c":
            await endpointInstance.Publish(new CakeOrderCanceled { OrderId = orderId });
            break;
        case "s":
            await endpointInstance.Publish(new CakeOrderShipped { OrderId = orderId });
            break;
        case "q":
            continueMessages = false;
            break;
        default:
            break;
    }

}

Things to try now

1. Run the applications and in the sender app try pressing some keys like p or c or s to see the messages being handled by the saga.
2. Try starting the applications in different orders and see how the saga handles the messages.

Things to Notice

Notice that the Saga can be started by 3 different events. Why would a saga be started by a cancel message? You can't cancel an order which hasn't even been placed yet - right? Well it turns out you can. Without some serious hoop jumping through the order of message delivery it not guaranteed. So in fact orders can be canceled or shipped before we get the message telling us the order has been placed. It is sort of mind-blowing.

• Kata 1 - Sending a message
• Kata 2 - Publishing a message
• Kata 3 - Switching transports
• Kata 4 - Long running processes
• Kata 5 - Timeouts
• Kata 6 - When things go wrong

One of the really nice things about NServiceBus is that it abstracts away a lot of the mess of dealing with transports. You can spend less time fiddling with plumbing and more time getting to the nitty gritty of building business value. We have a lot of options for transport and the ability to write new ones if needed. The official supported transports are

• Azure Service Bus
• Azure Storage Queues
• Amazon SQS
• RabbitMQ
• SQL Server
• MSMQ

Each of these has advantages and disadvantages which you can read about in some detail in the documentation. In a production environment I'd push heavily towards Azure Service Bus but it has a major disadvantage: no local emulator. There is a github issue open to add an emulator which has been open since 2018. Normally I'd grumble a bit about how this is never going to be solved but actually the team has dedicated resources to building one out and think they'll have one ready by the end of 2024. Only you, dear readers from the future, know if this came to fruition.

For now we're going to take advantage of the interchangeability of transports and switch out the LearningTransport for the RabbitMQ transport. This can be run with a container on docker.

The Kata

Take the solution developed already and switch out the learning transport for the RabbitMQ transport running inside of a container. Everything should continue to work as it did before but now using RabbitMQ.

My Solution

1. Download and run the RabbitMQ container

1

docker run -d --hostname nsbkata --name nsbkata-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-management

2. Install the RabbitMQ transport package in all the projects

1
2
3

dotnet add sender package NServiceBus.RabbitMQ
dotnet add receiver package NServiceBus.RabbitMQ
dotnet add anotherReceiver package NServiceBus.RabbitMQ

3. Modify the endpoint configurations across all the projects swapping

1

var transport = endpointConfiguration.UseTransport<LearningTransport>();

with

1
2
3

var transport = endpointConfiguration.UseTransport<RabbitMQTransport>();
transport.ConnectionString("host=localhost");
transport.UseConventionalRoutingTopology(QueueType.Quorum);

4. It is generally best that you take control of creating queues for NServiceBus but tooling is provided. Install that tooling

1

dotnet tool install -g NServiceBus.Transport.RabbitMQ.CommandLine

Next create the queues for the endpoints (and the delays queues)

1
2
3
4

rabbitmq-transport delays create  -c "host=localhost"
rabbitmq-transport endpoint create NServiceBusKataAnotherReceiver -c "host=localhost"
rabbitmq-transport endpoint create NServiceBusKataReceiver -c "host=localhost"
rabbitmq-transport endpoint create NServiceBusKataSender -c "host=localhost"

Things to try now

1. Run the sender, receiver and anotherReceiver projects. You should see the messages being sent and received as before.

2. Log into the RabbitMQ management console(running at http://localhost:15672/ with credentials guest/guest) you should see the queues created and in fact a message processed

• Kata 1 - Sending a message
• Kata 2 - Publishing a message
• Kata 3 - Switching transports
• Kata 4 - Long running processes
• Kata 5 - Timeouts
• Kata 6 - When things go wrong

The Problem

It is great being able to send a message from one system to another - you can instruct a remote system to take some action which you don't know how to do. For instance sending an email. In a large system lots of processes will likely result in an email but if you can centralize the logic around how to send an email it makes it very easy to do things like change email providers as that functionality is isolated and independent.

If you looked at my solution for the first Kata then you might have notice that I named my message EatCake this is named in the imperative form. This is a common pattern in messaging systems and is, I think, remarkably helpful to understand the command pattern. When you issue a command you know exactly who or what you're commanding and in the same way the command is sent to a known endpoint.

Sometimes, however, you don't have a particular target in mind and you just want to let other systems know that you have done something. In this case you can publish a message. Unlike a command you don't know who is going to act on it - it may be nobody or it may be multiple people. In a messaging system we call this an Event.

Let's extend our cake eating example to imagine some consequences of what might happen if I happily reported that I had eaten cake: CakeEaten. Finding out that the cake had been eaten the bakery might start baking more cake assuming that I'd want more (very likely). My gym might free up a treadmill knowing that my cake-induced guilt might drive me to run until I'd burned off the delicious butter-cream. It might also be of interest to my wife who can text me and see if I enjoyed the cake. I don't actually know who would be interested in the event so it is the responsibility of those who are interested to subscribe to the event.

The Kata

Using the solution to the first kata as a starting point extend the system to publish a CakeEaten message. Subscribe to the message in the sender and write out a message to the console. Also create another new project similar to the others and have it subscribe to the message.

My Solution

1. Create a new message in the messages assembly

1
2
3
4
5
6
7
8

namespace messages;

using NServiceBus;

public class CakeEaten : IEvent
{
    public DateTime EatingFinishedAt { get; set; }
}

2. Modify the Hander in the receiver to publish the new event. Notice that we have access to a IMessageHandlerContext which we can use to publish messages.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

using messages;

public class PlaceOrderHandler :
    IHandleMessages<EatCake>
{
    public async Task Handle(EatCake message, IMessageHandlerContext context)
    {
        Console.WriteLine($"Cake eaten, NumberOfCakes = {message.NumberOfCakes}; Flavour = {message.Flavour}");
        await context.Publish(new CakeEaten
        {
            EatingFinishedAt = DateTime.Now
        });
    }
}

3. Modify the sender to subscribe to the event by adding a Handler to the project.

1
2
3
4
5
6
7
8
9
10
11
12
13

using messages;

using NServiceBus;

public class CakeEatenHandler :
    IHandleMessages<CakeEaten>
{
    public Task Handle(CakeEaten message, IMessageHandlerContext context)
    {
        Console.WriteLine($"Uh oh, somebody ate a cake at {message.EatingFinishedAt}");
        return Task.CompletedTask;
    }
}

4. Optionally modify the sender to remain running so that it can receive the message. This is done by adding a Console.ReadLine() at the end of the Main method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

using NServiceBus;

Console.Title = "NServiceBusKata - Sender";

var endpointConfiguration = new EndpointConfiguration("NServiceBusKataReceiver");

// Choose JSON to serialize and deserialize messages
endpointConfiguration.UseSerialization<SystemJsonSerializer>();

var transport = endpointConfiguration.UseTransport<LearningTransport>();

var endpointInstance = await Endpoint.Start(endpointConfiguration);

Console.WriteLine("Press Enter to exit...");
Console.ReadLine();

await endpointInstance.Stop();

Pause here and ensure that when running the sender and receiver that you see both a message sent and published.

5. Create a new project to act as a second subscriber or receiver

1
2
3

dotnet new console -o anotherReceiver
dotnet add anotherReceiver reference ../messages
dotnet add anotherReceiver package NServiceBus

6. Add a subscriber class to the project

1
2
3
4
5
6
7
8
9
10
11
12
13

using messages;

using NServiceBus;

public class CakeEatenHandler :
    IHandleMessages<CakeEaten>
{
    public Task Handle(CakeEaten message, IMessageHandlerContext context)
    {
        Console.WriteLine($"Awesome, somebody ate a cake at {message.EatingFinishedAt}. Time to bake another");
        return Task.CompletedTask;
    }
}

7. Update the Program.cs to initiate NServiceBus

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

using messages;
using NServiceBus;

Console.Title = "NServiceBusKata - AnotherPublisher";

var endpointConfiguration = new EndpointConfiguration("NServiceBusKataAnotherReceiver");

// Choose JSON to serialize and deserialize messages
endpointConfiguration.UseSerialization<SystemJsonSerializer>();

var transport = endpointConfiguration.UseTransport<LearningTransport>();

var endpointInstance = await Endpoint.Start(endpointConfiguration);


Console.ReadLine();

await endpointInstance.Stop();

Things to try now:

1. Start the receiver and the another receiver then the sender - ensure you see messages flowing correctly
2. Start services in different orders and see which messages might be lost

Sending and receiving messages and publishing messages should now be working great. But to get this example going we've made some simplifications which we'll have to address in the next kata.

• Kata 1 - Sending a message
• Kata 2 - Publishing a message
• Kata 3 - Switching transports
• Kata 4 - Long running processes
• Kata 5 - Timeouts
• Kata 6 - When things go wrong

The Problem

Our goal is to very simply demonstrate reliable messaging. If you're communicating between two processes on different machines a usual approach is to send a message using HTTP. Problem is that sometimes the other end isn't reachable. Could be that the service is down, could be that the network is down or it could be that the remote location was hit by a meteor. HTTP won't help us in this case - what we want is a reliable protocol which will save the message somewhere safe and deliver it when the endpoint does show up.

For this we use a message queue. There are approximately 9 billion different messaging technologies out there but we're going to use NServiceBus. NServiceBus is a .NET library which wraps up a lot of the complexity of messaging. It is built to be able to use a variety of transport such as RabbitMQ and Azure Service Bus.

We want to make use of NServiceBus and a few C# applications to demonstrate reliable messaging.

The Kata

I like cake but I feel bad about eating it because it's not good for me. So in this kata you need to command me to eat cake. I can't refuse a command to eat cake so I can't possibly feel bad about it.

Create a sender application which sends a message to a receiver application. The receiver application should be able to receive the message and write it to the console. The sender application should be able to send the message and then exit. The receiver application should be able to start up and receive the message even if the sender application isn't running.

Now go do it!

Useful resources:

• NServiceBus getting started

My Solution

1. Create a new directory for the project

1
2

mkdir kata1
cd kata1

2. Create a new console project for the sender

1

dotnet new console -o sender

3. Create a new console project for the receiver

1

dotnet new console -o receiver

4. Create a new class library for the messages

1

dotnet new classlib -o messages

5. Add a reference to the messages project in the sender and receiver projects

1
2

dotnet add sender reference ../messages
dotnet add receiver reference ../messages

6. Add a reference to NServiceBus in all the projects

1
2
3

dotnet add sender package NServiceBus
dotnet add receiver package NServiceBus
dotnet add messages package NServiceBus

7. Create a new class in the messages project (and remove Class1.cs)

1
2
3
4
5
6
7
8
9

namespace messages;

using NServiceBus;

public class EatCake: ICommand
{
    public int NumberOfCakes { get; set; }
    public string Flavour { get; set; } = "Chocolate";
}

8. Update Program.cs in the sender project to send a message

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

using messages;
using NServiceBus;

Console.Title = "NServiceBusKata - Sender";

var endpointConfiguration = new EndpointConfiguration("NServiceBusKataSender");

// Choose JSON to serialize and deserialize messages
endpointConfiguration.UseSerialization<SystemJsonSerializer>();

var transport = endpointConfiguration.UseTransport<LearningTransport>();

var endpointInstance = await Endpoint.Start(endpointConfiguration);

await endpointInstance.Send("NServiceBusKataReceiver", new EatCake{
    Flavour = "Coconut",
    NumberOfCakes = 2 //don't be greedy
});

await endpointInstance.Stop();

9. Update Program.cs in the receiver project to be an endpoint

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

using NServiceBus;

Console.Title = "NServiceBusKata - Reciever";

var endpointConfiguration = new EndpointConfiguration("NServiceBusKataReceiver");

// Choose JSON to serialize and deserialize messages
endpointConfiguration.UseSerialization<SystemJsonSerializer>();

var transport = endpointConfiguration.UseTransport<LearningTransport>();

var endpointInstance = await Endpoint.Start(endpointConfiguration);

Console.WriteLine("Press Enter to exit...");
Console.ReadLine();

await endpointInstance.Stop();

10. Add a message handler to the receiver project

1
2
3
4
5
6
7
8
9
10
11

using messages;

public class EatCakeHandler :
    IHandleMessages<EatCake>
{
    public Task Handle(EatCake message, IMessageHandlerContext context)
    {
        Console.WriteLine($"Cake eaten, NumberOfCakes = {message.NumberOfCakes}; Flavour = {message.Flavour}");
        return Task.CompletedTask;
    }
}

Things to try now:

1. Run the sender project - it will send a message but the receiver won't be running so nothing will happen
2. Run the receiver project - it will start listening for messages and find the message which was left for it
3. Run the sender project again - it will send a message and the receiver will pick it up and write to the console

This demonstrates reliable messaging with NServiceBus

)

I just loved this mental model. Each peg in the board is an fork in the code and a different path can be taken from that point on. It's basically an execution tree but with a fun visual that's easy to explain to people.

1
2
3
4

flowchart TB
    A --> B & C
    B --> D & E
    C --> F & G

• How do I see the exceptions that happened?
• How do I configure an Azure Function app to save a PDF to a Google Drive folder

This is from the perspective of someone who doesn't need their app to be highly available/scalable/reliable/tenable/affable/inscrutable/explicable, which the documentation for all cloud products seems to assume (and, it must be said, rightfully so).

I like crossword puzzles. And I like doing them on paper. I subscribe to a few services, including the New York Times, AVCX, Matt Gaffney, the Muller Monthly Music Meta, and a couple of others, both free and paid. For years, I've maintained a bash file that downloads PDFs from various services that allow it (and some that likely don't) and merges them into a single document for printing. It works fine but I wanted to try my hand at moving it to an Azure Function app that ran on a timer rather than on demand like I do with the bash file.

So the app in question is pretty simple and the code itself isn't super interesting. Here are the logistics:

• Runs nightly
• Downloads PDFs from various crossword providers (three so far)
• Saves the PDF to Google Drive

The last one is only because that's where I keep them after they're printed. I have an archive of the puzzles I've downloaded going back a number of years. I've no idea why except that I have the storage available. Maybe I'll run through them again in retirement but the more likely scenario is that my children will mass delete them in a housecleaning exercise after I die.

With that in mind, let's get to the two areas where I struggled.

App Insights

By most accounts, App Insights on Azure is a powerful tool and telemetry in general is a big business. But I've always shied away from the Diagnose/Investigate/Metrics tabs because they're just plain overwhelming for an aging hillbilly who's used to scrolling through IIS log files. So when I wake up in the morning and the latest Times puzzle isn't there, I need to figure out where to go.

My solution (which may not align with the solution) is pretty simple in the end:

• Navigate to the Function app in Azure
• Go to Monitoring | Logs
• Query the exceptions table with no other qualifers and with an appropriate time range

This shows the exceptions and the stack traces within the time period. Given this runs nightly, there likely won't be more than a couple in the last 24 hours.

Connect to a Google Drive folder

Ugh...

I've said this on more than one occasion: I hate all things security-related including, but not limited to: certificates, OAuth, face ID, fingerprint scanners, digital signatures, encryption, authentication providers, 2FA, MFA, SSL, VPN, CVE, IAM, JWT, SSO, AES, TSA, passkeys, passwords, passports, key fobs, car alarms, and bike locks.

Setting up an Azure Function App to connect to Google Drive touches on several of these pet peeves and adds a few more. Here's what I eventually did to get this to work:

• Create a project in my Google Developer Console
• Enable the Google Drive API (it wasn't enabled by default)
• Create a service account credential
• Under the new service account, create a key. This downloads a file to your computer.
• Upload the file to the Azure Storage account for the Azure Functions app in a File Share folder called secrets
• Create two environment variables for the Azure Function:
  • GoogleApiSecretsFileName: the name of the file (include the .json extension) I downloaded for the key
  • GoogleDriveFolderId: The ID of the folder where the puzzles should be saved. (Navigate to the folder in a browser. The ID is everything after folder/ in the URL.)
• Share the Google drive folder with the Google API service account (with Editor access). The email address is on the Credentials page in the Google Developer Console.

Oh, and also write the actual code.

The Azure documentation and ChatGPT suggest strongly that OAuth2 credentials (instead of a service account) should work if you set up Google as an authenticator on the Azure Functions app. I ran into problems with this and I think it's because I was running locally. I have the app set up to do everything through a console app instead of through the Azure Functions host when running in DEBUG mode and I suspect the problems are because my local app hasn't gone through the necessary authentication process. Either way, a service account explicitly says it's for unattended server-to-server scenarios, which this is, so I've justified it in my head, even if the Azure documentation suggests something else.

It's a fairly simple process to view all of these calendars in a single place. All the major calendar providers generously provide iCal links at the very least to let you combine everything in a single view and you can even turn individual calendars on and off.

I have two problems with this.

First, I often don't want to have all of these calendars visible in a single view on my screen. Screen-sharing is common and on more than one occasion, I've said, "let's schedule a meeting" and switched to the shared calendar view where all my personal Dudeism meetings are there for all to see.

Second (and more important than risking an argument with a nihilist), people booking meetings with me don't see the unavailable slots on the other calendars, leading to double-bookings. I didn't know how good I had it in high school when this was...let's just say, "not a problem".

So I went on the hunt for a solution that met these criteria:

1. The ability to book an event on one calendar and have the time blocked on the other two.
2. Have the time on other calendar show as a generic "busy" or "unavailable". I.e. don't broadcast the details of the events from other calendars.
3. Have a centralized view somewhere of all the events without duplicates. I.e. without a bunch of "busy" or "unavailable" events.
4. Ideally, works with both Google and Microsoft calendars

)

The easiest way to achieve this, of course, is to do it yourself. When you book an event on one calendar, you go through the motions of creating a generic event on the other two. This meets all the criteria but depending on how heavily your client leans into the "all meetings/all the time" project management style, this gets old real fast. But to be fair, it's served me reasonably well for many years.

This led naturally to a search for a technical solution, which yielded the following options:

• OneCal
• CalendarBridge
• Fantastical
• Calendly
• Reclaim.ai
• Spike
• Syncing calendars

I have OneCal at the top because it's (currently) my favorite option. But I'll skim through the rest and provide reasons why I didn't go with them. I'll use my tried-and-true practice of claiming that my choice is the One True Way™ and everyone else is wrong as a means of encouraging people to comment with their own options, if only to prove me wrong. Which I'm quite happy to be. But still, that's not the case here. I'm right and you're wrong. Comment below if you disagree.

Back to the evaluations. I'll start with some of the quicker ones.

Syncing calendars

I'll be honest, I just kind of skimmed this article. I have kind of a vague idea of what they're going for but this seems geared more toward people that are managing someone else's calendar. Plus I don't think it meets the criteria of actually booking the time off on the other calendar. Either way, it looks kind of cumbersome to set up and to manage.

Spike

I didn't give this much more than a glance. It does way more than I want it to and calendar sync doesn't appear to be among their more popular features. In any case, the banner says "email and chat app" and the byline talks about teams and partners. I don't think I'm their target audience.

Reclaim.ai

Also has many features I don't need nor want though at least it's focused on calendars. Google Calendar specifically. Outlook is coming soon. So while calendar sync is one of the highlighted features, the lack of Outlook, as well as the focus on collaboration and AI and "smart" also suggests it's overkill.

Calendly

I've used Calendly before to send people links to schedule calls with me. It's a lovely app and the free version is quite powerful. I'd recommend it even over Google Calendar's built-in "scheduling slots" or "appointment schedules" or whatever they're calling the feature now.

That said, based on what I read, I don't think they offer what I need. It looks like the paid plans will let you connect to multiple calendars so that you can give someone a link with a more comprehensive view of your availability. But I need something for internal people who have direct access to my calendar.

Fantastical

This is just plain a gorgeous app, if you'll forgive an awkwardly-worded almost-oxymoron. I keep threatening to use it every couple of years but can't really justify it. The major selling points are syncing across multiple devices, the natural language parsing, the UX, and integration with a lot of other productivity tools I don't use. Seriously, you should check this app out...

...except if all you need is to sync events across multiple calendars. I'm still in the midst of testing the app but the process to ensure an event appears as a block of unavailable time on multiple calendars seems to be to duplicate it. Which, to be fair, they make really easy. And if you keep the details the same in all the clones, the unified calendar view removes the duplicates. But that doesn't meet criteria #2 for me above. And if you change the name, it shows as a separate event which is a bit clumsy. For this feature specifically, it's not much better than what I can achieve in Google Calendar natively. But still, give it a whirl.

Final verdict: OneCal (CalendarBridge runner up)

That leaves two options: OneCal and CalendarBridge. Both do the same thing though their respective marketing teams might squabble over technicalities. Their primary purpose is to sync multiple calendars and avoid double-bookings. Perfect.

Both services sync one calendar to another by cloning events between them and monitoring the source calendar for new events. Both give you control over what information you want to copy over and have similar options like excluding events of a given colour or excluding events where you're marked free (useful if you track birthdays in your calendar). Once syncing was set up, my calendars looked pretty much exactly how they did when I was doing it manually.

Neither service has a mobile app from what I can tell which is a shame, especially in the case of OneCal because unlike CalendarBridge, they offer a unified calendar view of all your calendars with the option to hide the cloned events. This is a nice touch and while the view is mobile friendly, it's missing some features, like a three-day view or the ability to put a widget on your phone.

Pricing is virtually identical on both services in that the one I would need runs about $100/year to sync three calendars. OneCal has the ability to do two-way syncs between calendars or to broadcast a one-way sync from one calendar to multiple other calendars, something that I don't think CalendarBridge does. In CalendarBridge, a two-way sync is done with two one-way syncs and their pricing reflects that. The basic plan for both essentially covers two calendars so I'd be looking at premium for both.

Setting up the syncs seemed nicer in OneCal to me though I can't 100% say why. Both essentially walk you through a three-ish-step wizard to do it. Maybe OneCal worded things better or laid things out more intuitively for me. I found myself wondering if I was doing it right a couple of times in CalendarBridge. OneCal also felt zippier in general.

Both apps give you booking links (a la Calendly) which is nice if you need that. OneCal apparently integrates with Zoom to automatically add a meeting link to appointments booked online but I didn't test that out since I don't need it. Neither requires a credit card to sign up which is an underrated feature these days.

A couple of little UX things. First, OneCal has another underrated feature: a big ol' Delete My Account button at the bottom of the settings page. This is great for someone who wants to test things out, decide it's not for them, and move on. I'm not planning to use CalendarBridge and I've cancelled my free trial but the account remains. There's no obvious way to delete it so I guess I'm a CalendarBridge user indefinitely now. I suppose that's good for them to pump up their user numbers in case someone wants to buy them out but doesn't do me any good.

Second thing is CalendarBridge's login mechanism. To log in, I enter my email address, they email me a code, I enter the code. It's basically the standard 2FA mechanism from fifteen years ago. There's no password, no Google/Apple/Facebook/etc option. It's kinda weird. Like the developers have a personal agenda against traditional authentication and this is their manifesto.

So both apps do what I want and cost roughly the same but I give the edge to OneCal for their usability and their unified Calendar view. The price point is high enough that I'll give it a few days before pulling the trigger so I can decide if I want this more than I want to watch seasons 2 of Silo and Severance.

The alternative is to set up an auto-decline automation for all meetings which I haven't fully taken off the table yet.

So what we want is to allow anybody to link into these and download them with a GET

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
...
var bsp = new BlobServiceProperties { HourMetrics = null, MinuteMetrics = null, Logging = null };
bsp.Cors.Add(new BlobCorsRule
{
    AllowedHeaders =  "*",
    AllowedMethods = "GET",
    AllowedOrigins = "*",
    ExposedHeaders = "*",
    MaxAgeInSeconds = 60 * 30 // 30 minutes
});

// from a nifty little T4 template
var connectionString = new ConnectionStrings().StorageConnectionString;
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
blobServiceClient.SetProperties(bsp);

Again, in hindsight I feel like these rules are overly permissive and I would probably want to lock them down a bit more.

Today my not problem was running a docker build wasn't copying the files I was expecting it to. In particular I had a themes directory which was not ending up in the image and in fact the build was failing with something like

1

ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref b1f3faa4-fdeb-41ed-b016-fac3862d370a::pjh3jwhj2huqmcgigjh9udlh2: "/themes": not found

I was really confused because themes absolutly did exist on disk. It was as if it wasn't being added to the build context. In fact it wasn't being added and, as it turns out, this was because my .dockerignore file contained

1

**

Which ignores everything from the local directory. That seemed a bit extreme so I changed it to

1
2

** 
!themes

With this in place the build worked as expected.