Target audience : junior and above

Intro

If you’ve worked with ASP.NET web applications you’re bound to have come across asynchronous code and the async/await pattern when handling HTTP requests. you’ve probably also been told that you should use async implementations whenever they are available, but have you ever wondered “what’s the point?”.

Truly asynchronous code

For web applications to benefit from asynchronous code, the library being consumed must be doing a “truly” asynchronous operation. Before we can explore synchronous vs asynchronous operations, we first need to gloss over CPU-bound vs I/O bound operations.

CPU-bound vs I/O bound operations

CPU-bound operations (e.g. complex calculations) are thread blocking by nature, as the work is being done by the CPU, but I/O operations (such as reading from a disk or making a network call) are not CPU-bound, so have no reason to block a thread.

Let’s say you are making a call to a database server on the same network as your ASP application as part of handling a HTTP request. From the perspective of the machine that is running the ASP application this is handled by the operating system, as network calls ultimately go through the network interface driver. This means the operation is I/O bound, rather than CPU-bound.

Synchronous vs Asynchronous code

When code for I/O bound operations is written synchronously, the thread handling the HTTP request gets blocked until this call had finished, meaning it cannot be used for anything else until the database returns a response, despite the operation itself being I/O bound.

        using var conn = new SqlConnection(ConnectionStrings.MsSqlConnection);
        conn.Open();

        using var command = new SqlCommand("EXEC dbo.MyProcedure", conn);
        command.ExecuteNonQuery();

        conn.Close();

When code for I/O bound operations are written asynchronously, what happens deep down is the OS creates an IRP (I/O request packet) and sends it to the driver like normal, but instead of blocking the thread while waiting for the response, the thread is returned to the thread pool. In the context of an ASP.NET application, this means the thread can be used to handle other HTTP requests while the OS waits for the operation to complete. This is where the increased scalability comes from. When the operation completes, the code then carries on where it left off (paraphrasing for simplicity - if you want to read more in depth about how asynchronous programming works in .NET, Stephen Cleary, the godfather of asynchronous programming in .NET, wrote an article on it here). It’s important to note that no new thread is created to handle anything that is truly asynchronous, so this is not to be confused with “multithreading”, which is a common misconception.

        using var conn = new SqlConnection(ConnectionStrings.MsSqlConnection);
        await conn.OpenAsync();

        using var command = new SqlCommand("EXEC dbo.MyProcedure", conn);
        await command.ExecuteNonQueryAsync();

        await conn.CloseAsync();

Let’s put this into an analogy. Imagine you are waiting in line at Starbucks and it is your turn to give your order to the cashier. After the cashier takes your order and you have paid, you don’t wait at the cash register until your order is ready, you move off to the side and wait in the unofficial waiting area until your order is ready. This allows the customer behind you to place their order. This system stops the queue going out the door and into the street. The coffee isn’t made any faster, nor are they making more coffees in parallel, but it’s allowing more “requests” to be taken while your order is being made. This analogy demonstrates that while the ‘work’ (coffee or database query) takes the same amount of time, the ‘cashier’ (server thread) is freed up more quickly in an asynchronous model.

Demonstration

Thankfully, this concept is really easy to demonstrate. I created a new ASP.NET application (targeting .NET 8). I have two API endpoints, using Thread.Sleep() to simulate a thread blocking operation - one being shorter running and the other longer running

[ApiController]
[Route("api/[controller]")]
public class HomeController : ControllerBase
{
    [HttpGet("Complex")]
    public IActionResult ComplexTask()
    {
        Thread.Sleep(1000);
        return Ok();
    }

    [HttpGet("Simple")]
    public IActionResult SimpleTask()
    {
        Thread.Sleep(50);
        return Ok();
    }
}

I then wrote a program that would continuously send as many concurrent API requests as possible to these two endpoints for 60 seconds - but it would allow any requests sent within that 60 seconds to wait for a response. The results are below.

The average response time for both endpoints was about 50 seconds - which is almost as long as the test was running for. This average was hindered by the requests that timed out - but it shows that when the server is under extreme load you have to wait longer for a response (not because the Task.Delay() was taking any longer, but because it took longer for a thread to be available in the thread pool to handle the request). You can also see a large amount of the requests timed out/were refused.

I then modified the API code to use Task.Delay() - which is asynchronous.

    [HttpGet("Complex")]
    public async Task<IActionResult> ComplexTask()
    {
        await Task.Delay(1000);
        return Ok();
    }

    [HttpGet("Simple")]
    public async Task<IActionResult> SimpleTask()
    {
        await Task.Delay(50);
        return Ok();
    }

I ran the same test again, and the results were staggering.

The asynchronous “Simple” endpoint served almost 2 million “simple” requests, and it did it within the 60 seconds run time - there was no “wait” for remaining tasks to finish, as all requests were handled concurrently without blocking.

Both average response times were also more representative of the simulated delay (the extra milliseconds can be accounted for by overhead of context switching, task scheduling and waiting for a thread to be free again to continue the request after the asynchronous operation had been completed, all of which are incurred by the code being asynchronous).

Another reason why there were so many more requests is because I am running the load test on the same PC as the API is running on. Although the load test was written asynchronously where possible as well, the thread blocking nature of the synchronous API code will have hindered my PC’s performance during the load test by blocking more threads to serve requests - which further highlights the scalability of asynchronous code.

You will also notice the actual elapsed time was pretty much 60 seconds - no extra time waiting for requests to finish, as they all were completed more or less within the expected time ,and no timeouts, all thanks to the ASP application being more efficient now its using asynchronous code.

There was also a reason why I made two endpoints, a longer running and shorter running one. This was to demonstrate another benefit of asynchronous code; while your application is waiting for a long running operation to finish, the thread is returned to the thread pool and can handle shorter running requests, which may even complete before the long running operation has completed!

Now, you might be thinking that this test is too “fake” or “simulated” as it uses Thread.Sleep() and Task.Delay() rather than an actual database or network calls, but it’s actually the perfect way to demonstrate this concept. Thread.Sleep blocks the thread in the same way that the synchronous database example does above, and Task.Delay() is asynchronous in the same way as the asynchronous database code snippet. However, in the interest of being a good sport, I also conducted a more “real” test.

I added some more endpoints to my application (yes, it appears that I’m calling my API from my API, but it’ll make sense in a minute):

    // single instance used across all calls to avoid socket exhaustion - this is an important detail! HttpClient is fully thread safe
    private readonly static HttpClient _client = new HttpClient
    {
        BaseAddress = new Uri(Urls.NetworkApi),
    };

    [HttpGet("SimpleApi")]
    public IActionResult CallApiSimple()
    {
        // GENERALLY VERY BAD TO CALL .Result ON A TASK IN ASP - THIS IS JUST FOR THREAD BLOCK DEMO PURPOSES!!!
        // See https://blog.stephencleary.com/2012/12/dont-block-in-asynchronous-code.html for more
        var result = _client.GetAsync("api/home/simple").Result;
        return Ok(new
        {
            result.StatusCode,
        });
    }

    [HttpGet("ComplexApi")]
    public IActionResult CallApiComplex()
    {
        // GENERALLY VERY BAD TO CALL .Result ON A TASK IN ASP - THIS IS JUST FOR THREAD BLOCK DEMO PURPOSES!!!
        // See https://blog.stephencleary.com/2012/12/dont-block-in-asynchronous-code.html for more
        var result = _client.GetAsync("api/home/complex").Result;
        return Ok(new
        {
            result.StatusCode,
        });
    }

    [HttpGet("SimpleApiAsync")]
    public async Task<IActionResult> CallApiSimpleAsync()
    {
        var result = await _client.GetAsync("api/home/simple");
        return Ok(new
        {
            result.StatusCode,
        });
    }

    [HttpGet("ComplexApiAsync")]
    public async Task<IActionResult> CallApiComplexAsync()
    {
        var result = await _client.GetAsync("api/home/complex");
        return Ok(new
        {
            result.StatusCode,
        });
    }

I then deployed this web app to my raspberry pi, running it inside a docker container. The Urls.NetworkApi set on HttpClient.BaseAddress points towards my main PC, where this web application is also running. The test is to load test the web application running on the raspberry pi, which makes a real network call to the API running on my PC. I am running the load test against the raspberry pi as my PC is far more powerful and I didn’t want to introduce a bottle neck had it been the other way round.

The latter is actually a nice segway into something you will need to be aware about when benefitting from the increased scalability of your web app. The services your web app uses also need to be able to handle the increased traffic! This underscores the importance of identifying where bottlenecks occur—async code helps the application scale, but only if underlying dependencies (databases, networks) can keep up.

My raspberry pi can handle nowhere near the same amount of traffic as my PC, so I had to limit the concurrency of requests to 40 per endpoint, which is still impressive performance from the pi! The results from the tests are below:

Synchronous

Asynchronous

The gains of the asynchronous versions are nowhere near as much as when I ran the tests against my PC due the lack of processing power the pi has, but there was still an improvement none the less and most importantly, no refused connections or timeouts when running asynchronously. The benefits of asynchronous code will scale with the processing power of the hardware it’s running on.

Conclusion

Truly asynchronous code frees up threads while waiting for non CPU-bound work to complete, enabling the application to handle other incoming requests simultaneously. The time it takes to complete the operation does not get faster, but it allows your application to do other things while it waits for the operation to complete. As your ASP application scales, make sure that your other backend services can scale with it!

CAUTION ⚠️

Not all APIs that are marked as async in .NET are truly non blocking underneath - some just wrap synchronous code in Task.Run() which causes another thread to be created to run the code. Make sure you research about the APIs you use before using them!