In Memory Caching on .NET 6.0
ASP.NET provides two types of caching that you can use to create high-performance Web applications. The first is called output caching, which allows you to store dynamic page and user control responses on any HTTP 1.1 cache-capable device in the output stream, from the originating server to the requesting browser. On subsequent requests, the page or user control code is not executed; the cached output is used to satisfy the request. The second type of caching is traditional application data caching, which you can use to programmatically store arbitrary objects, such as data sets, to server memory so that your application can save the time and resources it takes to recreate them. Today we’re going to see how to do application data caching using Microsoft.Extensions.Caching.Memory/IMemoryCache
Caching can significantly improve the performance and scalability of an app by reducing the work required to generate content. Caching works best with data that changes infrequently and is expensive to generate. Caching makes a copy of data that can be returned much faster than from the source.
Non-sticky sessions(client requests are directed to any server in a cluster of application servers rather than returning to the same server with each request for a given client) in a web farm require a distributed cache (We’ll look at this on another article) to avoid cache consistency problems. For some apps, a distributed cache can support higher scale-out than an in-memory cache. Using a distributed cache offloads the cache memory to an external process.
⚠ ️Keep in mind that..
The in-memory cache can store any object and can drastically improve your performance as mentioned earlier. But this is not ideal if your application is distributed, not to mention because this is “in-memory”, you need to limit your cache growth, otherwise you may not have enough memory for your application to do it’s work.
- Code should always have a fallback option to fetch data and not depend on a cached value being available.
- The cache uses a scarce resource, memory. Limit cache growth:
- Do not insert external input into the cache. As an example, using arbitrary user-provided input as a cache key is not recommended since the input might consume an unpredictable amount of memory.
- Use expiration to limit cache growth.
- Use SetSize, Size, and SizeLimit to limit cache size. The ASP.NET Core runtime does not limit cache size based on memory pressure. It’s up to the developer to limit cache size.
Let’s start.
🛠 Implementation
Source code for this story can be downloaded from here.
First of all as I mentioned earlier you need to install Microsoft.Extensions.Caching.Memory from dotnet CLI or package manager console.
Install-Package Microsoft.Extensions.Caching.Memory// If on dotnet CLI
// dotnet add package Microsoft.Extensions.Caching.Memory
In-memory caching is a service that’s referenced from an app using Dependency Injection. So you can request the IMemoryCache instance in the constructor like this:
If you’re creating a Web API you need to call builder.Services.AddMemoryCache() in Program.cs to register all dependencies.
After you have done this setting a cache and accessing it really easy. Take a look at the GetUsersCacheAsync() method in the UserRepository in the sample project.
As you can clearly see from this, what I have done is fairly simple. Called the _memoryCache.Get<TItem>(IMemoryCache, Object) extension method (Line 3), with a key “users”; which has the type List<User> Check here for all the methods and extension methods.
And if it contains any values I just return it (Line 5), If it’s NULL, which will be the case when we run our application for the first time, It will fetch all the users from a database or from any other data source. (Line 7–14) I have delayed the execution for 3 seconds to simulate this because database calls and / or 3rd party API calls can take time (Line 14), and once I have the data I’m setting a new cache entry using _memoryCache.Set<TItem>(IMemoryCache, Object, TItem, TimeSpan)
_memoryCache.Set("users", output, TimeSpan.FromMinutes(1));Let’s run this with benchmarks (You can find these in the Benchmark project, I have created 2 other methods, one without caching, another without caching but with async (this one will be the baseline)). Before you run these benchmark the solution configuration should be set to “Release”, read more here about benchmarks.
You can see that WithCachingWithAsync has a mean of 15 μs, where others took more than 3 seconds. No need to explain this anymore, you get the point.
🤔 But wait, “Didn’t you say to keep in mind 2 things?” you might ask, well yes, I told you that “code should always have a fallback option to fetch data and not depend on a cached value being available” and “cache should have a limit”
Fallback option
First one you already saw how to implement,
if (output is not null)
{
return output;
} else {
// Get data from store
}
For this purpose you can also use, TryGetValue() — to check if any value exists for a given key.
For the second warning (limiting cache growth), we can use expiration time intervals and cache limits.
Expiration
There are 2 types in this, and both are time based.
- Sliding Expiration — will expire the entry if it hasn’t been accessed in a set amount of time.
- Absolute Expiration — will expire the entry after a set amount of time.
So what this means is that sliding expiration has a condition; it will expire if it hasn’t been accessed in the set time, but if you set absolute expiration it will expire after the set time no matter it has been accessed or not.
Assume you set the sliding expiration to 30 seconds. Before that 30 seconds runs out, if no one make that same request again, cache will expire. If someone makes the request in the 25th second, the counter restarts and waits another 30 seconds.
You can set either sliding expiration or absolute expiration from MemoryCacheEntryOptions()
Similarly you can set absolute expiration like this.
But there’s a problem, if you look at this diagram again, you might be able to spot it.
If an item always gets accessed more frequently than it’s sliding expiration time (30 secs in this case), then there is a risk that this item would never expire. So the best practice is to you both sliding and absolute expiration combined.
When absolute expiration is specified with sliding expiration, a logical OR is used to decide if an item should be marked as expired. If either the sliding expiration interval or the absolute expiration time pass, the item is removed from the cache.
Limits
A MemoryCache instance may optionally specify and enforce a size limit. The cache size limit doesn't have a defined unit of measure because the cache has no mechanism to measure the size of entries. If the cache size limit is set, all entries must specify size. The ASP.NET Core runtime doesn't limit cache size based on memory pressure. It's up to the developer to limit cache size.
If SizeLimit isn’t set, the cache grows without bound. The ASP.NET Core runtime doesn’t trim the cache when system memory is low. Apps must be architected to:
The following code creates a unitless fixed size MemoryCache accessible by dependency injection:
The following code registers MyMemoryCache with the dependency injection container:
Also you can do this while calling AddMemoryCache(), (see this commit and this commit) options can be set to provide SizeLimit of cache store. Then while adding every individual items SetSize can be used to set approximate size of the item being added.
Compact
MemoryCache.Compact attempts to remove the specified percentage of the cache in the following order:
- All expired items.
- Items by priority. Lowest priority items are removed first.
- Least recently used objects.
- Items with the earliest absolute expiration.
- Items with the earliest sliding expiration.
Pinned items with priority NeverRemove are never removed. The following code removes a cache item and calls Compact to remove 25% of cached entries:
I’m gonna wrap things up there. Of course you can move this key “users” to an Enum or a constant and make your life easier, I’m not gonna do it because I want to keep this very simple. Hope you learned something new. Happy coding! 👋
