Memory Usage Optimization for Distributed Cache Cluster

You might find your distributed cache’s cluster hosting different types of caches: write-behind, read-through and cache-aside caches. The more your storage nodes hosts caches, the more memory it requires. So, let’s see what optimizations tweaks could be applied to allow your caches feel well and safe.

Count of Backups

In the most common use cases a cache is mapped to the DistributedCache service:

A distributed, or partitioned, cache is a clustered, fault-tolerant cache that has linear scalability. Data is partitioned among all the machines of the cluster.

Apart of holding a part of data, by default this service assumes storing backups from all other storage nodes. That means that having 3 storage nodes each of them is hosting ⅓ of data, and 2 backups from 2 other nodes (⅓ + ⅓).
So, if you’re caching non-critical data, and it’s acceptable to miss some piece of cached data, you can go ahead and disable backups holding for this particular service:

<distributed-scheme>
    <scheme-name>products</scheme-name>
    <service-name>DistributedCache</service-name>
    ...
    <backup-count>0</backup-count>
</distributed-scheme>

In this case you can assume your cache’s capacity boosted up to N times (N - is a number of storage nodes in the cluster) since it’s not tacking care of backups storing any more.
Just keep in mind that it’s not allowed to run different caches with different number of backups mapped to the same service. That means that all the caches mapped to the DistributedCache service should have either enabled or disabled data backups.

Thresholds Defining

It’s supposed to be a good practice to measure your storage node’s capacity and define thresholds for all the caches hosted by the node.
You can define a right amount of items allowed to be cached at the same time:

<distributed-scheme>
    <scheme-name>products</scheme-name>
    <service-name>DistributedCache</service-name>
    <backing-map-scheme>
        <local-scheme>
            <eviction-policy>LRU</eviction-policy>
            <high-units>10000</high-units>
            <expiry-delay>0</expiry-delay>
            <unit-calculator>FIXED</unit-calculator>
        </local-scheme>
    </backing-map-scheme>
    ...
</distributed-scheme>

Here you have to define an average object’s size and come up with total amount calculation. E.g. having 100KB per object and 1GB of heap memory, you can allow 10K objects to be cached.

Or you can limit your cache size by consumed memory, e.g. here we have 100MB uppper limit for distributed cache with the Least Recently Used expiration policy:

<distributed-scheme>
    <scheme-name>products</scheme-name>
    <service-name>DistributedCache</service-name>
    <backing-map-scheme>
        <local-scheme>
            <eviction-policy>LRU</eviction-policy>
            <high-units>100m</high-units>
            <expiry-delay>0</expiry-delay>
            <unit-calculator>BINARY</unit-calculator>
        </local-scheme>
    </backing-map-scheme>
    ...
</distributed-scheme>

So now your products cache won’t grow beyond 1GB, and because of defined Least Recently Used eviction policy, old cached items will be replaced with new ones.

Please note, that if your threshold is more than 3GB, you should use <unit-factor> element with value of 1048576 to count megabytes instead of bytes:

<distributed-scheme>
    <scheme-name>products</scheme-name>
    <service-name>DistributedCache</service-name>
    <backing-map-scheme>
        <local-scheme>
            <eviction-policy>LRU</eviction-policy>
            <high-units>4096</high-units>
            <unit-factor>1048576</unit-factor>
            <expiry-delay>0</expiry-delay>
            <unit-calculator>BINARY</unit-calculator>
        </local-scheme>
    </backing-map-scheme>
    ...
</distributed-scheme>

Summary

As you can see, it’s very important to take care about your cache’s capacity in advance and have it protected from overpopulation. So, you’ll have your distributed cache’s cluster very unlikely fails because of out of memory.