Addressing Cache Retrieval Failures on Azure App Service - Node-Cache Example

I recently encountered an issue when attempting to read cache values using the Node-Cache module on an app service running on multiple instances. However, the module worked fine on my local environment, single virtual machine, or single-instance app service.

This behavior occurs because Node-Cache and other private cache solutions are designed to store data in the memory of a single process. The private cache works properly when running on a single instance, as all requests are handled by the same process and have access to the same cache.

However, in a multi-instance environment like a multi-instance App Service, using a private cache can lead to inconsistent behavior and data inconsistency.

For example, you save a value in one cache in an instance. Then, a request tries to read this value from another instance. In this case, you may fail to get the value of this cache or get an incorrect value.

Here's a JavaScript code snippet for an Express application to demonstrate this behavior:

const express = require('express');
const NodeCache = require('node-cache');

const app = express();
const port = process.env.PORT || 3000;

const myCache = new NodeCache();
const options = {
  weekday: 'long',
  year: 'numeric',
  month: 'long',
  day: 'numeric',
  hour: 'numeric',
  minute: 'numeric',
  second: 'numeric'
};

app.get('/', (req, res) => {
  const unixTimestamp = Math.floor(Date.now() / 1000);
  const date = new Date();
  const friendlyDateTime = date.toLocaleString(undefined, options);
  myCache.set(unixTimestamp, friendlyDateTime);
  res.send(`${unixTimestamp}: ${friendlyDateTime} (instance name: ${process.env.WEBSITE_INSTANCE_ID})`);
});

app.get('/:unixTimestamp', (req, res) => {
  const unixTimestamp = parseInt(req.params.unixTimestamp);
  const friendlyDateTime = myCache.get(unixTimestamp);
  res.send(`${unixTimestamp}: ${friendlyDateTime} (instance name: ${process.env.WEBSITE_INSTANCE_ID})`);
});

app.listen(port, () => {
  console.log(`Server listening on port ${port}`);
});

You may observe the following behavior after deploying this application to a multi-instance app service:

1. When you send a GET request to the root URL for this application, the application will generate a key-value cache – the key is the current Unix timestamp, and the value is its date-time in a friendly format. It will be saved and displayed with the instance name together.
   

    

2. Then, if you open a new InPrivate window of your browser (or you have ARR infinity disabled) and browse to http://<your app service name>.azurewebsites.net/<unix timestamp>, you may occasionally see that the value is undefined by refreshing the page a few times. You will also see the instance is different from the one that saved the cache.
   

This issue may also occur with other private cache libraries for different languages, such as:

Python: cachetools, discache and joblib.Memory

C#: MemoryCache, ObjectCache

Java: Caffeine, Guava, Ehcache

To avoid this issue, it is recommended to use a distributed cache solution instead of private cache solutions. A distributed cache stores data in a centralized location accessible to all instances, ensuring that all requests have access to the same data. Azure provides several options for distributed caching, including Azure Cache for Redis and Azure Cosmos DB.

In summary, using private cache solutions like Node-Cache in multi-instance environments can cause issues. To ensure data consistency, it is recommended to use a distributed cache solution.