Use Rdbtools to Generate Memory Report for Azure Cache for Redis and Convert Dump Files to JSON

Scenario:
The rdbtools is a handy utility that can parse Redis dump.rdb files, analyze memory, and export data to JSON. This tutorial shows a quick example using rdbtools to generate report for Azure cache for Redis, on memory usage and number of keys categorized by data type and data encoding.

Objective:
To showcase how to use the rdbtools to analyze memory use and data dumps from an Azure Cache for Redis. It is easy to use and gives more insights than the Memory Usage metrics available on the Azure portal.

Pre-requisites:
For this example, you would need:

• An Azure Cache for Redis in the Premium, Enterprise, or Enterprise Flash tiers.

Steps:

1. Install rdbtools
2. Add some data in the Redis
3. Export data to dump
4. Generate memory report
5. Convert dump file to JSON

[STEP 1]: Install rdbtools

pip install rdbtools python-lzf

Note: Run the "pip install" command from the PowerShell. The "python-lzf" package accelerates the report generation. If you encounter the "Microsoft Visual C++ 14.0 is required" error, please install the C++ Build Tools from Visual Studio Installer as shown below. After installation in completed, add the script path which contains rdb.exe and redis-profiler.exe into the Path environment variable. For me, the script path is C:\Users\xxxxxx\AppData\Local\Programs\Python\Python39\Scripts.

[STEP 2]: Add some data in the Redis

Open "Console" from the Azure portal and run the following:

SET nba https://www.nba.com/
SET cnn https://edition.cnn.com/
SET nike https://www.nike.com/
MSET google https://www.google.com/ yahoo https://www.yahoo.com/
MSET user:charles:name Charles user:charles:password s3cret
HMSET user:charles name Charles password s3cret
RPUSH charles:favoritefruit apple orange banana
SADD sports swimming basketball tennis
SADD hobbies swimming hiking boxing
SUNIONSTORE exercise sports hobbies
ZADD visits 500 nba 9 cnn 9999 nike

Note: Here I added 13 keys with various data types such as strings, lists, hashes, sets, and sorted sets.

[STEP 3]: Export data to rdb dump

Browse to the cache in the Azure portal and select "Export data" from the left menu. Pick a blob name prefix, select an output storage container, and then click "Export". After export is done, browse to the storage container and download the dump file.

[STEP 4]: Generate memory report

PS C:\Users\xxxxxx\Downloads> redis-profiler .\charlesredis1 -f charlesredis1-report.html

charlesredis1-report.html:

[STEP 5]: Convert dump file to JSON

PS C:\Users\xxxxxx\Downloads> rdb -c json .\charlesredis1
[{
"yahoo":"https://www.yahoo.com/",
"nba":"https://www.nba.com/",
"hobbies":["hiking","swimming","boxing"],
"user:charles:name":"Charles",
"cnn":"https://edition.cnn.com/",
"nike":"https://www.nike.com/",
"charles:favoritefruit":["apple","orange","banana"],
"exercise":["tennis","basketball","swimming","boxing","hiking"],
"user:charles":{"name":"Charles","password":"s3cret"},
"sports":["swimming","tennis","basketball"],
"google":"https://www.google.com/",
"visits":{"cnn":"9","nba":"500","nike":"10000"},
"user:charles:password":"s3cret"}]

Conclusion:

This example shows how to use the rdbtools to generate a memory report from an Azure Cache for Redis and convert the exported dump to JSON. The rdbtools analyzes the memory consumption of Redis keys as well as their data types. When it comes to improving the memory efficiency of a Redis cache, the rdbtools becomes a useful utility.

References:

• https://pypi.org/project/rdbtools/
• https://www.cnblogs.com/zhoujinyi/p/13276697.html
• https://github.com/sripathikrishnan/redis-rdb-tools 
• https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-how-to-import-export-data#export