Redis Keys Statistics
Redis Keys statistics including Key Time-to-Live (TTL) statistics and Key sizes are useful for troubleshooting cache usage and performance, from client side. This article have two sections: First one, using a script to get statistics from Keys on Redis ( 1 Bash script + 1 LUA script - getKeyStats.sh + getKeyStats.lua ) Second one, using a script to filter and list key names ( 1 Bash script that includes a LUA script - listKeys.sh ) Key Time-to-Live (TTL): TTL may have impact on memory usage and memory available on Redis services. Data Loss on Redis services may happened unexpectedly due to some issue on backend, but may also happen due to Memory eviction policy, or Time-to-Live (TTL) expired. Memory eviction policy may remove some keys from Redis service, but only when used capacity (the space used by Redis keys) reach 100% on memory available. Not having any unexpected issue on Redis backend side or not reaching the maximum memory available, the only reason for having some keys removed from cache is due to TTL value. TTL may not be defined at all, and in that case the key remains in the cache forever (persistent) TTL can be set while setting a new key TTL can be set / re-set later after key creation TTL is defined in seconds or milliseconds, or with a negative value: -1, the key exists but has no expiration (it’s persistent); this happens when the TTL was not defined or it was removed using PERSIST command -2, if the key does not exist. any other value Related commands: SET key1 value1 EX 60 - defines TTL as 60 seconds SET key1 value1 PX 60000 - defines TTL as 60000 milliseconds (60 seconds) EXPIRE key1 60 - Set a timeout of 60 seconds on key1 TTL key1 - returns the current TTL value, in seconds PTTL key1 - returns the current TTL value, in milliseconds PERSIST key1 removes TTL from that key and make the key persistent Notes: TTL counts down in real time, but Redis expiration is lazy + active, so exact timing isn’t guaranteed to the millisecond. A TTL of 0 is basically a race condition, that usually are not seen, it because the key expires immediately. EXPIRE key 0 deletes the key right away. There is no guarantee the deletion happens exactly at expiration time. Redis lazy + active expiration means the key is checked only when someone touches it (lazy), but to avoid memory filling up with expired junk, Redis also runs a background job to periodically scan a subset of keys and delete the expired ones (active). So, some expired keys may survive a bit longer, not accessible anymore but still im memory. Example Redis lazy: at 11:59:00 SET key1 value1 EX 60 - 60 seconds expiration time key1 expires at 12:00:00 no one accesses it until 12:00:05 - when someone try to access key1 at 12:00:05, Redis identify the key1 expired and delete it. Example Redis active: for the same key1, after 12:00:00. if during the periodically background job Redis scan the subset of keys containing key1, that key1 will be actively deleted. For that reason, we may see some higher memory usage than the real memory used by active keys in the cache. For more information about Redis commands, check Redis Inc - Commands Key Sizes: Large key value sizes in the cache, may have high impact on Redis performance. Redis service is designed to 1KB response size, and Microsoft recommends to use up to 100KB on Azure Redis services, to get a better performance. Redis response size may not be exactly the same as key size, as Response size is the sum of the response from each operation sent to Redis. While the response size can be the size of only one key requested (like GET), we can see very often response size being a sum of more than one key, as result of multikey operations (like MGET and others). The scope of this article is the focus on each key size; so, we will not discuss on this article the implications of multikey commands. By design Redis service is a single thread system per shard, and this is not a Microsoft/Azure limitation but a Redis design feature. To be very quick on processing requests, Redis is optimized to work and process small keys, and for that is more efficient using a single thread instead of the need of context switching. In a multi threaded system, context switching happens when the processor stops executing one thread and starts executing another. When that happens, the OS saves the current thread’s state (registers, program counter, stack pointer, etc.) and restores the state of the next thread. To save time on that process, Redis service is designed to run in a single thread system. Due to the single thread nature, all operations sent to Redis service, are waiting in a queue to be processed. To minimize latency, all keys must remain small so they can be processed efficiently and responses can be transmitted to the client quickly over the network. For that reason, it's important to understand the key sizes we have on our Redis service, and maintain all keys as small as possible. Scripts Provided To help on identifying some specific TTL values and Keys sizes in a Redis cache, two solutions are provided below: 1. Get Key statistics - that scans all cache and return the amount of Redis keys with: Number of keys with TTL no set Number of keys with TTL higher or equal to a user defined TTL threshold Number of keys with TTL lower than a user defined TTL threshold Number of keys with value size higher or equal than a user defined Size threshold Number of keys with value size lower than a user defined Size threshold Total number of keys in the cache. It also includes start and end time, and the total time spent on the keys scan. 2. List Key Names - this script returns a list of Redis Keys names, based on parameters provided: No TTL set, or TTL higher or equal to a user defined TTL threshold, or TTL lower than to a user defined TTL threshold Key value size higher or equal than a user defined Size threshold, or Key value size lower than a user defined Size threshold Total number of keys in the cache It also includes start and end time, and the total time spent on the keys scan. WARNING: Due to the need to read all keys in the cache, both solutions can cause high workload on Redis side, specially for high datasets on the cache, with high number of keys. Both solutions are using LUA script that runs on Redis side, and depending on the amount of keys in the cache, may block all other commands to be processed, while the script is running. The duration time on the output from each script run, may help to identify the impact of the scripts to run. Run it carefully and do some tests first on your developing environment, before using in a production. YOU CAN RUN THE BELOW SCRIPTS AT YOUR OWN RISK. WE DON'T ASSUME ANY RESPONSABILITY FOR UNEXPECTED RESULTS. 1- Get Key statistics To get Redis key statistics, we use Linux Bash shell and Redis-cli tool to run LUA script on Redis side, to get TTL values and sizes from each key. This solution is very fast, but needs to scan all keys in the cache during the LUA script run. Despite very quick, the time depends of the amount of keys to scan. This may block Redis to process other requests, due to the single-thread nature of Redis service. The below script scans all cache and return only the amount of Redis keys with: Number of keys with TTL no set Number of keys with TTL higher or equal to a user defined TTL threshold Number of keys with TTL lower than a user defined TTL threshold Number of keys with value size higher or equal than a user defined Size threshold Number of keys with value size lower than a user defined Size threshold Total number of keys in the cache. It also includes start and end time, and the total time spent on the keys scan. Calling getKeyStats.sh return statistics from the existing keys in the cache, based on the two threshold values (optional), that can be passed on script command line parameters. This script can clarify questions like: “Do I have any key in the cache without TTL set?”, or “Why my keys are not expiring?” (any threshold values used will clarify this) “Do I have large keys in my cache, larger than 1KB (any TTL threshold can be used, with key size threshold 1024) “How many keys I have that will expire on next 1 hour?” (threshold 3600, with any key size threshold) Output: How to run: create the below getKeyStats.sh and getKeyStats.lua files on same folder, on your Linux environment (Ubuntu 20.04.6 LTS used) give permissions to run Shell script, with command chmod 700 getKeyStats.sh Call the script using the syntax: ./getKeyStats.sh host password [port] [ttl_threshold] [size_threshold] Script parameters: host (mandatory) : the URI for the cache password (mandatory) : the Redis access key from the cache port (optional - default 10000) : TCP port used to access the cache ttl_threshold (optional - default 600 - 10 minutes) : Key TTL threshold (in seconds) to be used on the results (use -1 to 1 to get Keys with no TTL set) size_threshold (optional - default 102400 - 100KB) : Key Size threshold to be used on the results If not provided, the default values will be used: Redis Port: 10000, ttl_threshold: 600 Seconds, size_threshold: 102400 Bytes (100KB). Tested with: Ubuntu 20.04.6 LTS redis-cli -v redis-cli 7.4.2 Redis services: Azure Managed Redis Balanced B0 OSSMode Azure Cache for Redis Standard C1 getKeyStats.sh #!/usr/bin/env bash #============================== LUA script version ================= # Linux Bash Script to get statistics from Redis Keys TTL values and Key value sizes # It returns the Number of: # - keys with TTL no set # - keys with TTL higher or equal to TTL_treshold # - keys with TTL lower TTL_threshold # - keys with value size higher or equal than Size_threshold # - keys with value size lower than Size_threshold # - total number of keys in the cache. #------------------------------------------------------- # WARNING: # It uses LUA script to run on Redis server side. # Use it carefully, during low Redis workoads. # Do your tests first on a Dev environment, before use it on production. #------------------------------------------------------- # It requires : # redis-cli v7 or above #-------------------------------------------------------- # Usage: # getRedisTTL.sh <cacheuri> <cacheaccesskey> [<accessport>(10000)] [<ttl_treashold>(600)] [<size_threshold>(102400)] #======================================================== #------------------------------------------------------ # To use non-ssl port requites to remove --tls parameter from Redis-cli command below #------------------------------------------------------ # Parameters REDIS_HOST="${1:?Usage: $0 <host> <password> [port] [ttl_threshold] [Size_Threshold]}" REDISCLI_AUTH="${2:?Usage: $0 <host> <password> [port] [ttl_threshold] [Size_Threshold]}" REDIS_PORT="${3:-10000}" # 10000 / 6380 / 6379 REDIS_TTL_THRESHOLD="${4:-600}" # 10 minutes REDIS_SIZE_THRESHOLD="${5:-102400}" # 100KB # Port number must be numeric if ! [[ "$REDIS_PORT" =~ ^[0-9]+$ ]]; then echo "ERROR: Redis Port must be numeric" exit 1 fi # TTL threshold must be numeric if ! [[ "$REDIS_TTL_THRESHOLD" =~ ^[0-9]+$ ]]; then echo "ERROR: TTL threshold must be numeric" exit 1 fi # Size threshold must be numeric if ! [[ "$REDIS_SIZE_THRESHOLD" =~ ^[0-9]+$ ]]; then echo "ERROR: Size threshold must be numeric" exit 1 fi echo "" echo "========================================================" echo "Scaning number of keys with TTL threshold $REDIS_TTL_THRESHOLD Seconds, and Key size threshold $REDIS_SIZE_THRESHOLD Bytes" # Start time start_ts=$(date +%s.%3N) echo "Start time: $(date "+%d-%m-%Y %H:%M:%S")" echo "------------------------" echo "" # Procesing result=$(redis-cli \ -h "$REDIS_HOST" \ -a "$REDISCLI_AUTH" \ -p "$REDIS_PORT" \ --tls \ --no-auth-warning \ --raw \ --eval getKeyStats.lua , "$REDIS_TTL_THRESHOLD" "$REDIS_SIZE_THRESHOLD" \ | tr '\n' ' ') read no_ttl nonexist ttl_high ttl_low ttl_invalid size_high size_low size_nil total <<< "$result" if [[ $result == ERR* ]]; then echo "Redis Lua error:" echo "$result" else echo "Total keys scanned: $total" echo "------------" echo "Keys with TTL not set : $no_ttl" echo "Keys with TTL >= $REDIS_TTL_THRESHOLD seconds: $ttl_high" echo "Keys with TTL < $REDIS_TTL_THRESHOLD seconds: $ttl_low" echo "Keys with TTL invalid/error : $ttl_invalid" echo "Non existent Keys : $nonexist" echo "------------" echo "Keys with Size >= $REDIS_SIZE_THRESHOLD Bytes: $size_high" echo "Keys with Size < $REDIS_SIZE_THRESHOLD Bytes: $size_low" echo "Keys with invalid Size : $size_nil" fi echo "" echo "------------------------" end_ts=$(date +%s.%3N) echo "End time: $(date "+%d-%m-%Y %H:%M:%S")" # Duration - Extract days, hours, minutes, seconds, milliseconds duration=$(awk "BEGIN {print $end_ts - $start_ts}") days=$(awk "BEGIN {print int($duration/86400)}") hours=$(awk "BEGIN {print int(($duration%86400)/3600)}") minutes=$(awk "BEGIN {print int(($duration%3600)/60)}") seconds=$(awk "BEGIN {print int($duration%60)}") milliseconds=$(awk "BEGIN {printf \"%03d\", ($duration - int($duration))*1000}") echo "Duration : ${days} days $(printf "%02d" "$hours"):$(printf "%02d" "$minutes"):$(printf "%02d" "$seconds").$milliseconds" echo "========================================================" getKeyStats.lua local ttl_threshold = tonumber(ARGV[1]) local size_threshold = tonumber(ARGV[2]) local cursor = "0" -- Counters local no_ttl = 0 local nonexist = 0 local ttl_high = 0 local ttl_low = 0 local ttl_invalid = 0 local size_high = 0 local size_low = 0 local size_nil = 0 local total = 0 repeat local scan = redis.call("SCAN", cursor, "COUNT", 1000) cursor = scan[1] local keys = scan[2] for _, key in ipairs(keys) do local ttl = redis.call("TTL", key) local size = redis.call("MEMORY","USAGE", key) total = total + 1 if ttl == -1 then no_ttl = no_ttl + 1 elseif ttl == -2 then nonexist = nonexist + 1 elseif type(ttl) ~= "number" then ttl_invalid = ttl_invalid + 1 elseif ttl >= ttl_threshold then ttl_high = ttl_high + 1 else ttl_low = ttl_low + 1 end if size == nil then size_nil = size_nil + 1 elseif size >= size_threshold then size_high = size_high + 1 else size_low = size_low + 1 end end until cursor == "0" return { no_ttl, nonexist, ttl_high, ttl_low, ttl_invalid, size_high, size_low, size_nil, total } Performance: Redis service used: Azure Managed Redis - Balanced B0 - OSSMode Scanning number of keys with TTL threshold 600 Seconds, and Key size threshold 102400 Bytes Total keys scanned: 46161 TTL not set : 0 TTL >= 600 seconds: 46105 TTL < 600 seconds: 56 TTL invalid/error : 0 Non existent key : 0 Keys with Size >= 102400 Bytes: 0 Keys with Size < 102400 Bytes: 46161 Keys with invalid Size : 0 Duration : 0 days 00:00:00.602 # ------------------ Redis service used: Azure Cache for Redis - Standard - C1 Scanning number of keys with TTL threshold 100 Seconds, and Key size threshold 500 Bytes Total keys scanned: 1227 TTL not set : 2 TTL >= 100 seconds: 1225 TTL < 100 seconds: 0 TTL invalid/error : 0 Non existent key : 0 Keys with Size >= 500 Bytes: 1225 Keys with Size < 500 Bytes: 2 Keys with invalid Size : 0 Duration : 0 days 00:00:00.630 # ------------------ WARNING: The above scripts uses LUA script, that runs on Redis side, and may block you normal workload. Use it carefully when have a large number of keys in the cache, and during low workload times. 2 - List Key Names Once we identify some amount of keys in the cache with some specific threshold, we may want to list that key names. The below script can help on that, and returns a list of Redis Keys names with: No TTL set TTL higher or equal to a user defined TTL threshold TTL lower than to a user defined TTL threshold Key value size higher or equal than a user defined Size threshold Key value size lower than a user defined Size threshold Total number of keys in the cache It also includes start and end time, and the total time spent on the keys scan. Calling getKeyStats.sh return just key names and respective TTL values and key sizes. The result depends of the threshold values used, that can be passed on script command line parameters or using the default ones. On this script, we can use a sign on both threshold values: “-“, if we want to return keys lower than that threshold; “+” or no sign, if we want to return keys higher than that threshold. This script can clarify questions like: “What are the keys I have in the cache without TTL?” (Threshold values: -1 0) “What are the keys I have in the cache with TTL, and size higher than 1KB”? (Threshold values: 0 1024) “What are the keys I have in the cache with TTL, and size smaller than 1KB ?” (Threshold values: 0 -1024) “What are the keys I have in the cache with that will expire on next 1 hour?” (Threshold values: -3600 0) “What are the keys I have in the cache with that will expire after 1 hour?” (Threshold values: +3600 0) “What are the keys I have in the cache with No TTL set and size larger than 200K?” (Threshold values: -1 204800) Output: How to run: create the below listKeys.sh file under some folder, on your Linux environment (Ubuntu 20.04.6 LTS used) give permissions to run Shell script, with command chmod 700 listKeys.sh Call the script using the syntax: ./listKeys.sh host password [port] [+/-][ttl_threshold] [+/-][size_threshold] Script parameters: host (mandatory) : the URI for the cache password (mandatory) : the Redis access key from the cache port (optional - default 10000) : TCP port used to access the cache [+/-] (optional) before ttl_threshold: indicates if we want return keys with lower "-", or higher TTL "+" than ttl_threshold ttl_threshold (optional - default 600 - 10 minutes) : Key TTL threshold (in seconds) to be used on the results (use -1 to get Keys with no TTL set) [+/-] (optional) before size_threshold: indicates if we want return keys with small size "-", or large size "+" than size_threshold size_threshold (optional - default 102400 - 100KB) : Key Size threshold to be used on the results If not provided, the default values that will be used: No TTL set (-1), and Key size threshold 102400 Bytes (100KB). Tips: use ttl_threshold = -1 to return key names with no TTL (ex: /listKeys.sh [port] -1 [+/-][size_Threshold]) use ttl_threshold = -500 to return key names with TTL below 500 seconds (ex: /listKeys.sh [port] -500 [+/-][size_Threshold]) use ttl_threshold = 500 to return key names with TTL above or equal to 500 seconds (ex: /listKeys.sh [port] 500 [+/-][size_Threshold]) use size_threshold = 0 to return key names with any size in the cache (ex: /listKeys.sh [port] [+/-][ttl_threshold] 0) use size_threshold = -1000 to return key names with size below 1000 Bytes (ex: /listKeys.sh [port] [+/-][ttl_threshold] -1000) use size_threshold = 1000 to return key names with size above or equal to 1000 Bytes (ex: /listKeys.sh [port] [+/-][ttl_threshold] 1000) use ttl_threshold = 0 AND size_threshold = 0 to return all key names with any TTL and any size in the cache (ex: /listKeys.sh [port] 0 0) use ttl_threshold = -1 AND size_threshold = 0 to return all key names with no TTL and any size in the cache (ex: /listKeys.sh [port] -1 0) Tested with: Ubuntu 20.04.6 LTS redis-cli -v redis-cli 7.4.2 Redis services: Azure Managed Redis Balanced B0 OSSMode Azure Cache for Redis Standard C1 listKeys.sh #!/usr/bin/env bash set -euo pipefail #============================== LUA script version ================= # Linux Bash Script to list Redis Keys names # It returns key names with: # - No TTL set # - with TTL higher or equal to TTL_treshold # - with TTL lower TTL_threshold # - with value size higher or equal than Size_threshold # - with value size lower than Size_threshold # - total number of keys in the cache. #------------------------------------------------------- # WARNING: # It uses LUA script (included on Bash code) to run on Redis server side. # Use it carefully, during low Redis workoads. # Do your tests first on a Dev environment, before use it on production. #------------------------------------------------------- # It requires : # redis-cli v7 or above #-------------------------------------------------------- # Usage: # listKeys.sh <cacheuri> <cacheaccesskey> [<accessport>(10000)] [+/-][<ttl_treashold>(-1)] [+/-][<size_treashold>(102400)] #======================================================== #------------------------------------------------------ # Using non-ssl port requires to remove --tls parameter on Redis-cli command below #------------------------------------------------------ sintax="<redis_host> <password> [redis_port] [+/-][ttl_threshold] [+/-][size_threshold]" REDIS_HOST="${1:?Usage: $0 $sintax}" REDISCLI_AUTH="${2:?Usage: $0 $sintax}" REDIS_PORT="${3:-10000}" # Redis port (10000, 6380, 6379) KEYTTL_THRESHOLD=${4:-"-1"} # -1, +TTL_threshold, TTL_threashold, -TTL_threshold KEYSIZE_THRESHOLD="${5:-102400}" # +Size_threshold, Size_threashold, -Size_threshold # Port number must be numeric if ! [[ "$REDIS_PORT" =~ ^[0-9]+$ ]]; then echo "ERROR: Redis Port must be numeric" exit 1 fi # Check if KEYTTL_THRESHOLD is a valid integer if ! [[ "$KEYTTL_THRESHOLD" =~ ^[-+]?[0-9]+$ ]]; then echo "Error: ttl_threshold $KEYTTL_THRESHOLD is not an integer" exit 1 fi # Check if KEYSIZE_THRESHOLD is a valid integer if ! [[ "$KEYSIZE_THRESHOLD" =~ ^[-+]?[0-9]+$ ]]; then echo "Error: Size_threshold $KEYSIZE_THRESHOLD is not an integer" exit 1 fi # Check if TTL Threasold is positive (or zero), or negative if [ "$KEYTTL_THRESHOLD" -ge 0 ]; then TTLSIGN="+" else TTLSIGN="-" fi # Check if Size Threshold is positive (or zero), or negative if [ "$KEYSIZE_THRESHOLD" -ge 0 ]; then SIZESIGN="+" size_text="larger than" else SIZESIGN="-" size_text="smaler than" fi # specific with no TTL set if [ "$KEYTTL_THRESHOLD" -eq -1 ]; then ttl_text="No TTL set" fi if [ "$KEYTTL_THRESHOLD" -ge 0 ]; then ttl_text="TTL above $KEYTTL_THRESHOLD Seconds" fi if [ "$KEYTTL_THRESHOLD" -lt -1 ]; then ttl_text="TTL below ${KEYTTL_THRESHOLD#[-+]} Seconds" fi # remove any sign KEYTTL_THRESHOLD="${KEYTTL_THRESHOLD#[-+]}" KEYSIZE_THRESHOLD="${KEYSIZE_THRESHOLD#[-+]}" echo "========================================================" echo "List all key names with $ttl_text, and Key size $size_text $KEYSIZE_THRESHOLD Bytes" # Start time start_ts=$(date +%s.%3N) echo "Start time: $(date "+%d-%m-%Y %H:%M:%S")" echo "------------------------" echo "" # Procesing redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDISCLI_AUTH" --tls --no-auth-warning EVAL " local cursor = '0' local ttl_threshold = tonumber(ARGV[1]) -- KEYTTL_THRESHOLD local ttl_sign = ARGV[2] -- TTLSIGN local size_threshold = tonumber(ARGV[3]) -- KEYSIZE_THRESHOLD local size_sign = ARGV[4] -- SIZESIGN local output = {} local count = 0 local totalKeys = 0 local strKeyTTL = '' local strKeySize = '' -- Scanning keys in the cache table.insert(output, '--------------------------------------') repeat local res = redis.call('SCAN', cursor, 'COUNT', 100) cursor = res[1] for _, k in ipairs(res[2]) do local ttl = redis.call('TTL', k) local size = redis.call('MEMORY','USAGE', k) totalKeys = totalKeys + 1 if (size_sign == '+' and size >= size_threshold) or (size_sign == '-' and size < size_threshold) then -- TTL == -1 → no expiration if ttl_sign == '-' and ttl_threshold == 1 then if ttl == -1 then table.insert(output, k .. ': TTL: -1, Size: ' .. size .. ' Bytes') count = count + 1 end -- TTL comparisons (exclude -1 and -2) else if ttl >= 0 then if ttl_sign == '-' and ttl < ttl_threshold then table.insert(output, k .. ': TTL: ' .. ttl .. ' seconds, Size: ' .. size .. ' Bytes') count = count + 1 elseif ttl_sign == '+' and ttl >= ttl_threshold then table.insert(output, k .. ': TTL: ' .. ttl .. ' seconds, Size: ' .. size .. ' Bytes') count = count + 1 end end end end end until cursor == '0' -- Adding summary to output table.insert(output, '--------------------------------------') if (size_sign == '+') then strKeySize = 'larger' else strKeySize = 'smaler' end strKeySize = 'size ' .. strKeySize .. ' than ' .. size_threshold .. ' Bytes' if ttl_sign == '-' and ttl_threshold == 1 then strKeyTTL = 'No TTL' elseif ttl_sign == '-' then strKeyTTL = 'TTL < ' .. ttl_threshold .. ' seconds' elseif ttl_sign == '+' then strKeyTTL = 'TTL >= ' .. ttl_threshold .. ' seconds' end strKeyTTL = ' keys found with ' .. strKeyTTL table.insert(output, 'Scan completed.') table.insert(output, 'Total of ' .. totalKeys .. ' keys scanned.') table.insert(output, count .. strKeyTTL .. ', and ' .. strKeySize) table.insert(output, '--------------------------------------') return output " 0 "$KEYTTL_THRESHOLD" "$TTLSIGN" "$KEYSIZE_THRESHOLD" "$SIZESIGN" echo " " end_ts=$(date +%s.%3N) echo "End time: $(date "+%d-%m-%Y %H:%M:%S")" # Duration - Extract days, hours, minutes, seconds, milliseconds duration=$(awk "BEGIN {print $end_ts - $start_ts}") days=$(awk "BEGIN {print int($duration/86400)}") hours=$(awk "BEGIN {print int(($duration%86400)/3600)}") minutes=$(awk "BEGIN {print int(($duration%3600)/60)}") seconds=$(awk "BEGIN {print int($duration%60)}") milliseconds=$(awk "BEGIN {printf \"%03d\", ($duration - int($duration))*1000}") echo "Duration : ${days} days $(printf "%02d" "$hours"):$(printf "%02d" "$minutes"):$(printf "%02d" "$seconds").$milliseconds" echo "========================================================" Performance: Redis service used: Azure Managed Redis Balanced B0 OSSMode # ------------------ Scan completed. Total keys listed: 46005 Duration : 0 days 00:00:01.437 # ------------------ Redis service used: Azure Cache for Redis - Standard - C1 Scan completed. Total keys listed: 1225 Duration : 0 days 00:00:00.545 # ------------------ WARNING: The above script uses LUA script, that runs on Redis side, and may block you normal workload. Use it carefully when have a large number of keys in the cache, and during low workload times. YOU CAN RUN THE BELOW SCRIPTS AT YOUR OWN RISK. WE DON'T ASSUME ANY RESPONSABILITY FOR UNEXPECTED RESULTS. References Azure Managed Redis Azure Best Practice for Development Redis Inc - Commands Redis LUA - Lua API reference Redis Inc - How Redis expires keys Redis CLI Bash Script xargs man page awk man page I hope this can be useful !!!
Azure Cache for Redis
Hello, I am trying to optimize the cost for Azure Cache for Redis. Right now i have an E100 sku deployed and checking if there is a possibility to right size the instance to a lower tier. I checked the utilization metrics and could find low CPU, memory and low server load. However i am struggling to understand the metrics for the connected clients as this is one of the important criteria for rightsizing. Can someone please tell me if i need to look the MAX or SUM aggregation for connected clients? As per azure pricing E100 include upto 100,000 connection but i would like to understand if it is concurrent connection during a month or summation of all connection during a month. Can someone please help me to understand this?
Setting Up WSL for Secure Redis CLI Connections to Azure Redis Cache on Windows.
This blog will guide you through the necessary steps to set up WSL on your Windows machine, enabling you to harness the power of Redis CLI and securely connect to an Azure Redis Cache instance by running a Linux environment with Redis CLI on your Windows machine, without the need for a separate virtual machine or dual booting.
Leveraging Redis Insights for Azure Cache for Redis
The blog talks about how to leverage Redis Insights GUI tool while working with Azure Cache for Redis. We will look at some of the option what will help us with some high-level connectivity troubleshooting and insights to our data present inside the cache. To start with, we can leverage this for testing the connectivity to our Redis cache instance. After clicking on Add Redis Database button, we can fill in the other fields ahead: Host: Complete FQDN or the completed Redis cache Endpoint For Basic, Standard & Premium Tier - <Cachename>.redis.windows.net For Enterprise Tier - <Cachename>.<regionname>.redisenterprise.cache.azure.net Port: 6380 or 6379 (depending on whether we are testing for SSL or non-SSL port respectively) / 10000 for Enterprise Tier Database Alias: Cache name Password: Access Key for your cache Use TLS: Option to be checked for testing with 6380 port and also Enterprise Tier cache. Post that, we can click on Test Connection button which will help us doing a high-level check whether the cache endpoint is reachable or not. SSL Port: Non-SSL Port: Enterprise Cache: Once the test connection is successful, you can click on Add Redis Database to start exploring the insights of your cache instance. Note: All the above demo has been done without any kind of firewall, private endpoints or VNET restrictions. In case you are having VNET or private endpoints configured, then you have test it from a VM which is part of the VNET configured. On clicking My Redis Database option, it will list down all the databases you have connected too from the Redis Insights along with some high-level details such as modules if any for the enterprise tier or OSS Cluster as connection type if clustering policy selected was OSS. If Enterprise, it shows as Standalone only. For this Demo, we took an empty cache and the below snippet demonstrate on how your can-do simple Set operations or add a key to your cache instance. We can add a new key by providing the key type such as Set, String, List, Hast etc, Key name, TTL etc. We have added 3 keys initially and it will start reflecting in the left-hand window section as depicted below: Similarly, we added the keys further and all of them started listing. Selecting any of the keys shall provide insights to that particular key on the right sight window such as value, TTL , key size etc. You can also use this to do any kind of pattern match as well. E.g. In the below snippet, we tried listing all the keys that start key name as testkey. There is a Bulk Actions button available as well which has mainly 2 option available: Perform bulk deletion. Execute multiple set of Redis commands in a sequential format which can be uploaded as a plain text file. Moving ahead, there is an Analysis Tool option which can be leveraged to gain insight to the data summary residing in our cache. There is a New Report button which will generate a report providing various kind of insights on data residing in the cache. Below are some of the highlights: It provides a high-level summary key based on type. It gives you a view of how much data in under No Expiry (no TTL set) and is expected to get freed in expected time (based on TTL set). In the below example, it points around ~450 bytes of memory to get freed in less than an hour while there is approx. ~1200 bytes of data which don’t have any kind of TTL set and will not expire. It further provides high level details of top keys based on TTL or Key size which can be used to identify larger size keys. There is also a Workbench option that provides a command line option like Redis CLI, using which we can execute commands. In the below example, we have used it to do PING-PONG test, set up the keys and other operations too. Disclaimer: Please note that tool is supported by REDIS and not Azure cache for Redis so we don’t control the behavior or features for the tool. Hope that helps!
Use Rdbtools to Generate Memory Report for Azure Cache for Redis and Convert Dump Files to JSON
The rdbtools is a handy utility that can parse Redis dump.rdb files, analyze memory, and export data to JSON. This tutorial shows a quick example using rdbtools to generate report for Azure cache for Redis, on memory usage and number of keys categorized by data type and data encoding.
Common causes of SSL/TLS connection issues and solutions
In the TLS connection common causes and troubleshooting guide (microsoft.com) and TLS connection common causes and troubleshooting guide (microsoft.com), the mechanism of establishing SSL/TLS and tools to troubleshoot SSL/TLS connection were introduced. In this article, I would like to introduce 3 common issues that may occur when establishing SSL/TLS connection and corresponding solutions for windows, Linux, .NET and Java. TLS version mismatch Cipher suite mismatch TLS certificate is not trusted TLS version mismatch Before we jump into solutions, let me introduce how TLS version is determined. As the dataflow introduced in the first session(https://techcommunity.microsoft.com/t5/azure-paas-blog/ssl-tls-connection-issue-troubleshooting-guide/ba-p/2108065), TLS connection is always started from client end, so it is client proposes a TLS version and server only finds out if server itself supports the client's TLS version. If the server supports the TLS version, then they can continue the conversation, if server does not support, the conversation is ended. Detection You may test with the tools introduced in this blog(TLS connection common causes and troubleshooting guide (microsoft.com)) to verify if TLS connection issue was caused by TLS version mismatch. If capturing network packet, you can also view TLS version specified in Client Hello. If connection terminated without Server Hello, it could be either TLS version mismatch or Ciphersuite mismatch. Solution Different types of clients have their own mechanism to determine TLS version. For example, Web browsers - IE, Edge, Chrome, Firefox have their own set of TLS versions. Applications have their own library to define TLS version. Operating system level like windows also supports to define TLS version. Web browser In the latest Edge and Chrome, TLS 1.0 and TLS 1.1 are deprecated. TLS 1.2 is the default TLS version for these 2 browsers. Below are the steps of setting TLS version in Internet Explorer and Firefox and are working in Window 10. Internet Explorer Search Internet Options Find the setting in the Advanced tab. Firefox Open Firefox, type about:config in the address bar. Type tls in the search bar, find the setting of security.tls.version.min and security.tls.version.max. The value is the range of supported tls version. 1 is for tls 1.0, 2 is for tls 1.1, 3 is for tls 1.2, 4 is for tls 1.3. Windows System Different windows OS versions have different default TLS versions. The default TLS version can be override by adding/editing DWORD registry values ‘Enabled’ and ‘DisabledByDefault’. These registry values are configured separately for the protocol client and server roles under the registry subkeys named using the following format: <SSL/TLS/DTLS> <major version number>.<minor version number><Client\Server> For example, below is the registry paths with version-specific subkeys: Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client For the details, please refer to Transport Layer Security (TLS) registry settings | Microsoft Learn. Application that running with .NET framework The application uses OS level configuration by default. For a quick test for http requests, you can add the below line to specify the TLS version in your application before TLS connection is established. To be on a safer end, you may define it in the beginning of the project. ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 Above can be used as a quick test to verify the problem, it is always recommended to follow below document for best practices. https://docs.microsoft.com/en-us/dotnet/framework/network-programming/tls Java Application For the Java application which uses Apache HttpClient to communicate with HTTP server, you may check link How to Set TLS Version in Apache HttpClient | Baeldung about how to set TLS version in code. Cipher suite mismatch Like TLS version mismatch, CipherSuite mismatch can also be tested with the tools that introduced in previous article. Detection In the network packet, the connection is terminated after Client Hello, so if you do not see a Server Hello packet, that indicates either TLS version mismatch or ciphersuite mismatch. If server is supported public access, you can also test using SSLLab(https://www.ssllabs.com/ssltest/analyze.html) to detect all supported CipherSuite. Solution From the process of establishing SSL/TLS connections, the server has final decision of choosing which CipherSuite in the communication. Different Windows OS versions support different TLS CipherSuite and priority order. For the supported CipherSuite, please refer to Cipher Suites in TLS/SSL (Schannel SSP) - Win32 apps | Microsoft Learn for details. If a service is hosted in Windows OS. the default order could be override by below group policy to affect the logic of choosing CipherSuite to communicate. The steps are working in the Windows Server 2019. Edit group policy -> Computer Configuration > Administrative Templates > Network > SSL Configuration Settings -> SSL Cipher Suite Order. Enable the configured with the priority list for all cipher suites you want. The CipherSuites can be manipulated by command as well. Please refer to TLS Module | Microsoft Learn for details. TLS certificate is not trusted Detection Access the url from web browser. It does not matter if the page can be loaded or not. Before loading anything from the remote server, web browser tries to establish TLS connection. If you see the error below returned, it means certificate is not trusted on current machine. Solution To resolve this issue, we need to add the CA certificate into client trusted root store. The CA certificate can be got from web browser. Click warning icon -> the warning of ‘isn’t secure’ in the browser. Click ‘show certificate’ button. Export the certificate. Import the exported crt file into client system. Windows Manage computer certificates. Trusted Root Certification Authorities -> Certificates -> All Tasks -> Import. Select the exported crt file with other default setting. Ubuntu Below command is used to check current trust CA information in the system. awk -v cmd='openssl x509 -noout -subject' ' /BEGIN/{close(cmd)};{print | cmd}' < /etc/ssl/certs/ca-certificates.crt If you did not see desired CA in the result, the commands below are used to add new CA certificates. $ sudo cp <exported crt file> /usr/local/share/ca-certificates $ sudo update-ca-certificates RedHat/CentOS Below command is used to check current trust CA information in the system. awk -v cmd='openssl x509 -noout -subject' ' /BEGIN/{close(cmd)};{print | cmd}' < /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem If you did not see desired CA in the result, the commands below are used to add new CA certificates. sudo cp <exported crt file> /etc/pki/ca-trust/source/anchors/ sudo update-ca-trust Java The JVM uses a trust store which contains certificates of well-known certification authorities. The trust store on the machine may not contain the new certificates that we recently started using. If this is the case, then the Java application would receive SSL failures when trying to access the storage endpoint. The errors would look like the following: Exception in thread "main" java.lang.RuntimeException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at org.example.App.main(App.java:54) Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:371) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:314) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:309) Run the below command to import the crt file to JVM cert store. The command is working in the JDK 19.0.2. keytool -importcert -alias <alias> -keystore "<JAVA_HOME>/lib/security/cacerts" -storepass changeit -file <crt_file> Below command is used to export current certificates information in the JVM cert store. keytool -keystore " <JAVA_HOME>\lib\security\cacerts" -list -storepass changeit > cert.txt The certificate will be displayed in the cert.txt file if it was imported successfully.
