Using Redis in Standalone and Cluster Modes with Kong Gateway and AWS ElastiCache

If you are new to the Kong gateway or custom plugin development, please follow the guide at https://konghq.com/blog/custom-lua-plugin-kong-gateway. And to learn more about our company follow Zelarsoft LLC

In a gist, Lua plugins are a type of plugin for Kong Gateway that allow you to execute Lua scripts as part of the request/response flow. In addition, these plugins will enable you to add custom functionality, such as authentication, rate limiting, and logging, to your API Gateway without modifying the underlying Nginx configuration.

Assuming you know the plugins’ basic setup, testing process, and deployment steps, the code samples below explain the next steps.

Redis in standalone mode

Connecting to Redis Instance in Lua

The most popular way of using Redis in Lua is with lua-resty-redis

This is a sample code to connect to Redis on local, VM, or docker.

local redis = require 'resty.redis'
local red = redis:new()
red:set_timeout(10000) -- in milliseconds

kong.log.info("redisConn:", red)

local sock_opts = {}
sock_opts.ssl = false

local ok, err = red:connect(host, port, sock_opts)
if not ok then
  kong.log.err("Failed to connect to Redis: ", err)
  return nil, err
end

-- put it into the connection pool of size 100,
-- with 10 seconds max idle time
local ok, err = red:set_keepalive(10000, 100)
if not ok then
  ngx.say("Failed to set keepalive: ", err)
  return
end

Connecting to Redis database in AWS (Cluster mode disabled)

Another popular library for using Redis is https://github.com/nrk/redis-lua

local redis = require 'redis'
local client = redis.connect('test.abc.0001.use1.cache.amazonaws.com', 6379)

local response = client:ping()       -- true
print(response)

client:set('name', 'TestUser')
client:set('city', 'London')

print(client:get('name'))
print(client:get('city'))

However, both these libraries only work for Redis in standalone mode, which is different from what I need to set up a more scalable enterprise-grade database setup using Kong gateway.

Redis in cluster mode

https://github.com/Kong/resty-redis-cluster

Connecting to Redis Instance in Lua

I am running Kong in the docker container using the below command.

The env. variable is mandatory here to connect to Redis in cluster mode. Otherwise, I kept getting a few errors, such as —

1. no slots information present, nginx might have never successfully executed cluster(“slots”)

2. Failed to connect to Redis: (failed to fetch slots: timeout; closed)

docker run -d --cpus="2" --name kong-demo \
  ……
  ……
  -e "KONG_NGINX_HTTP_LUA_SHARED_DICT=redis_cluster_slot_locks 100k" \
  kong-demo

Install the following luarocks

luarocks install kong-redis-cluster

luarocks install lua-resty-lock

local config = {
    dict_name = "test_locks",               --shared dictionary name for locks, if default value is not used
    refresh_lock_key = "refresh_lock",      --shared dictionary name prefix for lock of each worker, if default value is not used
    name = "testCluster",                   --rediscluster name
    serv_list = {                           --redis cluster node list(host and port),
        { ip = "172.18.0.5", port = 6379 },
        { ip = "172.18.0.6", port = 6379 },
    },
    keepalive_timeout = 60000,              --redis connection pool idle timeout
    keepalive_cons = 1000,                  --redis connection pool size
    connect_timeout = 1000,              --timeout while connecting
    max_redirection = 5,                    --maximum retry attempts for redirection
    max_connection_attempts = 1             --maximum retry attempts for connection
}

local redis_cluster = require "rediscluster"
local red_c = redis_cluster:new(config)

print("hello world")

local v, err = red_c:setnx("city", "London")
local v, err = red_c:get("city")

if err then
    ngx.log(ngx.ERR, "err: ", err)
else
    ngx.say(v)
end

Connecting to Redis database in AWS (Cluster mode enabled)

local config = {
    refresh_lock_key = "refresh_lock",      --shared dictionary name prefix for lock of each worker, if default value is not used
    name = "testCluster",                   --rediscluster name
    serv_list = {                           --redis cluster node list(host and port),
         { ip = "clustercfg.test.abc.use1.cache.amazonaws.com", port = 6379 },
    },
    keepalive_timeout = 60000,              --redis connection pool idle timeout
    keepalive_cons = 1000,                  --redis connection pool size
    connect_timeout = 1000,              --timeout while connecting
    max_redirection = 5,                    --maximum retry attempts for redirection
    max_connection_attempts = 1
}

This works fine as long as you disable the encryption in AWS for this cluster. If you need to enable the encryption for your ElastiCache Redis instance, add the following lines to the config in the plugin codes to make a successful connection.

local config = {
    ......
    connect_opts = {
        ssl = true,
    }
}

M