What is Redis?
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams.
Concept of Redis Cluster
Redis Clustering provides a consistent and resilient data service where data is automatically sharded across multiple Redis nodes. And it provides a master/slave setup for enhance availability in case of a failure and to support distributed environment.
Each slave replicates a specific master and can be reassigned to replicate another master or be elected to a master node as needed.
Redis Cluster Topology
Minimal cluster that works as expected requires to contain at least 3 master nodes in the cluster and Redis recommendation is to have at least one slave for each master.
Redis Cluster TCP ports
Every Redis Cluster node requires two TCP connections open. The normal Redis TCP port used to serve clients, plus the port obtained by adding 10000 to the data port.
This second high port is used for the Cluster bus, that is node-to-node communication channel using a binary protocol. The Cluster bus is used by for failure detection, configuration update, failover authorization and so forth. Clients should never try to communicate with the cluster bus port, but always with the normal Redis command port.
Note that for a Redis Cluster to work properly you need, for each node:
1. The normal client communication port used to communicate with clients to be open to all the clients that need to reach the cluster, plus all the other cluster nodes (that use the client port for keys migrations).
2. The cluster bus port (the client port + 1000) must be reachable from all the other cluster nodes.
Redis Cluster data sharding
Redis Cluster does not use consistent hashing, but a different form of sharding where every key is conceptually part of what we call a hash slot. There are 16384 hash slots in Redis Cluster.
Every node in a Redis Cluster is responsible for a subset of the hash slots, so for example you may have a cluster with 3 nodes, where:
· Node A contains hash slots from 0 to 5500.
· Node B contains hash slots from 5501 to 11000.
· Node C contains hash slots from 11001 to 16383.
In order to remain available when a subset of master nodes are failing or are not able to communicate with the majority of nodes, Redis Cluster uses a master-slave model where every hash slot has from 1(the master itself) to N replicas (N-1 additional slaves nodes).
When the cluster is created(or at a later time) we add a slave node to every master, so that the final cluster is composed of A, B, C that are masters nodes, and A1, B1, C1 that are slaves nodes, the system is able to continue if node B fails.
Node B1 replicates B, and B fails, the cluster will promote node B1 as the new master and will continue to operate correctly.
Fault tolerance and Recoverability
When a master fails or is found to be unreachable, the appointed slave will take the master’s place and continue to operate correctly as mentioned above.
When the failing master eventually rejoins the cluster, it will join as a slave and begin to replicate another master.
Redis Cluster is not be able to guarantee strong consistency. In practical terms this means that under certain conditions, it is possible that Redis Cluster will lose writes that were acknowledged by the system to the client.
The first reason why Redis Cluster can lose writes is because it uses asynchronous replication. This means that during writes the following happens:
· Your client writes to the master B.
· The master replies OK to your client.
· The master B propagates the write to its slaves B1, B2, and B3.
As you can see, B does not wait for an acknowledgement from B1, B2, and B3 before replying to the client, since this would be a prohibitive latency penalty for Redis, so if your client writes something, B acknowledges the write, but crashes before being able to send the write to its slaves, one of the slaves (that did not receive the write) can be promoted to master, losing the write forever.
There is another notable scenario where Redis Cluster will lose writes, that happens during a network partition where a client is isolated with a minority of instances including at least a master.
Any processing cycles are dedicated to serving users’ requests, which improves the overall performance. For example, each Redis shard in a Redis Cluster works as if it were a standalone Redis instance. The shard doesn’t need to monitor other Redis instances, has no need to deal with failure or partition events, and is unaware of which hash-slots are being managed.
Any configuration operation is allowed through a secure UI, CLI, or API interface that follows role-based authorization controls. The proxy-based architecture ensures that only certified connections can be created with each shard, and only certified requests can be received by Redis shards.
The re-sharding mechanism of Redis is based on Redis replication. Whenever a shard needs to be scaled out, the Redis Cluster launches another Redis instance and replicates to it half of the hash slots of the original shard. Once the new shard is populated, traffic is sent to both shards by the proxy in a manner completely transparent to the application.
There are two ways to scale database shards:
1. Scaling up by adding shards to your database without adding nodes to your cluster. This scenario is useful when there is enough un-utilized capacity in the cluster.
2. Scaling out by adding node(s) to the Redis Cluster, rebalancing and then re-sharding your database. This scenario is useful if more physical resources are needed in your cluster in order to scale your database.