Rico's Blog

Back

A subtle but painful bug recently cost me a significant amount of debugging time. This post documents what went wrong, what the root cause was, and how to avoid the same trap.

The Setup#

I have a Node.js application using a single persistent ioredis client shared across API endpoints. Two of those endpoints read from different Redis databases:

  • /endpoint0 reads from db 0 using a straightforward GET:
const res = await client.get(someKey)
js
  • /endpoint2 reads from db 2 using a pipeline with SELECT:
const [res1, res2] = await client.multi().select(2).hget(hash, key).exec()
js

/endpoint0 was added well after /endpoint2 was already in production. Shortly after it was introduced, an intermittent failure started appearing — calls to /endpoint0 would begin returning incorrect results some time after a fresh deployment, and a redeployment would temporarily resolve it.

The Investigation#

The bug manifested across all environments — dev, UAT, and production — which ruled out environment-specific configuration issues. I checked dependency declaration order, environment variable handling, and the deployment pipeline itself. None of those investigations turned up anything.

The breakthrough came when I inspected the output of CLIENT LIST directly on the Redis server:

id=501766576 addr=192.168.1.57:61819 fd=36 name= age=701 idle=623 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=hget

id=501692280 addr=192.168.1.57:24319 fd=10 name= age=3619 idle=4 flags=N db=2 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
log

The db=2 field on the second connection immediately stood out. A connection that should have been operating on db 0 was sitting on db 2 — which pointed directly at the select call in /endpoint2.

The Root Cause#

SELECT in Redis — and by extension client.select() or client.multi().select() in ioredis — is a stateful, connection-level operation. It does not scope to a single command or pipeline; it changes the active database for that connection and the change persists until another SELECT is issued.

When a shared connection is used, any call to SELECT affects every subsequent command on that connection, regardless of which endpoint or function issued them. In this case, once /endpoint2 ran and switched the connection to db 2, any subsequent call from /endpoint0 was unknowingly querying db 2 instead of db 0. Refer to this diagram for a better understanding of the problem:

mermaid chart explaining the bug

The Fix#

In this specific case, the simplest resolution was to migrate the data for /endpoint2 into db 0, eliminating the need for SELECT entirely:

const res = await client.hget(hash, key)
js

The more robust and general solution — especially in applications where multiple databases are genuinely required — is to create a dedicated ioredis client per database. Calling SELECT at runtime on a shared connection is a latent bug waiting to surface, and it will do so in ways that are difficult to reproduce and trace.

Takeaway#

If you are using ioredis with multiple Redis databases in a shared-connection setup, avoid SELECT at runtime. Provision one client per database instead, and make the database assignment explicit at initialization. It is a small upfront cost that eliminates an entire class of hard-to-debug, timing-dependent failures.