🔎 Spike: Investigate Migrating In-Cluster Redis to AWS ElastiCache

[jasonBirchall]

User Need

As a member of the AI Team and an SRE
I want to investigate replacing our in-cluster Redis instance with AWS ElastiCache
so that we can avoid memory constraints, reduce manual intervention, and ensure high availability for our application

---

Functional Requirements (What):

• Work with the AI SRE team to set up a spike to understand the feasibility and benefits of using AWS ElastiCache versus in-cluster Redis
• Get a working prototype of ElastiCache terraform module for users to insert into their applications.
• Identify potential changes required in our application code/configuration to point to ElastiCache
• Document the expected cost, performance implications, and any operational overhead
• Ensure the AI SRE team can connect their application environment to their ElastiCache instance

---

Non-Functional Requirements (How):

• Ian and Roch should have a clear understanding how to provision, access and maintain an ElastiCache instance.

---

Acceptance Criteria:

• A proof of concept or test environment has been configured to trial AWS ElastiCache
• A recommendation (and migration plan if proceeding) is presented to both the AI Team and Platform Engineering
• Any risks or blockers are identified, with clear next steps for resolution

---

Notes:

• This spike originates from a postmortem where our in-cluster Redis instance OOM’d and required manual scaling by Platform Engineering.
• The aim is to prevent similar incidents by offloading Redis responsibilities to a managed solution.
• The existing AI Team ticket about splitting read/write pods for Search API (Trello #4dC5b61X) can be considered separately, but this spike will inform that effort if caching is required.
• Once completed, we can apply learnings and rollout similar migrations to other teams running Redis in-cluster if beneficial.