Indian Caching Architectures: Correctness vs Latency vs Cost

Myntra, Zerodha, Flipkart, and JioHotstar all cache. They are also solving four different problems, and the word “cache” hides that.

Myntra's question is roughly: how stale can inventory get before we sell something that doesn't exist anymore?

Zerodha's question: how do we keep a trading screen fast and predictable when a few hundred thousand people refresh it at the same second?

Flipkart gets to ask a more relaxed question for some data: is it fine if a product review is a few seconds old, if that drops p99 from roughly 400ms to roughly 10ms?

And JioHotstar's question isn't even about latency. It's: how much traffic can we kill at the edge so it never touches origin at all?

Same word. Four architectures.

“Senior, the product API is slow. Should we add Redis?”

The senior looked up from his screen.

“What are you optimizing?”

“Latency.”

“Only latency?”

“And database load.”

“How stale can the response be?”

“I don't know.”

“What happens if Redis is down?”

“Haven't thought about it.”

“Can two users see different values?”

“Maybe?”

“Can we lose an update?”

“Definitely not.”

He smiled. “Then you don't have a Redis question. You have a correctness model you haven't written down yet.”

This is the mistake most of us make with caching, me included for the first few years. We start with the technology: Redis, Memcached, Caffeine, Hazelcast, a CDN, or just a Go map with a mutex around it. The technology is honestly the least interesting part. The real architecture comes from five questions:

“Let's start with Zerodha,” the senior said.

“Financial app. So the cache has to prioritize correctness?”

“Not exactly. The Kite read architecture they've written about is one of the most aggressive latency plays I've seen described publicly.”

Zerodha has written that every piece of data shown in Kite, including orders, positions and portfolios, was served from a hot Redis cache. A typical GET did an O(1) lookup, read an already-prepared byte payload, and wrote it straight to the HTTP connection. No database query, no business computation, not even JSON serialization on the hot path. A separate asynchronous service produced that data and placed it in Redis.

That's not cache-aside. It's closer to a materialized read model.

Copy
Classic cache-aside
request → Redis miss → database → build response → Redis → client

Zerodha's described common path
async projection → pre-serialized Redis
request → Redis → write bytes to HTTP connection

The database isn't on the interactive read path at all, not even as the miss fallback in the usual sense. The Redis instances backing this were explicitly ephemeral: Zerodha described disabling disk persistence for extra performance and running multiple independent instances for availability. They also cached one layer closer to the user: Kite APIs returned ETags, so web and mobile clients got an HTTP 304 Not Modified instead of re-downloading unchanged JSON. Zerodha said this saved terabytes of bandwidth a month.

“It would be,” the senior said, “if you confused the cached representation with the authoritative transaction state.”

Zerodha separately describes relational databases holding historical financial data: trades, P&L records, and ledger entries. Those go through end-of-day reconciliation against exchange data.

So the lesson is not “financial systems can trust Redis as the final authority.” The lesson is that a correctness-sensitive system can still run an aggressively optimized, throwaway read model, as long as the authoritative transaction path stays separate. From the public posts we can study the display and reporting side. We can't infer that order-placement decisions come off the same Redis values, and I'd bet money they don't.

What Zerodha is optimizing: predictable read latency, minimal database work on interactive paths, low CPU per response, less client bandwidth, and operational simplicity. The cache isn't storing database rows. It's storing a response that's already ready to transmit, which is a genuinely different idea.

“Now Myntra. Here correctness stops being abstract.”

Myntra's inventory system had to serve roughly 100 million inventory cache reads per minute. Their earlier central cache got inventory changes asynchronously, and the team listed the problems with that: momentary cache inconsistencies, overselling risk during sale events, duplicate operational ownership, a central-cache single point of failure, and maintenance pain whenever the data model changed.

Their first alternative looked like write-through: keep MySQL as the persistent ACID store, update it on writes, and synchronously update Redis so Redis serves the reads. Freshness improved. Then the sizing math came in.

To serve roughly 100 million requests per minute for only about 10 GB of data, they estimated the Redis design would need roughly 80 to 100 nodes. Read that again. The data fits in the RAM of a laptop. The expensive part was networked cache throughput, not storage.

“Do you see the problem?”

“The cache was being sized for request volume, not data size.”

“Exactly.”

Myntra then looked at an embedded application cache. Local memory kills the network round trip, but now you have cache warming, restarts, and synchronization to deal with. The design they landed on, per the engineering post, was a hybrid: MySQL as source of truth, change propagation into a Hazelcast cluster as a shared fallback, and a near cache inside each application instance holding the hot values.

Copy
MySQL (source of truth)
  → async change propagation → Hazelcast cluster (shared fallback)
    → near cache in every application instance → inventory reads

Periodic and business-triggered reconciliation repair drift.

Myntra reported the near-cache design cut required resources by about 80%, because most inventory reads never left application memory and a small shared cluster handled fallback traffic.

The final design still propagated committed MySQL changes to Hazelcast asynchronously. So Myntra added two repair mechanisms:

This is the part juniors skip past. They see a near cache and think latency. The actual architecture is a negotiated agreement: browsing inventory can tolerate a small inconsistency window, final inventory blocking gets a stronger check, propagation is allowed to fail temporarily, reconciliation has to eventually catch the drift, and MySQL stays the source of truth throughout.

What Myntra is optimizing: massive read throughput and lower infra cost, yes. But also controlled inventory inconsistency, with the critical inventory-blocking path protected separately.

The senior put it this way: “Myntra didn't solve correctness by finding a perfectly consistent cache. They solved it by deciding where cached inventory was acceptable and where the workflow had to check reality again.”

“Does every value need the same freshness?”

“Probably not.”

“Then why does every cache entry in your service have the same TTL?”

Ouch. Flipkart's engineering work is basically a long argument that cache policy should follow the semantics of the data, not the other way around.

Flipkart open-sourced a library called GraceKelly built around refresh instead of expiry. The problem it attacks is one every backend engineer has watched happen: a popular key expires, thousands of requests miss at once, all of them stampede the backend, latency climbs, downstream services wobble.

Instead of deleting an expired value, GraceKelly could keep serving the existing value while kicking off a request-driven background refresh. If the refresh failed, the old value could stick around or get its TTL extended.

Copy
fresh window              stale-but-served window                 hard expiry
|----------------|------------------------------------------|
return fresh      return stale + one background refresh          block on origin

This is the same idea HTTP standardized as stale-while-revalidate: return stale content immediately, refresh asynchronously, and hide origin latency from the caller. RFC 5861 also defines stale-if-error, which lets you serve stale data instead of a hard 5xx for a bounded period.

Flipkart's own writing makes the business split explicit. Stock availability should not tolerate stale data. Ratings and reviews can be stale for a short while. That's not a technical distinction. It's a product decision expressed as cache policy.

In a separate ratings-and-reviews architecture, Flipkart shrank and reorganized its cached data, then put the hottest values inside the application JVMs using Caffeine. During peak sale periods they reported over 95% hit rate in the app-level cache, p99 dropping from roughly 400 ms to roughly 10 ms, about 5x more throughput per host, distributed cached data falling from terabytes to a few hundred gigabytes, and a large Redis and Aerospike footprint replaced by a much smaller cluster.

None of that came from buying a bigger Redis cluster. It came from deleting duplicate data, encoding values more efficiently, storing fewer review pages, moving hot objects into process memory, and accepting that refresh and cold starts now needed explicit handling.

What Flipkart is optimizing: tail latency and predictable backend load, plus throughput per host, memory footprint, cost, and availability when refreshes fail. It works because a review being briefly old hurts less than a product page failing to load.

“Staleness,” the senior said, “isn't automatically a correctness bug. It becomes one when it breaks the product's contract.”

The junior pointed at the architecture diagram. “So Hotstar just puts a CDN in front?”

The senior laughed. “At tens of terabits per second, you do not ‘just put a CDN in front.’ The CDN is part of the application.”

Disney+ Hotstar documented scaling to a record 59 million concurrent streams. Client apps sent requests through CDNs acting as an external API gateway before anything hit internal gateways and backend services. Those edge nodes were tuned for cache-hit traffic. But once the same nodes had to do security checks, rate controls, request transformation and proxying at millions of requests per second, compute became the constraint, not just bandwidth.

So the team split APIs into cacheable and non-cacheable. Features like scorecards, concurrency data and key moments moved to a separate CDN domain with leaner security and routing rules. They also tuned refresh frequencies for scorecards and live feeds to take pressure off peak bandwidth and edge compute.

A later JioHotstar post described live events needing roughly 60 to 80 Tbps of network bandwidth, and a QoS routing manager that dynamically shifts cohorts of users across different CDNs based on location, network provider, observed video quality and capacity.

Copy
QoS routing control plane
          ↓
users → CDN A / CDN B / CDN C → shield → origin
          ↑
location, ISP, video quality, and capacity feedback

Hotstar has also warned about “TTL cloudbursts.” When lots of popular values expire around the same moment, requests that caches were quietly absorbing suddenly land on origin together. Traffic didn't grow; the cache just stopped soaking it up. For a normal backend that means extra database load. For a live streaming platform, one bad TTL can manufacture an origin event out of millions of requests.

That's why CDN caching here isn't about making one request faster. It controls origin request volume, edge compute, network bandwidth, failure isolation, delivery cost, and whether traffic spikes ever reach the backend at all. The team explicitly found that scaling CDN capacity for the originally projected API volume was neither cost-effective nor fast enough, which is what pushed them to separate and simplify the cacheable paths instead.

What JioHotstar is optimizing: origin protection and event capacity first. Then edge and network cost, multi-CDN resilience, last-mile playback quality, and predictability during spikes. For this architecture, cache-hit ratio isn't an application metric. It's the capacity plan.

The senior tapped the table. “Notice anything?”

“They're not choosing between Redis and Memcached.”

“Correct. They're choosing which failure mode their business can survive.”

To make these trade-offs measurable instead of theoretical, build a small Go repo with three strategies behind one API. The goal isn't to clone any company's internals. It's to reproduce the decisions: cache-aside, write-through, and TTL plus stale-while-revalidate.

Copy
cache-architecture-lab/
├── cmd/api/main.go
├── cmd/bench/main.go
├── internal/cache_aside.go
├── internal/write_through.go
├── internal/stale_revalidate.go
├── internal/cache/{redis.go,local.go}
├── internal/store/postgres.go
├── internal/fault/injector.go
├── internal/metrics/metrics.go
├── migrations/
├── docker-compose.yml
├── Makefile
└── README.md

One common interface keeps the comparison honest:

go
Copy
1type Strategy interface {
2    Get(ctx context.Context, key string) (Item, ReadMetadata, error)
3    Put(ctx context.Context, item Item) error
4}
5
6type ReadMetadata struct {
7    Source   string // cache, database, or stale-cache
8    Age      time.Duration
9    WasStale bool
10}

Expose the strategy as a query parameter:

Copy
GET /v1/items/42?strategy=aside
GET /v1/items/42?strategy=through
GET /v1/items/42?strategy=stale
PUT /v1/items/42?strategy=through

Copy
GET: read Redis → on miss read Postgres → populate Redis
PUT: update Postgres → delete Redis key

It's simple, uses cache space only for values people actually request, falls back to the database when Redis dies, and gives decent general-purpose read performance. That's why it's everyone's default.

The benchmark should expose its ugly side: cold-key latency, stampedes, stale repopulation races, database load spikes after eviction, and what happens when TTLs expire in sync.

There's one race in particular worth demonstrating:

Copy
Reader misses cache
Reader starts slow database query

Writer updates database
Writer deletes cache

Reader receives old database snapshot
Reader writes old value back into cache

The cache is now stale even though the writer invalidated it correctly. Which is why “delete cache after writing the database” is a pattern, not a proof of correctness.

Copy
PUT: update Postgres → synchronously update Redis → return success
GET: read Redis → on miss fall back to Postgres

You get warmer reads, fresher cached values, and fewer database reads after writes. You pay with extra write latency, cache storage for values nobody may ever read, two systems in every write, and a partial-failure problem that never really goes away.

Postgres succeeds, Redis fails: the source of truth moved and the cache didn't. Flip it, Redis succeeds and Postgres fails: now the cache holds a value that was never committed. A production version usually ends up needing an outbox, CDC, versioned cache entries or a reconciliation worker. The demo should inject these failures on purpose instead of hiding them.

Give each entry two deadlines:

Copy
Fresh until: now + 2 seconds
Stale until: now + 30 seconds

Read behavior:

Copy
Age < 2 seconds: return immediately
2 seconds ≤ Age < 30 seconds: return stale immediately; trigger one background refresh
Age ≥ 30 seconds: block and fetch from Postgres

Use request coalescing so only one goroutine refreshes a hot key while everyone else keeps getting the stale value.

This buys you tail latency, availability during database slowdowns, predictable origin load, and protection against synchronized expiration. It costs you deliberately stale responses, messier observability, refresh coordination, and a product conversation about maximum acceptable age.

Good fits: reviews, recommendations, public profiles, live-event metadata with bounded age, product descriptions. Bad fits: account balances used for authorization, final inventory reservation, permissions, security policy, anything financial and irreversible.

“Can I run one load test and compare p99?” the junior asked.

“No. A design that wins on steady reads can lose on writes, cold starts, or failures.”

Run several scenarios.

Copy
95% GET
5% PUT
Zipf-distributed product IDs
500 concurrent clients

Measure p50/p95/p99, hit rate, database QPS, cache ops per second, CPU and memory.

Cold cache burst. Flush everything, then hammer a small set of popular keys. Measure duplicate database queries, time to recover the hit rate, p99 during warm-up, and whether coalescing actually works.

Database tail-latency injection. Make most database calls take 8 ms, but inject occasional 200 to 500 ms responses. This shows cache-aside miss amplification, write-through write latency, and the tail-latency advantage of serving bounded-stale values.

Partial cache-write failure. Fail a configurable percentage of cache updates after the database commit. Measure stale reads, time to repair, version regressions, and whether the API happily reports success while the cache drifts.

Simultaneous expiration. Create thousands of popular keys with the same TTL. This is the Hotstar TTL cloudburst in miniature: expiry itself becomes a traffic generator. Then rerun with jitter:

go
Copy
1ttl := baseTTL + time.Duration(rand.Int63n(int64(jitter)))

Origin failure. Take Postgres down for a bit.

Don't report throughput alone. Also report:

Copy
correctness_error_rate
stale_read_rate
database_queries_total
cache_refresh_total
cache_refresh_failures_total
cache_write_failures_total
origin_offload_ratio
estimated_write_amplification

Otherwise the “fastest” strategy might just be the one returning the most wrong answers.

Before picking a cache, write this contract down for each data type.

The junior closed his laptop. “So which design should we use?”

“For what data?”

“Product information.”

“Which part?”

“Title, price, inventory and reviews.”

“That's four cache policies, not one.”

The title can probably do cache-aside with a long TTL. Reviews can do stale-while-revalidate. Displayed inventory may want a near cache with fast propagation and reconciliation behind it. Final reservation has to validate against the authoritative inventory workflow, no shortcuts. Price might need versioning, event propagation and stricter invalidation depending on what the purchase contract promises.

The senior finished the thought: “A cache's job isn't to make data fast. It's to make a carefully chosen version of the data fast, while keeping the resulting errors inside a boundary the business agreed to tolerate.”

That's what these four systems teach. Zerodha shows how an ephemeral, precomputed read model can strip nearly all work out of a latency-sensitive read path. Myntra shows how near caching delivers enormous throughput cheaply while reconciliation and critical-path checks keep inconsistency on a leash. Flipkart shows that bounded staleness can be a reliability feature, and that deleting data can beat adding cache nodes. JioHotstar shows that at live-event scale, edge caching is an origin-protection and capacity architecture, not a response-time trick.

So the next time someone says “let's add Redis,” don't ask which instance size to buy. Ask:

Which wrong answer are we willing to serve, for how long, and what do we get in exchange?

That is the real caching architecture.