HLive - Cache

Overview

HLive's Page can be given anything that implements its small Cache interface (Get/Set), and will use it to skip the SSR render pipeline for a request whose output hasn't changed since it was last cached. hlivekit ships three adapters over popular in-memory cache libraries: CacheRistretto, CacheOtter, and CacheTheine — pick whichever fits your app.

This only affects the initial HTTP/SSR render — the one that's a good candidate for a CDN in the first place, as covered in HTML vs WebSocket. It has no effect on WebSocket renders, which always run live since they reflect a specific session's current state.

Note: the value cached is a page's serialized node tree, so its size varies per page. hlivekit's Ristretto and Theine adapters pass a cost of 0 for every Set (matching the Cache interface, which has no room for a per-item cost), so out of the box neither bounds memory by the actual byte size of what's cached — only by entry count. If that matters for your app, either bound generously by count or fork the relevant adapter to compute a cost from len(value.([]byte)). Otter sidesteps this entirely: its size limit is configured once via Weigher, not passed on every write.

API

hlivekit.NewCacheRistretto(cache *ristretto.Cache[string, any]) *CacheRistretto — wrap a Ristretto cache.
hlivekit.NewCacheOtter(cache *otter.Cache[string, any]) *CacheOtter — wrap an Otter cache.
hlivekit.NewCacheTheine(cache *theine.Cache[string, any]) *CacheTheine — wrap a Theine cache.
l.PageOptionCache(cache l.Cache) func(*l.Page) — the l.NewPage option that installs any of them.

Use case

A high-traffic marketing or content page that's expensive to render but identical for most visitors — caching its SSR output means the render pipeline only has to run again once the underlying content actually changes, not on every request.

Ristretto

Ristretto is the original adapter hlivekit shipped with, now updated to its generics-based v2 API. It uses a Sampled LFU eviction policy behind a TinyLFU admission filter, and bounds the cache by a configurable cost total rather than a raw entry count.

ristrettoCache, err := ristretto.NewCache(&ristretto.Config[string, any]{
	NumCounters: 1e7,
	MaxCost:     1 << 30,
	BufferItems: 64,
})
if err != nil {
	log.Fatal(err)
}

cache := hlivekit.NewCacheRistretto(ristrettoCache)

f := func() *l.Page {
	return l.NewPage(l.PageOptionCache(cache))
}

http.Handle("/", l.NewPageServer(f))

Pros

The most battle-tested of the three — it's been in wide production use for years, including inside Dgraph/Badger.
Cost-based bounding fits a cache of variably-sized page trees well, once you wire up a real Cost function.
No extra dependency if your app already uses Ristretto elsewhere for a different cache.

Cons

Development has slowed and mostly tracks the needs of Dgraph/Badger rather than general-purpose caching.
Under high write contention, Set calls can be silently dropped rather than applied — tolerable here since a miss just costs an extra SSR render, but worth knowing.
Config has some sharp edges — NumCounters/MaxCost/BufferItems take some tuning to get right.
No built-in proactive TTL sweep; expired entries are only cleaned up lazily via TtlTickerDurationInSec.

Otter

Otter v2 is a newer cache built around Adaptive W-TinyLFU — the same policy family used by Java's Caffeine. It configures its size bound once, up front, rather than per write.

otterCache, err := otter.New(&otter.Options[string, any]{
	MaximumWeight: 1 << 30,
	Weigher: func(key string, value any) uint32 {
		b, ok := value.([]byte)
		if !ok {
			return 1
		}

		return uint32(len(b))
	},
	ExpiryCalculator: otter.ExpiryWriting[string, any](time.Hour),
})
if err != nil {
	log.Fatal(err)
}

cache := hlivekit.NewCacheOtter(otterCache)

f := func() *l.Page {
	return l.NewPage(l.PageOptionCache(cache))
}

http.Handle("/", l.NewPageServer(f))

Pros

Adaptive W-TinyLFU tends to hold a higher hit rate across a wider range of workloads than Ristretto's fixed TinyLFU/Sampled LFU pairing.
No dropped writes under contention — every Set is actually applied.
Weight-based sizing (the Weigher) is configured once, so it naturally bounds by actual byte size — the caveat above doesn't apply if you set one.
Built-in expiry calculators give you real, proactive TTLs without extra machinery.

Cons

Younger project with far less production mileage than Ristretto, despite strong published benchmarks.
Larger, more Caffeine-shaped API surface (loaders, refresh, bulk operations) than this simple Get/Set use case needs.

Theine

Theine also implements Adaptive W-TinyLFU, with an API shaped almost exactly like Ristretto's — a cost-bounded Set plus an explicit SetWithTTL — which makes it close to a drop-in swap. It's the cache behind Vitess.

theineCache, err := theine.NewBuilder[string, any](1 << 30).Build()
if err != nil {
	log.Fatal(err)
}

cache := hlivekit.NewCacheTheine(theineCache)

f := func() *l.Page {
	return l.NewPage(l.PageOptionCache(cache))
}

http.Handle("/", l.NewPageServer(f))

Pros

Adaptive W-TinyLFU hit rates, with an API close enough to Ristretto's that migrating an existing adapter is a small diff.
Proven in production as Vitess's cache, at a scale that exercises real workloads.
A proactive, hierarchical timer-wheel TTL implementation — good fit if SSR entries should expire promptly rather than just get evicted under memory pressure.

Cons

Same per-call cost caveat as Ristretto applies unless you compute a real cost from the cached value.
Smaller community and ecosystem than Ristretto's.
Its sharded-map design scales somewhat worse than Otter's across very high core counts.

Which one should you use?

For a new project, reach for Otter or Theine over Ristretto — both implement a newer eviction policy with better hit rates and no dropped writes, and either one's TTL support is a real win for SSR cache entries, which are dead weight within minutes of the page's WebSocket connecting. Pick Ristretto instead only if your app already depends on it for another cache and you'd rather not add a second cache dependency.

← PubSub