perf: stop encoding terms for adapters that store them natively

[MikaAK]

The problem

Cache.TermEncoder.encode/2 was called in the generic Cache macro on every put and decode/1 on every get. Because it lives in the macro, it applied to every adapter — including the ones that store Erlang terms natively, where the round trip buys nothing.

Adapter	Stores	Encoding needed?
Cache.ETS	native terms	no — pure waste
Cache.Agent	native terms	no
Cache.PersistentTerm	native terms, zero-copy by design	no — encoding defeated its entire purpose
Cache.ConCache	ETS-backed, native terms	no
Cache.Counter	integers	already passed through
Cache.DETS	terms, but a durable on-disk format	yes — see below
Cache.Redis	bytes on the wire	yes

The decode is not a rounding error. On a 500k-entry ETS table, a bare :ets.lookup of a ~10KB body costs 63 ns while binary_to_term of that same body costs ~29,000 ns. The decode was ~460x the cost of the lookup it was attached to, on the hot read path, for nothing.

The design

Adapters now declare how they store values through an optional callback on the Cache behaviour:

@callback native_term_storage?(adapter_opts :: Keyword.t()) :: boolean()
@optional_callbacks native_term_storage?: 1

The Cache macro resolves it at compile time from @cache_adapter and the configured opts, then compiles either a real encode or an identity shim — so there is zero runtime branch on the read/write path.

The callback is optional and defaults to false (encode), so any third-party adapter keeps working with no change.

Two details worth calling out:

• sandbox?: true resolves against the adapter it stands in for, not against Cache.Sandbox. Otherwise a value would round-trip differently in test than in prod.
• Runtime opts ({app, key}, MFA, zero-arity fun) fall back to encoding. The capability can't be read at compile time there, so it takes the safe path, which is exactly the old behaviour.

Composite adapters

• HashRing / RefreshAhead call the encoder directly, bypassing the macro. They now resolve the capability against the adapter they wrap, via TermEncoder.maybe_encode/3 and maybe_decode/3.
• MultiLayer needed no change. Each layer is a full Cache module that owns its own encoding, so a [ETS, Redis] stack already stores a term in the ETS layer and bytes in the Redis layer. There's now a test asserting exactly that, reading both underlying stores directly, plus one asserting a value backfilled from Redis lands in ETS as a term.

Benchmark

500k entries, ~10KB structured body (13,505 B encoded), 20k iterations, measured through the public API on main vs this branch:

Op	Before	After
get/1	29,981 ns	6,211 ns	4.8x faster
put/2	23,542 ns	10,804 ns	2.2x faster

The win is not confined to large payloads:

Payload	Before	After
small map (41 B)	2,105 ns	302 ns	7.0x
session (375 B)	2,179 ns	363 ns	6.0x
api body (10.6 KB)	28,284 ns	6,152 ns	4.6x

Cache.PersistentTerm also regains its zero-copy read — :erts_debug.same/2 now confirms two reads return the same term rather than two independently decoded copies.

The DETS decision: keep encoding

Cache.DETS still encodes. :dets.insert/2 takes terms and would serialise them itself, so this is a redundant serialisation — and I kept it anyway.

Every .dets file written by an earlier version holds term_to_binary blobs as its values, and those files outlive the upgrade — that is the entire point of a disk cache. Switching to native terms would make the first read after upgrading hand back a raw binary where the caller expects a term. Silently. With no error to trace it back to.

I considered a read-side fallback (try binary_to_term, fall back to raw) and rejected it: it is unbackable-out-of, it guesses, and it would misfire on any value that is legitimately a binary. A version marker means rewriting the file format for an adapter whose whole job is to not do that.

The cost of keeping the encode is one redundant serialisation of data already headed for a disk write, which the I/O dominates. The cost of dropping it is corrupting every existing on-disk cache. That trade isn't close. There's a test asserting a file written in the old format still decodes correctly.

The same reasoning applies to Cache.ETS with :rehydration_path — that option dumps the table via :ets.tab2file/2 and reloads it on the next boot, making it a durable format with the identical constraint. So native_term_storage?/1 returns false for ETS when that option is set. Plain ETS is unaffected. (The repo's existing rehydration test, which writes a hand-encoded value into the dump file, still passes untouched — which is itself the proof.)

compression_level

When set, encoding is deliberate, so compression_level forces encoding on any adapter, native or not. The user asked to compress; they get compression.

⚠️ Worth flagging separately: compression_level is currently dead config on main. It is not declared in any adapter's opts_definition, so NimbleOptions rejects it on compile-time opts; and via runtime opts the macro's @compression_level is only read when opts are a compile-time list, so it resolves to nil and never reaches the encoder. It's unreachable on every path. I have not fixed that here — it's an orthogonal bug and this PR is already load-bearing. The new code honours it correctly wherever it becomes reachable.

The TermEncoder landmine (fixed)

encode/2 regex-checks binaries against ~r/^{.*}$/ and passes JSON-looking ones through unencoded, while encoding everything else. decode/1 then has to guess. It's genuinely broken, and it's worse than "guessing":

# on main:
put(:k, ~s({"a": 1}))  |> get(:k)   #=> {:ok, %{"a" => 1}}   # put a String, got back a Map
put(:k, "{oops}")      |> get(:k)   #=> ** (Jason.DecodeError)  # crash on read

The first is a silent type change. The second is a crash on read for any brace-wrapped binary that isn't valid JSON.

I did not build on top of it. decode/1 now uses decode_json/1 (which falls back to returning the binary unchanged) instead of Jason.decode!/1, so the crash is gone. The remaining JSON-passthrough asymmetry only affects byte-storing adapters now, and its existing tests still pass — a full fix means picking a wire format and is a breaking change to Redis-stored data, which belongs in its own PR.

Other bugs this fixes

Both were direct consequences of encoding unconditionally:

• ConCache.get_or_store/3 + get/1 raised. get_or_store/3 writes through ConCache directly, bypassing the encode in put/3, so the matching get/1 tried to binary_to_term/1 a raw term and blew up. It works now.
• The raw ETS API was broken. Cache.ETS exposes match_object/1, select/1, tab2list/0, foldl/2 — all of which saw opaque encoded binaries instead of the terms that were put. You could not match on a value. They now see real terms.

Breaking change status

Minor-breaking. Version bumped to 0.5.0.

• get/1, put/3, delete/1 round-trip exactly as before — the public API is unchanged.
• It is observable if you read the underlying store directly (:ets.lookup/2, :persistent_term.get/1, ConCache.get/2) or through the raw ETS API. Those now return terms — which is what they were always meant to return.
• No data-migration hazard. DETS and rehydrated-ETS keep their on-disk format. ETS, Agent, PersistentTerm and ConCache don't survive a restart, so there is no upgrade path on which a stale encoded value can be read back by the new code.

Tests

21 new tests (test/cache/native_term_storage_test.exs + 2 in multi_layer_test.exs) covering: capability resolution per adapter; native adapters storing the raw term (asserted by reading :ets/Agent/:persistent_term/ConCache directly); arbitrary terms round-tripping (maps, structs, tuples, keyword lists, pids, floats, nil, raw binaries); Redis still storing an encoded binary; DETS still storing an encoded binary and old-format files still decoding; a MultiLayer of [ETS, Redis] doing the right thing per layer; compression_level still compressing on a native adapter; and both crash-on-read regressions.

Two hash_ring_test fixtures were updated: they hand-encoded a value to simulate a remote Cache.ETS node, which is now the wrong fixture — a real ETS node returns the term.

mix test      345 tests, 13 failures   # all 13 pre-existing on main: local Redis lacks the
                                       # RedisJSON module. Verified identical on main, and all
                                       # 13 pass against redis-stack (which has ReJSON).
mix credo --strict                     776 mods/funs, found no issues.
mix dialyzer                           Total errors: 0  (passed successfully)
mix compile --warnings-as-errors       clean

---

A latent race this surfaced (second commit)

The first CI run failed one test — RefreshAhead "global lock prevents refresh while lock is held" — which passes locally. It's a real race in the test, not flake to be re-run away:

assert {:ok, "original"} === LockedRefreshCache.get("locked_key")  # spawns a refresh task
assert true === :global.del_lock(lock_id, lock_nodes)              # releases the lock immediately
assert {:ok, "original"} === LockedRefreshCache.get("locked_key")  # <- failed here

The get spawns a refresh task that is still racing toward :global.set_lock. Nothing sequences it against the del_lock on the very next line, so it can reach set_lock after the lock is freed, acquire it, and refresh the value the next assertion expects to be untouched — failing with exactly the observed right: {:ok, "locked:locked_key"}.

The race predates this PR. Making get/1 ~5x faster widened the window by getting the test process to del_lock sooner, and a loaded CI runner did the rest.

Fixed by draining the in-flight task while the lock is still held — it loses the race, cleans up its tracker entry, and nothing is in flight when the lock is released. That makes the assertion deterministic rather than merely likely.

[codecov]

Codecov Report

❌ Patch coverage is 86.84211% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.20%. Comparing base (7ad44ee) to head (5d0f09f).

Files with missing lines	Patch %	Lines
lib/cache/term_encoder.ex	77.27%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #48      +/-   ##
==========================================
- Coverage   83.49%   83.20%   -0.29%     
==========================================
  Files          23       23              
  Lines         636      661      +25     
==========================================
+ Hits          531      550      +19     
- Misses        105      111       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.