perf: stop encoding terms for adapters that store them natively

CHANGELOG.md

@@ -1,3 +1,26 @@
+# 0.5.0
+
+## Performance
+
+- perf: values are no longer run through `:erlang.term_to_binary/1` for adapters that store Erlang terms natively (`Cache.ETS`, `Cache.Agent`, `Cache.PersistentTerm`, `Cache.ConCache`, `Cache.Counter`). The encode/decode round trip was pure overhead on those adapters, and the decode dominated the lookup it was attached to. On a 500k-entry ETS table, `get/1` of a ~10KB body drops from **29,981 ns to 6,211 ns (4.8x)** and `put/2` from **23,542 ns to 10,804 ns (2.2x)**. The win holds across payload sizes — a small map goes from 2,105 ns to 302 ns (7.0x).
+- `Cache.PersistentTerm` now hands out the stored term itself on every read, restoring the zero-copy property the adapter exists for.
+
+## Features
+
+- feat: added the optional `c:Cache.native_term_storage?/1` callback, letting an adapter declare that it stores Erlang terms natively. It is resolved at compile time, so there is no runtime branch on the read or write path. The callback is optional and defaults to encoding, so third-party adapters keep their current behaviour unchanged.
+
+## Bug Fixes
+
+- fix: `Cache.ConCache.get_or_store/3` followed by `get/1` no longer raises. `get_or_store/3` writes through ConCache directly, bypassing the encode in `put/3`, so the matching `get/1` tried to `binary_to_term/1` a raw term.
+- fix: a binary value wrapped in braces but not valid JSON (eg `"{oops}"`) no longer raises `Jason.DecodeError` on read. `Cache.TermEncoder.decode/1` used `Jason.decode!/1`, and now falls back to returning the binary unchanged.
+- fix: raw `Cache.ETS` operations (`match_object/1`, `select/1`, `tab2list/0`, `foldl/2`) now see the terms that were `put`, rather than the opaque encoded binaries they used to return.
+
+## Breaking Changes
+
+- Values held by native-term adapters are now stored as terms rather than encoded binaries. This is not observable through `get/1`, `put/3` and `delete/1`, which round-trip exactly as before. It is observable if you read the underlying store directly (`:ets.lookup/2`, `:persistent_term.get/1`, `ConCache.get/2`) or through the raw ETS API — those now return terms, which is what they were always meant to return.
+- `Cache.DETS` is unchanged and still encodes, so existing `.dets` files stay readable. `Cache.ETS` with `:rehydration_path` also still encodes, so existing table dumps stay loadable.
+- An in-memory cache populated by an older version and read by this one would return raw binaries, but ETS, Agent, PersistentTerm and ConCache do not survive a restart, so there is no upgrade path on which that can happen.
+
 # 0.4.9
 
 ## Performance

guides/explanation/architecture.md

@@ -37,11 +37,26 @@ When you `use Cache` in your module, the macro:
 
 ## Term Encoding and Compression
 
-ElixirCache includes internal term encoding functionality that handles serialization and deserialization of Elixir terms. This allows you to store complex Elixir data structures in any cache backend. The encoding system:
+ElixirCache lets you store any Elixir term in any backend. How a term gets there depends on
+what the backend can actually hold, and that is decided per adapter rather than globally.
 
-1. Uses Erlang's term_to_binary for efficient serialization
-2. Applies configurable compression to reduce memory usage
-3. Automatically handles decoding when retrieving values
+Adapters that store Erlang terms natively — `Cache.ETS`, `Cache.Agent`, `Cache.PersistentTerm`,
+`Cache.ConCache`, `Cache.Counter` — receive the term as-is. Serialising it first would cost a
+`term_to_binary/1` on every write and a `binary_to_term/1` on every read while buying nothing,
+and on a hot read path that decode dominates the lookup it is attached to.
+
+Adapters that cannot hold a term serialise it with `:erlang.term_to_binary/1`:
+
+- `Cache.Redis` stores bytes on the wire, so terms must be serialised before they leave the node.
+- `Cache.DETS` owns a durable on-disk format, and so does `Cache.ETS` when configured with
+  `:rehydration_path`. Both keep encoding so that files written by earlier versions stay readable.
+
+Setting `:compression_level` forces encoding on any adapter — an explicit request to compress
+is honoured even when the backend could have held the term directly.
+
+Adapters declare this through the optional `c:Cache.native_term_storage?/1` callback. It is
+resolved once at compile time, so there is no runtime branch on the read or write path.
+The callback is optional and defaults to encoding, so third-party adapters are unaffected.
 
 ## Sandboxing for Tests
 

lib/cache.ex

@@ -33,6 +33,22 @@ defmodule Cache do
               ErrorMessage.t_ok_res()
   @callback delete(cache_name :: atom, key :: atom | String.t()) :: ErrorMessage.t_ok_res()
 
+  @doc """
+  Returns true when the adapter stores Erlang terms natively, given its resolved options.
+
+  Adapters that hold terms in memory (`Cache.ETS`, `Cache.Agent`, `Cache.PersistentTerm`,
+  `Cache.ConCache`, `Cache.Counter`) do not need values to be run through
+  `:erlang.term_to_binary/1` first — doing so costs a full encode on every write and a
+  full decode on every read while buying nothing. Adapters that store bytes on the wire
+  (`Cache.Redis`) or that own a durable on-disk format (`Cache.DETS`) must return `false`.
+
+  This callback is optional. Adapters that do not implement it are assumed to need
+  encoding, which is what every adapter did before this callback existed.
+  """
+  @callback native_term_storage?(adapter_opts :: Keyword.t()) :: boolean()
+
+  @optional_callbacks native_term_storage?: 1
+
   defmacro __using__(opts) do
     sandbox_opt = Keyword.get(opts, :sandbox?, false)
 
@@ -288,6 +304,12 @@ defmodule Cache do
         @adapter_opts adapter_opts
         @compression_level if is_list(@adapter_opts), do: @adapter_opts[:compression_level]
 
+        # Resolved against the *configured* adapter and its *configured* opts rather than
+        # `@cache_adapter`/`@adapter_opts`, which are swapped out under `sandbox?: true`.
+        # A sandboxed cache must encode exactly like the adapter it stands in for, or a
+        # value would round-trip differently in test than in production.
+        @cache_encode_terms? Cache.TermEncoder.encoding_required?(opts[:adapter], opts[:opts])
+
         unquote(adapter_use_ast)
 
         def cache_name, do: @cache_name
@@ -334,8 +356,18 @@ defmodule Cache do
           @cache_adapter.child_spec({@cache_name, adapter_options()})
         end
 
+        @compile {:inline, encode_value: 1, decode_value: 1}
+
+        if @cache_encode_terms? do
+          defp encode_value(value), do: Cache.TermEncoder.encode(value, @compression_level)
+          defp decode_value(value), do: Cache.TermEncoder.decode(value)
+        else
+          defp encode_value(value), do: value
+          defp decode_value(value), do: value
+        end
+
         def put(key, ttl \\ nil, value) do
-          value = Cache.TermEncoder.encode(value, @compression_level)
+          value = encode_value(value)
           key = maybe_sandbox_key(key)
 
           :telemetry.span(
@@ -370,7 +402,7 @@ defmodule Cache do
                     res
 
                   {:ok, value} ->
-                    {:ok, Cache.TermEncoder.decode(value)}
+                    {:ok, decode_value(value)}
 
                   {:error, _} = error ->
                     error

lib/cache/agent.ex

@@ -10,6 +10,12 @@ defmodule Cache.Agent do
   @impl Cache
   def opts_definition, do: []
 
+  @doc """
+  Values are held as terms in the Agent's map, so encoding them is pure overhead.
+  """
+  @impl Cache
+  def native_term_storage?(_opts), do: true
+
   @impl Cache
   def start_link(opts \\ []) do
     with {:error, {:already_started, pid}} <- Agent.start_link(fn -> %{} end, opts) do

lib/cache/con_cache.ex

@@ -61,6 +61,15 @@ defmodule Cache.ConCache do
   @impl Cache
   def opts_definition, do: @opts_definition
 
+  @doc """
+  ConCache is ETS-backed and stores terms. Encoding is pure overhead, and it made
+  `get_or_store/3` unusable — that function writes through ConCache directly, bypassing
+  the encode in `Cache.put/3`, so a later `get/1` tried to `binary_to_term/1` a raw term
+  and raised.
+  """
+  @impl Cache
+  def native_term_storage?(_opts), do: true
+
   @impl Cache
   def start_link(opts) do
     cache_opts =

lib/cache/counter.ex

@@ -95,6 +95,13 @@ defmodule Cache.Counter do
   @impl Cache
   def opts_definition, do: @opts_definition
 
+  @doc """
+  Values are integers in an `:counters` array. `Cache.TermEncoder.encode/2` already
+  passed integers through untouched, so this only removes a no-op function call.
+  """
+  @impl Cache
+  def native_term_storage?(_opts), do: true
+
   @impl Cache
   def start_link(opts) do
     parent = self()

lib/cache/dets.ex

@@ -363,6 +363,23 @@ defmodule Cache.DETS do
   @impl Cache
   def opts_definition, do: @opts_definition
 
+  @doc """
+  DETS keeps encoding, even though `:dets.insert/2` takes terms and would serialise
+  them itself.
+
+  The reason is the file on disk. Every `.dets` file written by an earlier version of
+  this library holds `:erlang.term_to_binary/1` blobs as its values. Those files outlive
+  the upgrade — that is the entire point of a disk cache. If this adapter switched to
+  native terms, the first read after an upgrade would hand the caller a raw binary where
+  it expects a term, silently, with no error to trace it back to.
+
+  The cost of keeping it is one redundant serialisation of data that is already headed
+  for a disk write, which is dominated by the I/O. The cost of dropping it is corrupting
+  every existing on-disk cache. That trade is not close.
+  """
+  @impl Cache
+  def native_term_storage?(_opts), do: false
+
   @impl Cache
   def start_link(opts) do
     parent = self()

lib/cache/ets.ex

@@ -570,6 +570,20 @@ defmodule Cache.ETS do
   @impl Cache
   def opts_definition, do: @opts_definition
 
+  @doc """
+  `:ets.insert/2` stores terms, so encoding is pure overhead — and it actively breaks
+  the raw ETS API this adapter exposes (`match_object/1`, `select/1`, `tab2list/0`,
+  `foldl/2`), which sees opaque binaries instead of the terms that were put.
+
+  The exception is `:rehydration_path`. That option dumps the table to disk with
+  `:ets.tab2file/2` and reads it back on the next boot, so the table is a durable
+  format. Encoding is kept there for the same reason it is kept for `Cache.DETS`:
+  a table dumped by an older version holds `term_to_binary/1` blobs, and switching to
+  native terms would hand callers raw binaries where they expect terms.
+  """
+  @impl Cache
+  def native_term_storage?(opts), do: is_nil(opts[:rehydration_path])
+
   def opts_definition(opts) when is_list(opts) do
     if Keyword.keyword?(opts) do
       validate_and_normalize_ets_options(opts)

lib/cache/hash_ring.ex

@@ -167,8 +167,8 @@ defmodule Cache.HashRing do
       {:ok, nil} ->
         read_repair(cache_name, key, target_node, underlying_adapter, underlying_opts, rpc)
 
-      {:ok, encoded} ->
-        {:ok, Cache.TermEncoder.decode(encoded)}
+      {:ok, stored} ->
+        {:ok, Cache.TermEncoder.maybe_decode(stored, underlying_adapter, underlying_opts)}
 
       {:error, _} = error ->
         error
@@ -180,7 +180,7 @@ defmodule Cache.HashRing do
     target_node = key_to_node(cache_name, key)
     rpc = adapter_opts[:rpc_module] || :erpc
     underlying_opts = validate_underlying_opts(underlying_adapter, Keyword.drop(adapter_opts, @strategy_keys))
-    encoded = Cache.TermEncoder.encode(value, underlying_opts[:compression_level])
+    encoded = Cache.TermEncoder.maybe_encode(value, underlying_adapter, underlying_opts)
 
     if target_node === Node.self() do
       underlying_adapter.put(cache_name, key, ttl, encoded, underlying_opts)
@@ -231,8 +231,8 @@ defmodule Cache.HashRing do
       end)
 
     case result do
-      {:found, encoded, old_node} ->
-        value = Cache.TermEncoder.decode(encoded)
+      {:found, stored, old_node} ->
+        value = Cache.TermEncoder.maybe_decode(stored, underlying_adapter, underlying_opts)
         migrate_value(cache_name, key, value, current_node, old_node, underlying_adapter, underlying_opts, rpc)
         {:ok, value}
 
@@ -242,7 +242,7 @@ defmodule Cache.HashRing do
   end
 
   defp migrate_value(cache_name, key, value, current_node, old_node, underlying_adapter, underlying_opts, rpc) do
-    encoded = Cache.TermEncoder.encode(value, underlying_opts[:compression_level])
+    encoded = Cache.TermEncoder.maybe_encode(value, underlying_adapter, underlying_opts)
 
     if current_node === Node.self() do
       underlying_adapter.put(cache_name, key, nil, encoded, underlying_opts)

lib/cache/persistent_term.ex

@@ -27,6 +27,14 @@ defmodule Cache.PersistentTerm do
   @impl Cache
   def opts_definition, do: []
 
+  @doc """
+  `:persistent_term` exists to hand out zero-copy references to already-built terms.
+  Encoding a value into a binary and decoding it on every read defeats the entire
+  point of the adapter, so it never encodes.
+  """
+  @impl Cache
+  def native_term_storage?(_opts), do: true
+
   @impl Cache
   def start_link(opts) do
     Task.start_link(fn ->

lib/cache/redis.ex

@@ -200,6 +200,13 @@ defmodule Cache.Redis do
   @impl Cache
   def opts_definition, do: @opts_definition
 
+  @doc """
+  Redis stores bytes on the wire. Terms must be serialised before they leave the node,
+  so this adapter always encodes.
+  """
+  @impl Cache
+  def native_term_storage?(_opts), do: false
+
   @impl Cache
   def start_link(opts) do
     if !opts[:uri] do

lib/cache/refresh_ahead.ex

@@ -126,18 +126,16 @@ defmodule Cache.RefreshAhead do
         Keyword.drop(adapter_opts, [:refresh_before, :on_refresh, :lock_node_whitelist, :__cache_module__])
       )
 
-    compression_level = underlying_opts[:compression_level]
-
     case underlying_adapter.get(cache_name, key, underlying_opts) do
       {:ok, nil} ->
         {:ok, nil}
 
-      {:ok, encoded} ->
-        case Cache.TermEncoder.decode(encoded) do
+      {:ok, stored} ->
+        case Cache.TermEncoder.maybe_decode(stored, underlying_adapter, underlying_opts) do
           {value, inserted_at, ttl} ->
             maybe_refresh_async(
               cache_name, key, value, inserted_at, ttl,
-              underlying_adapter, adapter_opts, compression_level
+              underlying_adapter, adapter_opts, underlying_opts
             )
             {:ok, value}
 
@@ -159,9 +157,11 @@ defmodule Cache.RefreshAhead do
         Keyword.drop(adapter_opts, [:refresh_before, :on_refresh, :lock_node_whitelist, :__cache_module__])
       )
 
-    compression_level = underlying_opts[:compression_level]
     inserted_at = System.monotonic_time(:millisecond)
-    wrapped = Cache.TermEncoder.encode({value, inserted_at, ttl}, compression_level)
+
+    wrapped =
+      Cache.TermEncoder.maybe_encode({value, inserted_at, ttl}, underlying_adapter, underlying_opts)
+
     underlying_adapter.put(cache_name, key, ttl, wrapped, underlying_opts)
   end
 
@@ -179,21 +179,21 @@ defmodule Cache.RefreshAhead do
     underlying_adapter.delete(cache_name, key, underlying_opts)
   end
 
-  defp maybe_refresh_async(cache_name, key, _value, inserted_at, ttl, underlying_adapter, adapter_opts, compression_level)
+  defp maybe_refresh_async(cache_name, key, _value, inserted_at, ttl, underlying_adapter, adapter_opts, underlying_opts)
        when is_integer(ttl) do
     refresh_before = adapter_opts[:refresh_before]
     now = System.monotonic_time(:millisecond)
     age = now - inserted_at
 
     if age >= ttl - refresh_before do
-      maybe_spawn_refresh(cache_name, key, ttl, underlying_adapter, adapter_opts, compression_level)
+      maybe_spawn_refresh(cache_name, key, ttl, underlying_adapter, adapter_opts, underlying_opts)
     end
   end
 
-  defp maybe_refresh_async(_cache_name, _key, _value, _inserted_at, _ttl, _underlying_adapter, _adapter_opts, _compression_level),
+  defp maybe_refresh_async(_cache_name, _key, _value, _inserted_at, _ttl, _underlying_adapter, _adapter_opts, _underlying_opts),
     do: :ok
 
-  defp maybe_spawn_refresh(cache_name, key, ttl, underlying_adapter, adapter_opts, compression_level) do
+  defp maybe_spawn_refresh(cache_name, key, ttl, underlying_adapter, adapter_opts, underlying_opts) do
     tracker = tracker_name(cache_name)
 
     if safe_ets_insert_new(tracker, {key, true}) do
@@ -209,16 +209,15 @@ defmodule Cache.RefreshAhead do
 
             case invoke_refresh(on_refresh, cache_module, key) do
               {:ok, new_value} ->
-                underlying_opts =
-                  Keyword.drop(adapter_opts, [
-                    :refresh_before,
-                    :on_refresh,
-                    :lock_node_whitelist,
-                    :__cache_module__
-                  ])
-
                 new_inserted_at = System.monotonic_time(:millisecond)
-                wrapped = Cache.TermEncoder.encode({new_value, new_inserted_at, ttl}, compression_level)
+
+                wrapped =
+                  Cache.TermEncoder.maybe_encode(
+                    {new_value, new_inserted_at, ttl},
+                    underlying_adapter,
+                    underlying_opts
+                  )
+
                 underlying_adapter.put(cache_name, key, ttl, wrapped, underlying_opts)
 
               {:error, _} ->

lib/cache/sandbox.ex

@@ -55,6 +55,14 @@ defmodule Cache.Sandbox do
   @impl Cache
   def opts_definition, do: []
 
+  @doc """
+  The sandbox holds terms in an Agent map. It is never consulted directly to decide
+  whether to encode — `Cache` resolves that against the adapter the sandbox stands in
+  for, so a sandboxed cache round-trips values exactly like the real one.
+  """
+  @impl Cache
+  def native_term_storage?(_opts), do: true
+
   @impl Cache
   def start_link(opts \\ []) do
     with {:error, {:already_started, pid}} <- Agent.start_link(fn -> %{} end, opts) do