Semantic Caching¶

adk-redis provides semantic caching that skips LLM calls when a user sends a prompt that is similar (or identical) to one already answered. This reduces latency and cost without changing agent behavior.

Quick Reference¶

How It Works¶

flowchart TD
    U([User prompt]) --> BC[before_model_callback<br/>embed prompt, search cache]
    BC --> D{Cache hit?}
    D -->|Yes| CR([Return cached response<br/>no LLM call])
    D -->|No| LLM[Call LLM]
    LLM --> AC[after_model_callback<br/>store response in cache]
    AC --> R([Return LLM response])

    subgraph Cache [Redis Cache]
        SE[(Semantic index<br/>prompt embeddings)]
    end

    BC <--> Cache
    AC --> Cache

1. Before the LLM is called, LLMResponseCache embeds the prompt and searches for a semantically similar entry in the cache.
2. If the distance is below the configured threshold, the cached response is returned immediately (no LLM call).
3. If no match is found, the LLM runs normally and the response is stored in the cache for future hits.

Two Provider Options¶

Self-Hosted (RedisVL)¶

Use RedisVLCacheProvider when you run your own Redis instance and want full control over the vectorizer and cache index.

from redisvl.utils.vectorize import HFTextVectorizer

from adk_redis.cache import (
    LLMResponseCache,
    LLMResponseCacheConfig,
    RedisVLCacheProvider,
    RedisVLCacheProviderConfig,
)

vectorizer = HFTextVectorizer(model="redis/langcache-embed-v1")

provider = RedisVLCacheProvider(
    config=RedisVLCacheProviderConfig(
        redis_url="redis://localhost:6379",
        name="my_cache",
        ttl=3600,
        distance_threshold=0.1,
    ),
    vectorizer=vectorizer,
)

Requirements: pip install 'adk-redis[search]' and a running Redis instance.

Managed (LangCache)¶

Use LangCacheProvider with Redis LangCache for a fully managed service. No local vectorizer needed; embeddings are handled server-side.

from adk_redis.cache import (
    LLMResponseCache,
    LLMResponseCacheConfig,
    LangCacheProvider,
    LangCacheProviderConfig,
)

provider = LangCacheProvider(
    config=LangCacheProviderConfig(
        cache_id="your-cache-id",
        api_key="your-api-key",
        server_url="https://aws-us-east-1.langcache.redis.io",
        ttl=3600,
    ),
)

Requirements: pip install 'adk-redis[langcache]' and a LangCache account.

Wiring Into an Agent¶

Both providers use the same LLMResponseCache wrapper, which produces ADK-compatible callbacks:

from adk_redis.cache import create_llm_cache_callbacks

llm_cache = LLMResponseCache(
    provider=provider,
    config=LLMResponseCacheConfig(
        first_message_only=True,   # only cache the first user message
        include_app_name=True,     # scope cache keys by app
        include_user_id=True,      # scope cache keys by user
    ),
)

before_cb, after_cb = create_llm_cache_callbacks(llm_cache)

agent = Agent(
    model="gemini-2.0-flash",
    name="my_agent",
    before_model_callback=before_cb,
    after_model_callback=after_cb,
)

When to Use Which¶

Entry IDs and Targeted Invalidation¶

Both providers expose a stable identifier for each cache entry, so an application-level coordinator can retire exactly one stale entry without a semantic re-search and without clearing unrelated entries:

• provider.check(prompt) returns a CacheEntry whose entry_id field identifies the matched entry.
• provider.store(prompt, response) returns the identifier of the newly written entry.
• provider.delete_by_id(entry_id) deletes exactly that entry. Unlike provider.clear(), which removes every entry in the cache, delete_by_id targets a single entry.

entry_id = await provider.store("What is our refund policy?", "30 days.")

hit = await provider.check("What's the refund policy?")
if hit is not None:
    print(hit.entry_id)  # same entry

# The source document changed; retire only this entry.
await provider.delete_by_id(entry_id)

ID availability per backend:

For a given provider instance, identifiers returned by check() and store() are interchangeable inputs to delete_by_id(). LLMResponseCache does not use targeted invalidation itself; the simple read-through callback path is unchanged.

Configuration Options¶

Next Steps¶

• Semantic cache example for a runnable self-hosted demo.
• LangCache example for a runnable managed demo.
• Sessions + Memory services and Sessions + Memory MCP for the other Redis-backed features.
• ADK runtime options for adk web, adk run, and adk api_server.