Skip to main content

Quick Start

Get caching working in 30 seconds.

Step 1: Create a CacheManager​

from omnicache_ai import CacheManager, InMemoryBackend, CacheKeyBuilder

manager = CacheManager(
    backend=InMemoryBackend(),
    key_builder=CacheKeyBuilder(namespace="myapp"),
)

The CacheManager is the central hub. It wires together a backend (where data is stored), a key builder (how keys are generated), and optional policies.

Step 2: Cache a Value​

# Store with 60-second TTL
manager.set("my_key", b"hello world", ttl=60)

# Retrieve
value = manager.get("my_key")
print(value)  # b"hello world"

# Check existence
print(manager.exists("my_key"))  # True

# Delete
manager.delete("my_key")

Step 3: Use a Cache Layer​

Cache layers provide typed, domain-specific caching. Here's ResponseCache for LLM outputs:

from omnicache_ai.layers.response_cache import ResponseCache

rc = ResponseCache(manager)

messages = [{"role": "user", "content": "What is 2+2?"}]

# Cache miss — returns None
print(rc.get(messages, "gpt-4"))  # None

# Store a response
rc.set(messages, "4", "gpt-4")

# Cache hit — returns "4" instantly
print(rc.get(messages, "gpt-4"))  # "4"

Step 4: Use the Middleware Decorator​

Wrap any LLM function to cache automatically:

from omnicache_ai.middleware.llm_middleware import LLMMiddleware

middleware = LLMMiddleware(
    response_cache=rc,
    key_builder=CacheKeyBuilder(),
    model_id="gpt-4"
)

@middleware
def call_llm(messages):
    # This is your real LLM call
    return openai_client.chat.completions.create(
        model="gpt-4", messages=messages
    )

# First call → hits the API
result = call_llm([{"role": "user", "content": "Hello"}])

# Second call → served from cache instantly
result = call_llm([{"role": "user", "content": "Hello"}])

Step 5: Use a Framework Adapter​

from langchain_core.globals import set_llm_cache
from omnicache_ai.adapters.langchain_adapter import LangChainCacheAdapter

set_llm_cache(LangChainCacheAdapter(manager))
# Every LLM call is now cached automatically
from langgraph.graph import StateGraph
from omnicache_ai.adapters.langgraph_adapter import LangGraphCacheAdapter

saver = LangGraphCacheAdapter(manager)
graph = StateGraph(...).compile(checkpointer=saver)
from crewai import Crew
from omnicache_ai.adapters.crewai_adapter import CrewAICacheAdapter

crew = Crew(agents=[...], tasks=[...])
cached_crew = CrewAICacheAdapter(crew, manager)
result = cached_crew.kickoff(inputs={"topic": "AI"})

From Settings (Environment Variables)​

For production, configure via environment variables:

export OMNICACHE_BACKEND=disk
export OMNICACHE_DISK_PATH=/data/cache
export OMNICACHE_DEFAULT_TTL=3600
from omnicache_ai import CacheManager, OmnicacheSettings

manager = CacheManager.from_settings(OmnicacheSettings.from_env())

Next Steps​