Skip to content

Conversation

@mshr-h
Copy link

@mshr-h mshr-h commented Dec 14, 2025

Passing invalid memory_max_entries to Cache can crash cachetool with unclear errors. This PR validates it upfront and fails with clearer messages.

Repros:

from dataclasses import dataclass
from dspy.clients.cache import Cache

@dataclass
class DummyResponse:
    message: str
    usage: dict

if __name__ == "__main__":
    memory_cache = Cache(
        enable_disk_cache=False,
        enable_memory_cache=True,
        disk_cache_dir="",
        disk_size_limit_bytes=0,
        memory_max_entries=1, # negative entry size
    )

    request = {"prompt": "Hello", "model": "openai/gpt-4o-mini", "temperature": 0.7}
    value = DummyResponse(message="This is a test response", usage={"prompt_tokens": 10, "completion_tokens": 20})
    memory_cache.put(request, value)  # -> ValueError: value too large
from dataclasses import dataclass
from dspy.clients.cache import Cache

@dataclass
class DummyResponse:
    message: str
    usage: dict

if __name__ == "__main__":
    memory_cache = Cache(
        enable_disk_cache=False,
        enable_memory_cache=True,
        disk_cache_dir="",
        disk_size_limit_bytes=0,
        memory_max_entries=None, # None
    )

    request = {"prompt": "Hello", "model": "openai/gpt-4o-mini", "temperature": 0.7}
    value = DummyResponse(message="This is a test response", usage={"prompt_tokens": 10, "completion_tokens": 20})
    memory_cache.put(request, value)  # -> TypeError: '>' not supported between instances of 'int' and 'NoneType'

@mshr-h mshr-h changed the title Fixmemory_max_entries enforce positive integer constraint Fix memory_max_entries enforce positive integer constraint Dec 14, 2025
@mshr-h mshr-h changed the title Fix memory_max_entries enforce positive integer constraint Fix: enforce positivememory_max_entries for in-memory cache Dec 14, 2025
@mshr-h mshr-h changed the title Fix: enforce positivememory_max_entries for in-memory cache Fix: enforce positive memory_max_entries for in-memory cache Dec 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant