-
-
Notifications
You must be signed in to change notification settings - Fork 14
Complete Redis cache driver refactor with enhanced tagging #289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…yle consistent with operations classes
|
@albertcht Sorry about the size of the code review 😅 It's a complete overhaul of the driver so it's a big PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements a comprehensive architectural overhaul of Hypervel's Redis cache driver, transitioning from a monolithic structure to a modular, operation-based architecture while introducing an optional union-based "any" tagging mode for Redis 8.0+. The existing "all" mode benefits from pipelined operations, unified connection handling, smart serialization, and a critical memory leak fix via the new prune command.
Key changes:
- Extracted all Redis operations into dedicated, testable operation classes
- Introduced optional "any" tag mode with union flush semantics using Redis 8.0+ hash field expiration
- Fixed memory leak in tag implementation by adding
cache:prune-redis-stale-tagscommand - Added integration testing infrastructure with multi-vendor Redis testing (Redis 8.0 and Valkey 9.0)
Reviewed changes
Copilot reviewed 131 out of 197 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
src/redis/src/Traits/MultiExec.php |
Coroutine-safe pipeline/transaction trait ported from Hyperf |
src/redis/src/Operations/SafeScan.php |
Memory-efficient SCAN iterator fixing OPT_PREFIX double-prefixing bug |
src/redis/src/Operations/FlushByPattern.php |
Batched pattern-based key deletion using SafeScan |
src/redis/src/RedisConnection.php |
Added helper methods: client(), serialized(), compressed(), pack(), safeScan(), flushByPattern() |
src/redis/src/Redis.php |
Added MultiExec trait and flushByPattern() facade method |
src/cache/src/Redis/Support/StoreContext.php |
Unified connection handling with withConnection() pattern |
src/cache/src/Redis/Support/Serialization.php |
Connection-aware serialization preventing double-serialization |
src/cache/src/Redis/TagMode.php |
Enum centralizing all mode-specific logic and key patterns |
src/cache/src/Redis/Operations/ |
Extracted operations for Get, Put, Many, Add, Forever, Forget, Increment, Decrement, Flush, Remember |
src/cache/src/Redis/Operations/AllTag/ |
All-mode tagged operations with pipelined ZADD + SETEX |
src/cache/src/Redis/Operations/AnyTag/ |
Any-mode tagged operations using Redis 8.0+ HSETEX |
src/cache/src/Redis/Console/PruneStaleTagsCommand.php |
Fixes memory leak by cleaning stale tag entries |
src/cache/src/Redis/Console/DoctorCommand.php |
Environment diagnostics for Redis cache setup |
src/cache/src/Redis/Console/BenchmarkCommand.php |
Performance testing across both tag modes |
tests/Cache/Redis/Integration/ |
16 new integration test classes covering both modes |
tests/Cache/Redis/ |
49 unit test files with comprehensive mock-based testing |
tests/Support/RedisIntegrationTestCase.php |
Reusable integration test infrastructure |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This PR is a comprehensive architectural overhaul of Hypervel's Redis cache driver. It transitions from a monolithic
RedisStoreto a modular, operation-based architecture while fixing long-standing issues and introducing an optional union-basedanytagging mode for Redis 8.0+.Background: Problems Being Solved
Problem 1: Memory leak in tag Implementation
The current tag implementation uses sorted sets with TTL timestamps as scores:
Issues:
flushStale()method exists but nothing invoked itProblem 2: Intersection-only flush semantics
Current behavior requires all tags to match for flush:
Most tagged cache systems use union/OR semantics: tag with many categories, flush by any matching tag.
Problem 3: Chatty (sub-optimal) network operations
The original tagged cache made multiple separate calls for a single logical operation:
Each call potentially checked out a separate connection from the pool.
Problem 4: Inconsistent connection handling
The existing code mixed approaches:
$store->connection()(proxy via RedisFactory)What's New
For all users (no config changes required)
withConnection()pattern$conn->serialized()to avoid double-serializationevalShacaching for Lua scriptsputMany()with Luacache:prune-redis-stale-tagscommandcache:redis-doctorcommandcache:redis-benchmarkcommandKeyWritten,CacheHit, etc. fired explicitly after each operationFor users opting into "any" tag mode (Redis 8.0+ Required)
Architecture Overview
Key architectural changes
From monolith to operations:
The original
RedisStorecontained all logic inline. The new version delegates to dedicated operation classes, making each operation isolated, testable, and self-contained.No inheritance magic:
The original
RedisTaggedCacherelied onparent::put()andparent::add()from theTaggedCache→Repositoryinheritance chain. The newAllTaggedCachecalls$store->allTagOps()->put()->execute()directly, making data flow explicit.Cluster-aware operations:
Tagged operations detect cluster mode via
$context->isCluster()and switch from pipeline to sequential execution to handle cross-slot tags safely:Key structure
Both tagging modes use distinct key prefixes to avoid collisions and enable mode-specific cleanup operations.
All mode (default)
{prefix}{sha1(tags)}:{key}{prefix}_all:tag:{tagName}:entriesExample:
Any mode (opt-in, Redis 8.0+)
{prefix}{key}{prefix}_any:tag:{tagName}:entries{prefix}{key}:_any:tags{prefix}_any:tag:registryExample:
Key Structure Change from Previous Versions
{prefix}tag:{tagName}:entries{prefix}_all:tag:{tagName}:entries{prefix}_any:tag:{tagName}:entriesThis change enables clean separation between modes and consistent cleanup patterns. Users with existing tagged cache data must run
cache:clearafter upgrading - see Migration Guide.Detailed Changes
Redis package enhancements
RedisConnection new methods
File:
redis/src/RedisConnection.phpclient()\Redis/RedisClusteraccess for advanced operationsserialized()OPT_SERIALIZERis enabledcompressed()OPT_COMPRESSIONis enabledpack(array $values)safeScan(string $pattern)flushByPattern(string $pattern)Coroutine-safe pipelines & transactions
File:
redis/src/Traits/MultiExec.php(NEW)Added
MultiExectrait (ported from Hyperf) providing callback-based pipelines and transactions:Properly handles coroutine context - releases connections only if it acquired them, preventing connection leaks.
SafeScan: Fixing the OPT_PREFIX double-prefixing bug
File:
redis/src/Operations/SafeScan.php(NEW)phpredis
OPT_PREFIXcreates a subtle bug when scanning and deleting keys:Why this happens:
SafeScan handles this correctly and also supports RedisCluster by iterating all master nodes.
FlushByPattern: Batched key deletion
File:
redis/src/Operations/FlushByPattern.php(NEW)Efficient pattern-based key deletion using SafeScan:
Updated docblocks
Added ~40 new
@methodannotations for modern Redis commands:Redis 8.0+ Hash Field Expiration (used by "any" mode):
hsetex,hexpire,hpexpire,hexpireat,hpexpireathttl,hpttl,hexpiretime,hpexpiretime,hpersisthgetex,hgetdelOther Modern Commands:
_pack,_unpack,_digest(phpredis internals)serverName,serverVersionmsetex,delex,xdelexvadd,vsim,vrange,vcard,vdim, etc.Cache driver: Support classes
Directory:
cache/src/Redis/Support/StoreContextwithConnection()for scoped connection useSerialization$conn->serialized()to prevent double-serialization)File:
cache/src/Redis/TagMode.phpSingle source of truth enum for all mode-specific logic:
tagSegment(),tagKey(),reverseIndexKey())hasReverseIndex(),hasRegistry())Extracted operations
Directory:
cache/src/Redis/Operations/All Redis cache operations extracted to dedicated classes:
Get.phpMany.phpPut.phpPutMany.phpAdd.phpForever.phpForget.phpIncrement.php/Decrement.phpFlush.phpRemember.php/RememberForever.phpAll-mode tag operations
Directory:
cache/src/Redis/Operations/AllTag/Pipelined operations combining tag tracking + cache storage:
Put.phpPutMany.phpAdd.phpForever.phpIncrement.php/Decrement.phpRemember.php/RememberForever.phpAddEntry.phpGetEntries.phpFlushStale.phpFlush.phpPrune.phpAny-mode tag operations
Directory:
cache/src/Redis/Operations/AnyTag/Operations using Redis 8.0+ hash field expiration:
Put.phpPutMany.phpAdd.phpForever.phpIncrement.php/Decrement.phpRemember.php/RememberForever.phpGetTaggedKeys.phpGetTagItems.phpFlush.phpPrune.phpRenamed/moved classes
RedisTaggedCacheRedis/AllTaggedCacheRedisTagSetRedis/AllTagSetNew classes:
Redis/AnyTaggedCacheRedis/AnyTagSetNew commands
cache:prune-redis-stale-tagsAuto-detects tag mode and runs appropriate cleanup.
All mode output:
Any mode output:
cache:redis-doctorComprehensive environment diagnostics:
php artisan cache:redis-doctor [--store=redis]cache:redis-benchmarkPerformance testing:
Scales:
small(1K ops),medium(10K ops),large(100K ops),extreme(1M ops)Configuration
New Config Option
'all'(default)'any'Performance
Both tagging modes deliver similar performance, so developers can choose based on semantics rather than speed concerns.
Benchmark Results (Large Scale: 100K operations)
Key observations:
putMany()is slower because each item requires atomic Lua execution with multiple data structuresget()andadd()are faster due to simpler key lookup (no namespace computation)Migration guide
Users not using tags
No changes required. You automatically benefit from:
add(),putMany(),remember(),rememberForever()Recommended: Add prune command to scheduler for future-proofing:
Users using tags (staying with the default
allMode)One-time action required: Clear the cache after upgrading.
The internal tag key structure changed from
tag:{name}:entriesto_all:tag:{name}:entriesfor consistency between modes. Your application code doesn't change, but existing tag data won't be found by the new code.Critical: Add prune to scheduler to fix the existing memory leak:
Users wanting any mode
Verify environment:
Ensure Redis 8.0+ or Valkey 9.0+ with HSETEX support.
Update config:
Clear existing cache (any mode uses different key structure):
Update code if using
Cache::tags(['a'])->get('key'):Schedule cleanup (handles orphans from edge cases):
Breaking Changes
Internal only (won't developers unless they're overriding internal classes / methods)
RedisTaggedCache→Redis/AllTaggedCacheCache::tags()facadeRedisTagSet→Redis/AllTagSetRedisStore::serialize()/unserialize()removedSerializationclassIf you extended internal classes
If you extended
RedisTaggedCacheorRedisTagSet, update your imports:File Structure
Click to expand full file structure
Key design decisions
Single driver with config toggle (vs separate drivers)
Operations extracted to classes
withConnection()Pattern$conn->serialized()before PHP serializingTagMode enum as single source of truth
All mode-specific logic centralized:
tagSegment(),tagKey(),reverseIndexKey())hasReverseIndex(),hasRegistry())isAnyMode(),isAllMode())Makes it easy to understand all behavioral differences between modes in one place.
Lazy flush for any mode
Testing
This PR introduces comprehensive test coverage and establishes integration testing infrastructure for the Hypervel components repo.
Test coverage overview
Integration testing infrastructure
This PR introduces integration testing infrastructure that didn't previously exist in the repo:
New base classes:
tests/Support/RedisIntegrationTestCase.phptests/Cache/Redis/Integration/RedisCacheIntegrationTestCase.phpKey features:
TEST_TOKENto create unique prefixes per worker (e.g.,int_test_1:,int_test_2:). This makes the tests parallel-safe in case we want to use paratest in the future.envfor local testingflushByPattern('*')with OPT_PREFIX (only deletes test keys)RUN_REDIS_INTEGRATION_TESTS=trueCI: Multi-vendor Redis testing
The GitHub Actions workflow now runs integration tests against both Redis 8.0 and Valkey 9.0:
This ensures the cache driver works correctly on both major Redis-compatible platforms, catching any implementation differences between Redis and Valkey.
Integration test classes
BasicOperationsIntegrationTestBlockedOperationsIntegrationTestClusterFallbackIntegrationTestConcurrencyIntegrationTestEdgeCasesIntegrationTestFlushOperationsIntegrationTestHashExpirationIntegrationTestHashLifecycleIntegrationTestKeyNamingIntegrationTestPrefixHandlingIntegrationTestPruneIntegrationTestRememberIntegrationTestTagConsistencyIntegrationTestTaggedOperationsIntegrationTestTagQueryIntegrationTestTtlHandlingIntegrationTestRunning tests locally
Benefits for future development
The integration test infrastructure is designed to be reusable:
RedisIntegrationTestCasefor their Redis-dependent tests