Support non-JSON-serializable types in RedisPriorityQueue values

[yedidyakfir]

Summary

RedisPriorityQueue currently serializes values using json.dumps, which limits supported value types to JSON-serializable ones (str, int, float, bool). This issue tracks adding support for non-JSON-native types such as None, bytes, dict, list, and Pydantic models as priority queue values.

Features / Details

• Support None as a priority queue value
• Support bytes values (with proper encoding/decoding)
• Support dict values
• Support list values
• Support Pydantic model instances as values
• Ensure round-trip serialization/deserialization works correctly for all types
• Maintain backward compatibility with existing JSON-serializable types

Use Cases

• Storing complex structured data (dicts, lists) as priority queue entries
• Using Pydantic models as queue items for typed, validated payloads
• Supporting None as a sentinel or placeholder value
• Storing binary data (bytes) in priority-ordered queues

Implementation Considerations

• The current _serialize_value / _deserialize_value pipeline in rapyer/types/priority_queue.py relies on json.dumps/json.loads
• Pydantic's TypeAdapter with mode="json" already handles model serialization, but the pipeline needs to handle edge cases for None, bytes, etc.
• bytes values may need base64 encoding or a custom serialization strategy since Redis sorted set members are strings
• Consider aligning the approach with how RedisDict and other data structures handle non-JSON types

Related Issues

• Implement RedisPriorityQueue data structure #79 — Implement RedisPriorityQueue data structure
• Design special field infrastructure for data structures (Set, Priority Queue, etc.) #172 — Design special field infrastructure for data structures

List tasks

• Add support for None values in serialization/deserialization
• Add support for bytes values (e.g., base64 encoding)
• Add support for dict values
• Add support for list values
• Add support for Pydantic model values
• Add integration tests for all new value types
• Add unit tests for all new value types