Skip to content

Expose apipeline operation results to users #225

Description

@yedidyakfir

Summary

When users perform operations inside an apipeline context manager (deletes, updates, field assignments, TTL refreshes, etc.), the return values from those operations are currently not accessible. The pipeline's execute() is called internally on context exit and its results are discarded. Users need a way to access these results so they can know, for example, how many keys were deleted, what previous values were overwritten, or whether specific operations succeeded.

Features / Details

  • Provide a mechanism for users to retrieve the results of all actions performed during an apipeline block
  • Each operation queued in the pipeline (delete, update, field set, TTL refresh, etc.) should have its return value accessible after the pipeline executes
  • Results should be associated/identifiable with the originating action (not just an opaque flat list of Redis responses) so users can correlate which result belongs to which operation
  • Must work for both apipeline() (standalone) and the use_existing_pipe=True case
  • Must continue to work correctly with the ignore_redis_error and NOSCRIPT recovery paths (rapyer/base.py:1018)

Use Cases

  • A user calls adelete on multiple models inside apipeline and wants to know which actually existed and were deleted
  • A user updates fields inside apipeline and wants the previous values returned by the underlying Redis commands
  • A user refreshes TTLs inside apipeline and wants to confirm which keys had their TTL successfully updated
  • Bulk operations where the caller needs per-operation success/failure feedback without giving up the atomicity of a pipeline

Implementation Considerations

  • The pipeline currently executes inside _apipeline after the yield (rapyer/base.py:1037-1045), so results are only available post-yield — the API needs to surface them to user code that ran before the yield returned
  • Likely options: return a results object/handle that the user obtains inside the with block and reads after exit, or expose results via the context (e.g., attach to the yielded pipe or to the rapyer context)
  • Need to handle the NOSCRIPT retry path (rapyer/base.py:1058-1072) — results from the retry pipeline must replace/merge with the original results coherently
  • Need to decide behavior when ignore_redis_error=True swallows a ResponseError — results may be partial or absent
  • Operations performed via Lua scripts (EVALSHA) return script results; mapping these back to the high-level rapyer action that issued them needs design
  • Consider how this interacts with use_existing_pipe=True where the user may own the outer pipeline lifecycle

List tasks

  • Design the user-facing API for retrieving per-action results (handle object vs. attribute on yielded pipe vs. context-based accessor)
  • Define how each rapyer action (delete, update/setattr, TTL refresh, field assignment, bulk ops) registers its result slot at queue time
  • Implement result capture from pipe.execute() and map responses to the registered actions
  • Handle the NOSCRIPT recovery retry path so retried operation results are surfaced correctly
  • Handle ignore_redis_error=True — document and implement behavior for partial/absent results
  • Handle use_existing_pipe=True — define how results are surfaced when rapyer doesn't own pipeline execution
  • Add unit tests under tests/unit/pipeline/ covering result retrieval for each supported action type
  • Add integration tests under tests/integration/pipeline/ covering end-to-end result retrieval
  • Update documentation: docs/api/rapyer-functions.md, docs/documentation/atomic-actions.md, and docs/api/atomic-redis-model.md
  • Add a CHANGELOG entry

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpipelinePipeline operations

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions