ref(reprocessing): optionally write unprocessed payload to nodestore instead of redis

[matt-codecov]

Ref FS-533

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

[matt-codecov]

@sentry review
bugbot run

[linear-code]

FS-533

[sentry]

Bug: Unprocessed event payloads backed up to nodestore with a :u suffix are not cleaned up by the retention process, leading to a storage leak.
_{Severity: MEDIUM}

Suggested Fix

Modify the event deletion process to also remove the associated :u node from nodestore. This can be done by updating delete_events_from_nodestore to explicitly delete the Event.generate_node_id(project_id, event_id) + ":u" key. Alternatively, a TTL could be set on the :u node upon its creation to ensure it expires automatically.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/sentry/reprocessing2.py#L182

Potential issue: When an unprocessed event payload is backed up to nodestore, it is
stored under a new key with a `:u` suffix. This new node is created without a TTL and is
not cleaned up by the regular event retention mechanism. The retention task,
`delete_events_from_nodestore`, only deletes the main event node, not the separate `:u`
variant. This causes the `:u` node to be orphaned and accumulate indefinitely, leading
to a storage leak. The `delete_unprocessed_event` function does not address this, as it
is only called for discarded events, not for successfully saved events that are later
deleted by retention.

_{Did we get this right? 👍 / 👎 to inform future reviews.}