Skip to content

feat: Add retries and gzip compression#77

Open
rafaeelaudibert wants to merge 1 commit intomasterfrom
add-retries-and-gzip
Open

feat: Add retries and gzip compression#77
rafaeelaudibert wants to merge 1 commit intomasterfrom
add-retries-and-gzip

Conversation

@rafaeelaudibert
Copy link
Member

This will hopefully bring us up to standard, meaning we pass all the compliance tests. It's also cool since it saves us some bytess (at the cost of CPU of course)

This will hopefully bring us up to standard, meaning we pass all the compliance tests. It's also cool since it saves us some bytess (at the cost of CPU of course)
@github-actions
Copy link
Contributor

github-actions bot commented Feb 5, 2026

posthog-elixir Compliance Report

Date: 2026-02-05 13:42:36 UTC
Duration: 127981ms

⚠️ Some Tests Failed

27/29 tests passed, 2 failed


Capture Tests

⚠️ 27/29 tests passed, 2 failed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 258ms
Format Validation.Event Has Uuid 259ms
Format Validation.Event Has Lib Properties 258ms
Format Validation.Distinct Id Is String 259ms
Format Validation.Token Is Present 259ms
Format Validation.Custom Properties Preserved 258ms
Format Validation.Event Has Timestamp 259ms
Retry Behavior.Retries On 503 8070ms
Retry Behavior.Does Not Retry On 400 2262ms
Retry Behavior.Does Not Retry On 401 2261ms
Retry Behavior.Respects Retry After Header 5264ms
Retry Behavior.Implements Backoff 22109ms
Retry Behavior.Retries On 500 6080ms
Retry Behavior.Retries On 502 6081ms
Retry Behavior.Retries On 504 6081ms
Retry Behavior.Max Retries Respected 30108ms
Deduplication.Generates Unique Uuids 269ms
Deduplication.Preserves Uuid On Retry 6081ms
Deduplication.Preserves Uuid And Timestamp On Retry 13075ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 6185ms
Deduplication.No Duplicate Events In Batch 264ms
Deduplication.Different Events Have Different Uuids 261ms
Compression.Sends Gzip When Enabled 259ms
Batch Format.Uses Proper Batch Structure 259ms
Batch Format.Flush With No Events Sends Nothing 256ms
Batch Format.Multiple Events Batched Together 264ms
Error Handling.Does Not Retry On 403 2262ms
Error Handling.Does Not Retry On 413 2261ms
Error Handling.Retries On 408 6080ms

Failures

retry_behavior.respects_retry_after_header

Expected at least 2 requests, got 1

compression.sends_gzip_when_enabled

Header 'Content-Encoding' with value 'gzip' not found in requests

Copy link
Member Author

Apparently not working 100% yet, will work on it!

doc:
"Maximum number of retry attempts for failed batch requests."
],
gzip: [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, why make it configurable?

In general, the idea behind api_client_module option was to provide an avenue to override the whole HTTP client and therefore any option. Otherwise the top-level configuration is doomed to eventually include most options that usually come with HTTP: retries, timeouts, logging, telemetry, middleware, proxies.

If the only reason to configure gzip is to let cpu-sensitive folks disable it, I imagine they would be motivated enough to just define their own client!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

100% agree. This is me trying to get to 100% on this arbitrary set of "assumptions" we made and that all clients should fulfill. This was my first dumb way to implement it, but I think I'm tending towards just implementing a client (on the compliance adapter folder) that demonstrates that this can be supported by the implementation (by implementing your own client like one would in Elixir) rather than enabling it as a config.

I won't be merging this soon, I'll iterate on it slowly.

Question for you: how do you feel with gzip being turned on by default in something like Elixir?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, let me know if you want me to take a stab at any of this!

Answering your question, I don't know much about gzip to be honest. Are there any trade offs? Looking at Req code, it calls built-in :zlib module, which is probably a C NIF under the hood.

when attempt > max_retries,
do: :ok

defp send_with_retries(api_client, events, max_retries, attempt) do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Req itself comes with retries plugin and I think it's even enabled by default, although maybe not for POST requests. We should be able to just configure Req and change tracked client to be nice and not cut it off.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lovely, I'll do more research, thank you 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants