feat(python): eliminate SSE dependency 'httpx_sse' by hard-forking into core_utilities

[aditya-arolkar-swe]

Description

Generator change to yield this output: cohere-ai/cohere-python#700

Fixes FER-6826

Changes Made

• Hard forked httpx_sse, moved into core_utilities, made fixes to line parsing bugs and improved idiomatic class definition.
• updated parser to: copy http_sse code into output, use new in-house SSE EventStream class and better handle failed SSE parsing

Testing

• Unit tests added/updated
• Manual testing completed

[graphite-app]

The buffer processing logic at the end doesn't handle multi-line content correctly. If the final buffer contains multiple lines without a newline separator (e.g., data received with \r but no \n), only the entire buffer will be processed as a single line. This could cause event data to be incorrectly parsed.

Consider modifying this section to handle potential multi-line content in the remaining buffer, perhaps by splitting on \r characters and processing each line individually, similar to how the main loop processes lines.

Suggested change

	# Process any remaining data in buffer
	if buffer.strip():
	line = buffer.rstrip('\r')
	sse = decoder.decode(line)
	if sse is not None:
	yield sse
	# Process any remaining data in buffer
	if buffer.strip():
	# Split the buffer by \r and process each line
	for line in buffer.split('\r'):
	if line.strip():
	sse = decoder.decode(line)
	if sse is not None:
	yield sse

Spotted by Diamond



Is this helpful? React 👍 or 👎 to let us know.

[tjb9dc]

I think this is wrong, and the SSE spec is very explicit about the double newline splitting events

[graphite-app]

The buffer processing logic at the end of the function doesn't properly handle incomplete SSE events. When the stream ends with data that doesn't include a final newline, the code attempts to process it as a complete line, but doesn't signal to the decoder that this is the end of the stream.

Consider adding a final empty line decode after processing any remaining buffer content:

# Process any remaining data in buffer
if buffer.strip():
    line = buffer.rstrip('\r')
    sse = decoder.decode(line)
    if sse is not None:
        yield sse
    
    # Signal end of stream to finalize any pending event
    final_sse = decoder.decode('')
    if final_sse is not None:
        yield final_sse

This ensures that any partial event in the buffer gets properly finalized and emitted.

Suggested change

	# Process any remaining data in buffer
	if buffer.strip():
	line = buffer.rstrip('\r')
	sse = decoder.decode(line)
	if sse is not None:
	yield sse
	# Process any remaining data in buffer
	if buffer.strip():
	line = buffer.rstrip('\r')
	sse = decoder.decode(line)
	if sse is not None:
	yield sse
	# Signal end of stream to finalize any pending event
	final_sse = decoder.decode('')
	if final_sse is not None:
	yield final_sse

Spotted by Diamond



Is this helpful? React 👍 or 👎 to let us know.

[tjb9dc]

I feel like we should catch this one with a test?

[aditya-arolkar-swe]

This is actually intentional. If the SSE server doesn't send a final chunk that closes out an SSE event. we don't want to force the processing of an event, which is what the graphite suggested logic would do:

            # Signal end of stream to finalize any pending event
            final_sse = decoder.decode('')

The current version would be a stricter way to handle SSE chunks, where the server must end on /n/n to close out events or our SDK won't process it.

[tjb9dc]

did you test this with a multiline event (like a JSON blob containing a content field with extra lines)? This doesn't appear to handle that to me

[aditya-arolkar-swe]

I tested it out with Cohere's bug. The extra lines in the JSON blob must be escaped ("\n\n") and this works fine. If they are regular double new lines ("\n\n"), we will interpret it as a new event.

[tjb9dc]

Can we have it anchored in the user-facing outcome more? I also don't love "hard-forking" language in the changelog. I think the points we are trying to land are:

• Fix SSE handling of events longer than or containing escaped newlines
• Remove external dependency on httpx-sse

[tjb9dc]

So we could include this file (or a version of it) like the other as-is unit test files and generate it for all the customers who have SSE included in their APIs.

I have a moderate preference for doing that, but regardless this looks good :)