Skip to content

Stop introspecting rawdata after Python 3.14.7 internal buffering added. - #168

Open
ianjosephwilson wants to merge 2 commits into
t-strings:mainfrom
ianjosephwilson:ian/stop_introspecting_rawdata
Open

Stop introspecting rawdata after Python 3.14.7 internal buffering added.#168
ianjosephwilson wants to merge 2 commits into
t-strings:mainfrom
ianjosephwilson:ian/stop_introspecting_rawdata

Conversation

@ianjosephwilson

Copy link
Copy Markdown
Contributor

We were introspecting rawdata to try to detect at least this error:

https://html.spec.whatwg.org/multipage/parsing.html#parse-error-eof-in-tag

eof-in-tag

This error occurs if the parser encounters the end of the input stream in a start tag or an end tag (e.g., <div id=). Such a tag is ignored.

Especially in the case of a dangling quote: html(t'<div x="{1}></div>').

This PR just cuts all that out and we lose some user experience but drop our dependency on rawdata.

Comment thread tdom/parser.py
raise ValueError(
"Parser expects more data, is the template valid html?"
)
super().close()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An internal flush has to be forced here so super.close() is called at the top.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃憤

@ianjosephwilson

Copy link
Copy Markdown
Contributor Author

@davepeck I think I prefer this. We can start something up at python hq and see if there is an official mechanism that could be exposed and if that doesn't work out then we can bring back the other PR or something else.

Comment thread tdom/parser.py
raise ValueError(
"Parser expects more data, is the template valid html?"
)
super().close()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃憤

Comment thread tdom/parser.py
def close(self) -> None:
if self.waiting_for_data():
# We apply heuristics here to try to guess why the parser didn't finish.
if self.rawdata.count('"') % 2 == 1 or self.rawdata.count("'") % 2 == 1:

@davepeck davepeck Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind losing this.

That said, if we wanted to restore this behavior, I suppose we could:

  1. Join our entire (with-placeholders) string and call feed() exactly once, and remember its length
  2. Before calling close(), call getpos() and see if there's still a bit of content left
  3. Run something like this check on the remaining content

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The result is pretty confusing, especially something like html(t'''<div id='1"></div>''') == ''.

It also breaks our placeholder tracking. Kind of a bummer. Maybe it won't happen very often because most people will be using an editor that will highlight the asymmetry of the quotes. I think the fact we are parsing small fragments is a disadvantage in this situation though because it exacerbates this type of mistake. This catches that we didn't use the placeholder but the user probably doesn't understand why: html(t'''<div id='{True}">''') raises ValueError: Some placeholders were never resolved..

A short test of your idea and it seems to work(!) but we are still implicitly depending on the implementation because we assume the buffering doesn't take place until after the first feed. We know that... because it says so in the source! Although if the threshold was higher then it might not parse at all. Seems like a really big implementation change on the stdlib's part which makes me nervous. Maybe this will just have to be a "gotcha" for a while...

I wonder where a good place to bring this up would be, a feature request on the issue tracker seems like it would get put on ice forever. Maybe starting a discussion on discuss.python.org? I'm an optimist on Thursdays.

@davepeck

davepeck commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

@ianjosephwilson agree this is the better approach.

We should probably add a true regression test for 166:

def test_issue_166():
    template = t"<button disabled={True}>x</button><button disabled={True}>y</button>"
    expected = "<button disabled>x</button><button disabled>y</button>"
    assert html(template) == expected

Other than that, happy to merge this one (and vastly prefer it over #167 !)

@ianjosephwilson

Copy link
Copy Markdown
Contributor Author

I realized that this doesn't actually update .python-version. I'm not sure what the best way is to handle that or if its just too early for us to try to deal with some sort of test matrix in the CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants