Skip to content

Conversation

@krichprollsch
Copy link
Member

Working on #1239, I'm trying to figure out why Playwright returns a null response.

I did find yet, but I have added some changes in Network events.

@krichprollsch
Copy link
Member Author

krichprollsch commented Dec 11, 2025

I added an important change in e2682ab
I now return the Page.navigate response after the navigation.

Claude mentioned that timing issue:

  Timing Issue

  Chrome sequence:
  1. Line 91: Page.navigate sent
  2. Line 96: Network.requestWillBeSent received ← BEFORE navigate returns
  3. Line 99: Network.responseReceived received
  4. Line 100: Page.navigate returns

  Lightpanda sequence:
  1. Line 71: Page.navigate sent
  2. Line 77: Page.navigate returns ← BEFORE requestWillBeSent!
  3. Line 82: Network.requestWillBeSent received ← TOO LATE
  4. Line 83: Network.responseReceived received
  
[...]

  3. Send Network.requestWillBeSent BEFORE Page.navigate returns
    - The CDP event must be dispatched to Playwright before the Page.navigate command response is sent
    - This allows Playwright to associate the pending request with the navigation's documentId

⚠️ But it doesn't fix the actual issue, so I'm not sure if I should keep it...

@krichprollsch
Copy link
Member Author

krichprollsch commented Dec 11, 2025

TODO The final problem is Playwright expects requestId equals loaderId

   The requestId and loaderId still don't match!

  Looking at the Playwright code again in crNetworkManager.ts:348-349:

  const isNavigationRequest = requestWillBeSentEvent.requestId === requestWillBeSentEvent.loaderId &&
                              requestWillBeSentEvent.type === 'Document';
  const documentId = isNavigationRequest ? requestWillBeSentEvent.loaderId : undefined;

  For a request to be recognized as a navigation request, BOTH conditions must be true:
  1. ✅ type === 'Document' - You fixed this!
  2. ❌ requestId === loaderId - Still broken!

  Since "REQ-1" !== "LID-2", the request is NOT recognized as a navigation request, so documentId remains undefined, and the
  request is never associated with the navigation.

  Required Fix

  In line 76, change:
  "requestId": "REQ-1",
  "loaderId": "LID-2",

  To use the same value for both:
  "requestId": "LID-2",
  "loaderId": "LID-2",

  Or alternatively:
  "requestId": "REQ-1",
  "loaderId": "REQ-1",

  The key is: for the main document navigation request, requestId must equal loaderId.

  Compare with Chrome's log (line 96 of chrome.log):
  {
    "requestId": "0FEB712DC451FB8C15552FA23FAB3B16",
    "loaderId": "0FEB712DC451FB8C15552FA23FAB3B16"  // ← Same value!
  }

  This is the final piece needed for Playwright to return a non-null response from page.goto().

For all events regarding an HTTP request, the values of requestId
and loaderId must be the same.
@krichprollsch krichprollsch marked this pull request as ready for review December 12, 2025 16:06
@krichprollsch
Copy link
Member Author

@karlseguin I added the change to make loaderId == requestId for all events relative to a given request.

If you want to review this PR again please 🙏

@karlseguin
Copy link
Collaborator

PR looks fine. Just to document past knowledge ,puppeteer relies on the loaderId to change in order to notice that a navigation has happened. Its waitForNavigation waits until the old loaderId != new loaderId. Looking at this PR, it seems like the loaderId will still changing on such internal navigations, so it should be fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants