WebDriver BiDi needs the concept of navigation id in order link several events and commands participating in a navigation together. (browsingContext.navigate commands + browsingContext and network events, see BiDi spec)

This navigation id is supposed to be generated at step 12 of https://html.spec.whatwg.org/#beginning-navigation

Invoke WebDriver BiDi navigation started with targetBrowsingContext, and a new WebDriver BiDi navigation status whose id is navigationId, status is "pending", and url is url.

From discussions on https://bugzilla.mozilla.org/show_bug.cgi?id=1763122, we already know that there is no identifier on the platform side which would be a good match. One of the suggestion was to try to manually create such an id by relying on the progress listener.

But as we saw on https://bugzilla.mozilla.org/show_bug.cgi?id=1756595#c8, the progress listener starts observing a load too late compared to our expectations. STATE_START will be observed after beforeunload, whereas by spec we are supposed to emit WebDriver BiDi events before that, with a proper navigation id.

Would it be possible to update the Progress Listener in order to observe something that matches WebDriver BiDi's navigation started? Or is there something else we could use ? (LoadingSessionHistoryEntry was also mentioned on Bug 1763122, but it was already suspected to be too late?)

Hi Olli, if you have an idea how this could get done we would appreciate any kind of feedback. Thanks.

So you'd want to get which all notifications? One before beforeunload and another one - if navigation is cancelled - for cancelling. Anything else?

We don't currently have any such notifications. Should it be possibly some new method to nsIWebProgressListener ?

(In reply to Olli Pettay [:smaug][[email protected]] from comment #2)

So you'd want to get which all notifications? One before beforeunload and another one - if navigation is cancelled - for cancelling. Anything else?

What would be important for us is to know when a navigation actually starts. Not sure if there is something special to track as well before beforeunload and unload happens, but with getting a status change or whatever in the progress listener we could generate a navigation id for tracking purposes.

We don't currently have any such notifications. Should it be possibly some new method to nsIWebProgressListener ?

Yes, this would probably be fine given that all of our navigation logic is based on the progress listener. Do you think that this is hard to do?

What does "navigation starts" mean in this case? When an <a> element is clicked or when network connection is opened or what?

NavigationTiming code is notified here: https://searchfox.org/mozilla-central/rev/d31e56f7b3c2c18b8071a7b2a2fb6b4e01e3d3e8/docshell/base/nsDocShell.cpp#9446, right before beforeunload, is that the only thing needed, some new notification?

(In reply to Olli Pettay [:smaug][[email protected]] from comment #4)

What does "navigation starts" mean in this case? When an <a> element is clicked or when network connection is opened or what?

NavigationTiming code is notified here: https://searchfox.org/mozilla-central/rev/d31e56f7b3c2c18b8071a7b2a2fb6b4e01e3d3e8/docshell/base/nsDocShell.cpp#9446, right before beforeunload, is that the only thing needed, some new notification?

Does this match https://html.spec.whatwg.org/#beginning-navigation step 14?

Invoke WebDriver BiDi navigation started with targetBrowsingContext, and a new WebDriver BiDi navigation status whose id is navigationId, status is "pending", and url is url.

That's where we need to be notified.

Edit: to answer the initial question, I think this needs to be emitted before the request is created, since it should occur before onbeforeunload runs?

I still wonder how to notify.... nsIWebProgressListener doesn't quite work since there isn't any request yet at that point.
We could pass loadIdentifier to the callback, would that be enough? I guess then nsIWebProgressListener could have a new method, something like
onTentativeNavigation(uint64_t id) and for cancelling that before request is created we'd need also a new one, onTentativeNavigationCancelled.
(Implementing that cancelled part might be a bit tricky since navigation, which hasn't yet really started may get cancelled in many places)

But would that API work? I guess we might need to propagate loadIdentifier to some other places too. Currently it lives for example in the browsing context.

We could pass loadIdentifier to the callback, would that be enough? I guess then nsIWebProgressListener could have a new method, something like
onTentativeNavigation(uint64_t id) and for cancelling that before request is created we'd need also a new one, onTentativeNavigationCancelled.

This sounds good to me. If we have access to loadIdentifier across all WebProgressListener callbacks, then we should be able to use this as navigation id (probably with a 1:1 mapping to a UUID but that's a detail). Would this loadIdentifier remain stable during cross-process/cross-group navigations as well?

I guess we might need to propagate loadIdentifier to some other places too. Currently it lives for example in the browsing context.

Do you mean making the loadIdentifier accessible from the parent process? I am using the content process WebProgressListener for now, because I am unable to monitor some navigations with the parent process one. So if the loadIdentifier is available in parent process and if the parent process web progress can really monitor all navigations, it would simplify the implementation for us. But otherwise we can probably live with a content process solution for now.

nsIWebProgressListener doesn't quite work since there isn't any request yet at that point.

To make sure, you meant that it doesn't work right now, but we are still ok to add the callbacks you mentioned to nsIWebProgressListener?

Should we split this in several bugs: one for onTentativeNavigation, one for onTentativeNavigationCancelled and one to propagate the LoadIdentifier to the parent process?

loadIdentifier changes when redirect happens, I think.

And yes, I meant that nsIWebProgressListener doesn't work right now, since it expects an nsIRequest object to be around.

I think this should be all one bug.

Hi Olli,

Quick question, do you think the onTentativeNavigation you suggested to add to WebProgressListener would also be invoked in case of same document navigations (eg hash changes, document write usage?)

I wasn't thinking so.

Thanks.

Reading the spec, it makes sense. The fragment navigation does not trigger this BiDi navigation started event, instead it has its own dedicated event FragmentNavigated, and I think this is the only place in WebDriver BiDi where we will see the navigation id for a fragment navigation. So I guess as long as we have a unique id here we should be fine and we can probably just generate it on the fly for the FragmentNavigated event.

For document.write I think based on the spec, there is no navigation id created for this. So even though it will trigger a few events which could have a navigation id (browsingContext.domContentLoaded + browsingContext.load), I don't think we have to set it.

So we can probably keep this to be solely about actual navigations.