PerformanceEntries are affected by backgrounded status

[npm1]

When a tab is backgrounded, it can become throttled, it will never paint, etc. In general, browsers apply heuristics to mitigate the impact of backgrounded tabs on performance of the visible tab. However, this affects aggregation of Performance APIs: values computed during a background status are not really comparable to values computed during a visible status because the browser can behave differently in these cases.

Loading is the most affected because (1) most performance metrics involve loading in one way or another and (2) it's common to be backgrounded while loading. I think the most common case of this would be opening a new browser window with multiple tabs: all but one will load in the background.

A prominent example of this is PaintTiming. The paint timing timestamps are only computed once the process actually paints, which means that the timestamps can be inflated arbitrarily while the tab is in the background.

I think the spec needs to provide some tools to allow more easily separating visible vs backgrounded metrics or at least provide some guidance about the necessity to do so.

[tdresser]

@n8schloss - Ilya mentioned you've had some thoughts here in the past? Are you aware of any previous discussion?

[igrigorik]

@tdresser @npm1 apologies about the delay. The related thread is @ w3c/page-visibility#29

[yoavweiss]

I think the use case raised by @n8schloss is the one where we need an answer to the question: "was this page ever visible?". That seems slightly different than the use case brought here.

Here it seems like we need different entries to have different definitions to whether the were affected by the page being BG:

• PaintTiming: paint never actually happens, so no timings will be displayed.
• ResourceTiming: resources loaded while the page was in the background can be affected
• Same for Navigation Timing, but for the whole page's resources (at least the ones that block onload)
• EventTiming: dispatching time or available CPU can be impactful

So, in some cases, it may make sense to add a flag on the PerformanceEntry itself, indicating that it was impacted by the tab being in the background.
In other cases, (e.g. PaintTiming, but potentially also in RT, ET, etc.), there is no entry, or the entry can be significantly delayed. So in these cases it makes sense to indicate this elsewhere.

I'm not sure what's the best way to tackle this:

• Page Visibility already gives us a global indicating overall visibility time intervals. Existing entries can match up against that.
• At the same time, it's possible that being in the BG is preventing or delaying entries from appearing. That is information that the browser currently doesn't expose.

Maybe the missing piece here is a "was impacted by BG throttling", which each of the features would be responsible for filling in, or would be entirely UA defined. Does that make sense?

[npm1]

Page Visibility only has an event handler for visibility change. It does not say what the previous visibility state was, and to use in this case would require early event handler registration via Javascript. So the current API is not good for our use case.

[gi11es]

Based on what was discussed during the meeting, is it fair to say that being able to look up a reliable visibility history would solve this problem, by way of identifying "tainted" page views?

The current set of JS APIs doesn't allow catching all background tab status reliably, because the status during the timespan between the tab being open (possibly in the background) and when the first piece of JS can run remains unknown.

As @yoavweiss suggested, it would make sense to record "VisibilityEntries" in the performance timeline for visibility changes, as well as an origin event that would contain whether the tab was initially opened in the background of in the foreground.

[Zizzamia]

@gi11es great point!
In Perfume.js I added an onVisibilityChange to disable logging and analytics when the tab becomes hidden because any data during a visibility change become high unreliable.

[yoavweiss]

As @yoavweiss suggested, it would make sense to record "VisibilityEntries" in the performance timeline for visibility changes, as well as an origin event that would contain whether the tab was initially opened in the background of in the foreground.

To add to that, we discussed on the call that out-of-viewport may also be throttled in the future, so we would then need to change PageVisibility (if possible) to reflect that.
Another alternative is to include some kind of "was throttled" signal that may better reflect what we're trying to measure here (so instead of tying the signal to visibility, we could add a ThrottlingEntry which may correlate with visibility at first, but might diverge from it in the future and in different implementations).

[gi11es]

Indeed, entries marking throttling sound like a more generic and future-proof solution!

It probably requires having separate entries when the throttling starts and when it stops. Otherwise relying on the startTime/duration paradigm of PerformanceEntry, it would only get recorded when throttling stops, not letting us know if there is ongoing throttling if we are in the middle of it.

[npm1]

The only problem with ThrottlingEntry is that it would be harder to specify. I would propose starting with page visibility (possibly adding out-of-viewport), which would likely cover most cases?

[gi11es]

I’m not sure how « out of viewport » would work in regards to visibility entries. Pages are usually larger than the viewport, which means that there’s always something out of view.

I think it’s pretty clear what constitutes throttling, it’s whenever the browser slows down JS execution purposefully. Specific examples can be provided, like not honoring small timeout values in setTimeout, etc.

What I think is most useful about making the entries about throttling is that it represents what truly affects the performance metrics. Even right now, browsers don’t necessarily throttle background tabs, and it’s possible that when they do it’s not at the exact moment the tab is put in the background. There’s a correlation in some implementations, but all the people currently discarding performance metrics for any tab that’s been in the background at some point are discarding perfectly valid, unthrottled data out of caution. If the actual throttling was recorded, we wouldn’t have to discard more than we need to.

[igrigorik]

FWIW, I think we're converging on the ~same solution here that we came up with at last year's F2F: w3c/page-visibility#29 (comment).

One change I'd make to what we described in that issue is migrating away from buffers and to PerfObserver + buffering of events prior to onload.

[tdresser]

"I think it’s pretty clear what constitutes throttling, it’s whenever the browser slows down JS execution purposefully. Specific examples can be provided, like not honoring small timeout values in setTimeout, etc."

Throttling isn't a binary thing though. Browsers can use all kinds of heuristics to prioritize frames and tabs, many of which will slow down JS execution to some degree. I don't think it will be clear which of these should set the "throttled" bit. It certainly isn't all of them, as some heuristics will only slow things down by a minute amount. It's not unreasonable to imagine a browser in which effectively every page has it's JS throttled by some degree.

[gi11es]

In the case where every page is throttled, is it really that bad that they all get marked as such? It's important information when looking at performance metrics.

I'm not familiar enough with browser internals to know if these different types of throttling need to be categorized. But as a consumer of performance metrics, I would like to know when these deliberate slowdowns happen. If it's something large enough to affect the reliability of the RUM metrics, it should be flagged. Maybe that effect should be the criteria, rather than focusing on the nature of the slowdown.

[tdresser]

"If it's something large enough to affect the reliability of the RUM metrics, it should be flagged."

Yeah, I think this is roughly correct, but extremely difficult to spec, and it's tricky to define "large enough".

If the browsers have acceptably similar behavior for when "large enough" throttling takes place, it may be just as good (and a lot easier) to specify the signals browsers use to decide to do "large enough" throttling, instead of specifying what it means to be doing "large enough" throttling.