Capping macOS User Agent String on macOS 11

Update on 2021-03-02: Capping User Agent String - followup meeting

This is to keep track and document the sequence of events related to macOS 11 and another cascade of breakages related to the change of user agent strings. There is no good solution. One more time it shows how sniffing User Agent strings are both dangerous (future fail) and source of issues.

Brace for impact!

Lion foot statue

Capping macOS 11 version in User Agent History

2020-06-25 OPENED WebKit 213622 - Safari 14 - User Agent string shows incorrect OS version

A reporter claims it breaks many websites but without giving details about which websites. There's a mention about VP9

browser supports vp9

I left a comment there to get more details.
2020-09-15 OPENED WebKit 216593 - [macOS] Limit reported macOS release to 10.15 series.
```
    if (!osVersion.startsWith("10"))
    osVersion = "10_15_6"_s;
```
With some comments in the review:

preserve original OS version on older macOS at Charles's request

I suspect this is the Charles, the proxy app.

2020-09-16 FIXED
2020-10-05 OPENED WebKit 217364 - [macOS] Bump reported current shipping release UA to 10_15_7

On macOS Catalina 10.15.7, Safari reports platform user agent with OS version 10_15_7. On macOS Big Sur 11.0, Safari reports platform user agent with OS version 10_15_6. It's a bit odd to have Big Sur report an older OS version than Catalina. Bump the reported current shipping release UA from 10_15_6 to 10_15_7.

The issue here is that macOS 11 (Big Sur) reports an older version number than macOS 10.15 (Catalina), because the previous bug harcoded the string number.
```
    if (!osVersion.startsWith("10"))
    osVersion = "10_15_7"_s;
```
This is still harcoded because in this comment:

Catalina quality updates are done, so 10.15.7 is the last patch version. Security SUs from this point on won’t increment the patch version, and does not affect the user agent.

2020-10-06 FIXED
2020-10-11 Unity [WebGL][macOS] Builds do not run when using Big Sur

UnityLoader.js is the culprit.

They fixed it on January 2021(?). But there are a lot of legacy codes running out there which could not be updated.

Irony, there’s no easy way to detect the unity library to create a site intervention that would apply to all games with the issue. Capping the UA string will fix that.
2020-11-30 OPENED Webkit 219346 - User-agent on macOS 11.0.1 reports as 10_15_6 which is older than latest Catalina release.

It was closed as a duplicate of 217364, but there's an interesting description:

Regression from 216593. That rev hard codes the User-Agent header to report MacOS X 10_15_6 on macOS 11.0+ which breaks Duo Security UA sniffing OS version check. Duo security check fails because latest version of macOS Catalina is 10.15.7 but 10.15.6 is being reported.
2020-11-30 OPENED Gecko 1679929 - Cap the User-Agent string's reported macOS version at 10.15

There is a patch for Gecko to cap the user agent string the same way that Apple does for Safari. This will solve the issue with Unity Games which have been unable to adjust the code source to the new version of Unity.
```
// Cap the reported macOS version at 10.15 (like Safari) to avoid breaking
// sites that assume the UA's macOS version always begins with "10.".
int uaVersion = (majorVersion >= 11 || minorVersion > 15) ? 15 : minorVersion;

// Always return an "Intel" UA string, even on ARM64 macOS like Safari does.
mOscpu = nsPrintfCString("Intel Mac OS X 10.%d", uaVersion);
```
It should land very soon, this week (week 8, February 2021), on Firefox Nightly 87. We can then monitor if anything is breaking with this change.
2020-12-04 OPENED Gecko 1680516 - [Apple Chip - ARM64 M1] Game is not loaded on Gamearter.com

Older versions of Unity JS used to run games are broken when the macOS version is 10_11_0 in the user agent string of the browser.

The Mozilla webcompat team proposed to fix this with a Site Intervention for gamearter specifically. This doesn't solve the other games breaking.

2020-12-14 OPENED Gecko 1682238 - Override navigator.userAgent for gamearter.com on macOS 11.0

A quick way to fix the issue on Firefox for gamearter was to release a site intervention by the Mozilla webcompat team

"use strict";

/*
* Bug 1682238 - Override navigator.userAgent for gamearter.com on macOS 11.0
* Bug 1680516 - Game is not loaded on gamearter.com
*
* Unity < 2021.1.0a2 is unable to correctly parse User Agents with
* "Mac OS X 11.0" in them, so let's override to "Mac OS X 10.16" instead
* for now.
*/

/* globals exportFunction */

if (navigator.userAgent.includes("Mac OS X 11.")) {
console.info(
    "The user agent has been overridden for compatibility reasons. See https://bugzilla.mozilla.org/show_bug.cgi?id=1680516 for details."
);

let originalUA = navigator.userAgent;
Object.defineProperty(window.navigator.wrappedJSObject, "userAgent", {
    get: exportFunction(function() {
    return originalUA.replace(/Mac OS X 11\.(\d)+;/, "Mac OS X 10.16;");
    }, window),

    set: exportFunction(function() {}, window),
});
}

2020-12-16 OPENED WebKit 219977 - WebP loading error in Safari on iOS 14.3

In this comment, Cloudinary explains they try to avoid the issue with the system bug with UA detection.

Cloudinary is attempting to work around this issue by turning off WebP support to affected clients.

If this is indeed about the underlying OS frameworks, rather than the browser version, as far as we can tell it appeared sometime after MacOS 11.0.1 and before or in 11.1.0. All we have been able to narrow down on the iOS side is ≥14.0.

If you have additional guidance on which versions of the OSes are affected, so that we can prevent Safari users from receiving broken images, it would be much appreciated!

Eric Portis (Cloudinary) created some tests: * WebPs that break in iOS ≥ 14.3 & MacOS ≥ 11.1 * Tiny WebP

The issue seems to affect CloudFlare
2021-01-05 OPENED WebKit WebP failures [ Big Sur ] fast/images/webp-as-image.html is failing
2021-01-29 OPENED Blink 1171998 - Nearly all Unity WebGL games fail to run in Chrome on macOS 11 because of userAgent
2021-02-06 OPENED Blink 1175225 - Cap the reported macOS version in the user-agent string at 10_15_7

Colleagues at Mozilla, on the Firefox team, and Apple, on the Safari team, report that there are a long tail of websites broken from reporting the current macOS Big Sur version, e.g. 11_0_0, in the user agent string:

Mac OS X 11_0_0

and for this reason, as well as slightly improving user privacy, have decided to cap the reported OS version in the user agent string at 10.15.7:

Mac OS X 10_15_7
2021-02-09 Blink Intent to Ship: User Agent string: cap macOS version number to 10_15_7

Ken Russell sends an intent to cap macOS in the Chrome (blink) user agent string to 10_15_7 to follow on the steps of Apple and Mozilla. In the intent to ship, there is a discussion on solving the issue with Client Hints. Sec-CH-UA-Platform-Version would be a possibility, but Client Hints is not yet deployed across browsers and there is not yet a full consensus about it. This is a specification pushed by Google and partially implemented by Google on Chrome.
- Mozilla - 935216 - Implement Client-Hints HTTP header
- Apple. I didn't find any Client-Hints bug for WebKit, except this partial proposal to implement images DPR through Client-Hints. I'm not sure what is the status for Apple.
Masataka Yakura shared with me (Thanks!) two threads on the Webkit-dev mailing-list. One from May 2020 and another one from November 2020.

In May, Maciej said:

I think there’s a number of things in the spec that should be cleaned up before an implementation ships enabled by default, specifically around interop, privacy, and protection against UA lockouts. I know there are PRs in flight for some of these issues. I think it would be good to get more of the open issues to resolution before actually shipping this.

And in November, Maciej did another round of spec review with one decisive issue.

Note that Google has released, last year, on January 14, 2020 an Intent to Ship: Client Hints infrastructure and UA Client Hints and this was enabled a couple of days ago on February 11, 2021.

And I'm pretty sure the story is not over. There will be probably more breakages and more unknown bugs.

Otsukare!