There is an ongoing Firefox sprint (This looks like a URI that will die in the future. Sniff.) trying to identify issues in the browsers and report them on Webcompat.
We got a flood of new issues. Unfortunately a lot of them were invalid and were not part of what we qualify as Web Compatibility issues. This strains a lot on our team (usually Adam, Eric and myself doing triage, but for this occasion everyone had to join) and it is probably frustrating for the reporters to see their bugs closed as invalid. So in the spirit of "How do we make it better for everyone?", a couple of guidelines for people running the events related to Webcompat sprint.
What is a Webcompat issue?
Or more exactly what is not a Webcompat issue.
Our work focuses on identifying differences in between browsers when accessing a Web site. That said all differences are not necessary Web compatibility issues.
Before starting a Webcompat sprint (organizers)
- Fully understand this document. If you organizing an event and have questions unanswered in this page, please ask a question.
- Make sure to explain to participants that quality of the reports is the goal.
- Make sure that participants have created a fresh profile.
- This fresh profile should not contains any add-ons, except if needed the reporter extension. Developer Edition and Nightly already have the Report Site Issue button into the "•••" menu.
- This fresh profile can be set in a way that it resets itself at each restart.
- Encourage participants to write detailed steps for reproducing the issue. An idea is for example to work as a team. One finds an issue, write the form without submitting it. The team mate on another computer is trying to reproduce the issue with only the form instructions. If the person can't understand the given instructions and/or can't see where the issue is, then it's probably an incomplete report.
During the Webcompat sprint (participants)
Testing in Multiple Browsers
The most important thing of all. You need to test in at least two browsers. The more, the merrier. Being sure that it's actually not breaking elsewhere is a good for a webcompat issue. Sometimes it's better to ask someone else who is using a browser and compare the results.
Responsive Mode and Devices
Responsive mode on desktop browsers is a quick way to test if the website is ready for mobile devices. That said these responsive modes have their own limitations. They are not simulators, just emulators. You might want to double check on a real device before reporting.
Slow Network
If the network is slow. It is usually slow in any browsers. These do not make necessary a Web compatibility issue. Performance issues are very hard to decipher, because they are dependent on many external parameters.
- Try to reproduce in another browser. If it's blazing fast in another browser. There might be something interesting.
- Try to reload and see if the second load is faster (it should if the website has been correctly parametrized.)
Most of the time, this is not a Webcompat issue.
Flash plug-in
Some sites do not work or half work without a flash plugin. We can't really do anything about it. It might not be a good business decision but they chose it. It creates basically the same issue for all browsers. If you are unsastified with it, try to find their feedback or contact page and send them an email.
This is not a Webcompat issue.
Java Applets
Java Applets were used a lot in the 90s to provide application contexts in Web page when HTML was not enough. It has mostly disappeared in the context of a Web page. If you see a page dependent on Java, no need to report it.
This is not a Webcompat issue.
Tracking Protection List
Firefox and some other browsers have mechanisms to block trackers. If you are accessing a website with a strict tracking protection active. The site is likely to break in some fashions. Sites have a tendency to track all your actions. And some scripts fail when the tracker is not working.
This is not a Webcompat issue.
Ads Blockers/Script Blockers
This is a variation of the Tracking Protection list. Any addon you install has the potential to disrupt the normal operation of a website. It's even more acute when it's about blocking ads or scripts. ADB, uBlock, NoScript are the most current reasons for a website failing. Sometimes you will get a site completely blank or just with the text and no layout at all.
This is not a Webcompat issue.
Desktop site instead of mobile site
When testing on mobile, make sure to test on a device. Receiving a desktop site on a mobile device is frequent. Not all websites have specific versions of their site for mobile, or a website which adjusts itself to the screen context. That said, there are specific case which are worth reporting.
- Receiving a desktop site on a mobile device while on the same device, Chrome is receiving the mobile version. Very often this is the result of user agent sniffing. You can report it.
- Receiving a desktop site partially visible while Chrome seems to adjust the site to the current screen. This is likely a duplicate of the lack of virtual viewport on Firefox Android. You can report it.
Different mobile versions
Sometimes some websites are sending a different version to two browsers. For example a text only version to one browser and a very graphic one to another browser. It's probably the result of user agent sniffing. You can report it.
Fluid layout
Some websites have fluid or responsive layouts. These layouts adjust well more or less to small screens. When they don't adjust, you might, for example, see a navigation bar with items folding on a second line. These issues are difficult to identify. Your best lead is to test in another browser. If you get the same behavior in both Chrome, Firefox, Edge and Safari on mobile, then it's not a webcompat issue.
This is not a Webcompat issue.
Login/Password required / Credit card required
This one is quite hard and heart wrenching. Many sites require login and password to be able to interact with them. Think social networks, emails, school networks, bank accounts, etc. For a very common site, we might be able to reproduce because some of us have an account on the same sites. But in many occasions, we do not. For example, you want to report a webcompat issue for a private page on your bank. It's unlikely that we will be able to do anything without being able to access the actual page. We are not client on this site. We do not have a credit card for buying stuff you bought.
- If you are anonymous, do not report it. There's nothing we can do about it.
- If you report it as a github user on webcompat.com, be ready to followup. It's very likely we will not be able to access the page ourselves, but we might be able to guide you in analyzing the issue for finding out the issue.
It might be a Webcompat issue, but it might not be worth reporting it.
Browser features (Reader View, Tabs, etc.)
Browsers all have specific features accessible through what we call the chrome (different from the chrome browser). This is basically things which are part of the browser UI and assists in having a better usage. The best way to report an issue you noticed is to directly open a bug report on the browser vendor reporting system.
This is not a Webcompat issue.
SSL errors
Some sites have very poor security practices or outdated certificates. So when the browser blocks you to access a website with a message saying "An error occurred during a connection to …" is likely an issue with their SSL handling. You can test that yourself by entering the domain name into a validator. Another common reason is the HTTPSEverywhere add-on (and alike) which tried to force redirect to https for all websites. Some sites do not have an https version. Then the site will fail under https, but will work with http. Feel free to contact with a link to the SSL validation.
This is mostly not a Webcompat issue.
Private domain names / Proxies / Routers UI (not on the public internet)
Web UIs are used for many type of applications. Many of them are accessible only in the context of a company network, a local home network, etc.
- If the site is not accessible through the public internet and you are anonymous, do not report it. We will not be able to do anything about it.
- On the other hand, if you are a github user and you are ready to help us with followup questions and diagnosis, it might be worth reporting.
After the Webcompat sprint (organizers, participants)
It's not about quantity, it's about quality. We try to improve the Web. A larger number of issues might not necessary help us to fix a browser or a website faster. On the other hand, being able to followup on the issues when there are unknown details or additional questions from people triaging and diagnosing is invaluable. Reporting many issues without being able to answer questions afterward because you have no time, or you are not interested, might waste time of many people.
You might want to do additional meetings for specifically following up on these issues. It's a great opportunity to learn how to debug and understand what is happening in a Web page.
Otsukare!