What exactly does it mean to pass the Acid3 test?

There has been some confusion about the test and its importance. Some people have been saying things like ”my browser does not pass the test and I have no problems using it”. Quite a few other people seem to think that Webkit and Gogi (Opera’s internal build) passed the test already in March – despite the fact that neither team has made this claim.

To answer these misconceptions we need to address the issue of what exactly is being tested and how. The main part of test is automated through JavaScript, a sort of test harness that runs 100 subtests. Getting a score of 100 is not the same as passing Acid3 – a common misconception, or perhaps an oversimplification.

Many subtests are high on a developer’s wish list: Full CSS 3 selectors support, media queries, SVG fonts. Admittedly a few others test edge cases and more esoteric features – but the test was supposed to be a significant challenge!

The second part is a rendering test. Some of the scripted subtests produce results that affect the rendering, but there are also rendering issues that come in addition to these. Some of them are high on many designers’ wish list: Text shadow, downloadable fonts, and display: inline-block.

The third test is the so called “smoothness” criterion. It is basically a speed test. No subtest may take too long – and especially subtest 26 is challenging. Compared to Slickspeed, Sun Spider, the V8 test suite or Dromaeo Acid3 is not so thorough. It will give some indication of a browsers speed, though.

This is exactly as planned. Acid3 was not meant to be the one and only indication of a browser’s performance. In fact many other test suites are far more important. (We provide links to some of them below.)

Testing is really important. Without tests that check how well a certain browser follows standards, i.e. applies mark up and displays the result correctly, we can never guarantee an open, fully interoperable web.

A highly visible test like Acid 3 hopefully helps to promote such interoperability. One can also hope that all the other tests will receive the attention they deserve. Writing them is not a glamorous task, but highly essential.

Apart from improving its support for CSS in its browser, Microsoft has contributed 2524 test cases to the CSS 2.1 test suite. For that they deserve credit!

We all know that Internet Explorer currently lag a bit behind the other browsers in standards compliance. Indeed they are last of the big ones to pass Acid2 and they fail Acid3 more than any other browser. But can we declare Webkit as the best rendering engine now that they pass it?

Of course not. Since Acid3 is only one indicator of many. Webkit’s achievement is great – and there are many other really exciting things they are pioneering, like CSS transitions and transformations. And with Squirrelfish Extreme JavaScript performance looks really exciting as well.

In other regards Opera is a clear leader. It is the only browser that supports more than 90 % of the SVG test suite. It is the only browser that implements Web Forms 2.0, currently being merged into HTML 5. They supported media queries and SMIL long before Acid3 came out.

Gecko (with Spidermonkey) is no longer an underdog. Besides the fun of meeting the technical challenge it is not hard to guess that the Webkit team rushed to pass Acid3 also for marketing reasons – that they perhaps need a bit more than Mozilla. Mozilla concentrated on releasing Firefox 3 before Acid 3 received any real attention. Now that they are working on it they are impressive in another way, compared to Webkit. Looking at the discussions for bug 410460 and its related bugs, it is clear that any improvement must be rock solid. Work often continues even when a particular feature is good enough for Acid3.

In fact, there is actually one open issue still in Acid 3 that might temporarily cause Webkit to become incompliant again. http://lists.w3.org/Archives/Public/www-style/2008Sep/0218.html. I rest assured that a fix probably already is being made, though.

Perhaps one can compare this to a race where you are supposed to run a distance, with a bucket of water. One competitor crosses the finishing line first, the other, on the other hand, has not lost a single drop from his bucket. Both have done great. (By the way, internal builds of Firefox get a score of 97 now, and downloadable fonts work on Windows and Mac.)

In the end the winner is neither Webkit, Opera, Mozilla nor Microsoft, but developers who get more powerful features to work with and more consistency between browsers. And that means that in the long run they are able to focus on user experience, not browser shortcomings. This means that the true winner of Acid3 is anybody who surfs the web.

Some other test suites for your review:

• Cascading Style Sheets level 2 revision 1
• Selectors
• CSS Color Module
• Collection of DOM tests
• SVG
• Web Forms
• Canvas

This guidance has been developed to assist those delivering public sector websites to determine which web browsers to use for testing. Public sector websites have a responsibility to be inclusive and not exclude groups of users but it would be impractical to test websites on every available browser.

So far, so apparently reasonable. I’m pleased to see that the COI advises that browsers on Windows, Mac and Linux be tested, that assistive technologies be tested, and it’s good that the draft guidance recognises that different sites have different target audiences.

But the central premise of the draft guidelines is fundamentally flawed.

Playing the numbers game

The central message of the draft guidelines (which are not available in HTML, only Word (280K) and PDF (160K)) is that public sector webmasters need not test in less-popular browsers.

If a browser appears in visitor logs as being below an arbitrary percentage of total “unique visitors” then it should not be listed as being “fully supported” in the site’s accessibility or help pages.

It may be listed as "semi-supported", which is defined: "A browser is semi-supported if the content and navigation works but the website does not display as intended”. Intended by whom? Are we back to the bad old days when webmasters strove for pixel-perfect rendering, even on governmental sites which are largely content-driven rather than design-dependant?

This page is best viewed with Browser X

The COI says “Avoid using statements such as, ‘this page is best viewed with Browser X’", and then goes on to advocate pushing users to change their browser.

There are many reasons why people use minority browsers, like Opera, Flock, Camino, OminWeb, Konqueror and the like (full disclosure: I work for Opera). A large percentage of visitors to the WaSP website use browsers other than the Web’s most numerically popular browser, for professional or political reasons. Others use a smaller browser for accessibility reasons, or because it’s familiar, or because they prefer the user interface, or just because they like it. Choice and personal preference is the heart of an open Web.

But the COI believes that it’s legitimate to suggest that visitors to government websites should change their preference, working methods and computer setup because of its testing policy. The example browser support policy statement, that they advise publishing on an accessiiblity or help page, says (my emphasis):

[Website name] has been tested on a wide range of browsers:

[List supported browsers here]

We advise you to upgrade your browser version as far as your computer allows and if possible to one of those listed above. However, the following browsers should also provide access to all of the content and navigation on the site:

[List semi-supported browsers here]

In short, in order to save costs on testing, the COI is advocates browser upgrade. By encouraging people to move from minority browsers to majority browsers, it works against the minority browsers ever increasing their market share in that sector, and the public sector is a big sector. It also sets a terrible example to organisations outside the public sector who are only just being cajoled out of the “build for IE” mindset.

Swimming against the tide

In the introduction to a document that wishes to help webmasters save money by limiting the range of browsers used for testing, it says that "how to code for browser compatibility" is "out of scope". This is bizarre; how can the single most effective way to reduce the cost of development be considered out of scope? This goes completely against the recent rise of the standards movement led by the WaSP, which was founded in 1998 to fight for standards that reduce the cost and complexity of development.

The guidelines note

These guidelines do not advocate specific development methodologies, for example graceful degradation or progressive enhancement. However, it is widely accepted that sites conforming to open web standards such as XHTML and CSS are more likely to work well across a wide range of browsers.

This is entirely back-to-front. The guidelines should be advocating a specific development methodology: they should recommend designing to Web Standards. Costs will be driven down, even if testing is performed across more browsers, because there will be fewer inconsistencies and less recoding to fix inconsistencies.

The COI should also advocate publishing information in open formats, such as HTML, and practice what it preaches.

Respond to the consultation

All UK and European developers have the right to respond to this consultation, which closes October 17, 2008. We hope that you will do so, because UK government websites have a lamentable track record, and we invite you to post your response on your site and link to it in the comments below, to inspire others.

Though you may not have heard of the Accessibility Institute, you have been influenced by its work. Its late founder, Dr. John Slatin, was the former co-chair of the Web Content Accessibility Guidelines 2.0 (WCAG2), and was an influential mentor to many of the web standards evangelists, including myself and current WaSP group manager, Glenda Sims. If you’ve ever attended SXSW, you know Austin has one of the most vibrant web accessibility communities in the world, thanks to the hard work of Knowbility and the University of Texas Accessibility Institute. The knowledge shared by these groups has influenced web and software developers worldwide, resulting in a more accessible web used and enjoyed by all of us, disabled or not.

The importance of accessibility research and development was echoed this week by retailer Target’s decision to settle its web accessibility discrimination lawsuit by the National Federation of the Blind (NFB). The story was covered in many US and international news outlets, and the the outcome of the case is a timely wake up call to the business world that good design is accessible, universal design.

The Accessibility Institute’s influence for the greater good cannot be overstated. The decision to close it on the eve of the universal design revolution is a poor choice by the UT Administration. If you agree, please sign the petition to keep accessibility research and development alive and well.

http://www.petitionspot.com/petitions/SavetheInstitute

Update: This post has been translated into Polish.

Does the Americans with Disabilities Act apply to the web?

Sadly, accessibility’s ultimate cliffhanger once again reaches an awkward denouement, leaving us deflated, and looking at yet another boring sequel. The National Federation of the Blind v. Target lawsuit, which promised to be a landmark case in determining the applicability of the ADA, was settled on Wednesday. The key provisions of the settlement have Target paying $6 million in damages to the members of the class action (which consists of legally blind people who have been denied Target’s online services), and agreeing to remove accessibility barriers to blind users by February of 2009.

As with most settlements, however, Target admits no wrongdoing, and so the ADA’s applicability to the web remains fuzzy. (Especially to a non-lawyer such as myself; please don’t consider this as anything like legal advice.) The legal ramifications of this case may not be as clear-cut as some of us would have liked, but it’d be hard to argue that after this decision people with disabilities are in any way on shakier legal ground.

One twist in this case was the application of two California laws: the Disabled Persons Act and the Unruh Civil Rights Act. Both of these offer protections over and above those of the ADA, for California citizens, such as the named plaintiff, Bruce Sexton. Even if we ignore the ADA for a moment, this means that sites who do business in California could be liable under these laws for denying access.

Whatever the legal ramifications may be, those of us who advocate accessibility don’t want to make this into a series of legal battles. There are no winners there. (Okay, besides the lawyers.) We want people to realize that engaging with people with disabilities well before the threat of legal action arises is always the best approach. When a company stalls and takes a case to court, delays, public relations nightmares, and skyrocketing costs are all that happens. In this case, Target will pay out well over $6 million in damages, when one-tenth–maybe even a hundredth–of that amount could have paid a dream team of accessibility-savvy designers ready to solve the actual issues at hand.

The question that’s on our minds today–whether ADA applies or not–ultimately doesn’t make much difference. In fact, it’s a major distraction from the heart of the matter. People of all kinds want to participate in all the activities the web has to offer. And many disability advocacy groups are reaching out to site admins to raise awareness of the barriers they face. The best thing you can do is to prepare yourself and your site with a little education and some fine tuning. When you’re in a lawyer’s office talking about the ADA, or any other accessibility statute, chances are you’ve already missed out on the most important part of the conversation. And that’s going to cost you, whether you win or lose.

Update: This post has been translated into Polish.

In order to help educational institutions to identify and include material for these competencies, we are creating a set of foundation courses that can be readily adapted into an existing program at a college, school or university.

The framework will include a collection of tools:

• Course overviews
• Recommended course dependencies indicating what students will need to know before beginning each course
• Learning competencies describing what students must master in order to receive a passing grade
• Ideas for assignments and test questions that allow educators to measure a student’s mastery of each competency
• Recommended textbooks and readings, including articles from the Opera Web Standards Curriculum and other reputable sources
• A list of helpful resources, tools, and utilities specific to each course that will help both educators and students

Why is it called a framework? Given the velocity at which Web technology unravels, we recognize that required skill sets can change rapidly, and that the best way to keep this material useful is for the education community to enrich it with their expertise and experiences. In this way, the WaSP Curriculum Framework will be a “living curriculum” that we hope would be a knowledge base of required skills.

The framework will include guidelines to help educators around the world develop assignments and learning modules that address issues specific to their classrooms. These independently developed teaching materials can then be submitted back to the WaSP Curriculum Framework for review and potential inclusion in the project.

We are also actively working on connecting with other organizations and institutions to create as comprehensive a curriculum framework as possible.

We encourage everyone to get involved by contributing content to the framework upon its initial release in March 2009. In the meantime, join the WaSP Edu Facebook group to share your insights and participate in the discussion. Of course, there is always the WaSP EduTF public discussion list if Facebook isn’t your thing.

The survey was launched in the second quarter of 2007 and educators in both secondary and higher education were targeted. The survey ran for approximately three months. After many starts, stops, and delays (which included the recruitment of Industry and Educational Professionals to the Task Force), the results of the survey are available.

When Educational Professionals were asked what their biggest challenges to implementing a curriculum for best practices, including accessibility and Web standards, they indicated the lack of appropriate materials and reference materials. With both the Opera Web Standards Curriculum and the Web Standards Project Curriculum Framework in active development, this should no longer be an issue.

Later this year, the Education Task Force plans to run another iteration of the survey. We hope to have multiple translations of the survey at that time. If you are interested in translating the survey into another language, please contact the Education Task Force.

I’ll be honest: over the last 10 days, I’ve learned more about the Acid 2 Test than I ever wanted to know. If you want to do the same, you might start with Acid 2: The Guided Tour.

The short version is that part of Acid 2 is a test for the way a browser handles an <object> element when the data attribute references a URL that returns an HTTP status code of 404. A number of caching rules, mod_rewrite rules and redirects all collided to create a problem with our 404. The cached version of our 404 page was returning an HTTP status of 200. As you might expect, this basically makes the test useless.

Acid 2 was broken. Now it is not. Carry on.

Two years ago, the BSI was sponsored by the Disability Rights Commission to write a Publicly Available Specification (PAS) called PAS 78: Guide to Commissioning Accessible Websites. Publicly Available Specifications are written quickly and “expire” after two years, but because of the popularity of PAS 78, the BSI have decided to update it to become a full British Standard.

We’ve just started work on the draft, which doesn’t yet have a title, although our working title is “encouraging the development of fantastic user experiences for disabled people online”.

Consequently, it’s too early to say what will be in BS8878, which will be released next spring. I can say that it will not tread on the toes of whichever version of WCAG is live then, as it’s a document to help site owners rather than developers. Like PAS 78, it will encourage adherence to current web standards.

Neither can I say who else is on the committee, except that it’s chaired by Julie Howell, and there are representatives from all over industry—broadcasting, banking, legal, education and (crucially) representatives of disability groups, including groups working with those with cognitive disabilities.

Patrick and I gratefully acknowledge our employers, Opera Software and the University of Salford, who are supporting us by paying our travel expenses and giving us time off to attend meetings and write the drafts. They have nothing material to gain by supporting us, and are exercising no editorial control, but are helping to make disabled people’s experiences of the web better.

As a procedural footnote, now that Derek Featherstone has moved role within WaSP to be Group Lead, I’m working with Patrick to be co-lead of the Accessiblity Task Force. Our main projects will be the British Standard, continuing to work with the microformats community testing various date-time patterns with screenreaders, and monitoring the developments in HTML5.

Chris states:

We think it will be useful to anyone who wants to learn or teach client-side web design/development “the right way”, including students and teachers at schools or universities, trainers and employees inside companies, etc. It already has support from several universities and large companies, including Yahoo!

Translations and packaging of the curriculum as PDFs is on the to-do list.