Working together for standards The Web Standards Project

Buzz Archives

Recent Buzz

Acid3 receptions and misconceptions and do we have a winner?

By Lars Gunther | October 2nd, 2008

Acid3 progress and what it really means.

Acid3 is probably the most visible thing that WaSP has done the last year. When Google Chrome was launched almost every review included our little test as an indicator of standards support. It is often mentioned in blogs and articles. Now the Surfin Safari blog has announced that the team behind Webkit considers that they have passed the test in every aspect. And no doubt this is a great achievement. Congratulations to the Webkit team, but even more we would like to congratulate the average web user - who in a few years thanks to our test we hope will get a better experience!

What exactly does it mean to pass the Acid3 test?

There has been some confusion about the test and its importance. Some people have been saying things like ”my browser does not pass the test and I have no problems using it”. Quite a few other people seem to think that Webkit and Gogi (Opera’s internal build) passed the test already in March – despite the fact that neither team has made this claim.

To answer these misconceptions we need to address the issue of what exactly is being tested and how. The main part of test is automated through JavaScript, a sort of test harness that runs 100 subtests. Getting a score of 100 is not the same as passing Acid3 – a common misconception, or perhaps an oversimplification.

Many subtests are high on a developer’s wish list: Full CSS 3 selectors support, media queries, SVG fonts. Admittedly a few others test edge cases and more esoteric features – but the test was supposed to be a significant challenge!

The second part is a rendering test. Some of the scripted subtests produce results that affect the rendering, but there are also rendering issues that come in addition to these. Some of them are high on many designers’ wish list: Text shadow, downloadable fonts, and display: inline-block.

The third test is the so called “smoothness” criterion. It is basically a speed test. No subtest may take too long – and especially subtest 26 is challenging. Compared to Slickspeed, Sun Spider, the V8 test suite or Dromaeo Acid3 is not so thorough. It will give some indication of a browsers speed, though.

This is exactly as planned. Acid3 was not meant to be the one and only indication of a browser’s performance. In fact many other test suites are far more important. (We provide links to some of them below.)

Testing is really important. Without tests that check how well a certain browser follows standards, i.e. applies mark up and displays the result correctly, we can never guarantee an open, fully interoperable web.

A highly visible test like Acid 3 hopefully helps to promote such interoperability. One can also hope that all the other tests will receive the attention they deserve. Writing them is not a glamorous task, but highly essential.

Apart from improving its support for CSS in its browser, Microsoft has contributed 2524 test cases to the CSS 2.1 test suite. For that they deserve credit!

We all know that Internet Explorer currently lag a bit behind the other browsers in standards compliance. Indeed they are last of the big ones to pass Acid2 and they fail Acid3 more than any other browser. But can we declare Webkit as the best rendering engine now that they pass it?

Of course not. Since Acid3 is only one indicator of many. Webkit’s achievement is great – and there are many other really exciting things they are pioneering, like CSS transitions and transformations. And with Squirrelfish Extreme JavaScript performance looks really exciting as well.

In other regards Opera is a clear leader. It is the only browser that supports more than 90 % of the SVG test suite. It is the only browser that implements Web Forms 2.0, currently being merged into HTML 5. They supported media queries and SMIL long before Acid3 came out.

Gecko (with Spidermonkey) is no longer an underdog. Besides the fun of meeting the technical challenge it is not hard to guess that the Webkit team rushed to pass Acid3 also for marketing reasons – that they perhaps need a bit more than Mozilla. Mozilla concentrated on releasing Firefox 3 before Acid 3 received any real attention. Now that they are working on it they are impressive in another way, compared to Webkit. Looking at the discussions for bug 410460 and its related bugs, it is clear that any improvement must be rock solid. Work often continues even when a particular feature is good enough for Acid3.

In fact, there is actually one open issue still in Acid 3 that might temporarily cause Webkit to become incompliant again. http://lists.w3.org/Archives/Public/www-style/2008Sep/0218.html. I rest assured that a fix probably already is being made, though.

Perhaps one can compare this to a race where you are supposed to run a distance, with a bucket of water. One competitor crosses the finishing line first, the other, on the other hand, has not lost a single drop from his bucket. Both have done great. (By the way, internal builds of Firefox get a score of 97 now, and downloadable fonts work on Windows and Mac.)

In the end the winner is neither Webkit, Opera, Mozilla nor Microsoft, but developers who get more powerful features to work with and more consistency between browsers. And that means that in the long run they are able to focus on user experience, not browser shortcomings. This means that the true winner of Acid3 is anybody who surfs the web.

Some other test suites for your review:

• Cascading Style Sheets level 2 revision 1
• Selectors
• CSS Color Module
• Collection of DOM tests
• SVG
• Web Forms
• Canvas

Filed in Acid3, Browsers, Bugs | Comments (1)

More Buzz articles

Title	Author
UK government draft browser guidance is daft browser guidance	Bruce Lawson
Call-to-action: Save the UT Accessibility Institute	James Craig
What the Target settlement should mean to you	Matt May
Announcing the WaSP Curriculum Framework	Steph Troeth