Use of app instead of mobile Safari makes browser speed test flawed?

Yesterday we reported on a test that tracked the speed of the browser on the Apple iPhone 4 against that of the Android flavored Nexus S. The test was conducted 45,000 times on 1,000 different sites and allegedly showed that the Android browser was 52% faster than the iOS 4 browser.

Many were surprised by the results, especially in light of JavaScript improvements recently made to mobile Safari. According to those who studied the contest, the reason why the JavaScript improvements seemed to have no effect during the test was that mobile Safari was not used in the competition. Instead of using the actual browser, the testing was done using Apple's own proprietary app, UIWebView, that bypassed the improvements made to the iOS browser. Apple had developed the UIWebView to allow web browser functions to be used within an app but the app does not include the improvements made to the actual browser itself.

So now it seems that there is a legitimate challenge to the results. Blaze can re-do the tests using the actual Safari browser, or someone with plenty of time on his hands can develop their own test for measuring which browser is faster. In the meantime, the record book that shows the Android browser 52% faster than mobile Safari will have to include an asterisk next to the results for now.