5 August 2010 – This test has now been superseded by my iPhone 4 Speed Test once again comparing Telecom XT with Vodafone 3G.

DISCLAIMER

• This is NOT a test of which network is “best” or “fastest” out of Telecom XT and Vodafone 3G.
• This is an iPhone specific test, and should only be taken as such. It is a comparison of speeds using an iPhone 3G device, on one day in March (16 March 2010), in Dunedin.
• The test results should not be extrapolated beyond the scope of the test. However the results should be of interest to iPhone users in Dunedin, and to iPhone users generally. Your milage may vary if you use a different handset/device, or if you live in another city/town.
• I am not a telecommunications expert. I am simply an iPhone user with an interest in testing the claims of Telecom XT being “faster in more places”. I repeat – this is a hobby. I welcome any feedback about these results and will endeavour to correct any errors which have been made if they are pointed out to me.

Geekzoners may recall that I performed a (crude) test of iPhone mobile data speeds at several locations around Dunedin last August which concluded that the iPhone 3G was “significantly faster in more Dunedin places on Telecom XT (compared to Vodafone 3G)”.

There has been various work done in recent times on the Vodafone network in Dunedin, which the company believes should improve iPhone data speeds on the network around Dunedin. Therefore I set out to perform a repeat test to compare speeds on the XT and Vodafone 3G networks at various locations around the Dunedin CBD.

The SpeedTest.net iPhone app which was used in last year’s test has been panned as inaccurate, so after consultation with sbiddle from Geekzone, it was decided to use the following methodology for this repeat test:

Steve suggested that I use a tethered iPhone to download a 20Mb+ file which is hosted on an Akamai server (such as Microsoft updates). I was able to find 2 files which should be hosted from Akamai servers – IE8 for Windows XP (16.1Mb) and Apple’s Security Update 2010-001 for Snow Leopard (21.9Mb).

In an ideal world I would have been able to download the 21.9Mb file multiple times at various locations on both XT and Vodafone 3G networks, but unfortunately due to data restrictions I needed to limit the total data used to roughly 100Mb to avoid heavy additional data charges. This is why I chose to use a combination of the 21.9Mb and 16.1Mb files, and from the results that I got for the testing, I do not believe that repeating the download multiple times would have affected the final results dramatically.

I used an iPhone 3G (same device as used in last year’s test) running firmware 3.1.3 (7E18). The XT carrier bundle from www.iphonewzealand.co.nz was installed to the iPhone. The iPhone was tethered by USB to an Intel MacBook running 10.6.2, with all software updates applied. Safari 4.0.5 was used to download the files, with the Safari cache being deleted before and after each download, and Safari being reset to “factory settings” before and after each download was made. For the purpose of comparison to last year’s tests, I also ran the SpeedTest.net iPhone app before each download, for what that was worth.

Tests were conducted between 12pm and 3pm on Tuesday 16 March at four locations scattered across the Dunedin central city. After discussion with an anonymous blog commenter “Sean”, I decided to go back to these four locations later that evening to perform “ping” testing (using OSX’s Terminal ie. equivalent of Window’s Command Prompt). Even though these ping tests were done at a different time of the day (around 11pm on the same night), they should hopefully be able to be compared between networks as they were performed at the same time as each other (XT vs. Vodafone 3G).

First of all, here are the daytime download tests that were performed, along with the SpeedTest.net app results:

Location One – Nova Cafe, Octagon, Dunedin

Sitting at an outside table at Nova (next to the Art Gallery), sipping a delicious flat white, and waiting for Snow Leopard Security Update 2010-001 (21.9Mb, 22,960,515 bytes).

Telecom XT – Time taken 1:13 (73 seconds) = 2826Kbit/sec

Vodafone 3G – Time taken 9:09 (549 seconds) = 376Kbit/sec

Location Two – Dunedin Chinese Gardens

Sitting in the central pagoda as per last year’s test, waiting for IE8 for Windows XP to download from Microsoft Downloads (16.1Mb, 16,883,056 bytes).

Vodafone 3G – Time taken 6:48 (408 seconds) = 372Kbit/sec

Telecom XT – Time taken 0:51 (51 seconds) = 2974Kbit/sec

Location Three – Otago University Campus

The weather packed it in so I wasn’t able to do this from the Campus lawn like last year. I sat inside the Information Services Building in the central foyer where all the students gather and mingle, looking very out of place as I downloaded the Snow Leopard Security Update once more (21.9Mb, 22,960,515 bytes).

Telecom XT – Time taken 9:18 (558 seconds) = 370Kbit/sec (this file download seemed to stall at around the 6 minute mark – about 19.2Mb complete – and then picked up again slowly)

Vodafone 3G – Time taken 8:53 (533 seconds) = 387Kbit/sec

Location Four – Edgar Centre

A return to where it all started. The first test location in last year’s test is the last location to be tested this time around. I downloaded IE8 for Windows XP once again from Microsoft Downloads (16.1Mb, 16,883,056 bytes)

Vodafone 3G – Time taken 7:14 (434 seconds) = 349Kbit/sec

Telecom XT – Time taken 2:51 (171 seconds) = 887Kbit/sec

Ping results

Here are the results of pings made to www.stuff.co.nz, www.nzherald.co.nz, and www.nbr.co.nz. I set OSX Terminal to ping 10 times, and discarded any grossly high ping times or “time-outs”. What is listed below is the range of the ping times that were left over.

Otago University Campus

www.stuff.co.nz XT 141-155ms; Vodafone 3G 318-479ms

www.nzherald.co.nz XT 120-143ms; Vodafone 3G 210-527ms

www.nbr.co.nz XT 120-137ms; Vodafone 3G 438-558ms

Nova Cafe, Octagon

www.stuff.co.nz Vodafone 3G 219-537ms; XT 170-185ms

www.nzherald.co.nz Vodafone 3G 178-318ms; XT 155-170ms

www.nbr.co.nz Vodafone 3G 196-435ms; XT 123-164ms

Dunedin Chinese Gardens

www.stuff.co.nz XT 131-162ms; Vodafone 3G 213-378ms

www.nzherald.co.nz XT 124-128ms; Vodafone 3G 196-316ms

www.nbr.co.nz XT 135-138ms; Vodafone 3G 197-376ms

Edgar Centre

www.stuff.co.nz Vodafone 3G 223-529ms; XT 131-355ms

www.nzherald.co.nz Vodafone 3G 198-338ms; XT 120-125ms

www.nbr.co.nz Vodafone 3G 179-357ms; XT 118-137ms

Summary of Results

In looking back and comparing the SpeedTest.net results from August 2009 and today, it would appear that Vodafone 3G has made speed improvements. At the same time, Telecom XT results have also improved compared to last year. However a quick comparison of the results using the SpeedTest.net app and those obtained by timing the download of a large file suggests that the SpeedTest app is inaccurate, as claimed by many critics. Unfortunately this is the only data I have to go on when comparing Vodafone 3G speeds in March 2010 and August 2009, so no valid conclusions can be drawn about whether there has been any change to speeds on the Vodafone 3G network when accessed using an iPhone in Dunedin.

When we look at the timed downloads performed today, we can see that XT appears to have a clear speed advantage – sometimes dramatically faster. The one exception to this out of of the four locations tested today was the University Campus, and I am unsure why this would be the case. On campus, XT speeds were much slower than in the other three Dunedin locations visited today, and this was the only location in which a download was faster on Vodafone 3G (albiet marginally, and only one a single test – I would in fact call it a dead heat).

Finally, the ping results of the SpeedTest.net app have always been a bone of contention, so my thanks to Sean for supplying the simple but useful information on how to measure ping times using OSX Terminal. Once again it would appear that XT has a clear edge over Vodafone 3G, a difference which the SpeedTest.net app does not seem to be able to discern reliably.

Final Thoughts

As emphasised in opening, these results should not be extrapolated beyond what the scope of the test is able to demonstrate. These results simply suggest that in Dunedin, as at March 2010, iPhone users would appear to enjoy speed advantages in using the XT network rather than the Vodafone 3G network. Reasons for this are beyond the scope of this article, but include factors such as network congestion. Vodafone might argue that well documented XT problems are pushing more users onto their network, but ultimately it is the responsibility of a mobile company to ensure adequate capacity for their users. Most importantly though, users looking for speed may well prefer to be using a network that others are rushing to get away from, if only to enjoy the benefits of less congestion.

Finally a quick note about iPhone operating frequencies. Vodafone operates on the 2100Mhz band for urban centres such as Dunedin, but even if they were to roll out their 900Mhz band to Dunedin city, iPhone users would still not be able to take advantage of this because the iPhone operates only on 2100Mhz and 850Mhz frequencies (850Mhz being the freqency that XT operates on). Therefore, until/unless Apple releases an iPhone which also operates on the 900Mhz band, and Vodafone chooses to roll out 900Mhz into cities, users will still enjoy benefits such as better in-building signal coverage on XT (850/900Mhz frequencies are less prone to degradation when entering buildings). This was not so important in today’s test because all tests were performed either outdoors or near doors and windows, if in-building. I did not retest in the basement foodcourt of the Meridian Mall where I expect Vodafone 3G (2100Mhz) coverage to be non-existent like last year.

In summary, as of the current time, iPhone users in Dunedin interested in the fastest possible speeds would be hard pressed to look past Telecom XT. When it works, it would appear that XT is still faster in more Dunedin places for the iPhone 3G.

This entry was posted on Wednesday, March 17th, 2010 at 12:40 am and is filed under Gaming. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.