December 2007
www.veritest.com • info@veritest.com
Built In EV-DO Comparison for Dell,
Lenovo and HP
Test report prepared under contract from Dell Inc.
Executive Summary
Dell Inc. commissioned VeriTest, the
testing
service
of
Lionbridge
Technologies Inc., to conduct a
competitive analysis comparing the
performance of built-in EV-DO (Rev-A)
mobile broadband modules using
Verizon Wireless BroadbandAccess.
VeriTest conducted a series of tests to
find which notebook had the best
performance in terms of overall data
throughput speed and performance. The
notebooks tested in this study include
the following Business Class models.
•
Key Findings
Average Download Speed
Business Class
o Dell Latitude D630
o HP 6910p
o Lenovo T61
The Dell Latitude’s EV-DO Rev-A
module is manufactured by Novatel. The
HP and Lenovo EV-DO Rev-A modules
Chart 1. Average Kbps for FTP and Web Downloading.
are made by Sierra Wireless. Each of
the notebooks tested had comparable
On average, the Dell D630 download speeds were 36%
configurations (e.g., processor speed,
faster than the HP 6910p and 88% faster than the
memory, and system OS). All notebooks
Lenovo ThinkPad T61.
included Intel Core 2 Duo processors as
well as 2 GB (2 x 1GB) of random
access memory (RAM). For a list of complete specifications for each notebook, please refer to
Appendix A.
In order to best replicate the end user experience, each notebook’s operating system type was
installed according to what is used the most from a business perspective. Business class models
included Microsoft Windows XP Professional with Service Pack 2.
Integrated EV-DO Comparison for Dell, Lenovo and HP
1
Tests were conducted in the following US cities over a 29 day period using Verizon Wireless
BroadbandAccess.
•
•
•
San Diego, California
New York, New York
Chicago, Illinois
Three test locations were used in each city. All notebooks were tested on the same day for each
location. Each location was tested on separate days to account for network variability. The final
result for each test was calculated by averaging the encountered results from multiple test
iterations for each location. Six (6) to ten (10) iterations were conducted for each test.
The following tests were conducted:
A. Web Downloading Test - to measure throughput when downloading 1MB and 2MB
web files via HTTP protocol.
B. FTP Send and Receive Test - to measure throughput when transferring 1MB and
2MB files to and from web servers via FTP protocol
Upon completion, the study findings show that on average, the Dell Latitude D630 provided 36%
more throughput than the HP 6910p and 88% more throughput than the Lenovo ThinkPad T61 for
download speed.
The Dell Latitude D630 had an average throughput speed averaged across the FTP and Web
Downloading tests of 1023 Kbps. The HP 6910p had an average throughput of 753 Kbps and the
Lenovo T61 had an average throughput of 545 Kbps.
Please refer to the Test Methodology section of this report for complete details of how testing was
conducted. All units were tested using default factory settings. Lionbridge confirmed that all
notebook computers were new and unopened prior to testing and ensured all tests were
completed with impartiality.
Test Results
The following section details the results from testing.
Web Download Throughput
Goal: Benchmark throughput when performing web downloads via HTTP using a low traffic server
hosted by a local internet service provider.
Summary of Results: The Dell Latitude D630 was on average 43% faster than the HP 6910p and
74% faster than the Lenovo T61 when downloading a 1MB zip file from the web. The Dell D630
was on average 27% faster than the HP 6910p and 80% faster than the Lenovo T61 when
downloading a 2MB zip file from the web.
Integrated EV-DO Comparison for Dell, Lenovo and HP
2
Web Download Comparison
1100
1000
Kilobits Per Second
900
800
700
600
Dell
HP
Lenovo
500
400
300
200
100
0
1MB
2MB
Chart 2: Average Kbps for all test run averages for the Web Downloading Test.
Notebook
DELL D630
HP 6910p
LENOVO T61
1MB(Kbps)
2MB(Kbps)
1012.66
707.29
582.48
989.18
780.47
550.79
Table A. Average Throughput of Web Downloading Test
FTP Test
Goal: Benchmark throughput when using FTP to transfer files.
Summary of Results: The Dell D630 was on average 43% faster than the HP 6910p and 94%
faster than the Lenovo T61 when downloading a 1 MB zip file via FTP. The Dell D630 was on
average 32% faster than the HP 6910p and 106% faster than the Lenovo T61 when downloading
a 2 MB zip file via FTP.
Upload results displayed minimal variance in performance. The Dell D630 was on average 10%
faster than the HP 6910p and 5% faster than the Lenovo T61 when uploading a 1 MB zip file via
FTP. The Dell D630 was on average 13% faster than the HP 6910p and 3% faster than the
Lenovo T61 when uploading a 2 MB zip file via FTP.
Integrated EV-DO Comparison for Dell, Lenovo and HP
3
FTP Throughput Comparison
1100
1000
Kilobits Per Second
900
800
700
600
Dell
HP
Lenovo
500
400
300
200
100
0
1MB snd
2MB snd
1MB rcv
2MB rcv
Chart 3: Average throughput for all test run averages for the FTP Test.
Notebook
DELL D630
HP 6910p
LENOVO T61
1MB(Kbps)
2MB(Kbps)
1066.27
746.47
549.36
1024.86
776.21
497.78
Table B. Average Download Speed of FTP Test
Notebook
DELL D630
HP 6910p
LENOVO T61
1MB(Kbps)
2MB(Kbps)
631.13
571.36
601.45
623.39
549.68
602.39
Table C. Average Upload Speed of FTP Test
Test Methodology
Dell Inc. commissioned VeriTest to conduct a competitive analysis comparing performance of
built-in EV-DO (Rev A) mobile broadband modules. VeriTest conducted a series of throughput
tests to measure which notebook had the highest performance.
Notebook Configurations
Test conditions were designed for the notebooks to be as close to an out-of-the-box configuration
as possible, but some third party software, such as firewalls and virus protection, had to be
disabled. This ensured that some of this software did not affect download speed.
Integrated EV-DO Comparison for Dell, Lenovo and HP
4
The following software was required for testing which needed to be installed:
o
Wget for DOS (an application distributed under the GNU license) for the non-interactive
Web Downloading Test.
Test Locations
The test locations included areas with low, marginal, and strong coverage. Locations used were
deemed typical for users of built-in mobile broadband and included hotels and coffee shops.
Some locations provided unreliable connections resulting in data with such large standard
deviations as to be statistically useless. Data from these locations was not used in this report.
Several test runs would take 8 hours or more requiring the availability of AC power to keep the
notebooks charged, however, notebooks tested were not connected to AC power during testing.
Test Run Conditions
To keep the testing notebooks from possessing any advantages over each other based on
location, orientation or configuration, the test design called for the following conditions to be met
before testing could begin:
o
o
o
o
o
o
Each computer must be placed in the same point of origin and facing the same direction
Each LCD monitor must be tilted back at approximately 110 degrees
Each computer must be running on battery only, to simulate a typical user experience
Power settings must be set to Always On
All brightness settings must be maximized
Device radios are to be powered off unless when under test
There are many factors affecting actual throughput analysis (e.g. structures, latency, and network
traffic). Because these factors were out of our control and testing was not done in a controlled
lab the results may not be reproducible. The tests were designed to be objective by ensuring that
each notebook was affected by the same or similar environmental factors.
Test Run Details
After test setup was complete and all test run conditions were met, all notebook tests were
repeated from six (6) to ten (10) iterations. A completed iteration is defined as a single test
executed once on all three notebooks. Interleaving the tests from notebook to notebook helped
ensure that any network traffic or signal fluctuations would affect all notebooks equally minimizing
factors that can cause network variance such as network traffic. Only one computer was
connected to the Verizon Wireless BroadbandAccess service at a time. Only one notebook
module was powered on at a time. The test run was complete when all iterations were complete
for all tests.
Calculating Average
Once a test run was complete, the recorded data points for each test location were averaged.
These results can be found in Appendix C: Detailed Results.
Test Details
The following describes each test in detail and includes information on test development,
description of execution and data gathering.
Integrated EV-DO Comparison for Dell, Lenovo and HP
5
A. Web Downloading Test
Test Description
Wget for DOS was used to download a 1 megabyte and 2 megabyte zip file from a web
site. Zip files were used to eliminate the possibility of file transfer compression affecting
test results.
Test Development
The Wget application was installed on each notebook. A batch file executed the Wget
command with the logging option to download the two zip files and log the results. The
zip files were located on a low traffic web server from a local ISP.
Test Execution and Data Gathering
Double clicking the batch file would run the test. A log file was created on the desktop
upon completion. The start and stop time data as well as the Kb/s from each download
were recorded into an Excel spreadsheet to be averaged.
B. FTP Test
Test Description
FTP is one of the most common methods for transferring large files over the internet. To
test throughput speeds via FTP, Microsoft XP’s integrated FTP client was used to
download and upload a 1 megabyte zip file and a 2 megabyte zip file. Zip files were used
to eliminate the possibility of file transfer compression affecting test results.
Test Development
A batch file was written to log onto an FTP server and run put/get commands which
would send and receive the files via FTP then log the results to a log file on the Desktop.
The FTP server used for this test was provided by a low traffic local ISP.
Test Execution and Data Gathering
Double clicking the batch file would run the test. Upon completion a log file was created
on the desktop. The time in seconds it took for each download and upload to complete as
well as the Kb/s were copied into an Excel document in order to be averaged.
Integrated EV-DO Comparison for Dell, Lenovo and HP
6
Appendix A: System Configurations
System Information Dell Latitude D630
Processor/speed
Intel® Core™ 2 Duo 2.00GHz
System Ram/type/number of slots
2 GB (2x1GB)
Mother board manufacturer
Dell Latitude D630
Built-in EV-DO Rev A Module Manufacturer Novatel
OEM OS
Win XP + SP2
Table D. System configuration for Dell Latitude D630
System Information HP Compaq 6910p
Processor/speed
Intel® Core™ 2 Duo 1.80GHz
System Ram/type/number of slots
2 GB (2x1GB)
Mother board manufacturer
HP Compaq 6910p
Built-in EV-DO Rev A Module Manufacturer Sierra Wireless
OEM OS
Win XP + SP2
Table E. System configuration for HP Compaq 6910p
System Information Lenovo ThinkPad T61
Processor/speed
Intel® Core™ 2 Duo 2.00GHz
System Ram/type/number of slots
2 GB (2x1GB)
Mother board manufacturer
Lenovo ThinkPad T61
Built-in EV-DO Rev A Module Manufacturer Sierra Wireless
OEM OS
Win XP + SP2
Table F. System configuration for Lenovo ThinkPad T61
Appendix B: Testing Software Information
Software
Version
Associated Test
Microsoft File
Transfer Program
5.1.2600.2180
FTP Test
Wget
1.8.2
Web
Downloading
Test
Description
This is an application integrated into
Windows XP and was used to test
throughput via FTP protocol.
This is an application which is
distributed under the GNU General
Public license and was used to
retrieve files (non interactive) via
HTTP protocol.
Table G. Testing software information
Integrated EV-DO Comparison for Dell, Lenovo and HP
7
Appendix C: Detailed Results
The following section details the results from each test run. All numbers contained herein are
averages from all the test iterations executed during that run as described in the Test
Methodology section. Please refer to that section for more details.
San Diego, CA – Test Location 1:
Testing Location: Tuesday, November 6, 2007 – Starbucks (Mission Beach) – San Diego, CA
Address: 4343 Mission Blvd # 104, San Diego, CA 92109
Details: Testing took place inside on the southwest side of the lobby area. This area included a
nearby power outlet on the wall. The nearest Verizon Wireless tower is approximately 1.5 miles
north from this location. This location was observed as a low to moderate signal reception area.
HTTP Download
DELL
HP
LENOVO
AVG
STDEV
AVG
STDEV
AVG
STDEV
1MB Down
344.46
68.72
263.95
91.31
435.28
54.75
2MB Down
326.62
70.10
278.00
75.46
472.73
76.42
FTP Test
1MB Up
347.84
89.81
420.50
209.87
428.27
53.72
2MB Up
385.13
97.99
436.98
167.03
397.18
119.03
1MB Down
323.84
81.49
376.30
148.26
346.48
118.58
2MB Down
330.62
94.24
306.24
105.27
285.73
77.71
San Diego, CA – Test Location 2:
Testing Location: Thursday, November 8, 2007 – Archstone La Jolla Colony – San Diego, CA
Address: 7205 Charmant Dr. San Diego, CA 92122
Details: Testing took place in a covered outdoor area with access to AC power. This is located in
the southeast area of the complex. This location was deemed as a moderate to good signal
coverage area.
HTTP Download
DELL
HP
LENOVO
AVG
STDEV
AVG
STDEV
AVG
STDEV
1MB Down
1427.52
332.10
976.45
288.86
136.87
136.87
2MB Down
1333.84
324.52
1143.38
233.16
883.42
86.45
FTP Test
1MB Up
658.61
44.76
578.63
53.61
582.68
26.24
Integrated EV-DO Comparison for Dell, Lenovo and HP
2MB Up
653.16
30.67
642.08
22.69
629.12
48.97
1MB Down
1771.57
322.88
1346.77
611.42
975.55
136.00
2MB Down
1667.83
349.09
1514.65
450.60
882.99
107.59
8
San Diego, CA – Test Location 3:
Testing Location: Saturday, November 10, 2007 – Westfield Shopping Town – San Diego, CA
Address: 1640 Camino Del Rio N, San Diego, CA 92108
Details: Testing took place in the Westfield Mall in Mission Valley. This location is near the
National University Marine Corps Recruit Depot and DeVry University. There is an in-building
Verizon Wireless tower at this location. Notebooks were tested in the food court area of the mall
with nearby access to AC power.
HTTP Download
DELL
HP
LENOVO
AVG
STDEV
AVG
STDEV
AVG
STDEV
FTP Test
1MB Down
2MB Down
1MB Up
2MB Up
1MB Down
2MB Down
840.49
192.40
481.37
124.68
547.27
132.03
746.74
159.17
393.45
82.00
603.27
106.55
564.87
62.17
519.53
171.89
456.27
130.64
567.12
66.65
415.81
138.27
414.37
121.14
1214.65
642.26
374.65
112.55
393.81
88.02
1074.07
649.68
449.80
128.28
348.11
72.73
New York, NY – Test Location 1:
Testing Location: Wednesday, November 14, 2007 – Radisson Martinique Hotel – New York
nd
Address: 49 W.32 Street, New York, NY 10001
Details: Testing took place in the restaurant area of the hotel. This is on the first floor of the hotel
(street level). The Radisson Hotel is approximately ¼ mile from the nearest Verizon Wireless
tower, but there is a fair amount of structural interference as well as a high amount of network
traffic. This location was deemed as a good coverage area based upon preliminary throughput
tests.
HTTP Download
DELL
HP
LENOVO
AVG
STDEV
AVG
STDEV
AVG
STDEV
FTP Test
1MB Down
2MB Down
1MB Up
2MB Up
1MB Down
2MB Down
1458.48
158.09
665.04
203.25
441.43
48.67
1432.54
136.05
780.89
97.59
462.20
24.63
711.12
98.96
550.29
113.41
740.67
30.16
670.11
88.48
535.67
155.01
761.16
22.88
1262.25
93.93
767.00
186.36
487.35
38.10
1295.68
98.13
733.20
189.97
452.16
46.74
Integrated EV-DO Comparison for Dell, Lenovo and HP
9
New York, NY – Test Location 2:
Testing Location: Friday, November 16, 2007 – Staten Island Ferry Station – New York
Address: South Ferry – State Street & Whitehall Street, New York, NY
Details: Testing took place in the main entrance area. No nearby AC power was available. This
area is approximately 1/2 mile south of Wall Street as well as the Castle Clinton National
Monument. This location was observed to be a moderate to good signal coverage area.
HTTP Download
DELL
HP
LENOVO
AVG
STDEV
AVG
STDEV
AVG
STDEV
FTP Test
1MB Down
2MB Down
1MB Up
2MB Up
1MB Down
2MB Down
863.52
114.48
617.80
149.79
458.72
56.43
709.07
54.57
627.23
70.52
469.89
50.72
791.07
9.18
660.57
70.72
713.45
70.69
792.79
23.62
675.19
25.61
752.29
46.45
983.31
163.33
646.03
177.53
562.65
33.62
840.55
64.65
638.35
46.80
468.00
25.60
New York, NY – Test Location 3:
Testing Location: Sunday, November 18, 2007 – Starbucks – New York
th
Address: 378 6 Ave, New York, NY 10011
Details: Testing took place in the lobby area at the northwest corner of the building near the
entrance. This location is next to New York University and Washington Square Park. The nearest
Verizon Wireless tower is approximately 1 ¼ mile west of the location. This location was
observed as a low to moderate signal coverage area with average network traffic.
HTTP Download
DELL
HP
LENOVO
AVG
STDEV
AVG
STDEV
AVG
STDEV
FTP Test
1MB Down
2MB Down
1MB Up
2MB Up
1MB Down
2MB Down
1135.12
202.93
559.24
150.86
518.20
44.55
1031.90
146.48
607.28
91.21
510.27
45.61
416.32
66.21
301.09
149.78
302.61
106.79
426.20
69.70
316.08
140.30
324.88
94.66
1122.55
136.38
537.23
116.94
519.24
27.72
1101.01
110.92
588.31
225.21
482.77
31.53
Integrated EV-DO Comparison for Dell, Lenovo and HP
10
Chicago, IL – Test Location 1:
Testing Location: Thursday, November 29, 2007 – Starbucks – Chicago
Address: 15 S Prospect Ave, Park Ridge, IL 60068
Details: Testing took place in the lobby area of the building with close access to AC power. There
are approximately ten (10) schools within a ¼ mile radius of the location. The nearest Verizon
Wireless tower is approximately 200 yards east of this location. This location was observed as a
moderate to good coverage area.
HTTP Download
DELL
HP
LENOVO
AVG
STDEV
AVG
STDEV
AVG
STDEV
FTP Test
1MB Down
2MB Down
1MB Up
2MB Up
1MB Down
2MB Down
1135.12
79.94
1063.45
49.15
725.35
255.44
1164.27
41.93
1113.49
36.73
572.89
27.94
807.44
4.30
739.00
24.99
751.28
19.36
794.71
16.38
748.48
26.83
764.73
8.12
1152.08
45.08
1065.63
138.48
595.07
32.00
1172.11
45.79
1087.89
143.36
594.44
19.81
Chicago, IL – Test Location 2:
Testing Location: Saturday, December 1, 2007 – Radisson Hotel – Chicago
Address: 1450 East Touhy Avenue, Des Plaines, IL 60018
Details: Testing took place in the lobby area of the hotel in the east part of the building. This area
is approximately 2 miles from the nearest Verizon Wireless tower. This location was observed as
a moderate coverage area.
HTTP Download
DELL
HP
LENOVO
AVG
STDEV
AVG
STDEV
AVG
STDEV
FTP Test
1MB Down
2MB Down
1MB Up
2MB Up
1MB Down
2MB Down
955.09
106.24
647.75
66.41
605.44
21.32
901.94
70.00
708.01
40.27
535.97
57.89
766.36
10.68
776.55
9.40
757.03
24.51
753.89
13.71
758.25
23.17
760.37
17.72
1069.35
52.71
641.60
99.34
580.03
8.16
979.45
62.62
658.92
67.57
551.95
19.71
Integrated EV-DO Comparison for Dell, Lenovo and HP
11
Chicago, IL – Test Location 3:
Testing Location: Monday, December 3, 2007 – Airport, O’Hare – Chicago
Address: Not Available
Details: Testing took place in the upper level main area of Terminal 3. There was no access to
AC power. This area was observed as a low to moderate coverage area with high network traffic.
HTTP Download
DELL
HP
LENOVO
AVG
STDEV
AVG
STDEV
AVG
STDEV
FTP Test
1MB Down
2MB Down
1MB Up
2MB Up
1MB Down
2MB Down
1081.57
341.52
873.25
269.49
449.87
53.34
1242.37
179.74
1067.24
302.28
412.19
59.03
616.59
48.52
596.04
194.51
680.81
38.21
567.37
79.00
418.59
255.73
617.40
75.77
696.81
208.18
963.03
407.09
484.11
35.17
762.47
233.18
1008.51
348.34
413.91
89.10
Integrated EV-DO Comparison for Dell, Lenovo and HP
12
All references, in part or in full, to test results presented in this report must directly reference or
link to this complete report. All references should be made to this report which will be posted in
the competitive analysis test section of the Lionbridge web site.
Disclaimer of Warranties; Limitation of Liability:
VERITEST HAS MADE REASONABLE EFFORTS TO ENSURE THE ACCURACY AND VALIDITY OF ITS
TESTING, HOWEVER, VERITEST SPECIFICALLY DISCLAIMS ANY WARRANTY, EXPRESSED OR
IMPLIED, RELATING TO THE TEST RESULTS AND ANALYSIS, THEIR ACCURACY, COMPLETENESS
OR QUALITY, INCLUDING ANY IMPLIED WARRANTY OF FITNESS FOR ANY PARTICULAR PURPOSE.
ALL PERSONS OR ENTITIES RELYING ON THE RESULTS OF ANY TESTING DO SO AT THEIR OWN
RISK, AND AGREE THAT VERITEST, ITS EMPLOYEES AND ITS SUBCONTRACTORS SHALL HAVE NO
LIABILITY WHATSOEVER FROM ANY CLAIM OF LOSS OR DAMAGE ON ACCOUNT OF ANY ALLEGED
ERROR OR DEFECT IN ANY TESTING PROCEDURE OR RESULT.
IN NO EVENT SHALL VERITEST BE LIABLE FOR INDIRECT, SPECIAL, INCIDENTAL, OR
CONSEQUENTIAL DAMAGES IN CONNECTION WITH ITS TESTING, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES. IN NO EVENT SHALL VERITEST'S LIABILITY, INCLUDING FOR
DIRECT DAMAGES, EXCEED THE AMOUNTS PAID IN CONNECTION WITH VERITEST'S TESTING.
VERITEST AND THE VERITEST LOGO ARE REGISTERED TRADEMARKS OF LIONBRIDGE
TECHNOLOGIES. DELL, THE DELL LOGO, AND LATITUDE ARE REGISTERED TRADEMARKS OF DELL
INC. OTHER COMPANIES’ NAMES ARE USED HEREIN FOR IDENTIFICATION PURPOSES ONLY AND
BELONG TO THEIR RESPECTIVE OWNERS.
Integrated EV-DO Comparison for Dell, Lenovo and HP
13