How We Test
The PC World Test Center uses a variety of rigorous performance tests and tools to conduct a fair and impartial analysis of major products under review.Ramon G. McLeod, Editor, PCWorld.com
PC World does two major forms of testing of all products under consideration for review: hands-on testing by editors and writers who are experts in the product category, and formal lab testing by the PC World Test Center.
In the near future we will publish all of the methodology we use for our hands-on testing, which, like lab testing, follows rigorous guidelines to ensure fair and impartial analysis.
Following is a synopsis of the tests that the PC World Test Center performs for major product categories. Other major categories will be added soon, including our procedures for testing televisions.
Desktops and Notebooks
We test or retest all desktop and notebook systems under our WorldBench 5 suite, and we put them through graphics testing as well. Notebooks also undergo battery-life tests.
WorldBench 5 is an applications-based benchmark that runs various tests using 12 applications selected for their mass-market appeal, stability, market share, and variety. PC World has always believed that application-based benchmarking is a better way to quantify computer performance than esoteric, synthetic methods. WorldBench 5 runs--albeit in an extremely compressed way--many of the same tasks that average users perform on their PCs every day.
WorldBench 5 uses the following applications:
For further information on WorldBench, 5 visit our WorldBench 5 page.
Graphics tests measure the frames per second achieved for two popular games played at a variety of resolutions. The games were chosen to run on the widest variety of systems.
Battery tests measure battery life by repeating a 1-hour cycle of tasks until the battery is exhausted: typing and running a search-and-replace operation in Microsoft Word; typing and recalculating a worksheet in Microsoft Excel; doing a criteria search in Paradox; and playing Solitaire.
Printers
We test printers for the number of pages per minute they can produce for text, and when appropriate, the pages per minute they can produce for color graphics and photos. We also assess them on text, line-art, color-graphics, grayscale, and color-photo quality.
We test for speed using a ten-page text document; a three-page text document that contains images and different-size fonts; a black-and-white photo using the black cartridge; a two-page spreadsheet, a three-page presentation, and an image using color cartridges; and an image with the printer set for best color-photo quality.
Multifunction printers receive additional tests for scanning speed and quality, as well as for copying speed and quality.
Graphics Boards
We test frame rates on a variety of popular games. Tests run at 1024 by 768 and 1600 by 1200 resolution, with and without 4X antialiasing and 8X anisotropic filtering enabled. You can find a complete list of the games used in the tests on the Test Report page of any graphics board review.
Monitors
A panel of 12 judges rates how well each monitor displays 11 text and graphics images at native resolution. For 17-inch and 19-inch monitors, native resolution is 1280 by 1024; for 23-inch and 24-inch wide-screen monitors, native resolution is 1920 by 1200. Some of the test images are from DisplayMate for Windows.
Hard Drives
To assess performance of external hard drives, we time how long it takes to copy and read 3.1GB of data (first as a number of folders and files, and then as one large.zip file) between our reference test bed's hard drive and the external drive we're evaluating. We also time how long it takes to perform a virus scan on 14.1GB of data stored on the external drive, and we search the entire contents of the external drive for files containing a certain text string.
To test internal hard drives, we copy 3.1GB of data (first as a number of folders and files, and then as one large.zip file) from one location on the drive to another; time how long it takes to scan a folder containing 6.2GB of data and program files for viruses; and time how long it takes to do a find-file operation on the entire drive in Windows XP.
For both external and internal drives, we single out the drive's results on the ACDSee PowerPack 5.0, Nero Express 6.0.0.3, and WinZip 8.1 tests contained in the release version of WorldBench 5. We carry out all tests on two identical test beds with a 3.6-GHz Pentium 4 processor and Windows XP Home Edition.
Optical Drives
We test optical drives under Windows XP Home Edition on a PC with a 1.67-GHz Athlon XP 2000+ CPU and 512MB of SDRAM, using the drive's bundled disc-burning software and media supplied by the vendor or by Verbatim. We test the drives with the format specified by the vendor at review time. Some vendors may change drive suppliers without altering their model designations; our results reflect the configuration tested at the time of our review. Write speed is the time (in minutes:seconds) required to write 7.9GB to double-layer DVD+R DL media, 4.35GB to single-layer DVD+R media, and 694MB to CD-R media, and to format a blank rewritable DVD+RW and packet-write 3.4GB to that disc.
Cameras
Point-and-shoot models: To gauge picture quality, we take a series of shots, with and without flash, at the camera's highest resolution. We photograph a complex still life, a target resolution chart, and a mannequin to see how well each camera captures details and subtle coloring such as skin tones. A panel of judges reviews on-screen and printed photos and assigns image-quality scores; we then average those scores.
Advanced models and single-lens reflex models: To gauge picture quality, we take a series of shots, with and without flash, at the camera's highest resolution. We photograph a complex still life and a mannequin using automatic settings in Program/Full-Auto Mode to see how well each camera captures subtle color and exposure under its default settings. We then photograph the same still life and a resolution moiré chart with semiautomatic settings using aperture priority, custom white balance, and exposure bracketing. We pick the best shots of each of those two subjects for judging. We also test the camera's capability for minimizing noise using a range of ISO settings. We review the on-screen and printed photos and assign image-quality scores.
The image-quality rating of the camera is based on five categories: exposure, color, sharpness, distortion, and overall.