I'm shocked! The article begins well, with an explanation of when dual processors should provide benefit, then procedes with a set of completely inadequate benchtests for the subject. Coupled with mismatched hardware, and consequently draws completely the wrong conclusion.

My points:

Why was the single Athlon XP faster than the Athlon MP setup?
- Your single setup used a KT266a chipset, which is faster than the MP760 chipset.

But the dual Xeon was faster, how come?
- Well, you used 2 processors with 512k cache and I presume (but it was not mentioned) that the Pentium 4 had 256k cache. This would account for the increased Xeon performance. Also the 860 chipset used for the Xeons, has been tested as providing enhanced memory performance over the 850 chipset anyway.

Poor choice of benchmarks!!!
- What's the point of banging on how dual processors should be of benefit when the benchmarks you chose to test with, don't benefit from dual processors, merely the higher performing processor/chipset configurations mentioned above? There were several references to Photoshop benefiting from a dual processor setup... where was the test?

How could you have conducted a more accurate and better concieved test?
- Hardware platforms, make them compatible! You should have compared the 760 platform with single and dual processors. Same for the Intel setup, no point pitting a Xeon against a P4.
- Benchmarks, use ones that are at least multithreaded.

In the end, a supposed comparison of dual processor setups turned out to be a single processor/chipset test. THAT'S how wide of the mark it was!

Mark,

You sure you fully read the article and are familiar with the benchmarks used? SYSmark 2002 includes a number of SMP aware applications such as Adobe Photoshop and Premiere.

What I tried to do is determine whether an SMP system is something which a regular user would benefit from and not just the professional that uses software with 4, 5 or even 6-figure pricetags. Thus I used a number of benchmarks that include applications a normal user uses, not some exotic SMP benchmark that is only useful when comparing one SMP system to another.

The systems use the same hardware, apart from the motherboard/memory/CPU and are a good representation of what a typical system with that CPU looks like. Unfortunately you can't rule out the fact that some chipsets perform better than others, what do you think we're trying to investigate here? It is these performance differences we're after.

The point of the article is to investigate whether a regular user would benefit from upgrading to, or buying a whole new, SMP system, let me quote from the first page:

'In the next few pages we’ll be looking at the performance of the new Xeon platform in comparison to AMD’s SMP solution, and a two single CPU systems based on AMD’s Athlon and Intel’s Pentium 4, we’ll be trying to answer the question whether a dual CPU system is worth the higher price-tag and whether the AMD and Intel SMP systems offer a good price/performance ratio.'

Hope this answers some, and hopefully all, of your questions.

Regards,

The link between SYSmark and multithreading should have been made clearer. However, it is just a benchmark at the end of the day, what would be more useful would be realworld examples, like rendering times in Adobe Premiere. Multi-CPU systems at the moment, seem to be targeted to NLE users in the UK. Even with advent of realtime editing cards, e.g. Canopus Storm, they make the disitinction between how many effects/filters can be processed by sinlge CPU's and multiple CPU's. I understand that this may have been out of the scope or budget of this article

These "benchmarks" articles for duallies can be sooo unrealistic. Why are you only running one program/application at a time? Here is a question for you:
What would be the effect on your "benchmarks" if you ran them again, same setups, BUT.. run an instance of seti or crunch an RC5 at the same time with priority equal to "high". Then see where your numbers sit!
The point of having duallies at home is to be able to run multiple apps concurrently while retaining a "zippy" system.

Matthew,

Guess you too haven't read the article in full, let me quote from the article:

'But there’s more, a dual CPU system can schedule applications among the two CPUs much more efficiently, for example when one CPU is taxed to 90% and the other to just 10%, launching a new application will automatically have the OS assign that to the least taxed CPU. As a result the system is much better able to keep on working efficiently and to the user that translates into a system that is able to do a higher workload without noticeably slowing down.'

Regards,

Yes, I read that.. However, your benchmarks do not SHOW it.
Regards,
Matt

PS:
Thats all I meant... The benchmarks published never measure performance with multiple apps running concurrently.
Regards,
Matt

The article was sloppy. The benchmarks do not show what is explained, in some cases, explained more than once.

Its like writing an article about a Dodge Viper with a red paint job and then depicting just any old red car as an example.

Also, you compared a P4 with 256KB L2 with TWO P4 Xeons with 512KB L2 - thats plain dumb and I know how long you have been in this line of work Sander and even you have GOT to admit thats a damn bad example - you even SAID you kept everything else the same in order not to bias the benchmarks!!! You could have used a P4 with 512KB L2.

By the same token, I think you are CORRECT to compare the KT266A with the 760MPX chipset because IF you had a single AMD CPU then you would likely (or possibly) have a KT266A chipset and if you had a dual MP you would almost certainly have to have an MPX chipset. Perhaps you could have picked the SiS chipset, but thats a minor detail.

Another point is the cost difference between the two systems. Thats also a factor if you ask me.

And yet another is the clock speed difference. Maybe you should have compared dual MP2100s and the Intel P4 Xeon 1.8GHz - that was the CPUs are almost at the same speed, the MPs running a tad slower.

And YET another point - why, oh why CL2.5? Faster RAM is what the MPs need as they chipset doesnt provide enough memory BW anyhow, so CL2 and good RAM makes the difference. You could have used 512MB of registered CL2 DDR RAM to at least get some parity - and registered DDR RAM its not that much more expensive than unbuffered (at least not the like ~200% difference between DDR and RMDRAM).

I say rewrite this article and benchmark more similarly specced and priced systems. Its clear to me that something costing $2000 is likely no where near as high-end as something costing $5000. Don't compare apples and oranges - its pointless unless all you want is page-views.

Okay I'll bite, as some of you seem to think I am skating on thin ice concerning the benchmarks used and the conclusion I (dare) to make based on the results. I picked SYSmark2002 for a reason and that reason is that it simply is the only real world benchmark that makes use of actual applications rather than run some homebrew app. that the developer 'thinks' is a good approximation of a real world situation.

Let me explain as from several posts I've read it is obvious that many of those participating in this thread are not familiar with SYSmark2002. SYSmark2002 is a script based benchmark, all it does is simply open up an app, for example Photoshop, and perform an operation, for example apply a filter, and then time how long it takes the system to complete this operation. It does this by using the actual application so in order to be able to run SYSmark2002 the installer takes about 30-mins to install a host of applications on your system ranging from MS Word to Photoshop, Premiere, Dreamweaver and a whole range of others. SYSmark2002 use the ACTUAL applications, not some custom version and better yet, it runs up to six different applications at the same time, thus taxing the system as if someone was actually working on it and doing some heavy multitasking, the whole benchmark (when doing an official run) can take up to two hours to complete.

So besides the fact that SYSmark2002 includes SMP aware apps it also runs multiple apps at the same time. How is this not a good example of a real world situation? And more importantly, what other benchmark do you suggest that would give a better picture? I mean what closer than real world do you want, they're already using the actual apps. Sure, we can argue about what apps a typical user would have installed and use, not everybody uses Word or Netscape, or whether the filters and operations used in Photoshop tax the system enough. To me this is a benchmark that sets the standard for real world testing in a simulated typical user environment, and that was exactly what I was after, nothing more, nothing less. If I had wanted to get down to the nitty gritty I'd have taken at least 12 to 15 pages and a lot more benchmarks and would've made an article that isn't so easy to grasp as this one.

Same goes for 3D Mark2001 SE, everybody uses it, mostly for bragging rights, 'look here, my OC-ed Ti4600 does 11000 3D marks' I only tried to show that a 2nd CPU doesn't give you a magic boost in 3D marks, as many seem to think. Maybe I should've used Quake3 instead? But 3D Mark2001 SE is the only benchmark that has an online database for people to up their scores so they can compare them to others. Anybody that is interested can look for a similar system, or systems, like we used and conclude that what I wrote is indeed accurate. Again I'm just trying to address misconceptions, or rather myths here, not make a 15-page article about the in- and outs of SMP, I'll leave that to Ace's Hardware.

Some of you also criticized the systems we used, and that's something I don't understand. All systems use similar hardware, if I used a 512Kb Xeon I also used a 512Kb P4, c' mon I've been in this industry for far too long to not make sure I have a proper and well balanced benchmark lineup. I'm comparing typical systems based on a particular CPU type to one another using the exact same clockspeed and hardware. Typical chipset for the Athlon XP is the KT266A and naturally I used memory that's also most used on that platform; CL-2.5 266MHz DDR. The very same applies to the dual Athlon MP, again I used CL-2.5 266MHz DDR. And no, we're not comparing apples to apples, how can we as there's such enormous differences between chipsets and architectures. Whether that is fair is not the point, as we have no other alternative, there's no RDRAM chipset for the Athlon XP, as there's no Socket-462 Pentium 4. What we did is compare typical system configurations that were identical in clockspeed, HD, VGA-card, memory size, etc. all that was different were the motherboards, the memory types and the CPUs. If we'd done a comparison based on price you'd see some pretty weird results, as we'd be comparing a fully loaded Athlon XP to a crippled dual Xeon, or a mediocre Athlon MP to a high-end Pentium 4.

I hope this has shed some light on the reasoning behind the article and why we used the benchmarks we used. Again, all that we aspired to do is to start the discussion, this was by no means the definitive article on SMP, merely and introduction and some info on how it works and what to expect.

Have a good weekend,

I will concede that many people use CL2.5 DDR RAM on an XP system. I will not believe that people use CL2.5 DDR on MPX chipsets. I know a few people who have MPX boards and not one uses CL2.5. Hell, a lot of CL2.5 DDR wont run on MPX boards unless you reduce the timings to ~2.5-4-4.

Now, I understand that the next step in choosing hardware is a matter of perspective: You can doggedly pursue the sole difference between dual and single CPU on a given system. That would be OK, if stated clearly in the article and you followed through on it. Test first one then two MPs in a MPX board and one Xeon and then two Xeons in an i860 board.

If you want to strictly compare two types of system (i.e, not just the CPUs, but ALL of the typical hardware you may find in such a system), a single CPU (XP or 2x MP, P4 or Xeon), that is also absolutely OK. Then you must change the other hardware to comply with a decent reflection of what a dual- or uniprocessor configuration is, for BOTH the Inteland the AMDs.

Based on your various replies I truly believe your intent was there, but somehow it got muddled in the doing and you ended up mixing both scenarios.

A further point is that the benchmarks between Intel and AMD platform should be seperated. Again not mandatory, but more accurate and scientifically neat.

I would genuinely find it great if you could compare a single and dual MP (both an 1.6GHz) with a single and dual Xeon system at 1.6GHz. As those systems are often used in servers of performance workstations, equip them both with hardware that would really be found in such systems. Fast RMDRAM, good CL2 DDR (Corsair PC2400 is good), RAID, possibly SCSI (especially in the Intel system) and all the trimmings. Seperate the Intel stats for the AMD stats in some way, we are only comparing the difference between single and dual. As an after thought by all means post stats from two single CPU systems that are running in single-cpu boards with all the whistles and bells.

As an after thought; if you want to portray a realisting typical XP1800 system, with CL2.5 DDR, then compare that to a P4 with CL2.5 DDR. Thats what most P4 systems are, at least here, and I doubt thats any different anywhere else. A DDR P4 system doesnt even cost so much more than an AMD based system - so it makes a better comparison to boot.