Please register or login. There are 0 registered and 867 anonymous users currently online. Current bandwidth usage: 326.30 kbit/s August 20 - 02:49am EDT 
Hardware Analysis
      
Forums Product Prices
  Contents 
 
 

  Latest Topics 
 

More >>
 

    
 
 

  You Are Here: 
 
/ Forums / Different points of view? Entertaining new trains ...
 

  Walking on a thin line..... 
 
 Author 
 Date Written 
 Tools 
Robert Kropiewnicki Jun 08, 2001, 01:03pm EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List Replies: 7 - Views: 482
I think that the biggest problem that sites such as this one face is the fact that the people who come to these sites are very knowledgeable, at times even more so than the people running the site.

Take Ace's Hardware for example.....the expertise of many of the site's most passionate visitors is amazingly high. If Johan and Brian (site operators) were to put out an article that isn't well thought out, well reasoned, and provide hard data to back up their claims, they would be roasted to no end. To their credit, they put out some of best review articles I have ever seen on the net. There is always an abundance of benchmarking, design analysis, and testing methodology available in the reviews. Thus, even if you don't agree with their results or final opinion, you at least have a better understanding of how they got to their final conclusion and can more easily ask questions or voice criticisms. It makes for much more substantial debates.

What is not so easily tolerated, nor should it be, are "reviews" where a bunch of opinions are voiced based on benchmarks that are either A) absent, B) supposedly found elsewhere but never referenced, or C) just poorly done (scary how many sites still think Quake 3 is the end all/be all benchmark). Worse yet is when sites go out of their way to use only those benchmarks (or a large majority of them) that make their chosen favorite shine.

It even needs to be taken a step farther.....you cannot just throw out a bunch of benchmarks and simply declare a winner. It is important that the reviewer makes the end user understand to the best of his/her ability why a benchmark between two equally clocked processors would favor one over the other (say a 1.4GHz Athlon vs. a 1.4 GHz P4). This goes a long way in helping the reader decide which processor is better for what they need it for. Are they using a lot of SSE apps? Is memory latency critical to their applications? Do they require more bandwidth? A stronger FPU?

Possibly the most difficult part of this is the fact that sites are not always able to get comparable hardware. This is where the methodology used becomes very important. When doing a review with benchmarks, it is important that the process be kept as standardized as possible. Of course, the process will differ is you're doing a processor review vs. a chipset review vs. a motherboard roundup, etc. Things like processor steppings, BIOS levels, driver info should always be part and parcel of a review. One of the best known hardware sites out there, Tom's Hardware, was recently blasted (and rightly so) for a roundup they did of motherboards based on the Ali Magik chipset. Why you ask? Well, the BIOS revisions they used for the different boards were old compared to what was currently available from the manufacturers at the time. As anyone who has kept tabs on the Ali based boards knows, BIOS revisions have made a world of difference in the performance and stability of the chipset.

So, what to do? Well for one, if you don't have the hardware at hand, DO NOT DO A REVIEW!!!!! Or at the very least don't call it such. Call it an opinion piece, call it an editorial. Without having tested the hardware, it isn't a review, it's a guess. And if you do have the hardware, make sure your methodology is well thought out, your testing thorough, and your conclusions based at the evidence in hand, not some wild speculation.

Also, there will be those who still aren't fond of the testing you do. That's unavoidable as there will always be someone's favorite benchmark that wasn't included in the test. If you've done your homework, you should be readily able to explain why you didn't do said test. Perhaps it's just something you hadn't thought of doing. The readers aren't perfect, but then again, neither are the reviewers. There is always room for improvement.


Want to enjoy fewer advertisements and more features? Click here to become a Hardware Analysis registered user.
NickName Jun 08, 2001, 02:27pm EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List  
>> Good Point...
You make a very strong point there. I agree with you totally. The person who does the test, better know his stuff or he is going to get "Roasted."

Dan Mepham Jun 08, 2001, 02:33pm EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List  
>> [No Subject]
Hey guys. Thanks for taking the time to write. Although I feel you got a little off the topic of my piece (I'm not talking about any one site, or even a review site at all. Im talking about the way people react to new ideas), I agree with what you said. In a nutshell, if you don't know what you're talking about, don't pretend you do. Absolutely.

But that was precisely my point. My point is that, too often, people dismiss something as being written by someone who doesnt know what he/she is talking about, simply because THEY DON'T AGREE.

The message was not 'You have to listen to everyone, even if he's full of crap'. The message was 'Think very, very, VERY carefully before you accuse someone of being full of crap'. I'm not saying you have to listen to crap, I'm inviting us all to be more careful about what we declare as crap.

Anyway, thanks again for commenting. Take care.
Dan Mepham

Editor in Chief, Hardware Analysis
Email : dmepham@hardwareanalysis.com
Visit us at : http://www.hardwareanalysis.com

Dan Mepham
Robert Kropiewnicki Jun 08, 2001, 02:43pm EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List  
>> More to the point then.....
By the same token, sites have to be careful about posting stuff that is easily labled as "crap".

If two sites provided two different conclusions about a product, who are you more likely to believe; A) the site with conclusions derived from benchmarks? or B) the site with nothing but the reviewers personal feelings as to the product with nothing to back it up?

By the same token if those two sites both have benchmarks available yet still arrive at different conclusions, the questions change.

Why did they get different benchmark results?
Why did they use/fail to use certain benchmarks?
Do the conclusions they arrive at seem to mesh with the benchmark results?

Give me hard data, tell me what the environment is, tell me what you think.....I'll tell you if I agree with you.

Give me speculations, offer obscure references with no links or direct quotes.....I'll tell you it's crap. By the same token, if the response to the article fails to provide links, direct quotes, or personally run benchmarks, I'll tell you that's crap as well.

Dan Mepham Jun 08, 2001, 02:50pm EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List  
>> [No Subject]
Again, I agree completely. You're absolutely right.

But try to think on the bigger scale. Im not just talking about websites and reviews. I just used that as an example. Perhaps it's my fault for not making that clearer - I'll be more careful next time. Live and learn. :) The article was speaking far more generally about human behavior. If I could prove to you right now that the world IS flat, would you believe me? Even if I had solid scientific evidence?

Probably not. Most people wouldnt. They'd try to discredit me. I'd probably do the exact same thing (please don't think Im trying to say I'm above anyone here...that wasn't the point). My point was that, for a LOT of people, it doesn't matter if I have good, solid evidence, or if I'm just shouting it out with no real reason. And that's not the way it should be.

Cheers!
Dan Mepham

Editor in Chief, Hardware Analysis
Email : dmepham@hardwareanalysis.com
Visit us at : http://www.hardwareanalysis.com

Dan Mepham
Robert Kropiewnicki Jun 08, 2001, 02:57pm EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List  
>> Data will eventually win out.
You are right Dan, there are always people who will be convinced of something despite all evidence to the contrary. How many great thinkers/scientists of the past had been ridiculed, imprisoned, or put to death simply because what they had discovered through scientific and/or mathematical experimentation flew in the face of what was generally accepted as true?

The problem comes when we allow ourselves and others with actualy numbers to back our claim to be drowned out by the howling masses who are too lazy/stubborn/fanatical to accept that what they think is true may not be after all.


Dan Mepham Jun 08, 2001, 03:00pm EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List  
>> [No Subject]
THAT's what I'm after. :) Thank you!

Stop looking for ways other people could be wrong. Look for ways they could be right.

Take care!
Dan Mepham

Editor in Chief, Hardware Analysis
Email : dmepham@hardwareanalysis.com
Visit us at : http://www.hardwareanalysis.com

Dan Mepham
Reny Abrego Jun 09, 2001, 01:45am EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List  
>> [No Subject]
Dan
I came here by way of 3D.net I read the newsand all i wanted to make a post but not being a member i had to join to so i did... just to post this
you know there are always people who will take the other side no matter what, you can talk till your blue in the face it will not change them no matter what. Now i think you can chose to worry about this or you can chose to go on and keep writing things base on what you find thru test the choice is yours. No one in history hasw ever changed the mass so why not be content to write for those who seek the truth. We are not the minority WE ARE THE MAJORITY who seek the truth....
I will come back to read you reviews ....Reny abrego


Write a Reply >>


 

    
 
 

  Topic Tools 
 
RSS UpdatesRSS Updates
 

  Related Articles 
 
 

  Newsletter 
 
A weekly newsletter featuring an editorial and a roundup of the latest articles, news and other interesting topics.

Please enter your email address below and click Subscribe.