Please register or login. There are 0 registered and 630 anonymous users currently online. Current bandwidth usage: 326.30 kbit/s April 30 - 10:22pm EDT 
Hardware Analysis
      
Forums Product Prices
  Contents 
 
 

  Latest Topics 
 

More >>
 

    
 
 

  You Are Here: 
 
/ Forums / Hard Drives /
 

  Newegg Now "Co-Mingles" Review Stats--Potential Mislead 
 
 Author 
 Date Written 
 Tools 
john albrich Jan 09, 2014, 08:19pm EST Reply - Quote - Report Abuse
Private Message - Add to Buddy List

Edited: Jan 10, 2014, 08:13am EST

Replies: 5 - Views: 7320
.
This is just a word of caution.
Warning! This post includes Boring Info..but that's because in addition to simply bringing up the caution, I try to explain why it makes a difference.

If you haven't noticed this, it appears that newegg.com is now intentionally grouping together "related" device reviews and review statistics, even when the devices are fundamentally different in design and manufacture.

This type of data massaging is unfortunate and potentially costly to the consumer (in both money and time). It is particularly worrisome as they don't make it clear that it is what they are now doing in the statistics summary "1 thru 5 stars rating chart". (e.g. you look at the summary thinking it is for the actual device you are looking at, but it now also includes statistics from the customer reviews of different devices.)

It is important to understand that it can be still possible to make an informed decision about a specific device based on its customer reviews (but only when individual "This review is from" data are available), it's just that Newegg has taken this mis-step (temporary, I hope) and made it MUCH more difficult for us to do so.

Personally, even though I have a very long history with Newegg (only as a customer), and have had overwhelmingly very positive experiences with the products they offer and newegg's RMA process (when needed...very infrequently), I think this is important enough to bring it to your attention so one can make a more informed purchase decision.

Manipulation of statistics in this way can muddy the perception of a product's quality. In my personal opinion, it may even have a major impact on business operating costs (e.g. you buy what seem to be highly customer-rated drive "X" thinking it is well reviewed by a lot of users, only to find out the stats included more positive review ratings from models "Y" and "Z" devices as well). Later, you may experience that the installed "X" devices start failing faster and in greater numbers than you anticipated.

For example, when I looked at an nTB Western Digital Red "Intellipower" Hard Drive, Newegg had smushed together all the reviews and review statistics for the 1TB, 2TB, 3TB and 4TB "Red" hard drives. You are NOT provided the individualized review rating summary for the 3TB Red HDD as a single device, and these drives may differ quite a bit in design, manufacture, and firmware. They also operate with different temperature and power profiles...all of which affect reliability.

History shows us that the reliability of a given, single hard drive can vary significantly depending on capacity, often decreasing as capacity increases. The simple fact that they must add more components (platters, electronics, and more R/W assemblies) to an HDD to increase capacity inherently decreases reliability of an individual HDD*.

For one example, I've seen 1TB HDDs of one manufacturer's "family" of HDDs with 70% 1-star top-rated reviews...but the 3TB 1-star rated reviews in the same family fell WAY down to nearly 50%. Thus, when they intermingle the stats from different drives the 3TB appears to get artificially boosted positive review ratings, while the 1TB positive ratings fall. However, it's always possible they will intermingle rating stats for the 3TB drives just to make the 3TB HDDs appear to get much better reviews than in reality, but then NOT intermingle the rating stats for the 1TB drives...thus letting those drives keep their superior rating for marketing purposes.

Thus, when you make a merely cursory examination of the "review" statistics for a 3TB HDD inextricably mixed in with the stats for the 1TB, 2TB, and 4TB HDDs, you can NOT be assured that the stats are meaningful for the 3TB HDD.

However, for the time being, the individual reviews are still visible on newegg, and you may be able to determine whether a particular review and rating is for a given 1TB or 3TB HDD. But...you personally and manually have to go through perhaps hundreds or thousands of reviews in the review section, and manually separate the data and compile the results yourself. That's a LOT of work.

It is my hope that Newegg will change this back to provide a more accurate device-specific review summary for customers. (Or even better, perhaps Newegg would give customers the option to view the review section that applies ONLY to the specific individual device (the way it used to be), but then also lets the customer view an "overall" grouped review section that applies to the "family" of devices.)
.
*Note: to address one possible argument folks might bring up...while it is generally true that if you use 3 of device "X" instead of a single device "X", the combined reliability of those those 3 device X's as a group is lower than the reliability of a single device X (for example, 3x1TB drives v. 1x1TB drive). But, that's not what we're talking about here. The same can't be inferred when the 3 "X" devices are different in design and manufacture from the single device "Y" (for example, 3x1TB drives v. 1x3TB drive). There are factors which can make device "Y" much less reliable than even all 3 of the "X" devices. Then again, there's the possibility one didn't really need device "Y", and a single device "X" would have met requirements, but there may have been a marketing factor one might consider to purchase "Y" instead...e.g. $/GB can be compelling. In such instances, one might think "Y" is a better deal in terms of initial cost, but later find it was a false savings as "Y" fails more frequently or has higher maintenance requirements.



edit:
Changed subject line so all of it shows on the "latest" web page index
edit:
This doesn't affect just hard drives, for example, as of 20140110 this customer review summary chart on the TUH-300 model USB 3.0 4-Port USB Hub product page co-mingles all 5 (and perhaps more) of these completely different devices, including some USB2.0 ONLY Hubs, which makes the summary virtually useless to a customer when considering the purchase of the TUH-300:
...Tek Republic TUH-300 USB 3.0 4-Port Hub
...Tek Republic TUH-2700 USB 2.0 ONLY 7 Port Hub
...Tek Republic TUH-3400 Aluminum USB 3.0 4 Port Hub
...Tek Republic TUH-200 USB 3.0 + USB 2.0 4-Port Hub
...Tek Republic UH-CBC7-US USB 2.0 ONLY 4-Port Hub


Want to enjoy fewer advertisements and more features? Click here to become a Hardware Analysis registered user.
john albrich Jan 29, 2014, 01:44pm EST Reply - Quote - Report Abuse
Private Message - Add to Buddy List

Edited: Jan 29, 2014, 01:52pm EST

 
>> Breakdown of actual review details for 700W PSU
.
Similarly...

ONLY 78 of 729 user reviews listed on the webpage for this specific 700W PSU were actually for this specific PSU.
That's 11%

However, the casual reader would see the review summary and likely conclude that "579 (80%) out of 729" reviews were 4* and 5* rated...a very high rating for many products.
Even though this device SEEMS very highly rated by the reviews, the reality is very different.
50 (9%) of the ~570 4*+5* rated reviews were for this 700W PSU
28 (19%) of the ~150 1*+2*+3* rated reviews were for this 700W PSU

So, instead of apparently 80% being 4* and 5* rated, it was really 7% of the total reviews when you detail the stats for the actual 700W PSU.

And the 50 4*+5* reviews comprise just 64% of the 78 total reviews when you look at JUST the reviews for the 700W PSU.

One can see that co-mingling the reviews of other PSUs with this particular 700W PSU makes things confusing.


Note: for various reasons, source data may have changed since I compiled them for this post

john albrich Jan 29, 2014, 01:47pm EST Reply - Quote - Report Abuse
Private Message - Add to Buddy List

Edited: Jan 29, 2014, 01:47pm EST

 
>> Summary of actual review details for 128GB USB3.0 Memory Stick
.
ONLY 9 of 62 reviews for a 128GB USB3 memory stick were for that actual specific device.
That's 15%

The 53 other reviews were for different USB memory sticks, but co-mingled with the reviews for the 128GB device.


Note: for various reasons, source data may have changed since I compiled them for this post

john albrich Feb 03, 2014, 04:11am EST Reply - Quote - Report Abuse
Private Message - Add to Buddy List  
>> How It Should Be Done-Example
.
The summary review statistics chart actually means something for this specific device.

Just to show how review statistics SHOULD be compiled for a specific device, here's one device for which newegg's summary review statistics chart includes data for ONLY this device.
Seagate Hybrid Drives ST1000LM014 1TB

135 of 135 reviews for this hybrid (HDD/SSD) 1TB were for that actual specific device.
That's 100%

At this time, NO reviews for other devices were included in the summary statistic chart.

That means when the customer looks at the chart the data actually mean something when it comes to making a purchasing decision. You "know" that when the summary says that 67% of the 135 reviews rated this drive as 4 and 5 (5=best rating), and that 33% rated the drive as either 3, 2, or 1 (1=worst rating) they were all "talking" about this device...not some other drive.

That makes it easy for the customer to see what other customers think about THIS drive [assuming Newegg didn't "cut" some negative reviews from a larger than 135 reviews database]

Good going, Newegg! That straight-forward handling of review data helps increase trust between the customer and a company.
.
.
Note: for various reasons, source review data may have changed on Newegg since I compiled them for this post, so when looking at a specific device's review summary chart, the reader should always check to see if the methodology has changed for better or worse.

john albrich Mar 11, 2014, 05:32pm EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List

Edited: Mar 11, 2014, 05:34pm EDT

 
>> A Different Kind Of Review Stats Mislead
.
Just observed that the review stats and the reviews themselves are in some cases also being skewed when you select different search/display parameters (e.g. sort by "Date Posted" v. "Lowest Rated", or change the number of results per page from "10/page" to "100/page" and so on).

This particular problem however, is almost certainly due to an unintended database SQL-type error.

Not only the statistics "star" count summaries change, but some reviews that should be included due to the requested search or sorting criteria are dropped from the stats AND the list of actual reviews.

For example, one product showed 8 reviews when I requested sort by "Date Posted" and "10/page", inclouding 1x 5 "star" review. When I changed it to sort by "Lowest Rated", there were suddenly only 7 reviews. The 5 "star" rated review disappeared entirely from both the stats and the listed reviews. Then when I changed it back to sort by "Date Posted", that review was STILL missing. Only by reloading the product webpage with the default search/display parameters did that 5 "star" review get re-included in the displayed review data.

In another product page, even though some of the reviews were listed on the displayed page, they were excluded from the statistics summary chart (e.g. there was 1x 5-"star" rated review that showed 5 "stars", but the statistics summary showed there were ZERO 5-"star" rated reviews. It was left out of the statistics summary chart entirely.

But, the result is the same. Again, we can't rely on just the Newegg product review statistics summary chart when making purchase decisions. We must look through each listed review and create our own review statistics to make sure we're getting a true picture of other user's input on any given product.

Meats_Of_Evil Mar 11, 2014, 06:08pm EDT Reply - Quote - Report Abuse
Private Message - Add to Buddy List  
>> Re: Newegg Now "Co-Mingles" Review Stats--Potential Mislead
Oh man, that was enlightening. What sort of benefit can they hope to achieve with this new method of statistics, that the old one doesn't do? I like simplicity and even though I do not buy from Newegg due to shipping costs I always check my hardware reviews on there.

Thanks for this John, hope they don't mix and match anymore reviews.

-------------------------------------------------------------------------------------------------------------------------
Everything I write is Sarcasm.

Write a Reply >>


 

    
 
 

  Topic Tools 
 
RSS UpdatesRSS Updates
 

  Related Articles 
 
 

  Newsletter 
 
A weekly newsletter featuring an editorial and a roundup of the latest articles, news and other interesting topics.

Please enter your email address below and click Subscribe.