A feedback thread for the review process

[nigerian_businessman]

I'm not intending to make this thread as a bitch fest, so to all visiting it, if you have some feedback to offer keep it civil and constructive.

To those of you who run the site, I know that you guys do this because you love what you do, and you love giving back to the community. Getting paid to do this is the icing on the cake. It comes through in the articles you write and I firmly believe that this website is the most straightforward and honest. I also know that, although you can't please everyone, you've taken the 'real-world scenario' direction with your reviews in an attempt to help your readers make educated decisions when it comes to upgrading.

That said, I have but one issue with the review process, and I've seen other people bring this up but I have yet to see any staff address it, because it's usually buried in a 20+ page thread with a bunch of other responses to a review.

What I would love to see is for the review process to occasionally contain more outdated hardware. I know this site is geared more towards hardcore gamers, but just by looking in the sigs here, it is easy to see that many of your readers aren't on the bleeding edge of technology. Now, obviously this adds more time and effort to every review, so I don't think it would be necessary to put the old hardware in EVERY review. It would, however, be very helpful to rewind a year or two when it comes to new architectures, be they CPU or GPU. There are a lot of people like me who only upgrade every year or two, as opposed to every 6 months. There are also rarely, if ever, comparisons between bleeding edge hardware and older stuff, and that goes for tech websites in general.

To use myself as an example, I've got a 754 3700+ Clawhammer that, due to a power supply failure, needs replacing. I would LOVE to see how this processor, or at least something from the same time period, would stack up against a Conroe. I'm sure there are plenty of people chugging along on 3000+ venices who would love the same info, same goes for people with single core P4's and low end dual core P4's. All of these processors can still deliver a good or at least decent gaming experience, and because of that, many people have hung on to them for a long time, waiting for something to knock their socks off. Yet when that something comes out, we get left behind in the review process, when we're the ones who need the comparison most!

I really do like the direction of the reviews, moving away from canned benchmarks and giving more 'real-world' scenario evaluations. So please, if we could also get some consideration for real-world upgrade cycles, that would be a major improvement. If it is unreasonable to have reviews include older hardware, maybe a 'state of the hardware address' that takes a look at what CPU's and GPU's are in the computers of most of the mainstream, and compares them to bleeding edge stuff. It wouldn't even have to be too thorough.

I can look at a review of a 2.2 ghz 1mb skt 754 a64 and figure out where I'm at in comparison with my 2.4ghz 1mb 754. I can't, however, figure out where my processor stacks up against a 2.4ghz, dual core, 512k cache, socket AM2 processor, or something like Conroe, when it comes to real world usage. All I can do to get some kind of idea is bounce from review site to review site, each with different review methods, chipsets, power supplies, etc.

Likewise, if I dig long enough, I can find out how my x800pro compares to, say, a 6800 ultra. I can then compare a 6800 ultra to a 7800 gtx. Then I can compare a 7800GTX to a 7900GTX or X1900XT to get an idea -- but this is very difficult, time consuming, and very innacurate due to different test setups on different websites with different review processes. Having it all in one place would be great. Having that one place be this website? A godsend.

Thanks for reading, and I hope you don't take this as some giant rant. I really like the site, love the reviews, and I think you all do a great job. Just offering up some constructive criticism.

 

[DeadlyAura]

Oh I definately agree. I don't see the point in reviewing and comparing 4 items that are nearly identical.

I don't see the need to ONLY review the 7900GT, 7900GTX, and 7950GTX. The differneces are miniscule and usually no one is upgrading from a 7900GT to a 7900GTX.

To be honest, I like that you review and compare all 3 similar products because it allows us to see what advantages one would have over another when upgrading. However, it does not show a comparison between what we already have and the new product. I would like to see what kind of improvements I would see by upgrading my hardware, I don't need repetitive pictures showing the amazing graphics over and over again.

I would like to see side by side photo comparisons between...say...a 7950GT and a Radeon 9800Pro. To me, that would be much more realistic and provide MUCH more information when deciding on how ot upgrade.

As long as you add in one or two majorly used pieces of hardware in addition to the new ones right into the already existing charts.

It couldn't be that hard especially since you probably have the data from past tests when the hardware was new.

 

[drizzt81]

I think this point has been brought up before. I think it boils down to "time constraints". The reviews do not have an infinite amount of time and need to set priorities. I think more people are interested in the "do I need to spend the extra $100 to get the GTX or will the GT do fine" question. I would love to see some comparison to old(er) hardware just from the point of view: If I replace my x800pro, what will I gain?

 

[DukenukemX]

I agree the idea of only comparing new hardware doesn't help anybody. I laugh at reviews that have four video cards with the same graphics chip. What's just as funny is when there's a review of high end graphic cards that have a difference of 5 10 fps in most games. OMG an extra 5-10 fps so worth the extra $100 ZOMG!

Like many people I've given up on caring what Asus 7900GT and BFG 7900GT benchmarks look like. Do I really look like I care what .5 fps difference one card has over the other?

Benchmarks are always more interesting when you mix some outdated hardware in it. I myself still use an Athlon XP in both my machines and one has an unlocked Radeon 9500 into a Radeon 9700 Pro and the other has a Geforce 6800 that is also unlocked. So far the benchmarks are nothing more then an e-penis war.

 

[DeadlyAura]

I think this point has been brought up before. I think it boils down to "time constraints". The reviews do not have an infinite amount of time and need to set priorities. I think more people are interested in the "do I need to spend the extra $100 to get the GTX or will the GT do fine" question. I would love to see some comparison to old(er) hardware just from the point of view: If I replace my x800pro, what will I gain?

I know what you are saying about time, but they reviewed the older products when they came out... why not use the old data? Maybe slap in and older piece of hardware INSTEAD of one of the ones they would normally test.

I mean, I can see doing the 7900GT and 7950GT...but why the GTX? Just give me numbers, I don't need a side by side comparison of two pictures that look exactly the same...

 

[thedude42]

I know what you are saying about time, but they reviewed the older products when they came out... why not use the old data?

I know I stand to get hammered for this, but it boils down to journalistic integrity.

Reason being, you have a different board, different bios, different memory modules, different CPU... whatever, that provides a bias with the old info.

The hard choice then is, do you post the old data with a foot note about it being under a different platform? Or do you leave it out entirely? This is a hard decision because it overcomplicates the reader's ability to interperate the review, and in journalism the goal is to get the information out there as clear, complete and quick as possible. That is definitely not an easy task when you're talking about computer hardware reporting, but it can be done when you have experienced people working hard to put things in the right place in the right context. Unfortunately, without overcomplicating things, you can't always cover the details EVERYONE is interested in, so the choice is to keep it simple or run the risk of making it overly confusing.

I mean, I can see doing the 7900GT and 7950GT...but why the GTX? Just give me numbers, I don't need a side by side comparison of two pictures that look exactly the same...

The reason for this, as far as I understand, is to get a graphic comparrisson of the actual differences between the hardware, because often times you're talking about a $200+ difference in these video cards. For $200 bucks most people will want to know what they are going to get for their money.


Ok, those points being addressed, I do agree that a really, really, really good idea might be to create a quick reference page that dynamically links a repository of past benchmark graphs, completely seperate fromt he reviews. That way when a new review pops up and all the bench graphs are posted, the info is dynamically linked back to this main repository page where you can look at all the past review results.

Hell, why not go a step further and let you choose which items you want to look at individually, like how you can compare 2 people in the distributed computing stats pages, and like how other sites let you line up things like cars or cell phones next to eachother.

I think doing something like this would allow the reviews to retain their integrity, while allowing interested users the ability to easily mix and match graphic results. I mean, simply because so many people ONLY look at the graphs to begin with would make this a good idea, help reduce the bandwidth they take up going through entire reviews viewing the product images

 

[Cron0]

It would be awesome because an older system is more of a real world benchmark. Few have an fx-62.

I think a good way to do this is keep a older system on hand, with the same config for a span of a year or 2, keep all the base games installed that you are benchmarking, and just put in the new vid card and run the tests. In a box on the review page, just show the scores of the last 5 generations of cards. All the hardware is the same minus the vid card, so we can see real world proformace over a few generations.

Build a box something like this:

Athlon 3000
1 gig of middle of the road ram or vaule ram
mid range or budget nf4 mobo

Run all tests at lcd resolutions on high quality, and thats it. Keep the line up of games the same even if they are not the newest thing out there. Every year or 2 update the games and they system.

You don't even have to review the card on this system, just show some numbers so readers can get an idea of speed increases.

 

[CommanderFrank]

Suggestions are noted.