680M vs 7970M - Discussion on how to benchmark them.

[Brian 1977]

So after reading endless discussions on another forum with respect to the 7970M and 680M and which is truly the better buy, I wrote the following response:

With respect to benchmarks, especially games, there needs to be a more comprehensive system of testing and scoring. For example, static benchmarks like 3dmark 11/vantage etc should be weighed much less than video games (which they typically are). However, even among games, SP games should have much less weight than popular AAA MP titles which hold 100s of hours of replay.

For example, if a gamer finishes an 8 hr SP game, even if its an awesome DX 11 title, they likely won't go back to it or at the most once or twice unless it has a huge modding community behind it. So if nVidia 680M gets 55 fps @ 1080p with ultra settings and AMD 7970M gets 70 fps @1080p, the fact that it is a SP game should weigh into giving it a score on a performance scale.

On the other hand, if a game like BF3 with settings on ultra @ 1080p (which even stresses high end desktop cards) gets 60 fps with 680M on a heavy MP level (multiple runs of FRAPs should be run for accuracy) vs 45 fps on 7970M, then there should be more weight placed on a benchmark like this since it has a large community of players replaying the same title daily vs a SP game. This can extend to other MP titles as well but this is just one example.

So before arguing about benchmarks take this into consideration and then decide which brand really provides the best performance and value for gaming. Of course there is always feature support, 3rd party development support and drivers that also make a huge impact. Most people here just simplify it to overall benchmark % + cost and that is quite misleading.

This has been on my mind for quite sometime and once we rev up T|I reviews again in the future (we will likely have to hire full-time writers since admins are too busy), I plan to implement a scoring system similar to what I described above. I'd like to hear what our members think of my proposal as well as your own suggestions.

[unreal25 786]

I agree. Benchmarks like 3DMark11/Vantage are ok to get a rough feeling how a card performs in comparison to other cards, but that's about it. If you recall few years ago EVERYONE was asking for each card "How well does it run WoW?". It was and still is a not well optimized game (shadows ftl), so you could say "Oh but it's not card's fault it's not running perfect. Try something like Crysis". The thing is, so many more people played it instead of Crysis -- it doesn't matter !

I imagine right now for each new card, people are most likely be wondering how well it runs Diablo3, Guild Wars2 and such. Maybe it would be cool to see Intel HD4000 vs AMD Trinity comparisons for Diablo3 as those cards are going to be available on a lot (or all) new laptops and the integrated cards seem to have reached a point where you can use them play something.

[Meaker 16]

The importance of games is up to the user to decide.

Post unbiased as possible numbers and that's the best you can do.

A scoring system is too simplistic to implement.

[StamatisX 1066]

I don't think a weighted scoring system would be very accurate given that some games are more or less optimized for a specific vendor ie nVidia and I agree with @unreal25 that not everybody has the same preference on games (SP vs MP).

I agree with @Meaker that synthetic benchmarks provide unbiased results. Though that's not the whole picture, so I would like to see a list with games that score better with nVidia cards and a separate list of games that favor AMD and let the reader decide given his gaming preferences to choose among those two companies.

A unified scoring system seems hard (what criteria will be used for the weight of each title) to implement and I doubt it will be widely accepted.

[Meaker 16]

I did not say sythetics were the answer. Though they make comparisons easier.

Just a list of games and differences. A conclusion based on the average percentage is not even that accurate.

[Brian 1977]

The importance of games is up to the user to decide.

Post unbiased as possible numbers and that's the best you can do.

A scoring system is too simplistic to implement.

I don't think a weighted scoring system would be very accurate given that some games are more or less optimized for a specific vendor ie nVidia and I agree with @unreal25 that not everybody has the same preference on games (SP vs MP).

I agree with @Meaker that synthetic benchmarks provide unbiased results. Though that's not the whole picture, so I would like to see a list with games that score better with nVidia cards and a separate list of games that favor AMD and let the reader decide given his gaming preferences to choose among those two companies.

A unified scoring system seems hard (what criteria will be used for the weight of each title) to implement and I doubt it will be widely accepted.

Its easy to decide which games deserve more importance, those that are labeled AAA by the industry and have high replay value. It doesn't take rocket science to determine that and it gives better insight into which vendor optimizes best for the most popular titles. Both AMD and nVidia heavily optimize their drivers for synthetic benchmarks to the point that they are completely useless and do not represent actual gaming performance at all. They are just for number chasing and stress testing a system. Using popular MP titles is also easily testable with reliable results, we've done it in our reviews.

[Wei Lu 17]

I think 680M is a little faster than 7970m but it's double price. So the 7970m is the best choice.

[Brian 1977]

I think 680M is a little faster than 7970m but it's double price. So the 7970m is the best choice.

Its definitely not double the price. Maybe $150-$200 premium over the 7970M but that's about it. As for it being a little faster, I guess we'll know soon enough when I test my 680M vs 7970M in an M18x-R1.

[kzk5609 10]

I think 680M is a little faster than 7970m but it's double price. So the 7970m is the best choice.

well, both green and red are doing great this time, as for performance/price Amd sure is the best but for enthusiasts user who want more power the 680m will be the best choice available now.

[johnksss 720]

running the extreme version of vantage and 3dmark11 will give you a pretty good feel of the cards potential (and that would be the extreme preset and not performance). add on top of that for gpu scores when using extreme cpus and high over clocks.

and to be more to the point, quite using the points system and start using the actual fps it shows when finished.

example:

meaker lost to brian by 16 points on gpu? in reality...that's not a lost because the fps is dead even when you go back and look (speculating because i didn't go back and look yet). those fps translate in to gaming fps numbers. (planned gaming bench mark run) when doing mp, it will never be the same...because it's not just having the fastest setup on the planet to play...you also need the most stable connection...or in modern warfare 3 (mp) you need to be the guy in the middle, not the 4 bar or the 1 bar connection and you can still win using a budget notebook. might not look the best but they can sure up the fps by running some super low resolution.

so do an extreme run of vantage and a extreme run of your favorite game and see if it really is "out dated"

metro2033 would fit pretty good in there....so would gta4

no matter how one tries to come up with a system, there will always be people to complain about it. no matter how universal it may be.... so all you can do is grab some titles and let the masses make up their minds in the end....

[johnksss 720]

here is an example: 3DMARK11

#	USER	VER	CARD	LINK	SCORE	GPU	PHYSX	COMBINED	GT1 FPS	GT2 FPS	GT3 FPS	GT4 FPS	CLOCKS
1	Riri-Fifi	R2	HD7970M	LINK	P12117	13668	9682	8222	55.91	65.00	86.37	44.52	1055/1550
2	Grimy1	R2	HD7970M	LINK	P11883	12615	10874	9172	51.52	60.19	79.48	41.11	950/1470
3	Speedy Gonzales	R2	HD7970M	LINK	P11868	13423	9404	8046	55.73	63.91	84.61	43.24	CLOCKS
4	PimpFinger	R2	HD7970M	LINK	P11565	12859	9522	8074	51.35	60.63	83.22	42.44	1000/1500
5	Mr Fox	R1	HD7970M	LINK	P11563	12676	9704	8292	52.30	61.11	79.79	4.73	1000/1400
6	DumbDumb	R1	HD7970M	LINK	P11451	12161	10444	8859					950/1350
7	Sketch	R1	HD7970M	LINK	P11413	12375	9945	8383	51.92	58.73	77.56	39.74	950/1350
8	Slowbot	R2	GTX680M	LINK	10002	11130	8877	6373					719/1900
9	Johnksss	R1	GTX580M	LINK	P8858	8938	10861	6597	42.58	42.97	54.40	26.47	890/850
10	5150Joker	R1	GTX680M/1	LINK	P8745	9033	8733	7073	43.00	42.12	55.48	27.16	1132/1275