Jump to content

Gradual Degradation of 3DMark Fire Strike Results Produces Unreliable Results


Guest

Recommended Posts

For those that are not already aware of the issue, the folks at Futuremark seem to be struggling to keep a consistent product in the latest 3DMark benchmark. In particular, Fire Strike. Sometime around the release of Time Spy things started getting screwy with Fire Strike and now it seems with every Fire Strike GUI update the effect is lowered benchmark scores, and specifically the physics portion of the benchmark.

 

Kudos to @Papusan for noticing this months ago. He been going back and forth with Futuremark about the problem and it seems they are either ignoring him or don't care. Maybe because most people are not observant enough to notice or care.

 

Some people might say you cannot compare results across benchmark software versions, but that shouldn't hold water here. There is a leaderboard and searchable database of results and if there is not a high degree of consistency between GUI versions the results in their database will become irreleant, as will their leaderboard. The search filter does not have a field to filter by GUI version, so we can expect the results from the database and leaderboard to be misleading, inaccurate and unreliable.

 

You will notice from the examples posted below that with each new version of Fire Strike the scores get lower and lower. These examples are consecutive runs on the same day, same machine, and identical CPU and GPU settings. The only thing that changes is Fire Strike benchmark results. We need Futuremark to understand and correct this.

 

http://www.3dmark.com/compare/fs/11047304/fs/11047179/fs/11047154

 

 

Here is a similar example from @Papusanhttp://www.3dmark.com/compare/fs/11036017/fs/11035883

 

If you agree this is a problem and want it to be fixed, please complain to Futuremark and let them know they need to put the brakes on and not do anything else with 3DMark until they have this mess under control. Gimmicky features are one thing, but inconsistent benchmark results makes 3DMark unreliable.

 

 


This post has been promoted to an article
Link to comment
Share on other sites

6 hours ago, Mr. Fox said:

For those that are not already aware of the issue, the folks at Futuremark seem to be struggling to keep a consistent product in the latest 3DMark benchmark. In particular, Fire Strike. Sometime around the release of Time Spy things started getting screwy with Fire Strike and now it seems with every Fire Strike GUI update the effect is lowered benchmark scores, and specifically the physics portion of the benchmark.

 

Kudos to @Papusan for noticing this months ago. He been going back and forth with Futuremark about the problem and it seems they are either ignoring him or don't care. Maybe because most people are not observant enough to notice or care.

 

Some people might say you cannot compare results across benchmark software versions, but that shouldn't hold water here. There is a leaderboard and searchable database of results and if there is not a high degree of consistency between GUI versions the results in their database will become irreleant, as will their leaderboard. The search filter does not have a field to filter by GUI version, so we can expect the results from the database and leaderboard to be misleading, inaccurate and unreliable.

 

You will notice from the examples posted below that with each new version of Fire Strike the scores get lower and lower. These examples are consecutive runs on the same day, same machine, and identical CPU and GPU settings. The only thing that changes is Fire Strike benchmark results. We need Futuremark to understand and correct this.

 

http://www.3dmark.com/compare/fs/11047304/fs/11047179/fs/11047154

 

 

Here is a similar example from @Papusanhttp://www.3dmark.com/compare/fs/11036017/fs/11035883

 

If you agree this is a problem and want it to be fixed, please complain to Futuremark and let them know they need to put the brakes on and not do anything else with 3DMark until they have this mess under control. Gimmicky features are one thing, but inconsistent benchmark results makes 3DMark unreliable.

 

 


This post has been promoted to an article

Thanks bro Fox

 See https://www.techinferno.com/index.php?/articles/frontpage_news/software_news/gradual-degradation-of-3dmark-fire-strike-produces-unreliable-results-r94/&do=findComment&comment=154385

 

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use. We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.