• Sign in to follow this  
    Followers 0

    AMD: Is SYSmark a Reliable Benchmark?


    Brian

    We all love drama between technology companies and the latest deliciousness comes courtesy of AMD. They have a 4 minute video on YouTube that questions whether or not SYSmark is a reliable, objective benchmark to use when evaluating performance between different systems. 

     

    AMD's video opens with John Hampton, Director Computer Client Products, talking about how it is very important in choosing the right benchmark when evaluating a purchase because if you choose the wrong one , you can end up overpaying for technology or getting a lesser performing PC. To emphasize his point, he indirectly mentions Volkswagen's diesel debacle and uses the latter case as an example of how even established organizations can be misleading. 

     

    We are then introduced to Tony Salinas, an AMD Engineering Manager, who runs through a couple tests in SYSmark using two notebooks, one that has an Intel i5 CPU and the other with an AMD FX CPU. SYSmark returns a score of 987 for the Intel system and 659 for the AMD one which is a delta of 50%. AMD notes that this is misleading and not reflective of real life performance. To prove this, they run a different benchmark called PCMark 8 on both systems and the results show the Intel i5 system scoring 4199 and the AMD equipped one 3908 which is a delta of 7%. AMD says this is a far more realistic indicator of true performance as PCMark 8 tests the CPU, GPU and video subcomponents rather than emphasizing the CPU like SYSmark does.

     

    And just to drive the point home some more, AMD created their own in-house test using Microsoft Office Suite where they run a custom script on both notebooks and then measure the start and end of a task with a time stamp to gauge performance. Their in-house test shows the Intel system completing the task in approximately 61 seconds while the AMD based one finished it in roughly 64 seconds which is a delta of 6-7%.

     

    As AMD's results line up with those found with PCMark 8, they contend that PCMark 8 is a more fair and balanced indicator of overall performance while SYSmark is not and therefore should be discarded by any potential customers as a tool for evaluating a purchase. 

     

    amdinteltest.thumb.JPG.57191048d02a2763b

    Intel system using AMD in-house test: Test completed in ~61 seconds.

    amdinternaltest.thumb.JPG.b4b0576be9702f

    AMD system using AMD in-house test: Test completed in ~64 seconds.

     

    They also take the time to remind us that the FTC has required SYSmark published benchmarks to contain fine print that notes their benchmark may only be optimized for Intel. Of course, this may have something to do with the fact that AMD resigned from BAPco consortium. Finally, AMD notes that a transparent benchmark should meet very specific criteria which includes being realistic, unbiased, objective and transparent, all of which SYSmark is not apparently. 

     

    criteria.thumb.JPG.83070ca48ade93fb60255

     

    While it's fun to see these types of videos from AMD, they also highlight the fact that the company is facing an uphill battle on all fronts, whether it's CPU or GPU, and 2016 may be the year that makes or breaks them.

     

    Edited by Brian

    1 person likes this
    Sign in to follow this  
    Followers 0


    User Feedback


    There are no comments to display.



    Create an account or sign in to comment

    You need to be a member in order to leave a comment

    Create an account

    Sign up for a new account in our community. It's easy!


    Register a new account

    Sign in

    Already have an account? Sign in here.


    Sign In Now

  • Similar Content

    • By unrealmitch
      First, hello! Although I am new here, I have been a long time following this forum, and without all of you this hasn't been possible,so thank you!
       
      System: Hardware Dell 5430 14" Core i5 3320M 8GB RAM (1x1600Mhz) iGPU Intel HD 4000 Software Win 10 Pro 64-bit Versión 10.0.14393 [UEFI] GPU Driver AMD Radeon 17.2.1 (non-whql) EGPU MSI Radeon RX 480 GAMING X 8G BEST EXP GDC V8.4d [Express Card] PSU Deer 400W Configuration Just connect the egpu to PC via ExpressCard (PSU ON & PC Off), turn on the PC, and GPU was detected and it worked. [PTD: Off] Others: With 16.12.2 Drivers have some better performance, but my screen blinks sometimes Freesync works well This test have some OC to have similar performance that with whql drivers Video demo:    
      Hardware Info:  
      TEST & BenchMark 3DMark TimeSpy 1.0 3242 (GPU:3920) http://www.3dmark.com/3dm/18115631 3DMark FIRE STRIKE 1.1 7375 (GPU:12996) http://www.3dmark.com/3dm/18115814 Ungine Heaven 4.0 [Basic]: 3025 [Extreme]: 1495 Ungine Valley 1.0 [Extreme HD]: 1933 Games [FRAPS - 5Min ingame]: Game [1080] FPS AVG FPS Min FPS Max Config The Witcher 41.66 28 52 Ultra GTA V 23.66 13 38 Maximun Setting [Mod Cars] GTA V 32.2 20 72 High Skyrim 59 35 63 Ultra [Mod ENB Photorealistic]             Screenshots Bench:  
    • By 750dmg
      Hi guys,
      I succesfully flashed v2 vbios from prema and I tried 1v and and oc but I couldnt. I think it was power suply. Mine is 230w. And I tried 0.75 0.50 0.25 and 0 volt. But now I m trying game, benchmark screen goes dark and after beeping at 0 volt. Is this a hardware damage? I used just 1v max.(temperature max 70c)
      I tried
      cmos reset
      DDU
      Xtu uninstall
      And come back to stock vbios
      I think my card is dying
      Please help me!
       
    • By Sanjula
      Specs :-  Intel i5-2410m 2.3 GHz
                     8 GB ram
                     Intel hd 3000 (igpu)
                     AMD hd 6470m (dgpu)
                     Win 10
       
      # Use the mpcie slot for egpu
       
      GPU powers on and Epu setup 1.x recognized the graphics card. 
      Disabled the dgpu using setup 1.x. 
      Installed latest crimson drivers
      In device manager gpu shows with error 43. Disabled the egpu from device manager and re-enabled it again. Then it showed normally. 
       
      BUT THE PROBLEM IS I CANT CONNECT IT INTO A EXTERNAL DISPLAY. i USED THE DVI-SINGLE LINK PORT AND THE HDMI PORT BUT MY MONITOR DOESNT PICK UP THE SIGNAL.
       
      Help required !!!!
       
      01. First look at the Device manager.

       
       
      02. After re-enabling the gpu

    • By rusTORK
      I know, i know, i know... @svl7 said it's disabled, @octiceps said it's disabled and many other forum users said it's disabled.
       
      I am always want to double check everything and i did it this time too. And i would like to post here result of my research. But first thing first - my PC:
       
      Laptop Configuration: Lenovo Y500 SLI
       
      CPU: Intel i7-3630QM (2400-3400 MHz, Ivy Bridge, QuadCore)
      VGA: 2x 2048Mb NVIDIA GeForce GT650M (SLI Configuration)
      MB: Compal LA-8692P
      RAM: 2x 4Gb Ramaxel RMT3160ED58E9W1600 (11-11-11-28)
      HDD: 1TB Seagate Momentus 5400 (ST1000LM024 HN-M101MBB)
      SSD: 120 Gb Samsung 850 EVO (MGX S4LN062X01)
      Display: 15.6" LG\Philips LP156WF1-TLC1 (1920x1080)
      WIN: MS Windows 10 Pro (x64)
       
      Also i must say, that i have got UNLOCKED BIOS by @svl7. And for second VGA too.
       
      Research
       
      It's beginning from @geraldthread about his Ultrabay to PCI-E Adapter for Lenovo Y510p for useing Radeon Desktop VGAs as external VGA. It's performed well with Y510p, but there was zero info about how it perform on Y500 or will it perform at all (now we know that it's no, but it's another story). Unlocked BIOS didn't help with adapter and i decided to dig more deep in re-enabling integrated GPU in Intel CPU - Intel HD Graphics 4000. And new unlocked BIOS give me a lot of new options to play. But important thing is... some of them working, some of them no. So, here is what i am founded.
       
      First i go to new unlocked BIOS and saw there a lot new tabs.

       
      I go to Advanced => Video Configuration => Special Features


      This option already was Enabled (Switch Graphic Function)
       

      After that i go to Internal Graphic Device and Enable Internal Graphics Device.
       
      Then save changes and exit. Laptop is shutdown and after few seconds turn on again. I saw Lenovo loading screen and then nothing when loading complete. Windows didn't start.
      I am restarted laptop three times and it's started recovery mode. From recovery mode i am loaded Safe Mode and at that point i am finally was at Windows.
       
      My first way goes to Device Manager to see new device. And there was new device - Intel HD Graphics 4000.

       
      In safe mode windows can't access to it's current status, but there also was no error codes. First time there was no driver installed, but now it's even have last driver!

       
      After that i go to AIDA64 Engineer Edition and look what it's show me. It's saw iGPU!

       
      So, at this point i don't what to think about it. I can see iGPU in safe mode, but i can't boot with it in normal Windows. Is AIDA64 lie to me or even Device Manager?
       
      What do your think about it?
       
      P.S. Oh yeah, most menus on Russian language, but i hope your got the idea and trust me. =) It's no fake.
    • By danielxyz
      Is that possible use it without external display?
      It always mention
      Radeon Settings are currently not available.Please try again after connecting a display to AMD graphics and extending the display.