• Sign in to follow this  
    Followers 0

    AMD: Is SYSmark a Reliable Benchmark?


    Brian

    We all love drama between technology companies and the latest deliciousness comes courtesy of AMD. They have a 4 minute video on YouTube that questions whether or not SYSmark is a reliable, objective benchmark to use when evaluating performance between different systems. 

     

    AMD's video opens with John Hampton, Director Computer Client Products, talking about how it is very important in choosing the right benchmark when evaluating a purchase because if you choose the wrong one , you can end up overpaying for technology or getting a lesser performing PC. To emphasize his point, he indirectly mentions Volkswagen's diesel debacle and uses the latter case as an example of how even established organizations can be misleading. 

     

    We are then introduced to Tony Salinas, an AMD Engineering Manager, who runs through a couple tests in SYSmark using two notebooks, one that has an Intel i5 CPU and the other with an AMD FX CPU. SYSmark returns a score of 987 for the Intel system and 659 for the AMD one which is a delta of 50%. AMD notes that this is misleading and not reflective of real life performance. To prove this, they run a different benchmark called PCMark 8 on both systems and the results show the Intel i5 system scoring 4199 and the AMD equipped one 3908 which is a delta of 7%. AMD says this is a far more realistic indicator of true performance as PCMark 8 tests the CPU, GPU and video subcomponents rather than emphasizing the CPU like SYSmark does.

     

    And just to drive the point home some more, AMD created their own in-house test using Microsoft Office Suite where they run a custom script on both notebooks and then measure the start and end of a task with a time stamp to gauge performance. Their in-house test shows the Intel system completing the task in approximately 61 seconds while the AMD based one finished it in roughly 64 seconds which is a delta of 6-7%.

     

    As AMD's results line up with those found with PCMark 8, they contend that PCMark 8 is a more fair and balanced indicator of overall performance while SYSmark is not and therefore should be discarded by any potential customers as a tool for evaluating a purchase. 

     

    amdinteltest.thumb.JPG.57191048d02a2763b

    Intel system using AMD in-house test: Test completed in ~61 seconds.

    amdinternaltest.thumb.JPG.b4b0576be9702f

    AMD system using AMD in-house test: Test completed in ~64 seconds.

     

    They also take the time to remind us that the FTC has required SYSmark published benchmarks to contain fine print that notes their benchmark may only be optimized for Intel. Of course, this may have something to do with the fact that AMD resigned from BAPco consortium. Finally, AMD notes that a transparent benchmark should meet very specific criteria which includes being realistic, unbiased, objective and transparent, all of which SYSmark is not apparently. 

     

    criteria.thumb.JPG.83070ca48ade93fb60255

     

    While it's fun to see these types of videos from AMD, they also highlight the fact that the company is facing an uphill battle on all fronts, whether it's CPU or GPU, and 2016 may be the year that makes or breaks them.

     

    Edited by Brian

    1 person likes this
    Sign in to follow this  
    Followers 0


    User Feedback


    There are no comments to display.



    Create an account or sign in to comment

    You need to be a member in order to leave a comment

    Create an account

    Sign up for a new account in our community. It's easy!


    Register a new account

    Sign in

    Already have an account? Sign in here.


    Sign In Now

  • Similar Content

    • By d3nver
      Hi,
       
      I would like to ask you about Intel Rapid Storage Technology, since few weeks I have a problem with my RAID0 ( 2x 512GB SSD disks ) after I shut down Windows, and power on the laptop, I have a message that RAID is damaged and one of the disks is missing. So i figured out that I'll check it in BIOS settings, and in BIOS everything is OK, 2 DISKs are visible, RAID status is NORMAL, if I run Windows after BIOS, then everything works perfectly, but if I run windows in 1st place, there is always raid error, and one of the disks is missing. Have you met a similar problem, do you have any idea how to fix it? I have to admit that I had reinstall Windows, and it's not helping ( Windows 7 and 10 were tested ).
    • By RyanKK
      I am currently using gtx 1060 3gb with mpcie and no hotplug (or something with that name) setup due to locked bios. I am tired of using old graphic driver as it seems to be not giving the best frames for new titles and my windows constantly trying to update my driver via windows update. even if i turn it manually, it constantly brings problem. I heard this isn't the case for AMD, but which one is better for mpcie gpu setup without hotplug capability?
    • By unrealmitch
      First, hello! Although I am new here, I have been a long time following this forum, and without all of you this hasn't been possible,so thank you!
       
      System: Hardware Dell 5430 14" Core i5 3320M 8GB RAM (1x1600Mhz) iGPU Intel HD 4000 Software Win 10 Pro 64-bit Versión 10.0.14393 [UEFI] GPU Driver AMD Radeon 17.2.1 (non-whql) EGPU MSI Radeon RX 480 GAMING X 8G BEST EXP GDC V8.4d [Express Card] PSU Deer 400W Configuration Just connect the egpu to PC via ExpressCard (PSU ON & PC Off), turn on the PC, and GPU was detected and it worked. [PTD: Off] Others: With 16.12.2 Drivers have some better performance, but my screen blinks sometimes Freesync works well This test have some OC to have similar performance that with whql drivers Video demo:    
      Hardware Info:  
      TEST & BenchMark 3DMark TimeSpy 1.0 3242 (GPU:3920) http://www.3dmark.com/3dm/18115631 3DMark FIRE STRIKE 1.1 7375 (GPU:12996) http://www.3dmark.com/3dm/18115814 Ungine Heaven 4.0 [Basic]: 3025 [Extreme]: 1495 Ungine Valley 1.0 [Extreme HD]: 1933 Games [FRAPS - 5Min ingame]: Game [1080] FPS AVG FPS Min FPS Max Config The Witcher 41.66 28 52 Ultra GTA V 23.66 13 38 Maximun Setting [Mod Cars] GTA V 32.2 20 72 High Skyrim 59 35 63 Ultra [Mod ENB Photorealistic]             Screenshots Bench:  
    • By 750dmg
      Hi guys,
      I succesfully flashed v2 vbios from prema and I tried 1v and and oc but I couldnt. I think it was power suply. Mine is 230w. And I tried 0.75 0.50 0.25 and 0 volt. But now I m trying game, benchmark screen goes dark and after beeping at 0 volt. Is this a hardware damage? I used just 1v max.(temperature max 70c)
      I tried
      cmos reset
      DDU
      Xtu uninstall
      And come back to stock vbios
      I think my card is dying
      Please help me!
       
    • By Sanjula
      Specs :-  Intel i5-2410m 2.3 GHz
                     8 GB ram
                     Intel hd 3000 (igpu)
                     AMD hd 6470m (dgpu)
                     Win 10
       
      # Use the mpcie slot for egpu
       
      GPU powers on and Epu setup 1.x recognized the graphics card. 
      Disabled the dgpu using setup 1.x. 
      Installed latest crimson drivers
      In device manager gpu shows with error 43. Disabled the egpu from device manager and re-enabled it again. Then it showed normally. 
       
      BUT THE PROBLEM IS I CANT CONNECT IT INTO A EXTERNAL DISPLAY. i USED THE DVI-SINGLE LINK PORT AND THE HDMI PORT BUT MY MONITOR DOESNT PICK UP THE SIGNAL.
       
      Help required !!!!
       
      01. First look at the Device manager.

       
       
      02. After re-enabling the gpu