Tishers

As soon as any form of metrology (benchmark) starts to drift away from a reference standard it becomes impossible to draw comparisons from machine to machine or from one configuration to the next. Everything needs to be traceable to a reference piece of hardware that does not change; that means the reference computer does not get operating system updates, drivers changes to any device or even the environment it is operated in (we know that temperature really does affect performance). For any of these companies that make benchmarking systems they need to adopt some "reference box" and the scoring systems need to be compensated against that reference box of the software tool becomes meaningless. In electrical engineering test environments we use NIST traceable standards for everything from voltage, capacitance, resistance and timing. If you do not have a reliable reference then you are back in the days when "one foot" meant the size of the king's shoe.