next up previous
Next: Conclusions Up: Recent Historical Trends Previous: Recent Historical Trends

The Role of the SPEC Benchmark Suites

As noted previously, the SPEC92 suite has the unfortunate characteristic of being largely cacheable in a 4 MB cache. Some vendors have responded by including 4 MB caches into their systems, rather than significantly increasing the memory bandwidth that larger computationally intensive applications require.

The new SGI/MIPS R10000 and HP PA-7200 and PA-8000 appear to be the beginnings of a deliberate counter-trend, with an advertising emphasis on improving ``real-world'' performance by larger factors than the improvement in SPECfp92 performance -- in other words, by improving memory bandwidth. It is not clear that these attempts will actually succeed in improving the machine balance, or if they will simply stabilize the trend for a processor generation or two.

Fortunately, the SPEC92 suite has been superseded by SPEC95, which is characterized by much larger job sizes. Few SPEC95 results are currently available, but the initial indications suggest that SPEC95's emphasis on larger jobs has resulted in a significant correlation of SPECfp95 results with memory bandwidth. An example is a comparison of the HP J-200 and HP-9000/755 (99 MHz version). These have approximate the same peak performance (200 and 198 MFLOPS, respectively). The J-200 has a significantly improved memory interface that results in double the sustainable memory bandwidth of the 755. The SPECfp92 ratio of the J-200 is 1.33 times that of the 755, while the SPECfp95 ratio is 1.57 times larger. Similar results can be seen in the IBM line, for which the extra memory bandwidth of the Power2-based systems corresponds into greater SPECfp95 ratios than would be expected based on peak performance or SPECfp92 results.

John McCalpin
Tue Aug 20 20:43:16 PDT 1996