Dear Colleagues;
I would like to get your help in reviewing the initial release of an
open source C++ Performance Benchmark.
These benchmarks will test compiler optimizations, language features,
common programming idioms, as well as some compiler runtime and OS
runtime support.
This is not SPEC or any sort of executive summary benchmark. This is
about details: what is the penalty for using ___, does my compiler
optimize ____, etc.
The primary goals of these benchmarks are:
1) To help compiler vendors identify places where they may be able to
improve the performance of the code they generate.
2) To help developers understand the performance impact of using different
data types, operations, and language features with their target compilers
and OSes.
The initial tests included in this release are:
loop_unroll.cpp
Test to see if compilers will correctly unroll loops to hide
instruction latency. Some compilers have problems expanding the
templates, and most compilers tested have problems correctly
unrolling the loops for best performance.
simple_types_loop_invariant.cpp
A test to see if the compiler will move loop invariant
calculations
out of the loop. Most compilers I tested have room for
improvement.
stepanov_abstraction.cpp
An expanded version of the original abstraction penalty test,
answering "what happens to performance when I wrap a value in
curly
braces?" Almost all compilers do well on the original summation
tests, but they don't do nearly so well on simple sort routines
using the same abstractions.
machine.cpp
A utility program to print out information about the compiler
version, OS, and machine environment.
You can download the source from
http://stlab.adobe.com/performance/downloads/C++Benchmarks.zip
I'll be posting updates and news on
http://stlab.adobe.com/performance/index.html
and blogging results at
http://blogs.adobe.com/c++performance/
Where I need help:
More eyes checking the code for correctness.
Double checking the optimization tests, to verify that they really
test what they claim to test and don't run afoul of other
optimizations. The tests may expose bugs other than what they
are intended to test, but that's ok (as long as you fix the bugs).
Help testing and improving the portability of the code and
makefiles.
Help identifying and reporting more information about platforms
and compilers in machine.cpp. This is important in trying to
differentiate reports from different builds, compiler options,
etc.
This will be even more important when running against a one-off
build of your own compiler on one of the 30 machines in your
testing lab.
What is still in progress:
More tests, of course. Suggestions are welcome.
Extracting conclusions from the raw numbers.
I want results from more platforms before nailing down
the comparisons.
Completing the web site with results, conclusions, forum,
documentation, references, etc.
A public perforce depot for the current release source and all
work
in progress.
Thank you for your assistance,
Chris Cox
Adobe Systems
(yeah, our internal netnews server is down...)
|
