322 lines
6.2 KiB
ReStructuredText
322 lines
6.2 KiB
ReStructuredText
|
=====================
|
||
|
|
Test-Suite Extensions
|
||
|
|
=====================
|
||
|
|
|
||
|
|
.. contents::
|
||
|
|
:depth: 1
|
||
|
|
:local:
|
||
|
|
|
||
|
|
Abstract
|
||
|
|
========
|
||
|
|
|
||
|
|
These are ideas for additional programs, benchmarks, applications and
|
||
|
|
algorithms that could be added to the LLVM Test-Suite.
|
||
|
|
The test-suite could be much larger than it is now, which would help us
|
||
|
|
detecting compiler errors (crashes, miscompiles) during development.
|
||
|
|
|
||
|
|
Most probably, the reason why the programs below have not been added to
|
||
|
|
the test-suite yet is that nobody has found time to do it. But there
|
||
|
|
might be other issues as well, such as
|
||
|
|
|
||
|
|
* Licensing (Support can still be added as external module,
|
||
|
|
like for the SPEC benchmarks)
|
||
|
|
|
||
|
|
* Language (in particular, there is no official LLVM frontend
|
||
|
|
for FORTRAN yet)
|
||
|
|
|
||
|
|
* Parallelism (currently, all programs in test-suite use
|
||
|
|
one thread only)
|
||
|
|
|
||
|
|
Benchmarks
|
||
|
|
==========
|
||
|
|
|
||
|
|
SPEC CPU 2017
|
||
|
|
-------------
|
||
|
|
https://www.spec.org/cpu2017/
|
||
|
|
|
||
|
|
The following have not been included yet because they contain Fortran
|
||
|
|
code.
|
||
|
|
|
||
|
|
In case of cactuBSSN only a small portion is Fortran. The hosts's
|
||
|
|
Fortran compiler could be used for these parts.
|
||
|
|
|
||
|
|
Note that CMake's Ninja generator has difficulties with Fortran. See the
|
||
|
|
`CMake documentation <https://cmake.org/cmake/help/v3.13/generator/Ninja.html#fortran-support>`_
|
||
|
|
for details.
|
||
|
|
|
||
|
|
* 503.bwaves_r/603.bwaves_s
|
||
|
|
* 507.cactuBSSN_r
|
||
|
|
* 521.wrf_r/621.wrf_s
|
||
|
|
* 527.cam4_r/627.cam4_s
|
||
|
|
* 628.pop2_s
|
||
|
|
* 548.exchange2_r/648.exchange2_s
|
||
|
|
* 549.fotonik3d_r/649.fotonik3d_s
|
||
|
|
* 554.roms_r/654.roms_s
|
||
|
|
|
||
|
|
SPEC OMP2012
|
||
|
|
------------
|
||
|
|
https://www.spec.org/omp2012/
|
||
|
|
|
||
|
|
* 350.md
|
||
|
|
* 351.bwaves
|
||
|
|
* 352.nab
|
||
|
|
* 357.bt331
|
||
|
|
* 358.botsalgn
|
||
|
|
* 359.botsspar
|
||
|
|
* 360.ilbdc
|
||
|
|
* 362.fma3d
|
||
|
|
* 363.swim
|
||
|
|
* 367.imagick
|
||
|
|
* 370.mgrid331
|
||
|
|
* 371.applu331
|
||
|
|
* 372.smithwa
|
||
|
|
* 376.kdtree
|
||
|
|
|
||
|
|
OpenCV
|
||
|
|
------
|
||
|
|
https://opencv.org/
|
||
|
|
|
||
|
|
OpenMP 4.x SIMD Benchmarks
|
||
|
|
--------------------------
|
||
|
|
https://github.com/flwende/simd_benchmarks
|
||
|
|
|
||
|
|
PWM-benchmarking
|
||
|
|
----------------
|
||
|
|
https://github.com/tbepler/PWM-benchmarking
|
||
|
|
|
||
|
|
SLAMBench
|
||
|
|
---------
|
||
|
|
https://github.com/pamela-project/slambench
|
||
|
|
|
||
|
|
FireHose
|
||
|
|
--------
|
||
|
|
http://firehose.sandia.gov/
|
||
|
|
|
||
|
|
A Benchmark for the C/C++ Standard Library
|
||
|
|
------------------------------------------
|
||
|
|
https://github.com/hiraditya/std-benchmark
|
||
|
|
|
||
|
|
OpenBenchmarking.org CPU / Processor Suite
|
||
|
|
------------------------------------------
|
||
|
|
https://openbenchmarking.org/suite/pts/cpu
|
||
|
|
|
||
|
|
This is a subset of the
|
||
|
|
`Phoronix Test Suite <https://github.com/phoronix-test-suite/phoronix-test-suite/>`_
|
||
|
|
and is itself a collection of benchmark suites
|
||
|
|
|
||
|
|
Parboil Benchmarks
|
||
|
|
------------------
|
||
|
|
http://impact.crhc.illinois.edu/parboil/parboil.aspx
|
||
|
|
|
||
|
|
MachSuite
|
||
|
|
---------
|
||
|
|
https://breagen.github.io/MachSuite/
|
||
|
|
|
||
|
|
Rodinia
|
||
|
|
-------
|
||
|
|
http://lava.cs.virginia.edu/Rodinia/download_links.htm
|
||
|
|
|
||
|
|
Rodinia has already been partially included in
|
||
|
|
MultiSource/Benchmarks/Rodinia. Benchmarks still missing are:
|
||
|
|
|
||
|
|
* streamcluster
|
||
|
|
* particlefilter
|
||
|
|
* nw
|
||
|
|
* nn
|
||
|
|
* myocyte
|
||
|
|
* mummergpu
|
||
|
|
* lud
|
||
|
|
* leukocyte
|
||
|
|
* lavaMD
|
||
|
|
* kmeans
|
||
|
|
* hotspot3D
|
||
|
|
* heartwall
|
||
|
|
* cfd
|
||
|
|
* bfs
|
||
|
|
* b+tree
|
||
|
|
|
||
|
|
vecmathlib tests harness
|
||
|
|
------------------------
|
||
|
|
https://bitbucket.org/eschnett/vecmathlib/wiki/Home
|
||
|
|
|
||
|
|
PARSEC
|
||
|
|
------
|
||
|
|
http://parsec.cs.princeton.edu/
|
||
|
|
|
||
|
|
Graph500 reference implementations
|
||
|
|
----------------------------------
|
||
|
|
https://github.com/graph500/graph500/tree/v2-spec
|
||
|
|
|
||
|
|
NAS Parallel Benchmarks
|
||
|
|
-----------------------
|
||
|
|
https://www.nas.nasa.gov/publications/npb.html
|
||
|
|
|
||
|
|
The official benchmark is written in Fortran, but an unofficial
|
||
|
|
C-translation is available as well:
|
||
|
|
https://github.com/benchmark-subsetting/NPB3.0-omp-C
|
||
|
|
|
||
|
|
DARPA HPCS SSCA#2 C/OpenMP reference implementation
|
||
|
|
---------------------------------------------------
|
||
|
|
http://www.highproductivity.org/SSCABmks.htm
|
||
|
|
|
||
|
|
This web site does not exist any more, but there seems to be a copy of
|
||
|
|
some of the benchmarks
|
||
|
|
https://github.com/gtcasl/hpc-benchmarks/tree/master/SSCA2v2.2
|
||
|
|
|
||
|
|
Kokkos
|
||
|
|
------
|
||
|
|
https://github.com/kokkos/kokkos-kernels/tree/master/perf_test
|
||
|
|
https://github.com/kokkos/kokkos/tree/master/benchmarks
|
||
|
|
|
||
|
|
PolyMage
|
||
|
|
--------
|
||
|
|
https://github.com/bondhugula/polymage-benchmarks
|
||
|
|
|
||
|
|
PolyBench
|
||
|
|
---------
|
||
|
|
https://sourceforge.net/projects/polybench/
|
||
|
|
|
||
|
|
A modified version of Polybench 3.2 is already presented in
|
||
|
|
SingleSource/Benchmarks/Polybench. A newer version 4.2.1 is available.
|
||
|
|
|
||
|
|
High Performance Geometric Multigrid
|
||
|
|
------------------------------------
|
||
|
|
https://crd.lbl.gov/departments/computer-science/PAR/research/hpgmg/
|
||
|
|
|
||
|
|
RAJA Performance Suite
|
||
|
|
----------------------
|
||
|
|
https://github.com/LLNL/RAJAPerf
|
||
|
|
|
||
|
|
CORAL-2 Benchmarks
|
||
|
|
------------------
|
||
|
|
https://asc.llnl.gov/coral-2-benchmarks/
|
||
|
|
|
||
|
|
Many of its programs have already been integrated in
|
||
|
|
MultiSource/Benchmarks/DOE-ProxyApps-C and
|
||
|
|
MultiSource/Benchmarks/DOE-ProxyApps-C++.
|
||
|
|
|
||
|
|
* Nekbone
|
||
|
|
* QMCPack
|
||
|
|
* LAMMPS
|
||
|
|
* Kripke
|
||
|
|
* Quicksilver
|
||
|
|
* PENNANT
|
||
|
|
* Big Data Analytic Suite
|
||
|
|
* Deep Learning Suite
|
||
|
|
* Stream
|
||
|
|
* Stride
|
||
|
|
* ML/DL micro-benchmark
|
||
|
|
* Pynamic
|
||
|
|
* ACME
|
||
|
|
* VPIC
|
||
|
|
* Laghos
|
||
|
|
* Parallel Integer Sort
|
||
|
|
* Havoq
|
||
|
|
|
||
|
|
NWChem
|
||
|
|
------
|
||
|
|
http://www.nwchem-sw.org/index.php/Benchmarks
|
||
|
|
|
||
|
|
TVM
|
||
|
|
----
|
||
|
|
https://github.com/dmlc/tvm/tree/master/apps/benchmark
|
||
|
|
|
||
|
|
HydroBench
|
||
|
|
----------
|
||
|
|
https://github.com/HydroBench/Hydro
|
||
|
|
|
||
|
|
ParRes
|
||
|
|
------
|
||
|
|
https://github.com/ParRes/Kernels/tree/master/Cxx11
|
||
|
|
|
||
|
|
Applications/Libraries
|
||
|
|
======================
|
||
|
|
|
||
|
|
GnuPG
|
||
|
|
-----
|
||
|
|
https://gnupg.org/
|
||
|
|
|
||
|
|
Blitz++
|
||
|
|
-------
|
||
|
|
https://sourceforge.net/projects/blitz/
|
||
|
|
|
||
|
|
FFmpeg
|
||
|
|
------
|
||
|
|
https://ffmpeg.org/
|
||
|
|
|
||
|
|
FreePOOMA
|
||
|
|
---------
|
||
|
|
http://www.nongnu.org/freepooma/
|
||
|
|
|
||
|
|
FTensors
|
||
|
|
--------
|
||
|
|
http://www.wlandry.net/Projects/FTensor
|
||
|
|
|
||
|
|
rawspeed
|
||
|
|
--------
|
||
|
|
https://github.com/darktable-org/rawspeed
|
||
|
|
|
||
|
|
Its test dataset is 756 MB in size, which is too large to be included
|
||
|
|
into the test-suite repository.
|
||
|
|
|
||
|
|
C++ Performance Benchmarks
|
||
|
|
--------------------------
|
||
|
|
https://gitlab.com/chriscox/CppPerformanceBenchmarks
|
||
|
|
|
||
|
|
Generic Algorithms
|
||
|
|
==================
|
||
|
|
|
||
|
|
Image processing
|
||
|
|
----------------
|
||
|
|
|
||
|
|
Resampling
|
||
|
|
``````````
|
||
|
|
|
||
|
|
* Bilinear
|
||
|
|
* Bicubic
|
||
|
|
* Lanczos
|
||
|
|
|
||
|
|
Dither
|
||
|
|
``````
|
||
|
|
|
||
|
|
* Threshold
|
||
|
|
* Random
|
||
|
|
* Halftone
|
||
|
|
* Bayer
|
||
|
|
* Floyd-Steinberg
|
||
|
|
* Jarvis
|
||
|
|
* Stucki
|
||
|
|
* Burkes
|
||
|
|
* Sierra
|
||
|
|
* Atkinson
|
||
|
|
* Gradient-based
|
||
|
|
|
||
|
|
Feature detection
|
||
|
|
`````````````````
|
||
|
|
|
||
|
|
* Harris
|
||
|
|
* Histogram of Oriented Gradients
|
||
|
|
|
||
|
|
Color conversion
|
||
|
|
````````````````
|
||
|
|
|
||
|
|
* RGB to grayscale
|
||
|
|
* HSL to RGB
|
||
|
|
|
||
|
|
Graph
|
||
|
|
-----
|
||
|
|
|
||
|
|
Search Algorithms
|
||
|
|
`````````````````
|
||
|
|
|
||
|
|
* Breadth-First-Search
|
||
|
|
* Depth-First-Search
|
||
|
|
* Dijkstra's algorithm
|
||
|
|
* A-Star
|
||
|
|
|
||
|
|
Spanning Tree
|
||
|
|
`````````````
|
||
|
|
|
||
|
|
* Kruskal's algorithm
|
||
|
|
* Prim's algorithm
|