Control Software and Benchmarking

In the analysis of numerical methods and their implementation as numerical software it is extremely important to be able to test the correctness of the implementation as well as the performance of the method. This validation is one of the major steps in the construction of a software library, in particular if this library is used in practical applications.

In order to carry out such a test it is important to have tools that yield an evaluation of the performance of the method as well as the implementation with respect to correctness, accuracy and speed. Similar tools are needed to be able to compare different numerical methods, to test their robustness, and also to analyse the behaviour of the methods in extreme situation, i.e., on problems where the limit of the possible accuracy is reached.

In many application areas therefore benchmark collections have been created that can partially serve for this purpose. Such collections are heavily used. In our opinion, in order to have a fair evaluation and a comparison of methods and software, there should be a standardized set of examples on which newly developed methods and their implementation can be tested. It was one of the goals of WGS to create such testing and validation environments for the area of numerical methods in control, in particular it was planned to accompany the SLICOT library with benchmark collections for each of the major problem areas. In order to make such collections useful, it is important that they cover a wide range of problems and also problems difficult to solve in finite arithmetic are included. Such problems in particular drive the methods and their implementation to a limit. These are ideal test cases, since errors and failures usually occur only in extreme cases and these are often not covered by standard software validation procedures.

The SLICOT library currently contains such benchmark collections for:
- standard and generalized continuous-time and discrete-time systems models (see reports 2 and 3)
- continuous-time and discrete-time standard and generalized Lyapunov matrix equations (see reports 4 and 5)
- continuous-time and discrete-time algebraic Riccati matrix equations (see reports 6 and 7)
- identification (see report 8)
- model reduction of (high order) linear time invariant dynamical systems containing some useful "real world" examples reflecting current problems in applications can be found here (see also report 9)

Everybody is invited to submit such benchmark examples for the SLICOT benchmark collection. For submissions please contact This email address is being protected from spambots. You need JavaScript enabled to view it.

Available Reports

1. Volker Mehrmann and Thilo Penzl: Benchmark collections in SLICOT SLICOT Working Note 1998-5, June 1998.
2. Daniel Kressner, Volker Mehrmann and Thilo Penzl:  DTDSX - A collection of benchmark examples for state-space realizations of time-invariant discrete-time systems; SLICOT Working Note 1998-10: November 1998, revised June 1999.
3. Daniel Kressner, Volker Mehrmann and Thilo Penzl: CTDSX - A collection of benchmark examples for state-space realizations of time-invariant continuous-time systems; SLICOT Working Note 1998-9: November 1998.
4. Daniel Kressner, Volker Mehrmann and Thilo Penzl: DTLEX - A collection of benchmark examples for discrete-time Lyapunuv equations; SLICOT Working Note 1999-7: June 1999.
5. Daniel Kressner, Volker Mehrmann and Thilo Penzl: CTLEX - A collection of benchmark examples for continuous-time Lyapunuv equations; SLICOT Working Note 1999-6: June 1999.
6. Jörn Abels and Peter Benner: DAREX - A collection of benchmark examples for discrete-time algebraic Riccati equations (version 2.0); SLICOT Working Note 1999-16: December 1999.
7. Jörn Abels and Peter Benner: CAREX - A collection of benchmark examples for continuous-time algebraic Riccati equations (version 2.0); SLICOT Working Note 1999-14: December 1999.
8. Ad van den Boom, Ton Backx and Yucai Zhu: Benchmarks for identification; NICONET Report 1999-19: July 2000.
9. Younès Chahlaoui and Paul Van Dooren: A collection of benchmark examples for model reduction of linear time invariant dynamical systems; SLICOT Working Note 2002-2: February 2002.

Some Matrix Collections Suitable for Benchmarking:

• The Matrix Computation Toolbox (Nick Higham)

Ad van den Boom, March 4, 2002
Updated: Vasile Sima, August 31, 2005; June 15, 2006; June 8, 2009