next|prev|index

3. Current state of the compiler's test infrastructure

• Tests are run and driven by a set of makefiles
  • Modifying or extending them is hard and error-prone
  • Writing tests is hard
  • Developers are not encouraged to do it
• No clear distinction between test metadata and the code to run them
• Not much information (easily) available e.g. for failing tests:
  • No log of executed commands
  • Binary programs (program.byte / program.opt) lost
  • Test results are in a non-standard format -- they can not be easily integrated e.g. in a continuous integration system
  • Test reports have thus to be done with a dedicated tool (written in awk)