summary refs log tree commit diff
path: root/Rules
diff options
context:
space:
mode:
authorJoseph Myers <joseph@codesourcery.com>2014-03-07 03:25:57 +0000
committerJoseph Myers <joseph@codesourcery.com>2014-03-07 03:25:57 +0000
commit265d52abcb4181e06cf5a60465b0a9514ecc4be9 (patch)
tree9c4c35e0dd670733d1f4b13c21d48e05b51c97ca /Rules
parent54ee292da7a65acf72ac5655777555786899119c (diff)
downloadglibc-265d52abcb4181e06cf5a60465b0a9514ecc4be9.tar.gz
glibc-265d52abcb4181e06cf5a60465b0a9514ecc4be9.tar.xz
glibc-265d52abcb4181e06cf5a60465b0a9514ecc4be9.zip
Generate overall summary of test results.
This patch, an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00197.html>, makes
testsuite runs generate an overall summary of test results.

A new script merge-test-results.sh deals both with collecting results
within a directory to a file with all the results from that directory,
and collecting the results from subdirectories into a single overall
file (there's not much in common between the two modes of operation of
the script, but it seemed silly to have two separate scripts for
this).  Within a directory, missing results produce UNRESOLVED lines;
at top level, missing results for a whole directory produce an ERROR
line (since toplevel can't identify what the specific missing tests
are in this case).

Note that this does not change the rules for when "make" considers
there has been an error, or terminates, so unexpected failures will
still cause make to terminate, or, with -k, mean the commands for
"tests" don't get run because of failure of a dependency.

Tested x86_64, including that the summary does in fact reflect all the
tests with .test-result files.

	* scripts/merge-test-results.sh: New file.
	* Makefile (tests-special-notdir): New variable.
	(tests): Run merge-test-results.sh.
	(xtests): Likewise.
	* Rules (tests-special-notdir): New variable.
	(xtests-special-notdir): Likewise.
	(tests): Run merge-test-results.sh
	(xtests): Likewise.
Diffstat (limited to 'Rules')
-rw-r--r--Rules11
1 files changed, 11 insertions, 0 deletions
diff --git a/Rules b/Rules
index d37ae52b6c..feb304dbc0 100644
--- a/Rules
+++ b/Rules
@@ -102,6 +102,17 @@ tests: $(tests:%=$(objpfx)%.out) $(tests-special)
 xtests: tests $(xtests:%=$(objpfx)%.out) $(xtests-special)
 endif
 
+tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
+xtests-special-notdir = $(patsubst $(objpfx)%, %, $(xtests-special))
+tests:
+	$(..)scripts/merge-test-results.sh -s $(objpfx) $(subdir) \
+	  $(sort $(tests) $(tests-special-notdir:.out=)) \
+	  > $(objpfx)subdir-tests.sum
+xtests:
+	$(..)scripts/merge-test-results.sh -s $(objpfx) $(subdir) \
+	  $(sort $(xtests) $(xtests-special-notdir:.out=)) \
+	  > $(objpfx)subdir-xtests.sum
+
 ifeq ($(build-programs),yes)
 binaries-all-notests = $(others) $(sysdep-others)
 binaries-all-tests = $(tests) $(xtests) $(test-srcs)