about summary refs log tree commit diff
path: root/scripts
diff options
context:
space:
mode:
authorMaxim Kuvyrkov <maxim.kuvyrkov@linaro.org>2023-05-19 08:28:21 +0000
committerMaxim Kuvyrkov <maxim.kuvyrkov@linaro.org>2023-11-03 12:58:17 +0000
commit02976a4a4b2d01a524b33a508994664ffaf88d79 (patch)
treedfada326589381bb897fa649fb6285ac21be6583 /scripts
parent9fd3409842b3e2d31cff5dbd6f96066c430f0aa2 (diff)
downloadglibc-02976a4a4b2d01a524b33a508994664ffaf88d79.tar.gz
glibc-02976a4a4b2d01a524b33a508994664ffaf88d79.tar.xz
glibc-02976a4a4b2d01a524b33a508994664ffaf88d79.zip
Format test results closer to what DejaGnu does
The years of dealing with Binutils, GCC and GDB test results
made the community create good tools for comparison and analysis
of DejaGnu test results.  This change allows to use those tools
for Glibc's test results as well.

The motivation for this change is Linaro's pre-commit testers,
which use a modified version of GCC's validate_failures.py
to create test xfail lists with baseline failures and known
flaky tests.  See below links for an example xfails file (only
one link is supposed to work at any given time):
- https://ci.linaro.org/job/tcwg_glibc_check--master-arm-build/lastSuccessfulBuild/artifact/artifacts/artifacts.precommit/sumfiles/xfails.xfail/*view*/
- https://ci.linaro.org/job/tcwg_glibc_check--master-arm-build/lastSuccessfulBuild/artifact/artifacts/sumfiles/xfails.xfail/*view*/

Specifacally, this patch changes format of glibc's .sum files from ...
<cut>
FAIL: elf/test1
PASS: string/test2
</cut>
... to ...
<cut>
             === glibc tests ===

Running elf ...
FAIL: elf/test1

Running string ...
PASS: string/test2
</cut>.

And output of "make check" from ...
<cut>
FAIL: elf/test1
</cut>
... to ...
<cut>
FAIL: elf/test1
		=== Summary of results ===
      1 FAIL
      1 PASS
</cut>.

Signed-off-by: Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>
Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
Diffstat (limited to 'scripts')
-rwxr-xr-xscripts/merge-test-results.sh3
1 files changed, 3 insertions, 0 deletions
diff --git a/scripts/merge-test-results.sh b/scripts/merge-test-results.sh
index e4dcc2520a..8f9d81f6eb 100755
--- a/scripts/merge-test-results.sh
+++ b/scripts/merge-test-results.sh
@@ -50,7 +50,10 @@ case $type in
   -t)
     subdir_file_name=$1
     shift
+    echo "		=== glibc tests ==="
     for d in "$@"; do
+      echo
+      echo "Running $d ..."
       if [ -f "$objpfx$d/$subdir_file_name" ]; then
 	cat "$objpfx$d/$subdir_file_name"
       else