about summary refs log tree commit diff
path: root/benchtests/scripts/compare_bench.py
Commit message (Collapse)AuthorAgeFilesLines
* Update copyright dates with scripts/update-copyrightsPaul Eggert2024-01-011-1/+1
|
* Update copyright dates with scripts/update-copyrightsJoseph Myers2023-01-061-1/+1
|
* Update copyright dates with scripts/update-copyrightsPaul Eggert2022-01-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | I used these shell commands: ../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright (cd ../glibc && git commit -am"[this commit message]") and then ignored the output, which consisted lines saying "FOO: warning: copyright statement not found" for each of 7061 files FOO. I then removed trailing white space from math/tgmath.h, support/tst-support-open-dev-null-range.c, and sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following obscure pre-commit check failure diagnostics from Savannah. I don't know why I run into these diagnostics whereas others evidently do not. remote: *** 912-#endif remote: *** 913: remote: *** 914- remote: *** error: lines with trailing whitespace found ... remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
* benchtests: Fix validate_benchout.py exceptionsNaohiro Tamura2021-09-161-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch fixed validate_benchout.py two exceptions, 1) AttributeError if benchout_strings.schema.json is specified, and 2) json.decoder.JSONDecodeError if benchout file is not JSON. $ ~/glibc/benchtests/scripts/validate_benchout.py bench-memset.out \ ~/glibc/benchtests/scripts/benchout_strings.schema.json Traceback (most recent call last): File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in <module> sys.exit(main(sys.argv[1:])) File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main bench.parse_bench(args[0], args[1]) File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 139, in parse_bench do_for_all_timings(bench, lambda b, f, v: File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 107, in do_for_all_timings if 'timings' not in bench['functions'][func][k].keys(): AttributeError: 'str' object has no attribute 'keys' $ ~/glibc/benchtests/scripts/validate_benchout.py bench-math-inlines.out \ ~/glibc/benchtests/scripts/benchout_strings.schema.json Traceback (most recent call last): File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in <module> sys.exit(main(sys.argv[1:])) File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main bench.parse_bench(args[0], args[1]) File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 137, in parse_bench bench = json.load(benchfile) File "/usr/lib/python3.6/json/__init__.py", line 299, in load parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) File "/usr/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/usr/lib/python3.6/json/decoder.py", line 342, in decode raise JSONDecodeError("Extra data", s, end) json.decoder.JSONDecodeError: Extra data: line 1 column 17 (char 16) Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
* Update copyright dates with scripts/update-copyrightsPaul Eggert2021-01-021-1/+1
| | | | | | | | | | | | | | | | I used these shell commands: ../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright (cd ../glibc && git commit -am"[this commit message]") and then ignored the output, which consisted lines saying "FOO: warning: copyright statement not found" for each of 6694 files FOO. I then removed trailing white space from benchtests/bench-pthread-locks.c and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this diagnostic from Savannah: remote: *** pre-commit check failed ... remote: *** error: lines with trailing whitespace found remote: error: hook declined to update refs/heads/master
* Update copyright dates with scripts/update-copyrights.Joseph Myers2020-01-011-1/+1
|
* Prefer https to http for gnu.org and fsf.org URLsPaul Eggert2019-09-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also, change sources.redhat.com to sourceware.org. This patch was automatically generated by running the following shell script, which uses GNU sed, and which avoids modifying files imported from upstream: sed -ri ' s,(http|ftp)(://(.*\.)?(gnu|fsf|sourceware)\.org($|[^.]|\.[^a-z])),https\2,g s,(http|ftp)(://(.*\.)?)sources\.redhat\.com($|[^.]|\.[^a-z]),https\2sourceware.org\4,g ' \ $(find $(git ls-files) -prune -type f \ ! -name '*.po' \ ! -name 'ChangeLog*' \ ! -path COPYING ! -path COPYING.LIB \ ! -path manual/fdl-1.3.texi ! -path manual/lgpl-2.1.texi \ ! -path manual/texinfo.tex ! -path scripts/config.guess \ ! -path scripts/config.sub ! -path scripts/install-sh \ ! -path scripts/mkinstalldirs ! -path scripts/move-if-change \ ! -path INSTALL ! -path locale/programs/charmap-kw.h \ ! -path po/libc.pot ! -path sysdeps/gnu/errlist.c \ ! '(' -name configure \ -execdir test -f configure.ac -o -f configure.in ';' ')' \ ! '(' -name preconfigure \ -execdir test -f preconfigure.ac ';' ')' \ -print) and then by running 'make dist-prepare' to regenerate files built from the altered files, and then executing the following to cleanup: chmod a+x sysdeps/unix/sysv/linux/riscv/configure # Omit irrelevant whitespace and comment-only changes, # perhaps from a slightly-different Autoconf version. git checkout -f \ sysdeps/csky/configure \ sysdeps/hppa/configure \ sysdeps/riscv/configure \ sysdeps/unix/sysv/linux/csky/configure # Omit changes that caused a pre-commit check to fail like this: # remote: *** error: sysdeps/powerpc/powerpc64/ppc-mcount.S: trailing lines git checkout -f \ sysdeps/powerpc/powerpc64/ppc-mcount.S \ sysdeps/unix/sysv/linux/s390/s390-64/syscall.S # Omit change that caused a pre-commit check to fail like this: # remote: *** error: sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S: last line does not end in newline git checkout -f sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S
* Update copyright dates with scripts/update-copyrights.Joseph Myers2019-01-011-1/+1
| | | | | | | * All files with FSF copyright notices: Update copyright dates using scripts/update-copyrights. * locale/programs/charmap-kw.h: Regenerated. * locale/programs/locfile-kw.h: Likewise.
* benchtests: send non-consumable data to stderrLeonardo Sandoval2018-12-121-3/+4
| | | | | | | | | | | Non-consumable data, alias data not related to benchmarks, should be sent to the standard error, thus pipelines can work as expected. * benchtests/scripts/compare_bench.py (do_compare): write to stderr in case stat is not present. * benchtests/scripts/compare_bench.py (plot_graphs): write to stderr in case timings field is not present. Also string showing the output filename goes into the stderr.
* benchtests: include --stats parameterLeonardo Sandoval2018-12-121-9/+18
| | | | | | | | | | | | | | Allows user to pick a statistic, defaulting to min and mean, from command line. At the same time, if stat does not exit, catch the run-time exception and keep comparing the rest of benchmarked functions. Finally, take care of division-by-zero exceptions and as the latter, keep comparing the rest of the functions, turning the script a bit more fault tolerant thus useful. * benchtests/scripts/compare_bench.py (do_compare): Catch KeyError and ZeroDivisorError exceptions. * benchtests/scripts/compare_bench.py (compare_runs): Use stats argument to loop through user provided statistics. * benchtests/scripts/compare_bench.py (main): Include the --stats argument.
* benchtests: keep comparing even if function timings do not matchLeonardo Sandoval2018-12-121-1/+1
| | | | | | | Allows other functions to be processed, making the script a bit more fault tolerant thus useful. * benchtests/scripts/compare_bench.py (compare_runs): Continue instead of return.
* benchtests: Set float type on --threshold argumentLeonardo Sandoval2018-10-081-1/+1
| | | | | | | | | | | Otherwise, we see the following runtime error when using the parameter: File "./glibc/benchtests/scripts/compare_bench.py", line 46, in do_compare if d > threshold: TypeError: '>' not supported between instances of 'float' and 'str' * benchtests/scripts/compare_bench.py (main): set float type on threshold argument.
* benchtests: improve argument parsing through argparse libraryLeonardo Sandoval2018-07-191-21/+19
| | | | | | | | | | | | | The argparse library is used on compare_bench script to improve command line argument parsing. The 'schema validation file' is now optional, reducing by one the number of required parameters. * benchtests/scripts/compare_bench.py (__main__): use the argparse library to improve command line parsing. (__main__): make schema file as optional parameter (--schema), defaulting to benchtests/scripts/benchout.schema.json. (main): move out of the parsing stuff to __main_  and leave it only as caller of main comparison functions.
* Update copyright dates with scripts/update-copyrights.Joseph Myers2018-01-011-1/+1
| | | | | | | * All files with FSF copyright notices: Update copyright dates using scripts/update-copyrights. * locale/programs/charmap-kw.h: Regenerated. * locale/programs/locfile-kw.h: Likewise.
* Update copyright dates with scripts/update-copyrights.Joseph Myers2017-01-011-1/+1
|
* Update copyright dates with scripts/update-copyrights.Joseph Myers2016-01-041-1/+1
|
* benchtest: script to compare two benchmarksSiddhesh Poyarekar2015-06-011-0/+184
This script is a sample implementation that uses import_bench to construct two benchmark objects and compare them. If detailed timing information is available (when one does `make DETAILED=1 bench`), it writes out graphs for all functions it benchmarks and prints significant differences in timings of the two benchmark runs. If detailed timing information is not available, it points out significant differences in aggregate times. Call this script as follows: compare_bench.py schema_file.json bench1.out bench2.out Alternatively, if one wants to set a different threshold for warnings (default is a 10% difference): compare_bench.py schema_file.json bench1.out bench2.out 25 The threshold in the example above is 25%. schema_file.json is the JSON schema (which is $srcdir/benchtests/scripts/benchout.schema.json for the benchmark output file) and bench1.out and bench2.out are the two benchmark output files to compare. The key functionality here is the compress_timings function which groups together points that are close together into a single point that is the mean of all its representative points. Any point in such a group is at most 1.5x the smallest point in that group. The detailed derivation is a comment in the function. * benchtests/scripts/compare_bench.py: New file. * benchtests/scripts/import_bench.py (mean): New function. (split_list): Likewise. (do_for_all_timings): Likewise. (compress_timings): Likewise.