Commit graph

19 commits

Author SHA1 Message Date
Dustin J. Mitchell
dbeee0640a Add a hint to 'make test' see test problems 2022-12-18 12:01:12 -05:00
Tomas Babej
095d51e479
tests: Properly label and detect unexpected successes
Unexpected successes were being labeled as failures, which might be
related to [1]. This behaviour is indeed welcome - we want to detect if
tests are unexpectedly succeeding via test pipeline failures, however,
the current implementation of the simpletap did not properly implement
this.

[1] https://bugs.python.org/issue20165
2020-12-05 19:28:47 -05:00
Max Rossmannek
f35989be0e Enforce Python 3 for unittests
This commit updates all tests to enforce the Python3 executable. This is
necessary because the `assertRegex` function we use was renamed to this
name only in Python 3.2 [1]

For reference:
s;/usr/bin/env python;/usr/bin/env python3;g

[1]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertRegex
2020-11-30 00:51:54 -05:00
Paul Fenwick
138f6ffed3 test/problems: Updated to handle TAP spec output 2017-12-02 12:22:14 -05:00
Paul Beckingham
cfd997111b Tests: The 'run_all' script was not correctly using exit code 0 when all tests pass
- Thanks to Gordon Ball.
2016-02-06 12:08:31 -05:00
Daniel Shahaf
823e3d51d0 Test: Make 'make test' exit non-zero if tests failed 2015-11-07 17:45:55 -05:00
Paul Beckingham
66676539ce Test: Allow .exe in file test names 2015-09-15 22:07:18 -04:00
Tomas Babej
f22322bc18 problems: Return non-zero code when some tests fail 2015-09-11 07:50:37 -04:00
Renato Alves
1ad8aa9aec Test: Add unexpected successes to problems script 2015-07-28 16:57:51 +01:00
Renato Alves
ddccfa798c Test: More Python 3 compatibility changes in accessory files 2015-07-17 19:27:58 +01:00
Renato Alves
50fa772ce1 Test: problems no longer includes "expected failures" in skipped 2015-06-25 22:34:43 +01:00
Renato Alves
290674d7c1 Tests: problems script now outputs color and optional --summary
The problems script now outputs color on each of the test categories,
following the same rules used by simpletap.

It also now includes a --summary switch which outputs the same short
report seen when using ./run_all.
2015-06-25 18:57:56 +01:00
Renato Alves
ef477673d6 Tests: Convert problems to python 2015-06-09 17:12:16 +01:00
Renato Alves
1302e69d51 Expected failures are now treated as skipped tests
Before they were being treated as "passing" tests.
Since this might cause them to be silently ignored, classifying them
under "skipped" will ensure they won't go unnoticed.

Expected failures != skipped. The former will always be executed.
2015-05-25 18:15:31 +01:00
Renato Alves
53899296b1 List expected failures with test/problems 2015-05-25 17:57:00 +01:00
Paul Beckingham
11dab68fce Unit Tests
- Longer test script names need wider padding.
- Added counts for skipped tests.
2015-03-07 17:55:42 -05:00
Paul Beckingham
a1132f0028 Code Cleanup
- Cmake was not updating HAVE_ST_BIRTHTIME.
- NIBBLER_FEATURE_DATE was not properly applied everywhere.
- FEATURE_COLOR was not properly set.
- Some source files failed to include cmake.h, and therefore were not properly
- Removed inefficient use of std::string::substr for guaranteed single character
  strings.
- Integrated Directory::cd.
- Integrated File::ctime, ::btime.
- Integrated Path::operator+.
- Integrated Nibbler::getDigit{2,4,6}.
- Integrated HighResTimer.
  enabling/disabling code.
- All Path objects now expanded internally to absolute form.
- Modified unit tests to accomodate absolute paths.
- Merged new nibbler.t.cpp tests.
- Made various methods const.
- Includes removed from some files, added to others.
2013-05-05 08:33:52 -04:00
Owen Clarke
31769a1fa4 Portability
- Changed the interpreter of the test files to use /usr/bin/env to find
  perl in the user's path instead of hard-coding /usr/bin/perl
2012-07-30 18:49:50 +10:00
Paul Beckingham
5a2fba607e Unit Tests
- New 'problems' script to help identify problem areas in the test
  suite.
2011-08-11 18:25:10 -04:00