* fix path to task executable in pyton tests
The current approach would copy the current files into the `build/test`
directory. Updating the paths according to the custom user setup.
By the copy I appended `.py` to have a clear visible distingtion which
ones are the python tests.
As soon as a source file in the normal directory is changed, it is
copied over and the corresponding file is updated.
From now on the python tests would need to get run in the according
build directory.
* reflect the current build instruction in PR template
* update paths and globing in run_all
* add line break for every cpp test
* remove .gitignore in test folder
As now all the auxillary files such as `all.log` as well as the
executables are present in the `build` directory there is no longer a
need to ignore them.
* update paths in python test scripts and enable deactivated
* remove .py extension when copy to build
Further remove glob pattern for `*.t.py` tests.
* remove accidentally added template.t from test files
Per the TODO, use `__init__` to override the stream. There don't seem to
be any other substantial changes to the `run` method needed for TAP and
I'm assuming the additional code in the upstream `run` has been added
since this class was written.
My primary motivation was to set `result.buffer = self.buffer` in the
`run` method, which fixes the runner enough to be able to use
`breakpoint()`.
In addition, I updated the shebang in `tw-2189.t` for portability.
Unexpected successes were being labeled as failures, which might be
related to [1]. This behaviour is indeed welcome - we want to detect if
tests are unexpectedly succeeding via test pipeline failures, however,
the current implementation of the simpletap did not properly implement
this.
[1] https://bugs.python.org/issue20165
This allows tools like `prove` to operate on our test-suite, allowing us
to do things like run recently-failing tests first, randomising test
order, and fancier reporting options.
Changes made:
- TAP output always goes to STDOUT, never to STDERR
- Skipped tests signaled as `ok 1 # skip` as per standard
- Expected failures signaled as `not ok 1 # TODO` as per standard
- `.prove` added to .gitignore
In addition to the state of the test the description now includes the
location of the test, i.e. the filename that contains it.
Useful in central.tasktools.org.
Before they were being treated as "passing" tests.
Since this might cause them to be silently ignored, classifying them
under "skipped" will ensure they won't go unnoticed.
Expected failures != skipped. The former will always be executed.
