mirror of
https://github.com/GothenburgBitFactory/taskwarrior.git
synced 2025-06-26 10:54:26 +02:00
4f6c51e7ae
- Updated README with suggested work. - Removed obsolete TAP.py. - Added 'conversion' script to check on the Perl -> Python progress.
110 lines
3.6 KiB
Text
110 lines
3.6 KiB
Text
README
|
|
======
|
|
|
|
This is the task.git/test/README file, and contains notes about the Taskwarrior
|
|
test suite.
|
|
|
|
|
|
Running Tests
|
|
-------------
|
|
|
|
TL;DR cd test && make && ./run_all && ./problems
|
|
|
|
All unit tests produce TAP (Test Anything Protocol) output, and are run by the
|
|
'run_all' test harness.
|
|
|
|
The 'run_all' script produces an 'all.log' file which is the accumulated output
|
|
of all tests. Before executing 'run_all' you need to compile the C++ unit
|
|
tests, by running 'make' on the 'test' directory.
|
|
|
|
The script 'problems' will list all the tests that fail, with a count of the
|
|
failing tests.
|
|
|
|
Any TAP harness may be used.
|
|
|
|
|
|
Architecture
|
|
------------
|
|
|
|
There are three varieties of tests:
|
|
|
|
* Perl unit tests that use Test::More and the JSON module. We are phasing
|
|
these out, and will accept no new Perl tests. These tests are high level
|
|
and exercise Taskwarrior at the command line level.
|
|
|
|
* C++ unit tests that test low-level object interfaces. These are typically
|
|
very fast tests, and are exhaustive in nature.
|
|
|
|
* Python unit tests that are at the highest level, exercising the command
|
|
line, hooks and syncing. There is an example, 'template.t', that shows how
|
|
to perform various high level tests.
|
|
|
|
All tests are named with the pattern '*.t', and any other forms are not run by
|
|
the test harness. Additionally a test must be set executable (chmod +x) for it
|
|
to be run. In the case of Perl and Python tests one can still run them manually
|
|
by launching them with 'perl/python test.t'. It also allows us to keep tests
|
|
submitted for bugs that are not scheduled to be fixed in the upcoming release,
|
|
and we don't want the failing tests to prevent us from seeing 100% pass rate
|
|
for the bugs we *have* fixed.
|
|
|
|
|
|
Goals
|
|
-----
|
|
|
|
The test suite is evolving, and becoming a better tool for determining whether
|
|
code is ready for release. There are goals that shape these changes, and they
|
|
are:
|
|
|
|
* Migrate test suite to Python and C++, eliminating all Perl. The Python
|
|
test suite is more expressive and high level. Migrating reduces
|
|
dependencies.
|
|
|
|
* Increase test coverage by testing more features, more thoroughly.
|
|
|
|
* Write fewer bug regression tests. Over time, bug regression tests are less
|
|
useful than feature tests, and more likely to contain overlapping coverage.
|
|
|
|
* The Python test suite provides test isolation, such that each test is run
|
|
in a separate directory. This will allow eventual parallelization, but not
|
|
until the Perl tests are eliminated.
|
|
|
|
* Eliminate obsolete tests, which are tests that have overlapping coverage.
|
|
This means migrate bug-specific tests to feature tests.
|
|
|
|
* Categorize the tests, restructure the directories.
|
|
|
|
|
|
What Makes a Good Test
|
|
----------------------
|
|
|
|
A good test ensures that a feature is functioning as expected, and contains
|
|
both positive and negative aspects, or in other words looks for expected
|
|
behavior as well as looking for the absence of unexpected behavior.
|
|
|
|
|
|
How to Submit a Test Change/Addition
|
|
------------------------------------
|
|
|
|
Mail it to support@taskwarrior.org, or attach it to an open bug.
|
|
|
|
|
|
TODO
|
|
----
|
|
|
|
For anyone looking for test-related tasks to take on, here are some suggestions:
|
|
|
|
* Take tw-285.t and improve it to test more (if not all) virtual tags, then
|
|
rename it virtual-tags.t.
|
|
|
|
* Select a bug.*.t Perl test and convert it to Python using the template.
|
|
|
|
* Look at the latest todo.txt file format spec, and make sure that
|
|
import.todo.sh.t is testing the current format.
|
|
|
|
* Select a feature.*.t Perl test, convert it to Python using the template,
|
|
then rename it to <feature>.t
|
|
|
|
* Find and eliminate individual test that do the same thing.
|
|
|
|
|
|
---
|