From 560f41a930073cba3dd514420877f17191382fda Mon Sep 17 00:00:00 2001 From: Paul Beckingham Date: Fri, 4 Jul 2014 10:24:31 -0400 Subject: [PATCH] Documentation - Updated developer documentation for 2.4.0. --- DEVELOPER | 117 ++++++++++++++++++++++++++--------------------------- EXPOSITION | 98 ++++++++++++++++++++++---------------------- 2 files changed, 106 insertions(+), 109 deletions(-) diff --git a/DEVELOPER b/DEVELOPER index 46e0078e2..6e3c2bba2 100644 --- a/DEVELOPER +++ b/DEVELOPER @@ -4,7 +4,7 @@ How to Build $ git clone https://git.tasktools.org/scm/tm/task.git task.git $ cd task.git $ git checkout 2.4.0 # Dev branch - $ cmake -DCMAKE_BUILD_TYPE=debug . # debug or release. Default: neither. + $ cmake -DCMAKE_BUILD_TYPE=debug . # debug or release. Default: neither. $ make VERBOSE=1 # Shows details Running Test Suite: @@ -14,67 +14,72 @@ How to Build # Install vramsteg for blinkenlights $ ./problems # Find errors in all.log + Note that any development should be performed using a git clone, and the + current development branch. The source tarballs do not reflect HEAD, and do + not contain the test suite. + General Statement This file is intended to convey the current efforts, priorities and needs of - the codebase. It is for anyone looking for a way to start contributing. + the codebase. It is for anyone looking for a way to start contributing. While this is biased towards developers, anyone can contribute, and everyone - is encouraged to do so. Here are many ways to contribute that may not be + is encouraged to do so. Here are many ways to contribute that may not be immediately obvious to you: - - Use Taskwarrior, become familiar with it, and make suggestions. There are + - Use Taskwarrior, become familiar with it, and make suggestions. There are always ongoing discussions about new features and changes to existing - features. Join us in the forums. + features. - - Join us in the #taskwarrior IRC channel on freenode.net. Some great ideas, - suggestions, testing and discussions have taken place there. It is also + - Join us in the #taskwarrior IRC channel on freenode.net. Some great ideas, + suggestions, testing and discussions have taken place there. It is also the quickest way to get help, or confirm a bug. - Join https://answers.tasktools.org and help us by asking, answering and voting on questions and answers, directly helping those who ask, and - indirectly helping those who search for existing answers. + helping future users who search for existing answers. - Review documentation: there are man pages, online articles, tutorials and so on, and these may contain errors, or they may not convey ideas in the - best way. Perhaps you can help improve it. Contact us - documentation is + best way. Perhaps you can help improve it. Contact us - documentation is a separate effort from the codebase, and includes all web sites. - - Take a look at the bug database, and help triage the bug list. This is a + - Take a look at the bug database, and help triage the bug list. This is a review process that involves confirming bugs, providing additional data, - information or analysis. Bug triage is very useful and much needed. + information or analysis. Bug triage is very useful and much needed. - - Fix a bug. For this you'll need C++ and Git skills, but this is one of - the largest ways you can contribute. We welcome all bug fixes, provided + - Fix a bug. For this you'll need C++ and Git skills, but this is one of + the largest ways you can contribute. We welcome all bug fixes, provided the work is done well and doesn't create other problems or introduce new - dependencies. We recommend talking to us before embarking on this quest. + dependencies. We recommend talking to us before starting. - - Add unit tests. Unit tests are possibly the most useful contributions of + - Add unit tests. Unit tests are possibly the most useful contributions of all, because they not only improve the quality of the code, but prevent future regressions, therefore maintaining quality of subsequent releases. Plus, broken tests are a great motivator for us to fix the causal bug. + You'll need Python skills, as we are migrating from Perl to Python for our + test suite. - - Add a feature. Well, let's be very clear about this: adding a feature is + - Add a feature. Well, let's be very clear about this: adding a feature is not usually well-received, and if you add a feature and send a patch, it - will most likely be rejected. The reason for this is that there are many - efforts under way, in various code branches. There is a very good chance + will most likely be rejected. The reason for this is that there are many + efforts under way, in various code branches. There is a very good chance that the feature you add is either already in progress, or being done in a - way that is more fitting when considering other work in progress. So if - you want to add a feature, please don't. Start by talking to us, and find - out what is currently under way or planned. You might find that we've - already rejected such a feature for some very good reasons. So please + way that is more fitting when considering other work in progress. So if + you want to add a feature, please don't. Start by talking to us, and find + out what is currently under way or planned. You might find that we've + already rejected such a feature for some very good reasons. So please check first, so we don't duplicate effort or waste anyone's time. - Donate some server time to the testing effort, by participating in the - continuous integration of all changes, with our Flod software. See this - page for an example: http://tasktools.org/tinderbox - Ask us about running a Flod satellite, we're always looking for exotic - platforms to test on. + continuous integration of all changes, with our Flod software. See this + page for an example: http://central.tasktools.org. Ask us about running a + Flod satellite, we're always looking for exotic platforms to test on. - - Spread the word. Help others become more effective at managing tasks. + - Spread the word. Help others become more effective at managing tasks. - - Encouragement. Tell us what works for you, and what doesn't. It's all - good. + - Encouragement. Tell us what works for you, and what doesn't. Tell us about + your methodology for managing tasks. It's all good. - - Request a feature. This not only tells us that you think something is + - Request a feature. This not only tells us that you think something is missing from the software, but gives us insights into how you use it. Plus, you might get your feature implemented. @@ -82,11 +87,11 @@ General Statement Deprecated Code This is code that is going to be phased out soon, and therefore is not worth - fixing or documenting. Don't waste your time. + fixing or documenting. Don't waste your time. - - Shadow file support in core. It will migrate to become an external script. + - Shadow file support in core. It will migrate to become an external script. - - Priorities in core. This will be migrated to become a UDA as soon as we + - Priorities in core. This will be migrated to become a UDA as soon as we have the right support in place for custom sorting. - Attribute modifiers. @@ -95,14 +100,14 @@ New Code Needs This is code that needs to be written, usually down at the C++ function/method level. - - Need export_viz.yy script. Any language. This would have value as an + - Need export_viz.yy script. Any language. This would have value as an example, or template script serving as a starting-point for anyone who needed this format. - - Need new export_xxx.yy scripts - the more the better. Any language. + - Need new export_xxx.yy scripts - the more the better. Any language. - The JSON.{h,cpp} code is fast and efficient up to a point, but has a non- - linear performance curve, implying a slowdown due to fragmentation. Find + linear performance curve, implying a slowdown due to fragmentation. Find it, fix it. - - Need a function that can optimize color codes. For example, if a string + - Need a function that can optimize color codes. For example, if a string contains this: one two three four It can be shortened to: @@ -110,8 +115,8 @@ New Code Needs This is only possible if there is nothing between and and the colors match. - Need an external script that can locate and correct duplicate UUIDs in the - data file, as found by 'task diag'. This should check to see if there is - a suitable UUID generator installed. This should also be careful to + data file, as found by 'task diag'. This should check to see if there is + a suitable UUID generator installed. This should also be careful to properly handle recurring tasks. - Take a look at: @@ -122,25 +127,25 @@ New Code Needs by anyone who wishes to participate. Unit Tests Needed - There are always more unit tests needed. More specifically, better unit tests - are always needed. The convention is that there are four types of unit test: + There are always more unit tests needed. More specifically, better unit tests + are always needed. The convention is that there are four types of unit test: 1. High level tests that exercise large features, or combinations of commands. For example, dependencies.t runs through a long list of commands that test dependencies, but do so by using 'add', 'modify', 'done' and 'delete'. - 2. Regression tests that ensure certain bugs are fixed and stay fixed. These + 2. Regression tests that ensure certain bugs are fixed and stay fixed. These tests are named bug.NNN.t where NNN refers to the Redmine issue number. While it is not worth creating tests for small fixes like typos, it is for changes to the logic. - 3. Small feature tests. When small features are added, we would like small, + 3. Small feature tests. When small features are added, we would like small, low-level feature tests named feature.NNN.t, where NNN is the Redmine issue number. - 4. Code tests. These are tests written in C++ that exercise C++ objects, or - function calls. These are the lowest level tests. It is important that + 4. Code tests. These are tests written in C++ that exercise C++ objects, or + function calls. These are the lowest level tests. It is important that these kind of tests be extensive and thorough, because the software depends on this code the most. - The tests are mainly written in Perl, and all use TAP. We want them all to be + The tests are mainly written in Perl, and all use TAP. We want them all to be eventually replaced by Python, so we are now only accepting new tests that use the Python-based approach you can see in some of the existing tests. Here is how to get the code and run the test suite: @@ -174,17 +179,14 @@ Unit Tests Needed many issues, open and closed, have the "needsTest" label. These are things that we would like to see in the test suite, as regression tests. - The basic.t unit tests are a misnomer, and should be either removed or - renamed. We have long talked of 'basic functionality' that includes add, - delete, done, and list commands. We need unit tests that prove that basic + renamed. We have long talked of 'basic functionality' that includes add, + delete, done, and list commands. We need unit tests that prove that basic functionality is working, and the file containing them should be called basic.t. - Test propagation of modifications to recurring tasks. - Test regex support. - - Need unit tests for each bug in the issue list, if suitable. - - We have bugs in the unit tests that only manifest on days such as December - 31st. Clearly there is some bad date math going on, most likely in the - tests themselves, rather than in Taskwarrior, although with broken tests, - who knows? + - Need unit tests for each bug in the issue list that is marked with the + 'needsTest' label. * Note that running the unit tests requires the Perl JSON module to be installed. @@ -196,8 +198,6 @@ Work in Progress Things that are currently in flux, which is another way of saying leave it alone while it is being worked on. - - Command line parser (A3). - - Expression evaluation (E9). - All columns/Col*::validate methods. - New columns/Col*::modify methods. @@ -207,14 +207,11 @@ Current Codebase Condition - 2.3.0 Current release, locked. '2.3.1' branch: - - Bug fix branch. To be released soon. + - Bug fix branch. To be released soon. '2.4.0' branch: - Current development branch with new command line parser, expressions --- -2014-01-19 Updated for 2.4.0. -2014-04-13 Added answers.tasktools.org, corrected URLs. -2014-05-11 Added build info. -2014-05-31 Updated unit tests section. +2014-07-04 Updated for 2.4.0 diff --git a/EXPOSITION b/EXPOSITION index 25c00e7d9..ef2a7bdd2 100644 --- a/EXPOSITION +++ b/EXPOSITION @@ -3,26 +3,26 @@ Startup Context::initialize and Context::run methods. Context is a large object that holds all task information, both in terms of - the task data, and intermediate run-time data. Having one global Context - object means we don't have 50 global variables. Context is therefore just a + the task data, and intermediate run-time data. Having one global Context + object means we don't have 50 global variables. Context is therefore just a big global bucket of data. - Context::initialize sets up all the data and processes the command line. The + Context::initialize sets up all the data and processes the command line. The initialization process is a big chicken-and-egg problem, because the command line depends on configuration (aliases) and the command line can force a - reload of configuration (rc:foo). This is solved by look-ahead: the command + reload of configuration (rc:foo). This is solved by look-ahead: the command line is scanned for 'rc:xxx' and 'rc.data.location:xxx' arguments, then later for overrides. - The Context::run method handles all the debug output and exceptions. Its + The Context::run method handles all the debug output and exceptions. Its main purpose is to set up exception handling and call Context::dispatch. Command Line Parsing - Command line parsing is difficult because of all the ambiguity. The solution - is to make several passes over the command line. For example, the task - command determines whether subsequent arguments are interpreted as part of a - filter or set of modifications. + Command line parsing is difficult because of all the ambiguity. The solution + is to make multiple passes over the command line. For example, the command + determines whether subsequent arguments are interpreted as part of a filter or + set of modifications. Dispatch @@ -30,40 +30,40 @@ Dispatch look up a command object, then call its execute method. Context stores an associative map of command object pointers indexed by a - string. This means the 'done' string is an index to the CmdDone object that + string. This means the 'done' string is an index to the CmdDone object that implements the functionality. Command Objects - Every task command is implemented by a command object. The command object + Every task command is implemented by a command object. The command object provides metadata, usage and one-line help in addition to the ::execute method that implements the command. Column Objects There is a 1:1 correspondence between attributes stored in the data files and - the columns that may be reported. These are represented by column objects, + the columns that may be reported. These are represented by column objects, which are responsible for validating input, measuring space needed according to various formats, and for rendering data for reports. TDB2 - The TDB2 object is a layered, transactioned I/O manager. Its purpose is to - isolate code from file I/O, locking, synching, and parsing details. It is + The TDB2 object is a layered, transactioned I/O manager. Its purpose is to + isolate code from file I/O, locking, synching, and parsing details. It is also responsible for minimizing reads, writes and parsing of data files. - All input is assumed to be UTF8. All stored data is UTF8. + All input is assumed to be UTF8. All stored data is UTF8. GC Garbage Collection is the process that moves tasks between the pending.data - and completed.data files. It is also responsible for waking tasks out of the + and completed.data files. It is also responsible for waking tasks out of the wait state. Every command that displays task IDs will cause a GC to be run first, which - minimizes the number of changes necessary to the task IDs. This means that + minimizes the number of changes necessary to the task IDs. This means that when a report shows task IDs, those IDs will remain valid while subsequent - write commands are issued. The next report run may show different IDs. + write commands are issued. The next report run may show different IDs. Minimizing the size of pending.data is important for performance, because it is the file that is accessed most. @@ -71,7 +71,7 @@ GC Files The data files used are all kept in the rc.data.location directory, which - defaults to ~/.task. The files are: + defaults to ~/.task. The files are: pending.data completed.data @@ -85,30 +85,29 @@ Files The completed.data file accumulates data over time, and grows unbounded. - The undo.data file accumulates changes over time, and grows unbounded. It + The undo.data file accumulates changes over time, and grows unbounded. It provides all the necessary metadata to support the 'undo' command. The backlog.data file contains an accumulated set of changes that have not - been transmitted to Taskserver. It grows unbounded between 'sync' commands. + been transmitted to Taskserver. It grows unbounded between 'sync' commands. Filter A filter is simply a set of command line arguments, but is only a subset of - the complete command line. These arguments (Arg objects) are grouped into - a set by the A3 (Args) object according to whether the command found is a - read or write command. + the complete command line. These arguments are extracted from the parse tree + according to whether the command found is a read or write command. - There is a Command::filter method for applying a filter to a set of tasks, - yielding a result set. It does this by creating an expression from the - filter using the E9 object, then evaluating the expression for each task, - such that the result set contains only tasks for which the expression - evaluates to Boolean true. + There is a Filter::subset method for applying a filter to a set of tasks, + yielding a result set. It does this by creating an expression from the + parse tree using the Eval object, then evaluating the expression for each task, + such that the result set contains only tasks for which the expression evaluates + to Boolean true. Sorting - Sorting is performed on a set of tasks. More specifically, the list that is + Sorting is performed on a set of tasks. More specifically, the list that is sorted is a set of numeric indexes to tasks that are stored in a separate - list. This minimizes the amount of data copying involved to just integers + list. This minimizes the amount of data copying involved to just integers rather than Task objects, but at the expense of one level of indirection. Memory fragmentation is a bigger problem than the performance of vector indexing. @@ -118,16 +117,16 @@ Sorting Render - There are two rendering objects, ViewTask and ViewText. These both have the - same tabular grid rendering capabilities. ViewText maintains a 2D vector of + There are two rendering objects, ViewTask and ViewText. These both have the + same tabular grid rendering capabilities. ViewText maintains a 2D vector of strings to contain the data to be rendered, so it is used for things like the - help command output. ViewTask does not copy data, but assumes all data is + help command output. ViewTask does not copy data, but assumes all data is stored externally in a vector of Tasks, which minimizes data copying. ViewTask contains projection data in the form of a set of Column objects that - represent the X axis. The Y axis is represented by a vector of tasks. + represent the X axis. The Y axis is represented by a vector of tasks. - The rendering process is complex. It involves dynamically setting column + The rendering process is complex. It involves dynamically setting column widths based on (1) available terminal width, (2) the columns to be included in the output, (3) ability to wrap text for certain columns and (4) the size of the data to be rendered, which involves added complexity when UTF8 is used. @@ -138,23 +137,24 @@ Render Test Suite A strong and diverse test suite is critical to the successful release of any - software. With the complex command set and its myriad permutations, a test + software. With the complex command set and its myriad permutations, a test suite is the only way to ensure quality levels, and guarantee that big changes are sound. It is intended that the test suite continues growing, mostly adding more - regression tests (bug.*.t) and more feature tests (feature.*.t). The test are - mostly written in Perl, and utilize the Test::More module to generate TAP - output. Some tests are written in C++ and also generate TAP. + regression and feature tests. The test are mostly written in Perl, and utilize + the Test::More module to generate TAP output. This is changing though, and the + suite is slowly migrating to Python. Some tests are written in C++ and all + tets generate TAP output. - There are currently over 5,000 unit tests, that take a minute or two to run + There are currently about 9,000 unit tests, that take a minute or two to run in total. - Taskwarrior uses flod software to automate continuous integration across - many platforms. Code changes are automatically detected, propagated, built and + Taskwarrior uses flod software to automate continuous integration across many + platforms. Code changes are automatically detected, propagated, built and tested on a variety of participating platforms. Grid testing results are here: - http://tasktools.org/tinderbox/taskwarrior-2.4.0.html + http://central.tasktools.org/task-2.4.0.html When making code changes, it is important that the test suite be run to verify that functionality was not broken. @@ -168,26 +168,26 @@ Debugging http://tasktools.org/performance). - Data load times. - Terminal size, color capabilities. - - Command line parsing steps, shown in colorful diagrams. + - Colorful, confusing, command line parse tree. - TDB2 layer and I/O information. Patches - Patches are encouraged and welcomed. Either attach them to the appropriate - Redmine issue, or send them to support@taskwarrior.org. A good patch: + Patches are encouraged and welcomed. Either attach them to the appropriate + Redmine issue, or send them to support@taskwarrior.org. A good patch: - Maintains the MIT license, and does not contain code lifted from other sources. - Precisely addresses one issue only. - Doesn't break unit tests. - Doesn't introduce dependencies. - - Is accompanied by unit tests, where appropriate. + - Is accompanied by unit tests, where appropriate, written in Python. - Is accompanied by documentation changes, where appropriate. - Conforms to the prevailing coding standards - in other words, it should fit right in with the existing code. A patch may be rejected for not following the above guidelines, and more. - Bad patches may be accepted and modified depending on work load and mood. It + Bad patches may be accepted and modified depending on work load and mood. It is possible that a patch may be rejected because it conflicts in some way with plans or upcoming changes.