- Updates:
- 2020-08-25: new section "Links" with A simple GTD approach using Org mode and Org Edna | Plain DrOps
- 2020-09-02: Jouke found a solution to "how to achieve that a task is only scheduled when a set of other tasks is done completely?"
- 2020-09-23: I found out that in contrast to the (now unmaintained)
org-edna fork I was using until a few days ago, the original org-edna
requires
ids(foo bar)
to have quotes likeids("foo" "bar")
unlessuuidgen
IDs are used. So I adapted the example project below to meet this syntax. Good news: with org-edna v1.1.2 you can also useids("id:foo" "id:bar")
and get navigatable IDs. - 2020-09-26:
- Comments from yantar92
- Link to
org-linker-edna
- 2021-09-20:
org-linker-edna
proves to solve the issue of linking headings - 2021-11-21: reddit discussion on org-edna and visualization
I'm currently dreaming of a workflow that allows complex project management within Org mode. Please do read my article how I define projects within Org mode. I won't repeat those things in this article here.
So far, I was using task dependencies via org-depend, writing manual :TRIGGER:
and :BLOCKER:
properties. This works perfectly fine. However, when you want to use more advanced features, you need to look for something different. For example, I sometimes want to schedule a task on the day I finalize a related one. Or I want to schedule a follow-up task three working-days after marking a specific task as done. Org-depend only allows to set todo keywords and has no functionality to control scheduled dates or other advanced stuff.
With having defined task dependencies explicitly, a full-blown project management workflow is more than just a nice to have. For example, I generate all tasks related to organizing and conducting a lecture via a very long, complex and static yankpad template. This way, I only have to maintain the snippet template and get all tasks for a semester with all of their dependencies using a short keyboard command.
When you can not apply a static template to your project, you have to define the dependencies of the tasks manually. Using an advanced (and more complex) syntax, this is a tedious and error-prone task to do. Therefore, I want to automate and semi-automate as much as possible so that project management does not have to deal with manually write cumbersome property syntax.
The Basic Concept (My Requirements)
These are the basic concepts I want to follow when managing a project using task dependencies:
- Manual effort for defining dependencies should be minimized.
- Next tasks that might be worked on within a project are very easily
spotted. Only actionable tasks are visible on your agenda.
- Compare to the Getting Things Done method and how it is emphasizing the importance of being able to spot the next task to work on.
- A task is only scheduled and only has a TODO keyword when it is
ready to be considered in your workflow.
- See exceptions due to current technical limitations.
- This requires task dependencies being explicitly defined using Org Edna:
:BLOCKER:
properties are redundant and optional: I like to use them to visualize dependent tasks.:Effort:
properties are optional but handy for deriving (Gantt) charts and clocking.- Multiple entries for
:BLOCKER:
and:TRIGGER:
properties are merged to one single line. In real world, you will likely useM-x org-edna-edit
to interactively work with those properties.
- For me, deadlines are for hard deadlines only.
- Therefore, they are seldomly used in my world.
- Task dependencies should be visible within both tasks, the prerequisite as well as the follow-up task.
- Changes or delays should be applied with minimum effort.
- There should be the possibility to generate an on-demand project status overview.
The Draft Workflow (Method)
My desired workflow is not finalized or completed yet. See the "FIXXME" parts where I'd love to get volunteers interested in contributing. Whatever code I will come up by myself, most probably will be implemented in Python and not elisp.
- MANUALLY: Create a new Org mode heading for the project.
- SUPPORTED: Invoke
C-c C-x b
(org-tree-to-indirect-buffer()
) on project heading.- This is for focus only: ignore the rest of the potentially large Org buffer while working within a specific sub-hierarchy.
- Think of switching to the version from my configuration with improvements by alphapapa.
- SUPPORTED: Mark project heading as such.
- Via
my-mark-as-project()
from my configuration (and this blog article):- adding a
:project:
tag - property
COOKIE_DATA
set to "todo recursive" - leading progress indicator
- adding a
- FIXXME:
- add appropriate column view mode property
- ask for a descriptive project tag and add it
- ask for short project acceptance criteria and add "<criteria> → close project" heading that closes the generated project heading ID as well
- Via
- MANUALLY: Adding a descriptive project tag such as
GProj
for "Garage Project".- I like this to add some context in derived views such as my agenda.
- MANUALLY: Brain-storm new project as simple list.
- Move around list items in a flat list.
- SUPPORTED: Convert list items to headings.
- Via marking list and invoking
C-c *
→org-toggle-heading()
.
- Via marking list and invoking
- MANUALLY/OPTIONAL: Add task effort estimates if you need them for
other purpose.
- Switch to column view via
C-c C-x C-c
. - Add effort estimates via
e
(on a value). - Quit column view via
q
(on a value).
- Switch to column view via
- SEMI-MANUALLY: Add unique
:ID:
properties to the headings.- I prefer
:ID:
over:Custom_ID:
as described in this article. - I generate IDs via
my-id-get-or-generate()
from my configuration because I prefer human-readable IDs instead of auto-generated UUIDs that consists of arbitrary digits and characters.- FIXXME: automate missing ID generating with the future function for adding trigger dependencies (see below) as it can be done for org-super-links using that snippet.
- I prefer
- FIXXME: add trigger dependencies via a cool
function as described on this
(rejected) feature request.
- Adding
:TRIGGER:
and:BLOCKER:
properties in an interactive way + tab-completion of IDs. - Notice that
org-edna
is able to understand IDs with theid:
prefix starting with version 1.1.2. This results in navigable links/ - Update 2021-09-20: This feature is provided using
org-linker-edna
and has proven in my daily life.
- Adding
- MANUALLY: Mark non-depending initial tasks with an active TODO
keyword.
- I personally prefer
NEXT
for tasks I might start doing now andWAITING
for blocked tasks by external parties.
- I personally prefer
- Start working on the project tasks.
- FIXXME: derive a read-only status chart from the
current state of the headings any time.
- Respecting dependencies, effort and status.
- See PlantUML Use-Case diagram example below.
- FIXXME: writing a hydra with all kinds of cool
tools for supporting project management. Some ideas are:
- Insert a new task between two tasks that do have a dependency
defined: A→B then gets modified to A→NEW→B with modifying corresponding
:TRIGGER:
and:BLOCKER:
properties. - Remove a task with all references within
:TRIGGER:
and:BLOCKER:
properties.
- Insert a new task between two tasks that do have a dependency
defined: A→B then gets modified to A→NEW→B with modifying corresponding
Currently, four steps are at least partly supported by functions. Five steps are manual tasks and most likely stay that way. At least three steps can be dramatically improved by developing functions for them.
Here is a very primitive elisp macro I was using for defining a default :TRIGGER:
property with a id:<my-current-id>
from the kill-ring (I got the ID via my-id-get-or-generate()
from my configuration):
(setq last-kbd-macro [?\C-c ?\C-p ?\C-c ?\C-x ?p ?T ?R ?I ?G ?G ?E ?R return ?\C-y ?\C-a right right ?s ?\C-d ?\( ?\C-e ?\) ? ?t ?o ?d ?o ?\( backspace ?! ?\( ?N ?E ?X ?T ?\) ? ?s ?c ?h ?e ?u ?d backspace backspace ?d ?u ?l ?e ?d ?\( backspace ?! ?\( ?\" ?. ?\" ?\) return])
While this would reflect my personal use-case for project management very well, there are even more possible future extensions. You could assign one or more persons to a task which supports capacity planning tasks. Or you can integrate with external project management tools. I'm sure that different project managers have different requirements and ideas here.
An Example Project: The Garage Project
For everybody unfamiliar with org-edna
and task dependencies, I created an example project for refurbishing a garage. You can skip this section if you don't need details on the syntax I'm planning to use.
Please note that I'm not done with switching from org-depend
to org-edna
yet. Therefore, there might be some things an experienced org-edna
user does differently. In those cases, please send me a comment!
*** TODO [0/2] Example project: Garage Project :GProj:project: :PROPERTIES: :COLUMNS: %40ITEM %6Effort(Effort){:} %60BLOCKER :COOKIE_DATA: todo recursive :END: **** TODO Find local stores for paint :PROPERTIES: :Effort: 3h :TRIGGER: if ids("GProj-Choose-color-for-paint") !done? then ids("GProj-Buy-paint") todo!(NEXT) scheduled!(".") endif :ID: GProj-Find-local-stores-for-paint :END: **** TODO Choose color for paint :PROPERTIES: :Effort: 3h :ID: GProj-Choose-color-for-paint :TRIGGER: if ids("GProj-Find-local-stores-for-paint") !done? then ids("GProj-Buy-paint") todo!(NEXT) scheduled!(".") endif :END: **** Buy paint :PROPERTIES: :Effort: 3h :ID: GProj-Buy-paint :BLOCKER: ids("GProj-Find-local-stores-for-paint" "GProj-Choose-color-for-paint") :TRIGGER: ids("GProj-Move-car-to-nearby-parking-lot") todo!(NEXT) scheduled!(".") ids("GProj-Remove-bikes") todo!(NEXT) scheduled!(".") ids("GProj-Remove-stuff-from-shelves") todo!(NEXT) scheduled!(".") :END: "Buy paint" is only scheduled when both prior tasks are marked as DONE thanks to the "if ids(...) !done? then ids(...) ... endif" trick above. Until org-edna is going to support this, there is a viable workaround at: https://github.com/mskorzhinskiy/org-linked-tasks **** Move car to nearby parking lot :PROPERTIES: :Effort: 1h :ID: GProj-Move-car-to-nearby-parking-lot :BLOCKER: ids("GProj-Buy-paint") :TRIGGER: ids("GProj-Garage-is-empty") todo!(DONE) scheduled!(".") :END: Please notice that as of 2020-08-14 and org edna version 1.0.2, headings without an active keywords can be marked as DONE despite the fact that they're blocked. I've reported this bug already and hope for a fix. **** Remove bikes :PROPERTIES: :Effort: 1h :ID: GProj-Remove-bikes :BLOCKER: ids("GProj-Buy-paint") :TRIGGER: ids("GProj-Garage-is-empty") todo!(DONE) :END: **** Remove stuff from shelves :PROPERTIES: :Effort: 2d :ID: GProj-Remove-stuff-from-shelves :BLOCKER: ids("GProj-Buy-paint") :TRIGGER: ids("GProj-Remove-shelves") todo!(NEXT) scheduled!("++1d") :END: **** Remove shelves :PROPERTIES: :Effort: 1d :ID: GProj-Remove-shelves :BLOCKER: ids("GProj-Remove-stuff-from-shelves") :TRIGGER: ids("GProj-Garage-is-empty") todo!(DONE) :END: **** Garage is empty :PROPERTIES: :ID: GProj-Garage-is-empty :BLOCKER: ids("GProj-Move-car-to-nearby-parking-lot" "GProj-Remove-bikes" "GProj-Remove-shelves") :TRIGGER: ids("GProj-Paint-walls-and-floor") todo!(NEXT) scheduled!("++1d") :END: **** Paint walls and floor :PROPERTIES: :Effort: 1d :ID: GProj-Paint-walls-and-floor :BLOCKER: ids("GProj-Garage-is-empty") :TRIGGER: ids("GProj-Re-install-shelves") todo!(NEXT) scheduled!("++2d") :END: **** Re-install shelves :PROPERTIES: :Effort: 8h :ID: GProj-Re-install-shelves :BLOCKER: ids("GProj-Paint-walls-and-floor") :TRIGGER: ids("GProj-Bring-back-bikes-into-garage") todo!(NEXT) scheduled!(".") ids("GProj-Bring-back-car-into-garage") todo!(NEXT) scheduled!(".") :END: **** Bring back bikes into garage :PROPERTIES: :Effort: 1h :ID: GProj-Bring-back-bikes-into-garage :BLOCKER: ids("GProj-Re-install-shelves") :TRIGGER: ids("GProj-Celebrate-and-close-project") todo!(NEXT) scheduled!(".") :END: **** Bring back car into garage :PROPERTIES: :Effort: 1h :ID: GProj-Bring-back-car-into-garage :BLOCKER: ids("GProj-Re-install-shelves") :TRIGGER: ids("GProj-Celebrate-and-close-project") todo!(NEXT) scheduled!(".") :END: **** Celebrate and close project :PROPERTIES: :BLOCKER: consider(all) rest-of-siblings-wrap :ID: GProj-Celebrate-and-close-project :END:
For the sake of simplicity, I modified the generated IDs so that the leading ISO date-stamp is replaced by GProj-
instead.
As you can see, I'm not using absolute dates for task planning as classic project management often does. My personal projects don't have deadlines and they develop according to my spare time, spanning over weeks, months and often even years. Therefore, personal projects often require a stricter management than my business projects.
Note that "Find local stores for paint" and "Choose color for paint" do feature rather complicated :TRIGGER:
expressions. This is the technical solution that Jouke has found here to achieve, that "Buy paint" gets only "active" (scheduled + todo keyword) when both blocker tasks are marked as done and not just one of them. This can get even more complicated when "Buy paint" would be depending on more than two tasks.
As a consequence, I am currently not using efforts at all. The only reason I added effort estimates here is to play around with methods to visualize project status.
Visualizations
At first, I tried to derive a classic Gantt chart because it is the most obvious visualization when it comes to project management.
The new elgantt project won't support task estimations more fine-grained than days. PlantUML Gantt does have the same restriction although the forum lists some discussions to introduce hours and even smaller periods.
Since my personal projects are dealing with smaller tasks, planning on a day-basis is a no-go to me. However, I personally also do not need visualizations that map the tasks to a calendar or resources. I just want to get a brief overview as a nice-to-have thing, that's all. This is the reason why I most probably will not work with effort estimates in the long run for personal projects.
With the two Gantt tools failing to support my use-cases, I manually generated a simple chart using PlantUML Use-Case diagrams. Automating the chart generating for an arbitrary Org sub-hierarchy reflecting a project is only a matter of time and should be straight-forward. Elisp-savvy volunteers could provide a general solution here.
The PlantUML source has an initial/general section which defines colors, and the project itself. The following section with the first half of the nodes I ordered by "PlantUML link type" so that you're able to understand the different PlantUML syntax elements. The last section holds the remaining nodes and is arranged by Org heading which most likely reflects the parsing order and the results by generated code:
' skin settings for changing certain nodes according to their state skinparam usecase { BackgroundColor<< NEXT >> LightGray BackgroundColor<< FINAL NEXT >> LightGray BackgroundColor<< STARTED >> PaleGreen BackgroundColor<< FINAL STARTED >> PaleGreen BackgroundColor<< DONE >> LimeGreen BackgroundColor<< FINAL DONE >> LimeGreen BackgroundColor<< FINAL >> LightSkyBlue } ' this is default anyway top to bottom direction ' from the project heading ' (**TITLE**) as (ID) (**Garage Project**) as (GProj) ' ------------------------------------------------------ ' the items of the first nodes in logical order ' ------------------------------------------------------ ' a list of all heading titles with their IDs as alias '(TITLE\n//ESTIMATE//) as (ID) '(TITLE) as (ID) '(TITLE) as (ID) << TODOKEYWORD >> ' TODOKEYWORD = FINAL when no trigger is defined (Find local stores for paint\n//3h//) as (GProj-Find-local-stores-for-paint) << NEXT >> (Choose color for paint\n//3h//) as (GProj-Choose-color-for-paint) << NEXT >> (Buy paint\n//3h//) as (GProj-Buy-paint) ' all headings that got no blocker tasks: ' (PROJECTID) -down-> (HEADINGID) (GProj) -down-> (GProj-Find-local-stores-for-paint) (GProj) -down-> (GProj-Choose-color-for-paint) ' trigger properties ' (HEADINGID) -down-> (HEADINGID) : TODOKEYWORD (GProj-Find-local-stores-for-paint) -down-> (GProj-Buy-paint) : NEXT (GProj-Choose-color-for-paint) -down-> (GProj-Buy-paint) : NEXT (GProj-Buy-paint) -down-> (GProj-Move-car-to-nearby-parking-lot) : NEXT (GProj-Buy-paint) -down-> (GProj-Remove-bikes) : NEXT (GProj-Buy-paint) -down-> (GProj-Remove-stuff-from-shelves) : NEXT ' blocker properties ' (HEADINGID) .up.>> (HEADINGID) (GProj-Buy-paint) .up.> (GProj-Find-local-stores-for-paint) (GProj-Buy-paint) .up.> (GProj-Choose-color-for-paint) ' ------------------------------------------------------ ' the rest of the nodes in the order of parsing ' ------------------------------------------------------ (Move car to nearby parking lot\n//1h//) as (GProj-Move-car-to-nearby-parking-lot) (GProj-Move-car-to-nearby-parking-lot) .up.> (GProj-Buy-paint) (GProj-Move-car-to-nearby-parking-lot) -down-> (GProj-Garage-is-empty) : DONE (Remove bikes\n//1h//) as (GProj-Remove-bikes) (GProj-Remove-bikes) .up.> (GProj-Buy-paint) (GProj-Remove-bikes) -down-> (GProj-Garage-is-empty) : DONE (Remove stuff from shelves\n//2d//) as (GProj-Remove-stuff-from-shelves) (GProj-Remove-stuff-from-shelves) .up.> (GProj-Buy-paint) (GProj-Remove-stuff-from-shelves) -down-> (GProj-Remove-shelves) : NEXT ++1d (Remove shelves\n//1d//) as (GProj-Remove-shelves) (GProj-Remove-shelves) .up.> (GProj-Remove-stuff-from-shelves) (GProj-Remove-shelves) -down-> (GProj-Garage-is-empty) : DONE (Garage is empty) as (GProj-Garage-is-empty) (GProj-Garage-is-empty) .up.> (GProj-Move-car-to-nearby-parking-lot) (GProj-Garage-is-empty) .up.> (GProj-Remove-bikes) (GProj-Garage-is-empty) .up.> (GProj-Remove-shelves) (GProj-Garage-is-empty) -down-> (GProj-Paint-walls-and-floor) : NEXT ++1d (Paint walls and floor\n//1d//) as (GProj-Paint-walls-and-floor) (GProj-Paint-walls-and-floor) .up.> (GProj-Garage-is-empty) (GProj-Paint-walls-and-floor) -down-> (GProj-Re-install-shelves) : NEXT ++2d (Re-install shelves\n//8h//) as (GProj-Re-install-shelves) (GProj-Re-install-shelves) .up.> (GProj-Paint-walls-and-floor) (GProj-Re-install-shelves) -down-> (GProj-Bring-back-bikes-into-garage) : NEXT (GProj-Re-install-shelves) -down-> (GProj-Bring-back-car-into-garage) : NEXT (Bring back bikes into garage\n//1h//) as (GProj-Bring-back-bikes-into-garage) (GProj-Bring-back-bikes-into-garage) .up.> (GProj-Re-install-shelves) (GProj-Bring-back-bikes-into-garage) -down-> (GProj-Celebrate-and-close-project) : NEXT (Bring back car into garage\n//1h//) as (GProj-Bring-back-car-into-garage) (GProj-Bring-back-car-into-garage) .up.> (GProj-Re-install-shelves) (GProj-Bring-back-car-into-garage) -down-> (GProj-Celebrate-and-close-project) : NEXT (Celebrate and close project) as (GProj-Celebrate-and-close-project) << FINAL >>
The first chart reflects the situation right after planning the project. You can see that the project has an initial node at the top and gray nodes below which are ready to go. The yellow ones (default color) are not scheduled and do not have an todo keyword yet. They're not actionable at the moment. :TRIGGER:
dependencies are visualized and blocking back-links as well.
While this is nice to look at, a more value is derived from the visualization when the project is in an ongoing state:
If there is a dangling task that relates to an error when defining the dependencies, the visualization can help to spot issues. In the following example, I removed the :TRIGGER:
property of the "Remove bikes" task:
Next Steps
While project management is not one of my main use-cases when working with Org and my personal projects, it would improve my ability to follow long-running projects with many tasks more efficiently.
Now that I've written down all my thoughts and ideas, it's easier for me to communicate the bigger picture when joining discussions or creating feature requests.
Furthermore, there is a chance that you do have similar project management requirements and with your ability to code elisp, you might want to help automating some of the manual steps of that workflow.
Comment from yantar92
Yantar92 wrote me an email with some very good points. I'm quoting some of his arguments and my answer here.
First of all, he stated that project management consists of more than just defining tasks. He is absolutely right on this. This is true. I omitted it because in contrast to the title, my article's focus was not project management in general but Org mode task dependencies in the context of project management.
For a full-blown project management workflow, there are much more things missing such as resource tracking, cost tracking, and so forth.
Why don't just use capture template? This will automatically narrow the
buffer, apply the :project: tag, add properties, progress indicator,
prompt you for the descriptive project tag (via :%^G:
in your capture
template)
Also true.
The reason is that a large percentage of projects are not started as a project. Therefore, there are some tasks that "explode" into a project.
If the majority of my projects would be projects right from the start, I would use a yankpad snippet instead which automates some things I mentioned in my workflow.
I feel that you are over-complicating the dependency management. Let's consider your garage project example. Most of dependencies can be trivially defined by introducing hierarchy into your task list.
Note that I use different todo-keyword convention: - all the tasks must have a keyword - the tasks that can be done next are only the tasks withNEXT
keyword /and/ with all the parent tasks also markedNEXT
- Marking NEXT task DONE switches the following siblingTODO
task toNEXT
I once followed this pattern as well. Somehow, it ended up with a different approach at my side. I don't use TODO
any more, just NEXT
. The TODO
keyword was replaced by no keyword (and optional some TRIGGER
property waiting to fire to set it) or the SOMETIME
keyword.
I'd rather not say that one of those two approaches is inferior to the other though. Just different conventions for the same, I guess.
* TODO Garage project ** TODO Buy paint *** TODO Chose pain colour *** TODO Find local stores for the paint *** TODO Buy the chosen paint ** TODO Empty the garage *** TODO Remove bikes *** TODO Move car to nearby parking *** TODO Remove shelves **** TODO Remove staff from shelves **** TODO Remove the shelves ** TODO Paint walls and floor ** TODO Re-install shelves ** TODO Bring things back into the garage *** TODO Bring back the bikes *** TODO Bring back the car
The above project structure will let you manipulate the dependencies simply by moving the headlines around and changing their todo-keywords with built-in org-mode functionality.
Absolutely true for this example.
I'd love to have a very low-profile feature compared to the :ordered: t
property but with an additional "moving" keyword. org-depend offers something similar with chain-siblings(KEYWORD)
. To my surprise, org-edna does not offer the same feature in a simple way.
Of course, it does not allow complex task dependencies (when some task is triggered by completing multiple sub-tasks in different hierarchies). However, they are quite rare in practice (according to my experience).
In mine as well.
Automating dependency management in these rare cases may be still useful, but if things go away from initial plan in such complex projects, any universal automation workflow often turns out to be not suitable. I find it more practical using weekly review to resolve complex dependency (and other) problems, as advised by David Allen.
Perfectly well approach.
While my draft workflow reflects a certain complexity to demonstrate advanced dependency features of org-edna, most projects are straight forward as you have pointed out already.
However, in my case there are not so rare cases where I want to define a dependency between tasks that are not within the same project as well. For example when I want to take a look at a specific features of a tool which is not available on my slow-moving Debian GNU/Linux stable, I might define a dependency from the system upgrade task to the software test task. Those two headings are not even in the same Org mode file.
My example project demonstrates dependency management. I might have caused a certain expectation by choosing the project management story. Therefore, I hope I could explain my underlying thoughts with these lines a bit more.
Thank you for your great comments!
Promising Approach: org-linker-edna
The org-linker-edna project is a very promising approach to deliver many missing pieces to my workflow. It is able to create simple dependency links with :TRIGGER:
and :BLOCKER:
properties between headings.
Update 2021-09-20: Meanwhile, I can say that org-linker-edna
is the solution to go for me to define dependencies. It's trutstworthy and easy to use.
Links
Andy read this article and discussed some aspects with me via Mastodon. He came up with his own workflow he's describing on his blog article. I very much recommend you to read through his approach as he's following absolutely valid simplifications for projects that don't need all of the complexity I mentioned above.
2021-11-21: Please do read this page with a very interesting discussion by u/TeMPOraL\_PL on org-edna and task dependency with visualizations and a BLOCKER status.