* Tried to re-use the topological sort order to crawl the graph to find
the next node(s) to run. This is incorrect, we need to take into account
the fail/success of jobs and directionally crawl the graph.
* Leave workflow nodes with no related unified job template nodes
do_not_run = False. If we mark it True, we can't differentiate between
the actual want to not take that path vs. do not run this because I do
not have a valid related unified job template.
* Perform topological sort on graph nodes before looping over them to
mark do not run. This guarantees that parent nodes will be processed
before calling dependent child nodes. The complexity of the sorting is
N. The complexity of marking the the nodes is N*V
* Compute largest depth of each node and traverse graph by depth. This
allows us to check a node once, and only once, to determine if it needs
to be marked for do not run.
* Workflow Node without unified_job_template is treated as a job marked
as failure; when deciding what path to execute.
* Remove optimization of marking dnr nodes due to it making the
algorithm incorrect.
* When workflow job fails because a workflow job node doesn't have a
related unified_job_template note that with an error on the workflow
job's job_description
* When a workflow job fails because a failure path isn't defined, note
that on the workflow job job_description
* Instead of traversing the workflow graph to determine if a workflow is
done or has failed; instead, loop through all the nodes in the graph and
grab only the relevant nodes.
* From the root, the code was only going down the did run path to find
nodes to mark DNR. This is incorrect, Now, we traverse the entire graph
each time to find nodes to mark DNR.
* Getting parent nodes from child was inefficient. Optimize it with a
hash table like we did for the getting of children.
* Getting leaf nodes was inefficient. Optimize it like we did getting
root nodes. A node is assumed to be a leaf node until it gets a child.
* WorkflowDAG accepts workflow job template and workflow jobs for which
to build a graph out of the nodes. The optimized query for each is
different. This changeset adds the differing queries for a workflow job.
* If a child has a parent that is not in the finished state then do not
propogate the DNR to the child in question.
* If a parent is in a finished state; do not propogate the DNR to the
child if the path to the child is traversed (based on the parent job
status).
Relates #264.
This PR proposed and implemented a way of defining workflow failure
state:
A workflow job fails if one of the conditions below satisfies.
* At least one node runs into states `canceled` or `error`.
* At least one leaf node runs into states `failed`, but no child node is
spawned to run (no error handler).
Signed-off-by: Aaron Tan <jangsutsr@gmail.com>
