External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-01-13 11:00:03 -03:30
Code Issues Packages Projects Releases Wiki Activity

check job status before marking as failed

Browse Source 
* When rectifying celery <-> db job status, re-check the job status to
ensure it did not finish between the time that we pulled the list of
celery tasks from celery and the time in which we rectify.
This commit is contained in:
Chris Meyers 2017-02-09 15:27:08 -05:00
parent aeee6db868
commit dfb7b08380
1 changed files with 6 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
awx/main/scheduler/__init__.py
View File

@ -346,10 +346,14 @@ class TaskManager():
for task in all_running_sorted_tasks:
if (task['celery_task_id'] not in active_tasks and not hasattr(settings, 'IGNORE_CELERY_INSPECTOR')):
# NOTE: Pull status again and make sure it didn't finish in
# the meantime?
# TODO: try catch the getting of the job. The job COULD have been deleted
task_obj = task.get_full()
# Ensure job did not finish running between the time we get the
# list of task id's from celery and now.
# Note: This is an actual fix, not a reduction in the time
# window that this can happen.
if task_obj.status is not 'running':
continue
task_obj.status = 'failed'
task_obj.job_explanation += ' '.join((
'Task was marked as running in Tower but was not present in',