mirror of
https://github.com/ansible/awx.git
synced 2026-02-16 18:50:04 -03:30
0e29f3617d
Running orphaned task cleanup within its own scheduled task via celery-beat causes a race-y lock contention between the cleanup task and the task scheduler. Unfortunately, the scheduler and the cleanup task both run at similar intervals, so this race condition is fairly easy to hit. At best, it results in situations where the scheduler is regularly delayed 20s; depending on timing, this can cause situations where task execution is needlessly delayed a minute+. At worst, it can result in situations where the scheduler is never able to schedule tasks. This change implements the cleanup as a periodic block of code in the scheduler itself that tracks its "last run" time in memcached (by default, it performs a cleanup every 60 seconds) see: #6534
32 lines
617 B
Python
32 lines
617 B
Python
|
|
# Python
|
|
import logging
|
|
|
|
# Celery
|
|
from celery import task
|
|
|
|
# AWX
|
|
from awx.main.scheduler import TaskManager
|
|
|
|
logger = logging.getLogger('awx.main.scheduler')
|
|
|
|
# TODO: move logic to UnifiedJob model and use bind=True feature of celery.
|
|
# Would we need the request loop then? I think so. Even if we get the in-memory
|
|
# updated model, the call to schedule() may get stale data.
|
|
|
|
|
|
@task
|
|
def run_job_launch(job_id):
|
|
TaskManager().schedule()
|
|
|
|
|
|
@task
|
|
def run_job_complete(job_id):
|
|
TaskManager().schedule()
|
|
|
|
|
|
@task
|
|
def run_task_manager():
|
|
logger.debug("Running Tower task manager.")
|
|
TaskManager().schedule()
|