32119 Commits

Author SHA1 Message Date
Alan Rominger
278db2cdde
Split reaper for running and waiting jobs
Avoid running jobs that have already been reapted

Co-authored-by: Elijah DeLee <kdelee@redhat.com>

Remove unnecessary extra actions

Fix waiting jobs in other cases of reaping
2022-08-17 10:53:29 -04:00
Alan Rominger
64157f7207
Merge pull request #12584 from AlanCoding/lazy_workers
Wait 60 seconds before scaling down a worker
2022-08-17 10:18:19 -04:00
Alan Rominger
9e8ba6ca09
Merge pull request #12494 from AlanCoding/revival
Register system again if deleted by another pod
2022-08-17 10:12:39 -04:00
Alan Rominger
268ab128d7
Merge pull request #12527 from AlanCoding/offline_db
Further resiliency changes, specifically focused on case of database going offline
2022-08-17 10:10:50 -04:00
Alan Rominger
fad5934c1e
Merge pull request #12356 from AlanCoding/copytree_neo
Replace git shallow clone with shutil.copytree
2022-08-17 10:07:28 -04:00
Jessica Steurer
6a19aabd44
feature_request_form_update (#12625)
* Feature_update

* Feature_update

* update-feature-request

* update-edit
2022-08-17 08:52:30 -03:00
Hao Liu
7c885dcadb
add help command to make (#12669)
add `make help`
that prints all available make targets
help text generated from comments above the make target starting with `##`

Signed-off-by: Hao Liu <haoli@redhat.com>
2022-08-16 20:36:47 -04:00
John Westcott IV
b84a192bad
Altering events relationship to hosts to increase performance (#12447)
Removing cascade on delete at model level that could cause locking issues.
2022-08-16 12:03:05 -04:00
Elijah DeLee
35afb10add fix use of distinct on query that UI
When on the screen in the UI that loads the job events, the ui includes
a filter to exclude job events where stdout = ''. Because this is a
TextField and was not in the allow list, we were applying DISTINCT to
the query. This made it very unperformant for large jobs, especially
on the query that gets the count and cannot put a LIMIT on the query.

Also correctly prefetch the related job_template data on the view to
cut down the number of queries we make from around 50 to under 10.

We need to analyze other similar views for other prefetch type
optimizations we should make.
2022-08-16 10:08:33 -04:00
Alan Rominger
f1bd1f1dfc
Merge pull request #12658 from AlanCoding/more_panels
Add more graphs for task manager refactor
2022-08-15 16:07:43 -04:00
Sarah Akus
67c9e1a0cb
Merge pull request #12650 from matburt/fix_default_adhoc_verbosity
Fixed a bug where the initial form value of verbosity isn't respected
2022-08-15 15:48:49 -04:00
Alan Rominger
f6da9a5073
Add more graphs for task manager refactor 2022-08-15 15:29:34 -04:00
Seth Foster
38a0950f46
Merge pull request #12656 from fosterseth/metrics_tm_on_commit
Add metric for task manager on_commit calls
2022-08-15 13:54:34 -04:00
Seth Foster
55d295c2a6
Add metric to measure task manager transaction, including on_commit calls 2022-08-15 12:44:29 -04:00
Elijah DeLee
be45919ee4 have postgres log to console in dev env
also log slow queries and link to documentation for other possible
settings
2022-08-15 12:09:17 -04:00
mabashian
0a4a9f96c2 Explicitly set value for verbosity to 0 as the default value which corresponds to 0 (Normal) 2022-08-12 14:03:36 -04:00
Matthew Jones
1ae1da3f9c Fix a bug where the form value of verbosity isn't respect 2022-08-12 09:29:31 -04:00
Keith Grant
cae2c06190
Complex schedules UI (#12445)
* refactor ScheduleFormFields into own file

* refactor ScheduleForm

* wip complex schedules form

* build rruleset from inputs

* update schedule form validation for multiple repeat frequencies

* add basic rrule set parsing when opening schedule form

* complex schedule bugfixes, handle edge cases, etc

* fix schedule saving/parsing for single-occurrence schedules

* working with timezone issues

* fix rrule until times to be in UTC

* update tests for new schedule form format

* update ouiaIds

* tweak schedules spacing

* update ScheduleForm tests

* show message for unsupported schedule types

* default schedules to browser timezone

* show error type/message in ErrorDetail

* shows frequencies on ScheduleDetails view

* handles nullish values
2022-08-11 16:55:52 -04:00
John Westcott IV
993dd61024
Forcing an unbind for a django-auth-ldap sticky session to the LDAP server (#12367)
* Forcing an unbind for a django-auth-ldap sticky session to the LDAP server

* Focring _connection_bound to false after closing and modifying exceptino logging
2022-08-11 16:46:41 -03:00
John Westcott IV
268a4ad32d
Modifying reaper of administrative work units to allow for change from Controller to Hybrid nodes (#12614) 2022-08-11 09:03:35 -03:00
Sean Sullivan
3712af4df8
update role to provide better error messages (#12599) 2022-08-11 07:09:11 -04:00
Sean Sullivan
8cf75fce8c
Update awx collection workflow nodes to look for type (#12597) 2022-08-11 07:08:27 -04:00
Alan Rominger
46be2d9e5b
Replace git shallow clone with shutil.copytree
Introduce build_project_dir method
  the base method will create an empty project dir for workdir

Share code between job and inventory tasks with new mixin
  combine rest of pre_run_hook logic
  structure to hold lock for entire sync process

force sync to run for inventory updates due to UI issues

Remove reference to removed scm_last_revision field
2022-08-10 16:18:56 -04:00
Alan Rominger
998000bfbe
Surface correct error from bulk_create on unrecoverable error 2022-08-10 16:16:57 -04:00
Alan Rominger
43a50cc62c
Fix event counting in error handling path 2022-08-10 16:16:57 -04:00
Alan Rominger
30f556f845
Further resiliency changes focused on offline database
Make logs from database outage more manageable

Raise exception if update_model never recovers from problem
2022-08-10 16:16:57 -04:00
Alan Rominger
c5985c4c81
Change lazy worker method name and adjust log 2022-08-10 16:12:03 -04:00
Alan Rominger
a9170236e1
Wait 60 seconds before scaling down a worker 2022-08-10 16:12:03 -04:00
Seth Foster
85a5b58d18
Merge pull request #12629 from fosterseth/task_manager_refactor_squashed
Task manager refactor
2022-08-10 16:02:05 -04:00
Seth Foster
6fb3c8daa8
Merge pull request #44 from AlanCoding/one_of_seths_own
Inherit from our own APIView, not rest framework
2022-08-10 15:38:14 -04:00
Alan Rominger
a0103acbef
Inherit from our own APIView, not rest framework 2022-08-10 15:31:19 -04:00
Alan Rominger
f7e6a32444
Optimize task manager with debug toolbar, adjust prefetch (#12588) 2022-08-10 10:05:13 -04:00
Alex Corey
7bbc256ff1
Merge pull request #12637 from AlexSCorey/12636-WorkflowApprovalTranslations
Fixes lack of translation on workflow approval list item actions
2022-08-09 15:47:34 -04:00
Alex Corey
64f62d6755 fixes translation issue 2022-08-09 15:30:08 -04:00
Alex Corey
b4cfe868fb
Merge pull request #12546 from mabashian/6018-node-alias
Fix bug where node alias is not remaining after changing the template on a wf node
2022-08-09 10:16:46 -04:00
Alex Corey
8d8681580d
Merge pull request #12548 from AlexSCorey/12512-UpdateWorkflowApprovalToolbar
Refactors and redesigns workflow approval to impove UX
2022-08-09 10:02:27 -04:00
Alex Corey
8892cf2622 Adds toast to workflow approval on cancel 2022-08-09 09:40:34 -04:00
Alan Rominger
585d3f4e2a
Register system again if deleted by another pod
Avoid cases where missing instance
  would throw error on startup
  this gives time for heartbeat to register it
2022-08-08 22:36:17 -04:00
Alex Corey
2c9a0444e6
Easier review workflow output (#12459)
* Adds new tab component and positions it properly on screen

* Adds filtering, and navigation to node outputs
2022-08-08 16:13:51 -04:00
Alan Rominger
279cebcef3
Merge pull request #12586 from AlanCoding/connections_graph
Add a graph to show database connections being used
2022-08-08 15:49:20 -04:00
Seth Foster
e6f8852b05
Cache task_impact
task_impact is now a field on the database
It is calculated and set during create_unified_job

set task_impact on .save for adhoc commands
2022-08-05 14:33:47 -04:00
Alan Rominger
d06a3f060d
Block sliced workflow jobs on any job type from their JT (#12551) 2022-08-05 14:33:45 -04:00
Seth Foster
957b2b7188
Cache preferred instance groups
When creating unified job, stash the list of pk values from the
instance groups returned from preferred_instance_groups so that the
task management system does not need to call out to this method
repeatedly.

.preferred_instance_groups_cache is the new field
2022-08-05 14:33:28 -04:00
Alan Rominger
b94b3a1e91
[task_manager_refactor] Move approval node expiration logic into queryset (#12502)
Instead of loading all pending Workflow Approvals in the task manager,
  run a query that will only return the expired apporovals
  directly expire all which are returned by that query

Cache expires time as a new field in order to simplify WorkflowApproval filter
2022-08-05 14:33:27 -04:00
Elijah DeLee
7776a81e22
add job to dependency graph in start task
We always add the job to the graph right before calling start task.
Reduce complexity of proper operation by just doing this in start_task,
because if you call start_task, you need to add it to the dependency
graph
2022-08-05 14:33:26 -04:00
Elijah DeLee
bf89093fac
unify call pattern for get_tasks 2022-08-05 14:33:26 -04:00
Elijah DeLee
76d76d13b0
Start pending workflows in TaskManager
we had tried doing this in the WorkflowManager, but we decided that
we want to handle ALL pending jobs and "soft blockers" to jobs with the
TaskManager/DependencyGraph and not duplicate that logic in the
WorkflowManager.
2022-08-05 14:33:26 -04:00
Elijah DeLee
e603c23b40
fix sliced jobs blocking logic in depedency graph
We have to look at the sliced job's unified_job_template_id
Now, task_blocked_by works for sliced jobs too.
2022-08-05 14:33:26 -04:00
Alan Rominger
8af4dd5988
Fix unintended slice job blocking 2022-08-05 14:33:25 -04:00
Seth Foster
0a47d05d26
split schedule_task_manager into 3
each call to schedule_task_manager becomes one of

ScheduleTaskManager
ScheduleDependencyManager
ScheduleWorkflowManager
2022-08-05 14:33:25 -04:00