External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-09-16 00:40:11 -02:30
Code Issues Packages Projects Releases Wiki Activity
32,672 Commits 63 Branches 102 Tags
da9b8135e88823ec86cf072c4142b72c8ee87b9c
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alan Rominger a0b8215c06 Merge pull request #13296 from AlanCoding/signing_bug 
Fix bug, sign work based signing, not verification
 2022-12-07 08:44:57 -05:00
Alan Rominger f88b993b18 Fix bug, sign work based signing, not verification 2022-12-06 16:21:17 -05:00
Alan Rominger 239959a4c9 Merge pull request #13213 from AlanCoding/execution_signing 
Fix fallout from turning off work signing in docker-compose
 2022-11-18 15:22:18 -05:00
Alan Rominger 84f2b91105 Fix fallout from turning off work signing in docker-compose 2022-11-18 13:25:05 -05:00
Alan Rominger 2fdce43f9e Bulk save facts, and move to before status change (#12998) 
* Facts scaling fixes for large inventory, timing issue

Move save of Ansible facts to before the job status changes
  this is considered an acceptable delay with the other
  performance fixes here

Remove completely unrelated unused facts method

Scale related changes to facts saving:
  Use .iterator() on queryset when looping
  Change save to bulk_update
  Apply bulk_update in batches of 100, to reduce memory
  Only save a single file modtime, avoiding large dict

Use decorator for long func time logging
  update decorator to fill in format statement
 2022-11-15 15:18:06 -05:00
Alan Rominger 1f939aa25e Merge pull request #12884 from AlanCoding/is_testing 
[tech debt] Move the IS_TESTING method out of settings
 2022-11-09 15:29:35 -05:00
Alan Rominger d6004fd2d3 Merge pull request #13153 from AlanCoding/cleanup_args 
Refactor test_get_cleanup_task_kwargs_active_jobs and add new test
 2022-11-08 16:06:13 -05:00
Alan Rominger 8e5af2b5f2 Merge pull request #13092 from AlanCoding/tags_no_record 
Fix error when applying taggit app tags to objects
 2022-11-07 23:07:59 -05:00
Alan Rominger 893dba7076 Merge pull request #13114 from AlanCoding/use_seth 
Remove ssh_key_data fix, handled in runner now
 2022-11-02 15:41:00 -04:00
Alan Rominger d571b9bbbc Refactor test_get_cleanup_task_kwargs_active_jobs and add new test 
This takes some logic out of the queryset logic,
  using some established assumptions about the task manager
  if a job lands on a hybrid node (or is a project update) then
  it will have the same controller and execution node

With that established, the queryset can be simplified
 2022-11-02 15:14:16 -04:00
Alan Rominger 6538d34b48 Remove ssh_key_data fix, handled in runner now 2022-10-31 11:01:28 -04:00
Alan Rominger ed318ea784 Merge pull request #13082 from AlanCoding/health_check_stdout 
Include stdout from health check if it is not nothing
 2022-10-28 13:59:21 -04:00
Alan Rominger d6fea77082 Include stdout from health check if it is not nothing 2022-10-26 16:26:59 -04:00
Alan Rominger 487efb77ce Fix error when apply taggit app tags to objects 2022-10-21 14:53:31 -04:00
Alan Rominger 192f45bbd0 Make canceling view non-atomic to fix 500 errors with job bursts (#13072) 
* Make canceling view non-atomic to fix 500 errors with job bursts

* Update test calls for cancel method changes
 2022-10-20 15:02:54 -04:00
Alan Rominger e013d25e2d Merge pull request #13073 from AlanCoding/max_conn_deadlock 
Fix dispatcher connection deadlock w scheduler and cleanup
 2022-10-19 14:06:12 -04:00
Alan Rominger cba780a8f8 Fix dispatcher connection deadlock w scheduler and cleanup 2022-10-19 12:12:15 -04:00
Alan Rominger 3fc67dc76c Merge pull request #13081 from AlanCoding/raw_string 
Use raw string to satisfy linter rules
 2022-10-19 12:08:02 -04:00
Alan Rominger 6f85aef5fe Use raw string to satisfy linter rules 2022-10-19 11:07:43 -04:00
Alan Rominger 4d9b8400da Merge pull request #12887 from AlanCoding/more_rules 
[tech debt] Add new flake8 rules to do some meaningful corrections
 2022-10-18 20:27:00 -04:00
Alan Rominger a0594c8948 Merge pull request #13035 from AlanCoding/mike_patch 
Only perform prompts validation if prompts fields are submitted
 2022-10-12 15:42:55 -04:00
Alan Rominger 0f150aa3b3 Only perform prompts validation if prompts fields are submitted 2022-10-11 10:50:03 -04:00
Alan Rominger 7849c0fb1e Merge pull request #12694 from AlanCoding/whoami 
Shortcut Instance.objects.me when possible
 2022-10-06 15:25:29 -04:00
Alan Rominger 805091cfc1 Merge pull request #12969 from AlanCoding/code_deletions 
[tech debt] Remove imports and references that are no longer needed
 2022-10-06 09:33:03 -04:00
Alan Rominger a64467c5a6 Shortcut Instance.objects.me when possible 2022-10-05 09:11:42 -04:00
Alan Rominger 58772d79c7 Remove unnecessary imports by deleting NOQA markers 2022-10-05 09:09:03 -04:00
Alan Rominger 235ed2f0d0 Remove current_user variable no longer used 2022-10-05 09:09:03 -04:00
Alan Rominger cfce31419d Move the IS_TESTING method out of settings 2022-09-28 11:19:10 -04:00
Alan Rominger d3eb2c1975 Add new flak8 rules to do some meaningful corrections 2022-09-27 20:36:42 -04:00
Alan Rominger 1a696c4f25 Merge pull request #12864 from AlanCoding/project_groups 
Avoid cache warning for dispatching control type tasks
 2022-09-27 20:00:12 -04:00
Alan Rominger 5648d9d96f Avoid cache warning for dispatching control type tasks 2022-09-27 15:18:13 -04:00
Alan Rominger ff6fb32297 Merge pull request #12875 from ansible/feature-prompt-on-launch-on-templates 
Feature prompt on launch on templates
 2022-09-23 09:16:02 -04:00
Alan Rominger e231e08869 Fix bug with missing parent field and diff with parent 
Remove corresponding views for job instance_groups

Validate job_slice_count in API

Remove defaults from some job launch view prompts
  the null default is preferable
 2022-09-22 16:08:23 -04:00
Alan Rominger 61093b2532 Treat instance_groups prompt as template-less 2022-09-22 16:08:22 -04:00
Alan Rominger 816e491d17 Fix another bug applying extra_vars to incompatible job types 2022-09-22 16:08:22 -04:00
Alan Rominger b38e08174a Write logic to combing workflow labels, IGs with nodes 
Additionally, move the inventory-specific hacks of yesteryear
  into the prompts_dict method of the WorkflowJob model
  try to make it clear exactly what this is hacking and why

Correctly summarize label prompts, and add missing EE

Expand unit tests to apply more fields

adding missing fields to preserve during copy to workflow.py

Fix bug where empty workflow job vars blanked node vars (#12904)

* Fix bug where empty workflow job vars blanked node vars

* Fix bug where workflow job has no extra_vars, add test

* Add empty workflow job extra vars to assure fix
 2022-09-22 16:08:07 -04:00
Alan Rominger 64dad61b29 Add support for instance_groups and labels on schedule create 2022-09-22 16:08:06 -04:00
Alan Rominger ef90adb67e Complete consolidation of the label views 2022-09-22 16:08:06 -04:00
Alan Rominger 34e8087aee DRY edits to access classes for new prompts 
Remove if-not-data conditional from WFJTnode.can_change
  these are cannonical for can_add, but this looks like a bug

Change JTaccess.can_unattach to call same method in super()
  previously called can_attach, which is problematic

Better consolidate launch config m2m related checks

Test and fix pre-existing WFJT node RBAC bug

recognize not-provided instance group list on launch, avoiding bug where it fell back to default

fix bug where timeout field was saved on WFJT nodes after creating approval node

remove labels from schedule serializer summary_fields

remove unnecessary prefetch of credentials from WFJT node queryset
 2022-09-22 15:58:15 -04:00
Alan Rominger 68e11d2b81 Add WorkflowJob.instance_groups and distinguish from char_prompts 
This removes a loop that ran on import
  the loop was giving the wrong behavior
  and it initialized too many fields as char_prompts fields

With this, we will now enumerate the char_prompts type fields manually
 2022-09-22 15:39:49 -04:00
Alan Rominger 15964dc395 Merge pull request #11745 from AlanCoding/cancel_rework_no_close 
Close database connections while processing job output
 2022-09-06 15:45:29 -04:00
Alan Rominger 430f1986c7 Merge pull request #12830 from AlanCoding/dev_stuff 
Fix LDAP volume conditional, better metrics interval
 2022-09-06 11:51:51 -04:00
Alan Rominger f512971991 Add project sync to job cancel chain 2022-09-05 22:29:19 -04:00
Alan Rominger 53de245877 Fix LDAP volume conditional, better metrics interval 2022-09-04 22:33:12 -04:00
Alan Rominger 725d6fa896 Merge pull request #12820 from AlanCoding/five_seconds 
Make the metrics default sampling interval 5s
 2022-09-02 15:21:57 -04:00
Alan Rominger ccbc8ce7de Make the metrics default sampling interval 5s 2022-09-02 13:38:49 -04:00
Alan Rominger c59bbdecdb Refactor canceling to work through messaging and signals, not database 
If canceled attempted before, still allow attempting another cancel
in this case, attempt to send the sigterm signal again.
Keep clicking, you might help!

Replace other cancel_callbacks with sigterm watcher
  adapt special inventory mechanism for this too

Get rid of the cancel_watcher method with exception in main thread

Handle academic case of sigterm race condition

Process cancelation as control signal

Fully connect cancel method and run_dispatcher to control

Never transition workflows directly to canceled, add logs
 2022-09-01 15:20:31 -04:00
Alan Rominger 310e354164 Merge pull request #12769 from AlanCoding/self_conn 
Fix sanity check to use the relevant active connection
 2022-08-29 20:36:48 -04:00
Alan Rominger 6d207d2490 Merge pull request #12754 from kdelee/fix_metrics_consumed_capacity 
calcuate consumed capacity in same way in metrics
 2022-08-29 16:37:53 -04:00
Alan Rominger 01037fa561 Fix sanity check to use the relevant active connection 2022-08-29 16:33:07 -04:00
Alan Rominger 61f3e5cbed Merge pull request #12702 from AlanCoding/poll_cancel 
Check exit conditions in loop waiting for project flock
 2022-08-29 16:29:39 -04:00
Alan Rominger 44995e944a Merge pull request #12766 from AlanCoding/lazy_no_more 
Revert "Merge pull request #12584 from AlanCoding/lazy_workers"
 2022-08-29 16:06:50 -04:00
Alan Rominger 2437a84b48 Minor changes to instance loop structure 2022-08-29 14:28:50 -04:00
Alan Rominger 974f845059 Revert "Merge pull request #12584 from AlanCoding/lazy_workers" 
This reverts commit 64157f7207, reversing
changes made to 9e8ba6ca09.
 2022-08-28 23:04:13 -04:00
Alan Rominger edc3da85cc Revert "Fix errors in websocket code due to missing template" 2022-08-20 19:09:57 -04:00
Alan Rominger 2357e24d1d Merge pull request #12701 from AlanCoding/no_more_schedules 
Make schedule teardown more reliable
 2022-08-20 07:05:21 -04:00
Alan Rominger 88bf03c6bf Check exit conditions in loop waiting for project flock 2022-08-19 16:08:56 -04:00
Alan Rominger 4b8a56be39 Make schedule teardown more reliable 2022-08-19 15:42:00 -04:00
Alan Rominger 2aa99234f4 Merge pull request #12692 from AlanCoding/mop_up 
Fix errors in websocket code due to missing template
 2022-08-19 14:46:10 -04:00
Alan Rominger 704e4781d9 Fix errors in websocket code due to missing template 2022-08-18 14:05:06 -04:00
Alan Rominger 4a8613ce4c Avoid updating modified_by from None to None (#11838) 
This should help the case of inventory updates in particular
  where imported hosts are managed by the system
 2022-08-18 11:39:29 -04:00
Alan Rominger e87fabe6bb Submit job to dispatcher as part of transaction (#12573) 
Make it so that submitting a task to the dispatcher happens as part of the transaction.
  this applies to dispatcher task "publishers" which NOTIFY the pg_notify queue
  if the transaction is not successful, it will not be sent, as per postgres docs

This keeps current behavior for pg_notify listeners
  practically, this only applies for the awx-manage run_dispatcher service
  this requires creating a separate connection and keeping it long-lived
  arbitrary code will occasionally close the main connection, which would stop listening

Stop sending the waiting status websocket message
  this is required because the ordering cannot be maintained with other changes here
  the instance group data is moved to the running websocket message payload

Move call to create_partition from task manager to pre_run_hook
  mock this in relevant unit tests
 2022-08-18 09:43:53 -04:00
Alan Rominger 532aa83555 Merge pull request #11833 from AlanCoding/facts_update_fields 
Use update_fields for Ansible facts update
 2022-08-17 22:37:45 -04:00
Alan Rominger d87bb973d5 Merge pull request #12090 from AlanCoding/mind_your_own_business 
Avoid parent instance update when status was unchanged
 2022-08-17 22:29:31 -04:00
Alan Rominger a72da3bd1a Merge pull request #12582 from AlanCoding/clean_and_forget 
Move reaper logic into worker, avoiding bottlenecks
 2022-08-17 18:53:47 -04:00
Alan Rominger 56df3f0c2a Merge pull request #12671 from AlanCoding/cut_the_line 
Avoid dependency manager for jobs with no deps
 2022-08-17 18:50:52 -04:00
Alan Rominger e0c59d12c1 Change data structure so we can conditionally reap waiting jobs 2022-08-17 16:00:30 -04:00
Alan Rominger 7645cc2707 Remove mocks for reap method that was removed 2022-08-17 15:43:29 -04:00
Alan Rominger 6719010050 Add back in cleanup call 2022-08-17 15:42:48 -04:00
Alan Rominger ccd46a1c0f Move reaper logic into worker, avoiding bottlenecks 2022-08-17 15:42:47 -04:00
Alan Rominger 4fca27c664 Merge pull request #12289 from AlanCoding/idle_help 
Correct help text for job idle timeout
 2022-08-17 13:55:44 -04:00
Alan Rominger 51be22aebd Merge pull request #12668 from AlanCoding/graph_tweaks 
Remove an old metrics field and add a new one to dashboard
 2022-08-17 13:49:17 -04:00
Alan Rominger 54b21e5872 Avoid dependency manager for jobs with no deps 2022-08-17 13:32:59 -04:00
Alan Rominger 85beb9eb70 Merge pull request #12676 from AlanCoding/forward_picks 
Stability fixes, and related logging for slowdowns in dispatcher task processing
 2022-08-17 13:32:34 -04:00
Alan Rominger 56739ac246 Use delay_update to set error message, according to merge note 2022-08-17 11:45:40 -04:00
Alan Rominger 1ea3c564df Apply a failed status if cancel_flag is not set 2022-08-17 11:42:09 -04:00
Alan RomingerandElijah DeLee 621833ef0e Add extra workers if computing based on memory 
Co-authored-by: Elijah DeLee <kdelee@redhat.com>
 2022-08-17 11:41:59 -04:00
Alan RomingerandShane McDonald 43a53f41dd Add logs about heartbeat skew 
Co-authored-by: Shane McDonald <me@shanemcd.com>
 2022-08-17 11:33:59 -04:00
Alan Rominger a3fef27002 Add logs to debug waiting bottlenecking 2022-08-17 11:33:49 -04:00
Alan Rominger cfc1255812 Merge pull request #12442 from AlanCoding/waiting_reaper 
Fix false reaper false-positives of waiting jobs that are waiting for worker
 2022-08-17 11:20:05 -04:00
Alan RomingerandElijah DeLee 278db2cdde Split reaper for running and waiting jobs 
Avoid running jobs that have already been reapted

Co-authored-by: Elijah DeLee <kdelee@redhat.com>

Remove unnecessary extra actions

Fix waiting jobs in other cases of reaping
 2022-08-17 10:53:29 -04:00
Alan Rominger 64157f7207 Merge pull request #12584 from AlanCoding/lazy_workers 
Wait 60 seconds before scaling down a worker
 2022-08-17 10:18:19 -04:00
Alan Rominger 9e8ba6ca09 Merge pull request #12494 from AlanCoding/revival 
Register system again if deleted by another pod
 2022-08-17 10:12:39 -04:00
Alan Rominger 268ab128d7 Merge pull request #12527 from AlanCoding/offline_db 
Further resiliency changes, specifically focused on case of database going offline
 2022-08-17 10:10:50 -04:00
Alan Rominger fad5934c1e Merge pull request #12356 from AlanCoding/copytree_neo 
Replace git shallow clone with shutil.copytree
 2022-08-17 10:07:28 -04:00
Alan Rominger c9e3873a28 Use update_fields for Ansible facts update 2022-08-17 08:22:41 -04:00
Alan Rominger 11e63e2e89 Remove an old metrics field and add a new one to dashboard 2022-08-16 22:37:27 -04:00
Alan Rominger f1bd1f1dfc Merge pull request #12658 from AlanCoding/more_panels 
Add more graphs for task manager refactor
 2022-08-15 16:07:43 -04:00
Alan Rominger f6da9a5073 Add more graphs for task manager refactor 2022-08-15 15:29:34 -04:00
Alan Rominger ea07aef73e Correct help text for job idle timeout 2022-08-11 09:39:29 -04:00
Alan Rominger 46be2d9e5b Replace git shallow clone with shutil.copytree 
Introduce build_project_dir method
  the base method will create an empty project dir for workdir

Share code between job and inventory tasks with new mixin
  combine rest of pre_run_hook logic
  structure to hold lock for entire sync process

force sync to run for inventory updates due to UI issues

Remove reference to removed scm_last_revision field
 2022-08-10 16:18:56 -04:00
Alan Rominger 998000bfbe Surface correct error from bulk_create on unrecoverable error 2022-08-10 16:16:57 -04:00
Alan Rominger 43a50cc62c Fix event counting in error handling path 2022-08-10 16:16:57 -04:00
Alan Rominger 30f556f845 Further resiliency changes focused on offline database 
Make logs from database outage more manageable

Raise exception if update_model never recovers from problem
 2022-08-10 16:16:57 -04:00
Alan Rominger c5985c4c81 Change lazy worker method name and adjust log 2022-08-10 16:12:03 -04:00
Alan Rominger a9170236e1 Wait 60 seconds before scaling down a worker 2022-08-10 16:12:03 -04:00
Alan Rominger a0103acbef Inherit from our own APIView, not rest framework 2022-08-10 15:31:19 -04:00
Alan Rominger f7e6a32444 Optimize task manager with debug toolbar, adjust prefetch (#12588) 2022-08-10 10:05:13 -04:00
Alan Rominger 585d3f4e2a Register system again if deleted by another pod 
Avoid cases where missing instance
  would throw error on startup
  this gives time for heartbeat to register it
 2022-08-08 22:36:17 -04:00
Alan Rominger 279cebcef3 Merge pull request #12586 from AlanCoding/connections_graph 
Add a graph to show database connections being used
 2022-08-08 15:49:20 -04:00