Commit Graph
100 Commits
Author SHA1 Message Date
Alan Rominger 06808ef4c4 Merge pull request #13608 from AlanCoding/keepalive
Use ansible-runner change to get periodic keep-alive messages in K8S
2023-03-06 14:34:37 -05:00
Alan RomingerandShane McDonald 90f54b98cd Update keepalive setting help_text to be more direct
Co-authored-by: Shane McDonald <me@shanemcd.com>
2023-02-28 09:04:07 -05:00
Alan Rominger 6fa22f5be2 Add UI for the new setting 2023-02-28 09:04:07 -05:00
Alan Rominger d5de1f9d11 Make use of new keepalive messages from ansible-runner
Make setting API configurable and process keepalive events
  when seen in the event callback

Use env var in pod spec and make it specific to K8S
2023-02-28 09:04:07 -05:00
Alan Rominger ccaace8b30 Merge pull request #13541 from npithonDR/devel
Fix error for byweekday in schedule_rruleset
2023-02-27 10:24:48 -05:00
Alan Rominger 3051384f95 Follow suggestion from comment, split if NOT list 2023-02-23 12:05:32 -05:00
Alan Rominger 811ecb8673 Follow suggestion from comment, split if NOT list 2023-02-23 12:05:21 -05:00
Alan Rominger 4499a50019 Merge pull request #13595 from sean-m-sullivan/devel
fix inventory prompt on launch for workflow nodes
2023-02-22 10:23:02 -05:00
Alan Rominger 3fe46e2e27 Merge pull request #13606 from AlanCoding/copy_login
Give proper 401 code to user not logged in
2023-02-21 16:31:23 -05:00
Alan Rominger 6d3f39fe92 Give proper 401 code to user not logged in 2023-02-21 13:34:29 -05:00
Alan Rominger a3233b5fdd Merge pull request #13594 from AlanCoding/approval_collection
Add integration test and docs for workflow_approval module
2023-02-21 09:03:17 -05:00
Alan Rominger b5f240ce70 Add integration test and docs for workflow_approval module 2023-02-17 15:10:59 -05:00
Alan Rominger 7f50679e68 Do not create setting with invalid value in data migration (#13576)
* Do not create setting with invalid value in data migration

* Add test for conf app data migration
2023-02-15 14:54:46 -05:00
Alan Rominger 6772fb876b Merge pull request #13522 from AlanCoding/no_events
Skip callback receiver bulk_create with 0 events
2023-02-06 12:02:20 -05:00
Alan Rominger 51112b95bc Add test for callback events flush with nothing in the buffer 2023-02-05 22:46:50 -05:00
Alan Rominger 6c1d4a5cfd Skip callback receiver bulk_create with 0 events 2023-02-04 12:10:39 -05:00
Alan Rominger 2e9106d8ea Merge pull request #13516 from AlanCoding/github_ci_runner
Attempt to consolidate CI logic with github_ci_runner target
2023-02-03 15:39:39 -05:00
Alan Rominger 84822784e8 Get rid of label because it is confusing 2023-02-03 14:24:43 -05:00
Alan Rominger 0f3adb52b1 Add help comments and reorg targets for separation 2023-02-03 14:24:43 -05:00
Alan Rominger 59da9a29df Delete everything about py_version in CI workflow 2023-02-03 14:24:43 -05:00
Alan Rominger a949ee048a Consolidate CI logic with github_ci_runner target
Delete outright the step to install python

Fix typo that failed to label stage
2023-02-03 14:24:43 -05:00
Alan Rominger 9b2725e5fe Merge pull request #13500 from AlanCoding/group_options
Fix OPTIONS permissions bug in groups list
2023-02-02 12:55:04 -05:00
Alan Rominger 1af955d28c Merge pull request #13267 from philipsd6/feature/complex_extra_vars
Enable support for injecting complex extra vars
2023-02-02 10:13:49 -05:00
Alan Rominger 6997876da6 Fix OPTIONS permissions bug in groups list 2023-02-01 16:19:24 -05:00
Alan Rominger 93d84fe2c9 Merge pull request #13502 from AlanCoding/new_black
Update to comply with new black rules
2023-02-01 16:18:50 -05:00
Alan Rominger f5785976be Update to comply with new black rules 2023-02-01 14:59:38 -05:00
Alan Rominger a2f528e6e5 Fix syntax bug that came from fixing sanity tests (#13473) 2023-01-31 15:55:20 -05:00
Alan Rominger 9777ce7fb8 Touchup of validation logic from testing 2023-01-23 11:01:08 -05:00
Alan Rominger 8a4059d266 Workaround for events with NUL char, touch up error loop (#13398)
* Workaround for events with NUL char, touch up error loop

This fixes an error where some events would not save
  due to having the 0x00 character which errors in postgres
  this adds a line to replace it with empty text

Hitting that kind of event put us in an infinite error loop
  so this change makes a number of changes to prevent similar loops
  the showcase example is a negative counter,
  this is not realistic in the real world but works for unit tests

These error loop fixes seek to esablish the cases where we clear the buffer
Some logic is removed from the outer loop, with the idea that
ensure_connection will better distinguish flake

* From review comments, delay NUL char sanitization to later

Use pop to make list operations more clear

* Fix incorrect use of pop
2023-01-19 13:36:23 -05:00
Alan Rominger ac9871b36f Merge pull request #13361 from relrod/sanity
[collection] Run sanity tests outside of our container
2022-12-21 11:00:21 -05:00
Alan RomingerandJohn R Barker f739908ccf Add comment about Ansible-core being installed by default
Co-authored-by: John R Barker <john@johnrbarker.com>
2022-12-21 09:57:00 -05:00
Alan Rominger cf1ec07eab Changes to run sanity tests locally
Use a Makefile arg for the ansible-test sanity CLI args
  defaults to --docker
  in the future we probably need to customize python versions

Copy the rule exception for Ansible 2.15
  this helps people who are running from Ansible devel
2022-12-21 09:53:22 -05:00
Alan Rominger 41f3f381ec Merge pull request #13352 from AlanCoding/dont_pass_subtasks
Remove `subtasks` keyword arg that can exceed pg_notify max message length
2022-12-20 16:25:39 -05:00
Alan Rominger ac8cff75ce Run collection sanity tests in CI (#13356)
* Run collection sanity tests in CI

This requires adding a Makefile install of ansible-core

Fake the version to make semver check happy

* Fixes from ansible-test sanity failures

* Exclude the export module due to awxkit requirement

* Fix broken ansible-test rule exceptions

remove Ansible 2.14 exclusions that make ansible-test ERROR, saying they are not needed
2022-12-20 16:06:25 -05:00
Alan Rominger 94b34b801c Avoid unbounded kwargs by fetching subtasks inside handle_work_error
Update tests to new handle_work_error call pattern

Handle blame correctly with multiple serial deps
  add new test case corresponding to this scenario
2022-12-19 16:02:51 -05:00
Alan Rominger d07076b686 Merge pull request #13330 from AlanCoding/ask_me_for_tags
Fill in rest of ask_tags handling for WFJT module
2022-12-15 10:59:17 -05:00
Alan Rominger 51f8e362dc Add tags prompt to integration test 2022-12-14 09:10:15 -05:00
Alan Rominger beaf6b6058 Fill in rest of ask_tags handling for WFJT module 2022-12-13 16:38:25 -05:00
Alan Rominger fe768a159b Merge pull request #13295 from AlanCoding/raw_instance_data
Remove un-editable Instance fields from pre-filled edit data in API browser
2022-12-13 15:16:34 -05:00
Alan Rominger c1ebea858b Merge pull request #13291 from AlanCoding/policy_want_a_cracker
Add missing disassociate trigger for policy task
2022-12-13 11:35:22 -05:00
Alan Rominger a0b8215c06 Merge pull request #13296 from AlanCoding/signing_bug
Fix bug, sign work based signing, not verification
2022-12-07 08:44:57 -05:00
Alan Rominger f88b993b18 Fix bug, sign work based signing, not verification 2022-12-06 16:21:17 -05:00
Alan Rominger 4a7f4d0ed4 Remove uneditable Instance fields from API browser 2022-12-06 15:20:04 -05:00
Alan Rominger 6e08c3567f Add missing disassociate trigger for policy task 2022-12-06 14:43:13 -05:00
Alan Rominger 239959a4c9 Merge pull request #13213 from AlanCoding/execution_signing
Fix fallout from turning off work signing in docker-compose
2022-11-18 15:22:18 -05:00
Alan Rominger 84f2b91105 Fix fallout from turning off work signing in docker-compose 2022-11-18 13:25:05 -05:00
Alan Rominger 2fdce43f9e Bulk save facts, and move to before status change (#12998)
* Facts scaling fixes for large inventory, timing issue

Move save of Ansible facts to before the job status changes
  this is considered an acceptable delay with the other
  performance fixes here

Remove completely unrelated unused facts method

Scale related changes to facts saving:
  Use .iterator() on queryset when looping
  Change save to bulk_update
  Apply bulk_update in batches of 100, to reduce memory
  Only save a single file modtime, avoiding large dict

Use decorator for long func time logging
  update decorator to fill in format statement
2022-11-15 15:18:06 -05:00
Alan Rominger 1f939aa25e Merge pull request #12884 from AlanCoding/is_testing
[tech debt] Move the IS_TESTING method out of settings
2022-11-09 15:29:35 -05:00
Alan Rominger d6004fd2d3 Merge pull request #13153 from AlanCoding/cleanup_args
Refactor test_get_cleanup_task_kwargs_active_jobs and add new test
2022-11-08 16:06:13 -05:00
Alan Rominger 8e5af2b5f2 Merge pull request #13092 from AlanCoding/tags_no_record
Fix error when applying taggit app tags to objects
2022-11-07 23:07:59 -05:00
Alan Rominger 893dba7076 Merge pull request #13114 from AlanCoding/use_seth
Remove ssh_key_data fix, handled in runner now
2022-11-02 15:41:00 -04:00
Alan Rominger d571b9bbbc Refactor test_get_cleanup_task_kwargs_active_jobs and add new test
This takes some logic out of the queryset logic,
  using some established assumptions about the task manager
  if a job lands on a hybrid node (or is a project update) then
  it will have the same controller and execution node

With that established, the queryset can be simplified
2022-11-02 15:14:16 -04:00
Alan Rominger 6538d34b48 Remove ssh_key_data fix, handled in runner now 2022-10-31 11:01:28 -04:00
Alan Rominger ed318ea784 Merge pull request #13082 from AlanCoding/health_check_stdout
Include stdout from health check if it is not nothing
2022-10-28 13:59:21 -04:00
Alan Rominger d6fea77082 Include stdout from health check if it is not nothing 2022-10-26 16:26:59 -04:00
Alan Rominger 487efb77ce Fix error when apply taggit app tags to objects 2022-10-21 14:53:31 -04:00
Alan Rominger 192f45bbd0 Make canceling view non-atomic to fix 500 errors with job bursts (#13072)
* Make canceling view non-atomic to fix 500 errors with job bursts

* Update test calls for cancel method changes
2022-10-20 15:02:54 -04:00
Alan Rominger e013d25e2d Merge pull request #13073 from AlanCoding/max_conn_deadlock
Fix dispatcher connection deadlock w scheduler and cleanup
2022-10-19 14:06:12 -04:00
Alan Rominger cba780a8f8 Fix dispatcher connection deadlock w scheduler and cleanup 2022-10-19 12:12:15 -04:00
Alan Rominger 3fc67dc76c Merge pull request #13081 from AlanCoding/raw_string
Use raw string to satisfy linter rules
2022-10-19 12:08:02 -04:00
Alan Rominger 6f85aef5fe Use raw string to satisfy linter rules 2022-10-19 11:07:43 -04:00
Alan Rominger 4d9b8400da Merge pull request #12887 from AlanCoding/more_rules
[tech debt] Add new flake8 rules to do some meaningful corrections
2022-10-18 20:27:00 -04:00
Alan Rominger a0594c8948 Merge pull request #13035 from AlanCoding/mike_patch
Only perform prompts validation if prompts fields are submitted
2022-10-12 15:42:55 -04:00
Alan Rominger 0f150aa3b3 Only perform prompts validation if prompts fields are submitted 2022-10-11 10:50:03 -04:00
Alan Rominger 7849c0fb1e Merge pull request #12694 from AlanCoding/whoami
Shortcut Instance.objects.me when possible
2022-10-06 15:25:29 -04:00
Alan Rominger 805091cfc1 Merge pull request #12969 from AlanCoding/code_deletions
[tech debt] Remove imports and references that are no longer needed
2022-10-06 09:33:03 -04:00
Alan Rominger a64467c5a6 Shortcut Instance.objects.me when possible 2022-10-05 09:11:42 -04:00
Alan Rominger 58772d79c7 Remove unnecessary imports by deleting NOQA markers 2022-10-05 09:09:03 -04:00
Alan Rominger 235ed2f0d0 Remove current_user variable no longer used 2022-10-05 09:09:03 -04:00
Alan Rominger cfce31419d Move the IS_TESTING method out of settings 2022-09-28 11:19:10 -04:00
Alan Rominger d3eb2c1975 Add new flak8 rules to do some meaningful corrections 2022-09-27 20:36:42 -04:00
Alan Rominger 1a696c4f25 Merge pull request #12864 from AlanCoding/project_groups
Avoid cache warning for dispatching control type tasks
2022-09-27 20:00:12 -04:00
Alan Rominger 5648d9d96f Avoid cache warning for dispatching control type tasks 2022-09-27 15:18:13 -04:00
Alan Rominger ff6fb32297 Merge pull request #12875 from ansible/feature-prompt-on-launch-on-templates
Feature prompt on launch on templates
2022-09-23 09:16:02 -04:00
Alan Rominger e231e08869 Fix bug with missing parent field and diff with parent
Remove corresponding views for job instance_groups

Validate job_slice_count in API

Remove defaults from some job launch view prompts
  the null default is preferable
2022-09-22 16:08:23 -04:00
Alan Rominger 61093b2532 Treat instance_groups prompt as template-less 2022-09-22 16:08:22 -04:00
Alan Rominger 816e491d17 Fix another bug applying extra_vars to incompatible job types 2022-09-22 16:08:22 -04:00
Alan Rominger b38e08174a Write logic to combing workflow labels, IGs with nodes
Additionally, move the inventory-specific hacks of yesteryear
  into the prompts_dict method of the WorkflowJob model
  try to make it clear exactly what this is hacking and why

Correctly summarize label prompts, and add missing EE

Expand unit tests to apply more fields

adding missing fields to preserve during copy to workflow.py

Fix bug where empty workflow job vars blanked node vars (#12904)

* Fix bug where empty workflow job vars blanked node vars

* Fix bug where workflow job has no extra_vars, add test

* Add empty workflow job extra vars to assure fix
2022-09-22 16:08:07 -04:00
Alan Rominger 64dad61b29 Add support for instance_groups and labels on schedule create 2022-09-22 16:08:06 -04:00
Alan Rominger ef90adb67e Complete consolidation of the label views 2022-09-22 16:08:06 -04:00
Alan Rominger 34e8087aee DRY edits to access classes for new prompts
Remove if-not-data conditional from WFJTnode.can_change
  these are cannonical for can_add, but this looks like a bug

Change JTaccess.can_unattach to call same method in super()
  previously called can_attach, which is problematic

Better consolidate launch config m2m related checks

Test and fix pre-existing WFJT node RBAC bug

recognize not-provided instance group list on launch, avoiding bug where it fell back to default

fix bug where timeout field was saved on WFJT nodes after creating approval node

remove labels from schedule serializer summary_fields

remove unnecessary prefetch of credentials from WFJT node queryset
2022-09-22 15:58:15 -04:00
Alan Rominger 68e11d2b81 Add WorkflowJob.instance_groups and distinguish from char_prompts
This removes a loop that ran on import
  the loop was giving the wrong behavior
  and it initialized too many fields as char_prompts fields

With this, we will now enumerate the char_prompts type fields manually
2022-09-22 15:39:49 -04:00
Alan Rominger 15964dc395 Merge pull request #11745 from AlanCoding/cancel_rework_no_close
Close database connections while processing job output
2022-09-06 15:45:29 -04:00
Alan Rominger 430f1986c7 Merge pull request #12830 from AlanCoding/dev_stuff
Fix LDAP volume conditional, better metrics interval
2022-09-06 11:51:51 -04:00
Alan Rominger f512971991 Add project sync to job cancel chain 2022-09-05 22:29:19 -04:00
Alan Rominger 53de245877 Fix LDAP volume conditional, better metrics interval 2022-09-04 22:33:12 -04:00
Alan Rominger 725d6fa896 Merge pull request #12820 from AlanCoding/five_seconds
Make the metrics default sampling interval 5s
2022-09-02 15:21:57 -04:00
Alan Rominger ccbc8ce7de Make the metrics default sampling interval 5s 2022-09-02 13:38:49 -04:00
Alan Rominger c59bbdecdb Refactor canceling to work through messaging and signals, not database
If canceled attempted before, still allow attempting another cancel
in this case, attempt to send the sigterm signal again.
Keep clicking, you might help!

Replace other cancel_callbacks with sigterm watcher
  adapt special inventory mechanism for this too

Get rid of the cancel_watcher method with exception in main thread

Handle academic case of sigterm race condition

Process cancelation as control signal

Fully connect cancel method and run_dispatcher to control

Never transition workflows directly to canceled, add logs
2022-09-01 15:20:31 -04:00
Alan Rominger 310e354164 Merge pull request #12769 from AlanCoding/self_conn
Fix sanity check to use the relevant active connection
2022-08-29 20:36:48 -04:00
Alan Rominger 6d207d2490 Merge pull request #12754 from kdelee/fix_metrics_consumed_capacity
calcuate consumed capacity in same way in metrics
2022-08-29 16:37:53 -04:00
Alan Rominger 01037fa561 Fix sanity check to use the relevant active connection 2022-08-29 16:33:07 -04:00
Alan Rominger 61f3e5cbed Merge pull request #12702 from AlanCoding/poll_cancel
Check exit conditions in loop waiting for project flock
2022-08-29 16:29:39 -04:00
Alan Rominger 44995e944a Merge pull request #12766 from AlanCoding/lazy_no_more
Revert "Merge pull request #12584 from AlanCoding/lazy_workers"
2022-08-29 16:06:50 -04:00
Alan Rominger 2437a84b48 Minor changes to instance loop structure 2022-08-29 14:28:50 -04:00
Alan Rominger 974f845059 Revert "Merge pull request #12584 from AlanCoding/lazy_workers"
This reverts commit 64157f7207, reversing
changes made to 9e8ba6ca09.
2022-08-28 23:04:13 -04:00
Alan Rominger edc3da85cc Revert "Fix errors in websocket code due to missing template" 2022-08-20 19:09:57 -04:00
Alan Rominger 2357e24d1d Merge pull request #12701 from AlanCoding/no_more_schedules
Make schedule teardown more reliable
2022-08-20 07:05:21 -04:00
Alan Rominger 88bf03c6bf Check exit conditions in loop waiting for project flock 2022-08-19 16:08:56 -04:00
Alan Rominger 4b8a56be39 Make schedule teardown more reliable 2022-08-19 15:42:00 -04:00