Commit Graph

8433 Commits

Author SHA1 Message Date
Elijah DeLee
b26d2ab0e9 fix looking at wrong id for wf allow_simultaneous 2022-08-05 14:33:25 -04:00
Elijah DeLee
7eb0c7dd28 exit task manager loops early if we are timed out
add settings to define task manager timeout and grace period

This gives us still TASK_MANAGER_TIMEOUT_GRACE_PERIOD amount of time to
get out of the task manager.

Also, apply start task limit in WorkflowManager to starting pending
workflows
2022-08-05 14:33:24 -04:00
Elijah DeLee
236c1df676 fix lint errors 2022-08-05 14:33:24 -04:00
Seth Foster
ff118f2177 Manage pending workflow jobs in Workflow Manager
get_tasks uses UnifiedJob
Additionally, make local overrides run after development settings
2022-08-05 14:31:48 -04:00
Elijah DeLee
29d91da1d2 we can do all the work in one loop
more than saving the loop, we save building the WorkflowDag twice which
makes LOTS of queries!!!

Also, do a bulk update on the WorkflowJobNodes instead of saving in a
loop :fear:
2022-08-05 14:31:48 -04:00
Elijah DeLee
ad08eafb9a add debug views for task manager(s)
implement https://github.com/ansible/awx/issues/12446
in development environment, enable set of views that run
the task manager(s).

Also introduce a setting that disables any calls to schedule()
that do not originate from the debug views when in the development
environment. With guards around both if we are in the development
environment and the setting, I think we're pretty safe this won't get
triggered unintentionally.

use MODE to determine if we are in devel env

Also, move test for skipping task managers to the tasks file
2022-08-05 14:31:24 -04:00
Seth Foster
431b9370df Split TaskManager into
- DependencyManager spawns dependencies if necessary
- WorkflowManager processes running workflows to see if a new job is
  ready to spawn
- TaskManager starts tasks if unblocked and has execution capacity
2022-08-05 14:29:02 -04:00
Rick Elrod
87f729c642 [FieldLookupBackend] limit iexact to string fields (#12569)
Change:
- Case-insensitive search only makes sense on strings, so check the
  type of the field we are searching and ensure it is a string field
  (TextField, CharField, or some subclass thereof).

- This prevents a 500 error when a user uses iexact on, e.g., an
  integer field. Now, a 400 Bad Request is returned instead.

Test Plan:
- Added simple unit tests for iexact

Tickets:
- Fixes #9222

Signed-off-by: Rick Elrod <rick@elrod.me>
2022-07-26 12:46:50 -05:00
John Westcott IV
62fc3994fb Modifying SAML adapter to not auto-add default galaxy creds to orgs on login (#12504)
* Modifying SAML adapter to not auto-add default galaxy creds to orgs on login

* Adding test, fixing old tests and moving add_default_galaxy_credential to pipeline
2022-07-25 17:16:22 -03:00
Alan Rominger
2d310dc4e5 Optimize object creation by getting fewer empty relationships (#12508)
This optimizes the ActivityStreamSerializer by only getting many-to-many
  relationships that are speculatively non-empty
  based on information we have in other fields

We run this every time we create an object as an on_commit action
  so it is expected this will have a major impact on response times for launching jobs
2022-07-19 14:27:51 -04:00
John Westcott IV
389c4a3180 Adding fields to job_metadata for workflows and approval nodes (#12255) 2022-07-18 16:53:49 +02:00
Lila
aeffd6f393 Bumped up version number of the collector. 2022-07-13 09:59:41 -04:00
djyasin
ab6b4bad03 Merge branch 'ansible:devel' into devel 2022-07-13 09:53:22 -04:00
John Westcott IV
37ff9913d3 Adding GOOGLE_APPLICATION_CREDENTIALS env var (#12389)
* Adding GOOGLE_APPLICATION_CREDENTIALS env var
* Updating tests
2022-07-12 08:51:02 -04:00
Lila
1e57c84383 Added forks to unified jobs table.
Co-authored-by: sarabrajsingh <singh.sarabraj@gmail.com>
2022-07-01 10:30:48 -04:00
Alan Rominger
fd671ecc9d Give specific messages if job was killed due to SIGTERM or SIGKILL (#12435)
* Reap jobs on dispatcher startup to increase clarity, replace existing reaping logic

* Exit jobs if receiving SIGTERM signal

* Fix unwanted reaping on shutdown, let subprocess close out

* Add some sanity tests for signal module

* Add a log for an unhandled dispatcher error

* Refine wording of error messages

Co-authored-by: Elijah DeLee <kdelee@redhat.com>
2022-06-30 13:20:08 -04:00
Seth Foster
d2013bd416 Merge pull request #12366 from fosterseth/remove_update_on_project_update
Remove deprecated field update_on_project_update
2022-06-28 13:15:57 -04:00
Jeff Bradberry
ae7be525e1 Fix a ugettext deprecation that snuck back in
at some point after the Django 3.2 upgrade.
2022-06-27 13:27:35 -04:00
jainnikhil30
5062ce1e61 add database connection to the metrics endpoint (#12427)
* add database connection to the metrics endpoint

* bump the counts collector version to 1.2

* check for postgresql as database so to not break the tests
2022-06-27 09:37:23 -04:00
Seth Foster
0522233892 remove update_on_project_update from InventorySource 2022-06-24 15:27:08 -04:00
Alan Rominger
783b744bdb Pass combined artifacts from nested workflows into downstream nodes (#12223)
* Track combined artifacts on workflow jobs

* Avoid schema change for passing nested workflow artifacts

* Basic support for nested workflow artifacts, add test

* Forgot that only does not work with polymorphic

* Remove incorrect field

* Consolidate logic and prevent recursion with UJ artifacts method

* Stop trying to do precedence by status, filter for obvious ones

* Review comments about sets

* Fix up bug with convergence node paths and artifacts
2022-06-23 16:54:53 -03:00
Seth Foster
c92619a2dc Subsystem metrics reset_values should remove all redis keys 2022-06-16 16:54:37 -04:00
Elijah DeLee
7cbe112e4e possible work around for 500 on /api/v2/metrics (#12376)
we've observed this in development and some users have reported experiencing 500's on /api/v2/metrics because of a key error here where a metric is missing from a certain instance
2022-06-16 13:15:25 -04:00
Jessica Steurer
116dc0c480 Merge pull request #12340 from john-westcott-iv/shedule_timezone_12255
Add documentation around schedule timezone change
2022-06-15 15:34:49 -03:00
Seth Foster
2f82b75748 Add subsystem metrics for task manager 2022-06-14 11:00:11 -04:00
John Westcott IV
fddf292d47 Additional changes from review 2022-06-10 10:26:24 -04:00
John Westcott IV
48ebcd5918 Fixing assertion of schedule_zoneinfo 2022-06-10 10:26:23 -04:00
John Westcott IV
62dabcae63 Removing unneeded function 2022-06-10 10:26:23 -04:00
John Westcott IV
c836fafb61 modifying schedules API to return a list of links 2022-06-10 10:26:23 -04:00
Seth Foster
550d9d5e42 detect if job events are tree-like and collapsable in the UI 2022-06-09 14:25:39 -04:00
Alan Rominger
4543f6935f Only do substitutions for container path conversions with resolved paths (#12313)
* Resolve paths as much as possible before doing replacements

* Move unused method out of main code, test symlink
2022-06-09 11:36:29 -04:00
Alan Rominger
78d3d6dc94 Merge pull request #12219 from AlanCoding/really_skip
Change Demo Project status to successful
2022-06-09 11:19:57 -04:00
Jessica Steurer
bf7ab1ede7 Merge pull request #12315 from djyasin/job_tag_characters
Job tag characters
2022-06-08 12:09:18 -03:00
Jessica Steurer
f5102ed24d Merge pull request #12102 from john-westcott-iv/allow_fqcn
Respect optional fully qualified collection name (ansible.builtin.) for playbook identification
2022-06-07 16:44:36 -03:00
Rebeccah Hunter
76ffdbb993 Merge pull request #12308 from rebeccahhh/job_event_lag
Metrics for callback receiver job event lag
2022-06-07 11:50:17 -04:00
John Westcott IV
f37951249f Adding options fqcn (ansible.builtin.) to playbook identification 2022-06-06 17:32:37 -04:00
Jeff Bradberry
faa5df19ca Merge pull request #12252 from jbradberry/fix-analytics-unicode
Double escape all unicode escape sequences in job events data
2022-06-06 16:41:06 -04:00
Rebeccah
5f9326b131 added average event processing metric (in seconds) that can be served to
grafana via prometheus.

This metric is a good indicator of how far behind the callback receiver
is. The higher the load the further behind/the greater the number of
seconds the metric will display.

This number being high may indicate the need for horizontal scaling in
the control plane or vertically scaling the number of callback
receivers.
2022-06-06 15:14:56 -04:00
Lila
5f783fd5ee Revised job_tags to handle more than 1024 characters. 2022-06-06 13:28:22 -04:00
nixocio
5e700c992d Add controller_node to job details page
Add controller_node to job details page. Modify serializers to make
controller_node available to the UI.

See: https://github.com/ansible/awx/issues/11196
Also: https://github.com/ansible/awx/issues/12132
2022-06-02 11:21:06 -04:00
Alan Rominger
8366386126 Merge pull request #12260 from AlanCoding/callback_status
Fix the callback receiver --status command
2022-05-24 15:26:02 -04:00
djyasin
2041665880 Merge pull request #12227 from ansible/vaultcredentialsbug
Prevent edit of  vault ID once credential is created.
2022-05-19 15:13:41 -04:00
Alan Rominger
1e6ca01686 Fix the callback receiver --status command 2022-05-19 15:00:49 -04:00
Jeff Bradberry
973facebba Double escape all unicode escape sequences in job events data
when collecting it for analytics.
2022-05-18 12:00:03 -04:00
Lila
40279bc6c0 Wrote corresponding tests.
Updated verbiage to be more in line with existing messages.
2022-05-16 16:55:49 -04:00
Alan Rominger
bca6e00e37 Change Demo Project status to successful 2022-05-12 16:14:09 -04:00
Seth Foster
eba4a3f1c2 in case we fail a job in task manager, we need to add the project update to the inventoryupdate.source_project field 2022-05-12 15:21:17 -04:00
Seth Foster
0ae9fe3624 if dependency fails, fail job in task manager 2022-05-12 14:00:13 -04:00
Seth Foster
1b662fcca5 SCM inv source trigger project update
- scm based inventory sources should launch project updates prior to
running inventory updates for that source.
- fixes scenario where a job is based on projectA, but the inventory
source is based on projectB. Running the job will likely trigger a
sync for projectA, but not projectB.

comments
2022-05-12 14:00:12 -04:00
Shane McDonald
9537d148d7 Merge pull request #12175 from TheRealHaoLiu/change-ee-container-volume-selinux-label
change SELinux label for EE volume mount
2022-05-11 16:00:02 -04:00