Commit Graph
998 Commits
Author SHA1 Message Date
Ryan Petrello 6b4219badb more ansible runner isolated cleanup
follow-up to https://github.com/ansible/awx/pull/6296
2020-04-08 01:18:05 -04:00
Ryan Petrello 301d6ff616 make the job event bigint migration chunk size configurable 2020-03-27 09:28:10 -04:00
Ryan Petrello c8044b4755 migrate event table primary keys from integer to bigint
see: https://github.com/ansible/awx/issues/6010
2020-03-26 15:54:38 -04:00
Bill Nottingham ac68e8c4fe Preserve symlinks when copying a tree.
This avoids creating a recursive symlink tree.
2020-03-20 13:41:16 -04:00
chris meyers 47f5c17b56 log when notifications fail to send
* If a job does not finish in the 5 second timeout. Let the user know
that we failed to even try to send the notification.
2020-03-20 09:11:01 -04:00
Ryan Petrello d40a5dec8f change when we send job notifications to avoid a race condition
success/failure notifications for *playbooks* include summary data about
the hosts in based on the contents of the playbook_on_stats event

the current implementation suffers from a number of race conditions that
sometimes can cause that data to be missing or incomplete; this change
makes it so that for *playbooks* we build (and send) the notification in
response to the playbook_on_stats event, not the EOF event
2020-03-19 10:01:52 -04:00
Ryan Petrello 1caa2e0287 work around a limitation in postgres notify to properly support copying
postgres has a limitation on its notify message size (8k), and the
messages we generate for deep copying functionality easily go over this
limit; instead of passing a giant nested data structure across the
message bus, this change makes it so that we temporarily store the JSON
structure in memcached, and look it up from *within* the task

see: https://github.com/ansible/tower/issues/4162
2020-03-18 16:10:20 -04:00
chris meyers dc6c353ecd remove support for multi-reader dispatch queue
* Under the new postgres backed notify/listen message queue, this never
actually worked. Without using the database to store state, we can not
provide a at-most-once delivery mechanism w/ multi-readers.
* With this change, work is done ONLY on the node that requested for the
work to be done. Under rabbitmq, the node that was first to get the
message off the queue would do the work; presumably the least busy node.
2020-03-18 16:10:16 -04:00
chris meyers 2a2c34f567 combine all the broker replacement pieces
* local redis for event processing
* postgres for message broker
* redis for websockets
2020-03-18 16:10:15 -04:00
Ryan Petrello f8818730d4 consolidate isolated event handling code into one function
make the non-isolated *and* isolated event handling share the same
function so we don't regress on behavior between the two
2020-03-13 10:05:48 -04:00
AlanCoding 5dba49a7bc Lower level of log about skipped project update 2020-02-27 14:20:36 -05:00
AlanCoding 7b880c6552 Cancel jobs if they were deleted in the database 2020-02-27 14:12:47 -05:00
Seth Foster 31dbf38a35 Prevent failing a successful project update if inventory update fails
Scenario - job is launched and spawns inventory update and project update.
If the inventory update fails, then it will fail the job and the project update.
It will fail the project update even if that update already ran and was successful.

This code change will not fail the project update if it has already ran successfully.

In cases where other jobs depend on that project update (but not the failed inventory
update), then we don't want those jobs to fail.
2020-02-24 11:35:57 -05:00
AlanCoding a5e3d9558f Handle case of deleted inventory 2020-02-13 08:29:52 -05:00
Bill Nottingham f3e2caeaa7 Bypass memcached to get last gather time to avoid reading cached values. 2020-02-06 21:41:41 -05:00
Ryan Petrello 7055460c4c fix broken project update secret filtering for external logging 2020-02-03 10:27:31 -05:00
AlanCoding d759aff4e9 Do not allow state where no Galaxy servers are enabled 2020-01-28 16:01:55 -05:00
Gabe Muniz a264b1db1f made licensing a warning and not trigger on periodic scheduler 2020-01-23 14:08:23 -05:00
Bill Nottingham b2a0b3fc29 Fix timedelta comparison to account for large intervals
It would fail if you set the interval to > 1 day.
2020-01-21 16:14:33 -05:00
Bill Nottingham 44e176dde8 Change how analytics is gathered to only gather once per interval 2020-01-21 11:40:51 -05:00
softwarefactory-project-zuul[bot] 6d075b8874 Merge pull request #5448 from ryanpetrello/remove-computed-group-and-host-fields
remove computed inventory fields from Host and Group

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-01-15 19:53:30 +00:00
Jake McDermott 0d98a1980e Add a configurable limit for job forks 2020-01-15 13:51:59 -05:00
Ryan Petrello 568606d2c8 remove computed inventory fields from Host and Group 2020-01-14 16:37:16 -05:00
Ryan Petrello 306f504fb7 optimize the callback receiver to buffer writes on high throughput
additionaly, optimize away several per-event host lookups and
changed/failed propagation lookups

we've always performed these (fairly expensive) queries *on every event
save* - if you're processing tens of thousands of events in short
bursts, this is way too slow

this commit also introduces a new command for profiling the insertion
rate of events, `awx-manage callback_stats`

see: https://github.com/ansible/awx/issues/5514
2020-01-14 12:04:26 -05:00
AlanCoding 8d4425f056 Revert "Reduce API response times by caching migration flag"
This reverts commit 5433af6716.
2020-01-02 09:08:51 -05:00
AlanCoding 5433af6716 Reduce API response times by caching migration flag 2019-12-15 22:56:57 -05:00
AlanCoding 419d32d3e3 Fix project sync errors when project branch is commit 2019-12-05 14:26:24 -05:00
AlanCoding 922ea67541 Fix situation were error happened before lock was released 2019-12-05 10:41:23 -05:00
softwarefactory-project-zuul[bot] 3b49dd78bf Merge pull request #5392 from ryanpetrello/fix-system-jobs
fix a few bugs with the session and oauth2 cleanup scheduled jobs

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2019-11-27 17:25:56 +00:00
Ryan Petrello 632810f3a8 fix a few bugs with the session and oauth2 cleanup scheduled jobs
see: https://github.com/ansible/tower/issues/3940
2019-11-26 13:17:46 -05:00
Christian Adams 4f8b624b96 Make spelling of canceled consistent 2019-11-26 00:31:15 -05:00
Alan Rominger 8e7d607a47 Only turn off Galaxy cert verification via toggle (#3933) 2019-11-19 08:54:40 -05:00
AlanCoding f64d0dde5a Use tags to reduce project update output
Handle folder deletion as tag

remove -v use by default

Change meaning of roles_enabled playbook var to
 value of AWX global setting
2019-11-12 12:52:39 -05:00
softwarefactory-project-zuul[bot] c605705b39 Merge pull request #5182 from wenottingham/for-this-to-actually-be-useful-we-would-need-a-sendmail.cf-and-lol-not-doing-that
[RFC] Remove admin alerts, there are better mechanisms for this

Reviewed-by: Ryan Petrello
             https://github.com/ryanpetrello
2019-10-31 14:52:37 +00:00
Bill Nottingham 5cdf2f88da Remove admin alerts, there are better mechanisms for this 2019-10-30 19:35:45 -04:00
Bill Nottingham 93e940adfc Remove extraneous leftover conditional import 2019-10-30 19:21:02 -04:00
Alan Rominger acba5306c6 Fix bug where SCM inventory did not have a collections destination (#3795)
* update inventory path to be in tmp project clone

* copy project folder for inventory scm launch type

* Optionally accept inventory collection paths from ansible.cfg
2019-10-29 11:24:14 -04:00
Ryan Petrello ccaaee61f0 improve cleanup of anonymous kubeconfig files 2019-10-29 11:24:12 -04:00
Graham Mainwaring b2b33605cc Add UI toggle to disable public Galaxy (#3867) 2019-10-29 11:24:11 -04:00
Ryan Petrello 16812542f8 implement a simple periodic pod reaper for container groups
see: https://github.com/ansible/awx/issues/4911
2019-10-17 17:06:36 -04:00
Ryan Petrello 570ffad52b clean up pods for all k8s execution, not just playbook runs
see: https://github.com/ansible/awx/issues/4908
2019-10-17 15:29:44 -04:00
Bill Nottingham c2743d8678 Merge pull request #3802 from wenottingham/oh-noes-not-again
Check the user's ansible.cfg for role/collection paths.
2019-10-15 14:21:18 -04:00
Ryan Petrello d5bdf554f1 fix a programming error when k8s pods fail to launch 2019-10-10 15:54:00 -04:00
Shane McDonald 8f75382b81 Implement retry logic for container group pod launches 2019-10-10 15:53:56 -04:00
Shane McDonald b93164e1ed Prevent pods from failing if the reason is because of a resource quota
Signed-off-by: Shane McDonald <me@shanemcd.com>
2019-10-10 15:53:50 -04:00
Bill Nottingham 31bdde00c9 Check the user's ansible.cfg for role/collection paths.
There's no other way to add our new paths reliably without breaking things.
2019-10-10 15:09:26 -04:00
AlanCoding c09039e963 Add setting for auth_url
Also adjust public galaxy URL setting to
allow using only the primary Galaxy server

Include auth_url in token exclusivity validation
2019-10-07 14:02:43 -04:00
AlanCoding 922e779a86 Rename private to primary in galaxy settings
use a setting for the public galaxy URL
2019-10-07 14:02:43 -04:00
AlanCoding 093bf6877b Finish adding settings to UI 2019-10-07 14:02:42 -04:00
AlanCoding c566c332f9 Initial env var implementation of private galaxy server 2019-10-07 14:02:42 -04:00