Commit Graph
100 Commits
Author SHA1 Message Date
Bill Nottingham fd658d44c9 Note that we need to match python versions.
(Some libraries don't have the same deps across python versions.)
2021-03-11 15:48:41 -05:00
Bill Nottingham 0505e38071 Remove ansible venvs & collection infrastructure. 2021-03-04 13:06:06 -05:00
Bill Nottingham fb07be36b5 Make email not required for a user in the UI.
It's already not required in the API.
2021-02-22 13:28:03 -05:00
Bill Nottingham e93518a030 Fix Openstack credential region implementation.
The injector wasn't using the same variable name as the model.
2021-02-12 17:44:44 -05:00
Bill Nottingham 7b757f17a9 Add schedule info from summary fields to allowed notification content. 2021-02-05 13:56:56 -08:00
Bill Nottingham a1934823ba Flip the instance ID var for AWS to instance_id
This is always returned by the plugin, ec2_id was what the old script returned.
2020-12-15 16:56:08 -05:00
Bill Nottingham 12edbdab11 Remove old radius library from requirements
If it's not required by django-radius, we probably don't need it.
2020-11-30 15:12:52 -05:00
Bill Nottingham 52db0bf0c0 Allow combining multiple subscriptions if they are in a manifest.
The rules:
- subs must be of same SKU type (all MCTxxxx, all SERxxxx, etc)
- when combining, we take the earliest end date
2020-11-25 17:03:12 -05:00
Bill Nottingham 13788c4568 Clean up some old sourve var settings in unit tests
Also if you want to use assert, use assert.
2020-11-20 16:32:47 -05:00
Bill Nottingham 7b04fa114e Revert portions of 40a10dcc5f
Keep some of the collectors, while removing ones we calculate
in other ways.
2020-11-13 14:50:04 -05:00
Bill Nottingham a092406543 Use the exported count in the manifest for the node count, if present. 2020-11-11 12:20:32 -05:00
Bill Nottingham 28a70ced56 Put everyone in the 'anonymous' pendo bucket regardless of setting
We no longer have the key detailed accounts are indexed on.
2020-11-04 15:18:12 -05:00
Bill Nottingham 7a9eff7e65 Don't write a tower_version file in awx 2020-11-02 12:54:51 -05:00
Bill Nottingham b9960abea6 Add a help popover that points to the docs 2020-10-30 16:40:01 -04:00
Bill Nottingham 2388758f8a Update the license wording and UX 2020-10-30 16:40:01 -04:00
Bill Nottingham 309396f199 flake8 2020-10-30 16:39:59 -04:00
Bill Nottingham 393e1b75e9 Return more user-friendly errors for assorted manifest failures 2020-10-30 16:39:59 -04:00
Bill Nottingham 48934e8544 Enable source vars for all inventory types
Update plugin popover to point to the plugin docs.
2020-10-30 16:39:57 -04:00
Bill Nottingham 0fbc02864e Enable source vars for all inventory types
Update plugin popover to point to the plugin docs.
2020-10-28 19:58:14 -04:00
Bill Nottingham bb38940638 Add a help popover that points to the docs 2020-10-26 20:04:18 -04:00
Bill Nottingham a72a688506 Update the license wording and UX 2020-10-26 18:35:16 -04:00
Bill Nottingham bfb8e384a8 flake8 2020-10-23 10:58:29 -04:00
Bill Nottingham 9bc17db45d Return more user-friendly errors for assorted manifest failures 2020-10-23 10:57:43 -04:00
Bill Nottingham f1d2d79f00 Move to rhel8 PG10 container for persistent PG
Merge in updated liveness checks from upstream template
2020-10-22 17:08:16 -04:00
Bill Nottingham bc22fa56dc Fix 500 when required LDAP group parameters aren't set. 2020-10-05 22:15:20 -07:00
Bill Nottingham 85b694410b Adjust included licenses 2020-09-25 15:51:55 -04:00
Bill Nottingham d0ba59735c Replace certifi with an alternate version
This version just uses the system cert store.
2020-09-25 14:39:16 -04:00
Bill Nottingham 18c27437b7 Fix analytics logging
The analytics change PR adjusted the logging for awx.analytics,
which solved the issue, but should have used the targeted awx.main.analytics.

Also flip a couple of loggers to use the regular awx.analytics (awx analytics)
logger instead of awx.main.analytics (the automation anayltics task system).
2020-09-17 13:39:14 -04:00
Bill Nottingham f1c59477c0 Make ansible venv psutil match awx venv version 2020-09-11 12:02:00 -04:00
Bill Nottingham 13802fcf2b Don't return error messages for license errors
Just log the exception and return None.
2020-09-10 21:19:07 -04:00
Bill Nottingham 05ad85e7a6 Remove the model for the now unused TowerAnalyticsState. 2020-09-09 20:18:04 -04:00
Bill Nottingham a604ecffb8 Adjust query_info to set the collection time based on what's passed. 2020-09-09 20:05:48 -04:00
Bill Nottingham 09f7d70428 Use isoformat() rather than strftime
Reformat SQL in unit tests because sqlite.
2020-09-09 20:05:32 -04:00
Bill Nottingham d4ba62695f Put awx analytics logs also in the task system logger
Errors/warnings when gathering analytics are about 50/50 split between
the gathering code in analytics and the task code that calls it, so
they should be in the same place for debugging sanity.
2020-09-09 17:42:40 -04:00
Bill Nottingham c753324872 Move back to less frequent collections, and split large event tables
This should ensure we stay under 100MB at all times.
2020-09-09 17:42:40 -04:00
Bill Nottingham 9f67b6742c Fail more gracefully if analytics.ship() is called with a bad path,
or it's deleted out from under us.
2020-09-09 17:42:40 -04:00
Bill Nottingham 1a15f18be3 Stop using the TowerAnalyticsState solo model
This is now tracked in the AUTOMATION_ANALYTICS_LAST_GATHER setting.
2020-09-09 17:42:40 -04:00
Bill Nottingham 40309e6f70 Ensure we do not send large bundles, or empty bundles
Collect expensive collectors separately, and in a loop
where we make smaller intermediate dumps.

Don't return a table dump if there are no records, and
don't put that CSV in the manifest.

Fix up unit tests.
2020-09-09 17:42:40 -04:00
Bill Nottingham 1c4b06fe1e Refactor analytics collectors.
- Only have one registration class
- Add description fields
- Add automation collector information to /api/v2/config
2020-09-09 17:10:14 -04:00
Bill Nottingham bd6cac966a Create marginally more realistic event data with firehose 2020-08-03 20:57:10 -04:00
Bill Nottingham c6eb8cf59b Cache downloaded roles & collections
Populate the cache the first time the job is run for a revision
that needs them, and for future runs for that revision just
copy it into the private directory.

Delete the cache on project deletion.

Invalidate the cache on a new project revision

Also download roles/collections during the sync job

Since we're writing into a per-revision cache, we can do this easily now.

Don't try and install content if there aren't any requirements expecting it

Adjust pathing to the proper location.

Force install if doing a manual sync.

Requirements may be unversioned.

Remove the cache when delete-on-update is set

Integrate content caching with existing task logic

Revert the --force flags

use the update id as metric for role caching

Shift the movement of cache to job folder from rsync task to python

Only install roles and collections if needed

Deal with roles and collections for jobs without sync
Skip local copy if roles or collections turned off

update docs for content caching

Design pivot - use empty cache dir to indicate lack of content

Do not cache content if we did not install content

Test changes to allay concerns about reliability of local_path

Do not blow away cache for SCM inventory updates

Remove project update vars no longer used

Remove job pre-creation of content folders

code style edit, always use cache_id as property in tasks

Fix log message
2020-07-28 13:57:28 -04:00
Bill Nottingham 72848fa097 Remove the cache when delete-on-update is set 2020-07-20 19:45:20 -04:00
Bill Nottingham 85ec3dd5c4 Force install if doing a manual sync.
Requirements may be unversioned.
2020-07-20 19:45:20 -04:00
Bill Nottingham 98de4cb35f Adjust pathing to the proper location. 2020-07-20 19:45:20 -04:00
Bill Nottingham dab7889715 Don't try and install content if there aren't any requirements expecting it 2020-07-20 19:45:19 -04:00
Bill Nottingham 95e69124af Also download roles/collections during the sync job
Since we're writing into a per-revision cache, we can do this easily now.
2020-07-20 19:45:19 -04:00
Bill Nottingham 1f4c46d873 Invalidate the cache on a new project revision 2020-07-20 19:45:18 -04:00
Bill Nottingham d272ee3521 Cache downloaded roles & collections
Populate the cache the first time the job is run for a revision
that needs them, and for future runs for that revision just
copy it into the private directory.

Delete the cache on project deletion.
2020-07-20 19:45:15 -04:00
Bill Nottingham 46734f5c10 Add artifacts field of jobs to the allowed list for notifications 2020-07-10 16:34:25 -04:00
Bill Nottingham a88f03b372 Reintroduce label filtering
Labels are visible if you have a role on the org they are in, or
on a job template they're attached to.
2020-07-06 13:50:00 -04:00
Bill Nottingham a33c303765 Remove active_counts_by_org
I was trying to parse the difference between this and the
(directly above) org_active_count from the comment, and then I
grepped and realized this function appears unused.
2020-06-10 15:25:14 -04:00
Bill Nottingham 64e3135754 Only build one awx image instead of separate web and task images 2020-06-03 10:48:48 -04:00
Bill Nottingham 559d917184 Make supervisor more consistent
Configs go in /etc, sockets in /var/run/supervisor for all invocations.
2020-06-02 19:51:16 -04:00
Bill Nottingham 0f0e401c98 Hardcode --kubeconfig and therefore only support OpenShift 3.11+
Avoid trying to parse inconsitent oc --version output
2020-05-29 13:41:49 -04:00
Bill Nottingham 131f5ff018 Remove dev env futzing of supervisord.conf permissions
If we just link it into the dev env, we don't need to copy it at
startup, and we don't need write permissions on it.
2020-05-28 13:12:57 -04:00
Bill Nottingham 4c499b2d80 Always check configuration before gathering data.
We shouldn't perform expensive operations if we won't be able to send it.
Only log at debug level, otherwise every node will log this every 5
minutes.
2020-05-20 17:24:38 -04:00
Bill Nottingham 17139f1f82 Always check configuration before gathering data.
We shouldn't perform expensive operations if we won't be able to send it.
Only log at debug level, otherwise every node will log this every 5
minutes.
2020-05-20 11:03:13 -04:00
Bill Nottingham 63494c94b7 Fix rsyslog in the dev env by using a local dir volume
We broke this in the image refactor.
2020-05-19 17:00:09 -04:00
Bill Nottingham 057933f030 Remove more unused settings from defaults (and some cases, the database) 2020-05-18 18:20:05 -04:00
Bill Nottingham 73b9d25371 Remove gather interval setting
This is a) the wrong settings key name
b) the same as the default in awx/main/conf.py anyway.
2020-05-18 17:55:07 -04:00
Bill Nottingham 73b0506e96 Remove obsolete setting.
This hasn't been used for years now.
2020-05-18 17:41:15 -04:00
Bill Nottingham 6924466c0b Also precreate /venv in /dev with modifiable permissions. 2020-05-14 15:33:21 -04:00
Bill Nottingham 1808bed2e8 Add nss for the chrome UI unit tests 2020-05-14 15:32:06 -04:00
Bill Nottingham d5817fd87a Remove repo files no longer used in build process 2020-05-14 15:02:32 -04:00
Bill Nottingham 8b51e8eb82 Stage rsyslog.conf from existing version so we don't have two copies. 2020-05-14 15:02:32 -04:00
Bill Nottingham 66245d3094 Reorganize both dev and prod container builds
Use only one (templated) docker file for both builds.

Move to two-stage container builds in order to save space in the
final image.
2020-05-14 15:02:32 -04:00
Bill Nottingham a057277823 Install config-watcher as 0755 rather than chmod'ing it later 2020-05-14 15:02:11 -04:00
Bill Nottingham caa4e90fa0 Remove extraneous re-install of rsyslog 2020-05-11 22:24:16 -04:00
Bill Nottingham 725437571d Set a default pycurl SSL backend. 2020-05-11 15:25:44 -04:00
Bill Nottingham 345f1db994 Don't uninstall pycurl, but do build it from source.
oVirt requires something newer, but we can't use the wheel.
2020-05-11 13:11:19 -04:00
Bill Nottingham 8cdffd0dd0 Remove pexpect_timeout setting for ansible-runner
PEXPECT_TIMEOUT has never(?) been an actual setting, and 5 is the
runner default anyway.
2020-05-08 16:31:11 -04:00
Bill Nottingham 4c6cac90fd Remove ssh version checking.
6.6 was added in RHEL 7.1, which is well before our supported platforms.
2020-05-07 15:30:55 -04:00
Bill Nottingham 5d24acf613 Allow unsigned certs in logging if cert verification is disabled. 2020-04-28 11:16:54 -04:00
Bill Nottingham 3f5820adf8 Don't override ANSIBLE_SSH_ARGS for ad-hoc commands.
This reverts 8beed7f4, which says:
    Disable ControlPersist for ad hoc commands, should avoid any
    issues with proot or needing to clean up sockets afterwards.

Given we've switched to the much less finicky bwrap for process
isolation, along with runner-based process killing, this probably
isn't needed any more.
2020-04-22 20:46:18 -04:00
Bill Nottingham 47fef1dcf3 Don't use SCLs for openshift.
This may be a mis-merge?
2020-04-22 17:16:44 -04:00
Bill Nottingham 58c821f3e1 De-flake the collector test. 2020-04-21 16:32:33 -04:00
Bill Nottingham 5b0bb4939f Allow subsets of table gathering for unit tests.
sqlite does not like some of our PG-isms.
2020-04-21 20:21:20 +02:00
Bill Nottingham 11b1d0e84c Flip CSRF_COOKIE_SECURE docs.
I think this was backwards.
2020-04-16 15:34:38 -04:00
Bill Nottingham 8e3931de37 Remove 'rampart' from a user-facing string. 2020-04-15 15:00:11 -04:00
Bill Nottingham f99a43ffa6 Collect information on inventory sources
Also remove one minor query from smart inventory collection that will
never return anything.
2020-04-14 19:15:19 -04:00
Bill Nottingham cd6d2ed53a Move the comma so unit test can filter things properly. 2020-04-14 13:12:03 -04:00
Bill Nottingham a296f64696 Cast the start/end times with timezone. 2020-04-14 10:53:57 -04:00
Bill Nottingham d651786206 Collect task timing, warnings, and deprecations from job events
Timing information requires ansible-runner >= 1.4.6.
2020-04-09 17:27:19 -04:00
Bill Nottingham 4f3213715e Allow disassociating any user from a credential role.
This is preventing removing roles from users no longer in the organization.
2020-04-03 13:39:28 -04:00
Bill Nottingham 96576b0e3d Don't return different fields for smart vs non-smart inventories 2020-03-30 17:15:55 -04:00
Bill Nottingham ac68e8c4fe Preserve symlinks when copying a tree.
This avoids creating a recursive symlink tree.
2020-03-20 13:41:16 -04:00
Bill Nottingham b875c03f4a Clean up a few more cases where we checked the license for features. 2020-03-17 17:19:33 -04:00
Bill Nottingham d0c510563f Allow scm_branch in notifications. 2020-03-17 12:09:35 -04:00
Bill Nottingham 871695ea5e Remove SCL python from awx-python
This really should be created at build time.
2020-03-09 15:51:56 -04:00
Bill Nottingham 341c6ae767 Also remove some dangling rax from awxkit and collection modules. 2020-02-25 15:18:26 -05:00
Bill Nottingham 3e6b6c05a6 Remove the rax support specified in the linked TODO 2020-02-25 15:03:05 -05:00
Bill Nottingham e2fb83db98 Tweak workflow error message 2020-02-21 02:37:03 -05:00
Bill Nottingham f20859c85f Tweak labels for workflow pending approval message 2020-02-20 22:19:34 -05:00
Bill Nottingham 4c8a4013cc Remove requirements_setup_requires and related handling.
Adjust requirements to keep docutils in awx requirements; these
are downstream packaging adjustments. Override azure wheel
dependency to a version that can install bcrypt properly.
2020-02-18 22:34:59 -05:00
Bill Nottingham 71ef7cdec1 Use AWX_TASK_ENV when connecting to Red Hat services 2020-02-07 14:29:42 -05:00
Bill Nottingham f3e2caeaa7 Bypass memcached to get last gather time to avoid reading cached values. 2020-02-06 21:41:41 -05:00
Bill Nottingham ea8a91893a Fix help for new analytics gather interval 2020-01-23 16:26:15 -05:00
Bill Nottingham b2a0b3fc29 Fix timedelta comparison to account for large intervals
It would fail if you set the interval to > 1 day.
2020-01-21 16:14:33 -05:00
Bill Nottingham 44e176dde8 Change how analytics is gathered to only gather once per interval 2020-01-21 11:40:51 -05:00
Bill Nottingham 4e46d5d7cd Fix some lint 2020-01-20 17:15:27 -05:00