Commit Graph
22 Commits
Author SHA1 Message Date
Seth Foster 09e5093b96 flake 8 version test 2020-05-13 13:32:11 -04:00
Seth Foster 4da0e0dd80 Vendor collections for isolated jobs to work in ansible 2.10
kubectl and synchronize are now part of community.kubernetes
and ansible.posix collections, respectively. This change installs
these collections to a local directory to be used in inventory and
isolated management playbooks.

awx issue #6930
2020-05-13 10:41:01 -04:00
Seth Foster 6652464e25 Unset old instance IP when conflicting new instance IP
With AWX_AUTO_DEPROVISION_INSTANCES on, instances
are registered with an ip address. However, new
instances might try to register before old instances
are deprivisioned. In this case old IPs can conflict with
the new ones. This will check for an ip conflict and unset
the IP of conflicting instance (set to None)

ansible/awx issue 6750
2020-04-28 10:52:15 -04:00
Seth Foster 843c22c6b1 Allow orphaned user to be added to org
Fixed bug where an org admin was not able to add
an orphaned user to the org, in the case where the
orphan had an ancestor role that matched one of the
roles for of the org admin.

scenario to fix -- sue is member of cred1, where cred1 is
part of org1. org1 admin cannot add sue to org1, because
the cred1 role for sue has an ancestor to org1 role. The org1
admin cannot change or attach sue to org1.

tower issue #4198 and #4197
2020-04-02 14:24:55 -04:00
Seth Foster 88fb30e0da Delete jobs without loading objects first
The commit is intended to speed up the cleanup_jobs command in awx. Old
methods takes 7+ hours to delete 1 million old jobs. New method takes
around 6 minutes.

Leverages a sub-classed Collector, called AWXCollector, that does not
load in objects before deleting them. Instead querysets, which are
lazily evaluated, are used in places where Collector normally keeps a
list of objects.

Finally, a couple of tests to ensure parity between old Collector and
AWXCollector. That is, any object that is updated/removed from the
database using Collector should be have identical operations using
AWXCollector.

tower issue 1103
2020-03-19 14:14:02 -04:00
Seth Foster 31dbf38a35 Prevent failing a successful project update if inventory update fails
Scenario - job is launched and spawns inventory update and project update.
If the inventory update fails, then it will fail the job and the project update.
It will fail the project update even if that update already ran and was successful.

This code change will not fail the project update if it has already ran successfully.

In cases where other jobs depend on that project update (but not the failed inventory
update), then we don't want those jobs to fail.
2020-02-24 11:35:57 -05:00
Seth Foster 9b4b2167b3 TaskManager process dependencies only once
This adds a boolean "dependencies_processed" field to UnifiedJob
model. The default value is False. Once the task manager generates
dependencies for this task, it will not generate them again on
subsequent runs.

The changes also remove .process_dependencies(), as this method repeats
the same code as .process_pending_tasks(), and is not needed. Once
dependencies are generated, they are handled at .process_pending_tasks().

Adds a unit test that should catch regressions for this fix.
2020-02-06 11:47:33 -05:00
Seth Foster 7873d08311 Update pip and setuptools in requirements txt
Versions selected to be pre-19 pip
due to unresolved issues with the build systems

Upgrade everything, party on

document new process

rotate license files

fix Swagger schema generation target

Remove --ignore-installed flag
2020-01-07 17:14:32 -06:00
Seth Foster b26b8e7097 Prevent running jobs from blocking inventory updates
A running job that has an inventory source will block
that inventory update from running. This fix removes
the block.

The test creates a job in running state, and an inventory
update in pending state. The test asserts that the
task manager and dependency graph .is_job_blocked method
returns False for the inventory update (i.e. update can
run).

issue #4809
2019-12-16 15:15:23 -05:00
Seth Foster 63e9aed601 Prevent running jobs from blocking project updates
A running job that has a project update will block
that update from running. This fix removes
the block.
Adds a functional test that sets up a job in "running" state, and
starts a project update that is in "pending" state. Assert that
the task manager and dependency graph .is_job_blocked methods both
return False.

issue #5153
2019-12-16 13:43:42 -05:00
Seth Foster e1e7e9047d Fix filtering azure inventory based on user-specified tags
Users can specify a list of keys, or a list of key:value pairs under
source variables. e.g. tags: Creator, peanutbutter or tags:
Creator:jmarshall, peanutbutter:jelly. If provided, only hosts that have
all keys or key:value pairs in the list will be returned.

inventory.py sets up the azure_rm.yml that provides
exclude_host_filters. This code adds a line for each key in the list, as
well as an additional line in the case of a key:value pair.

e.g.

exclude_host_filters:
- "'Creator' not in tags.keys()"
- tags['Creator'] != 'jmarshall'

Each line is a conditional, and if any conditionals is true, then the
host is filtered out.

fix for issue # 5044
2019-11-15 14:18:20 -05:00
Seth Foster 7e83ddc968 Fix URLField to allow numbers in top level domain
Add a custom regex to URLField that allows numbers to be present in the
top level domain, e.g. https://towerhost.org42

Set by variable allow_numbers_in_top_level_domain in URLField __init__,
and is set to True by default. If set to False, it will use the regex
specified in the built-in django URLValidator class.

This solution was originally implemented in LDAPServerURIField, but is
now implemented in URLField to support this behavior more generally. The
changes in LDAPServerURIField are longer needed and have been removed in
this commit.

Adds unit testing to make sure URLField changes handle regex input
and settings correctly.
2019-10-28 13:47:01 -04:00
Seth Foster 58e84a40e5 Fix 500 error when creating a job schedule
- 500 error occurs when a non-admin user attempts to add an invalid
  credential during schedule creation
- This change checks that the user can add the object to
  serializer.validated_data, instead of serializer.initial_data
- The invalid credential field is purged in .validated_data, so the
  request passes through cleanly
- Fix for awx issue #4147
2019-10-23 14:22:07 -04:00
Seth Foster 9c04e08b4d Fix secret lookup links when credentials are copied
- When a credential that contains secret lookups (e.g. HashiCorp Vault
  Secret Lookup) is copied, the lookup fields are not properly copied
- This change adds the necessary fields to FIELDS_TO_PRESERVE_AT_COPY
  for both Credential and CredentialInputSource classes to ensure a
  proper copy
2019-10-22 17:49:10 -04:00
Seth Foster 8e26e4edd5 Allow oauth2 settings to be set in the ui and api
Oauth2 settings were initialized early in the awx import stage, and
those settings were not modifiable. This change allows oauth2 to check
for settings in django.conf settings, which are dynamically updated
through api calls at runtime. As a result, oauth2 settings will match
the values in django.conf settings at any point in time.
2019-10-14 11:38:20 -04:00
Seth Foster bbd625f3aa update help_text to include information about REFRESH_TOKEN_EXPIRE_SECONDS 2019-10-02 17:16:01 -04:00
Seth Foster 8b22c86b10 Register default settings for OAUTH2_PROVIDER app
Grab AUTHORIZATION_CODE_EXPIRE_SECONDS from oauth2_settings
rather than hard code.

Add REFRESH_TOKEN_EXPIRE_SECONDS to valid_key_names
in OAuth2ProviderField class
2019-10-02 15:29:45 -04:00
Seth Foster db1ad2de95 Set REFRESH_TOKEN_EXPIRE_SECONDS
- Set OAUTH2 REFRESH_TOKEN_EXPIRE_SECONDS to 1 month
  (2628000 seconds)
- If not set, awx-manage cleartokens, or cleanup_tokens,
  will not work properly
- Once cleartokens is run, this setting is the amount of
  time after an access token expires that we keep its
  refresh token in the database
2019-10-02 13:51:46 -04:00
Seth Foster b6ffde75ef check expired sessions only if User exists
- Indent rest of code into the conditional
  that checks for expired sessions of that User
- If user doesn't exist, no need to check
2019-09-27 12:31:10 -04:00
Seth Foster 94fa745859 removed duplicate import of User 2019-09-27 11:47:44 -04:00
Seth Foster c94ebba0b3 Saving user session checks if User exists
- Check that model User object exists with id=user_id
  before attempting to save to database
- UserSessionMembership saves to the database using
  foreign key, User
- However, User with matching id might not exist if
  browser sends request with stale cookies
- Change made in regards to issue #4334
2019-09-27 10:52:53 -04:00
Seth Foster ca5de6378a Fix LDAPServerURIField number in domain
- Bug: API error if LDAPServerURIField contains a number in the top level domain
- Add custom regex in LDAPServerURIField class that is passed to django
  URLValidator
- The custom regex allows for numbers to be present in the top level domain
- Unit tests check that valid URIs pass through URLValidator, and that
  invalid URIs raise the correct exception
- Related to issue #3646
2019-09-20 10:36:47 -04:00