Compare commits

..

19 Commits

Author SHA1 Message Date
Hao Liu
849f5f796c Restore oauth_token backward compatibility for collection token auth (#16500)
* Restore oauth_token backward compatibility for collection token auth

The aap_token rename (c8981e321e) restored module-level token auth but
left two interfaces from earlier collection releases broken:

- The lookup (controller_api) and inventory (controller) plugins
  previously declared an oauth_token option. Add oauth_token as an
  alias of aap_token in the auth_plugin doc fragment and in
  AUTH_ARGSPEC so query(..., oauth_token=...) and inventory YAML keys
  keep working.

- tower_cli.cfg-style config files used an oauth_token key under
  [general]; it was silently ignored after the rename, quietly
  degrading auth. load_config() now also reads the legacy oauth_token
  key and maps it to aap_token, with the new aap_token key winning when
  both are present. aap_token remains the canonical attribute used by
  _parse_aap_token() and the Bearer header logic.

Also make the test helper compatible with ansible-core 2.21+, which
requires a serialization profile alongside _ANSIBLE_ARGS, and extend
the tests to cover the oauth_token alias and legacy config file key.

No changelog fragment added: awx_collection has no changelogs/
directory on devel.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Document oauth_token alias in module auth doc fragment

The oauth_token alias was added to aap_token in AUTH_ARGSPEC but not to
the module doc fragment, failing the validate-modules sanity check
(undocumented argument alias).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Generalize version references in compat comments

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 18:32:07 +00:00
Hao Liu
c8981e321e Make aap_token functional for collection token auth (#16498)
The aap_token parameter was added to the collection argspec and docs
in #16025, but nothing consumed it after token auth was removed in
#15623: modules silently ignored the token and fell back to basic
auth, breaking token authentication through the AAP gateway.

Wire it up so requests authenticate with the provided token (e.g. one
issued by the AAP gateway, which validates it and proxies to the
controller):

- Send "Authorization: Bearer <token>" in make_request when aap_token
  is set, skipping the basic-auth login probe; basic auth is unchanged
  when no token is given
- Accept the token as a string or as the dict set as a fact by the
  ansible.platform.token module ({token: ..., id: ...}), which is the
  documented cross-collection mint/use/delete workflow
- Restore controller_oauthtoken and tower_oauthtoken as aliases for
  back-compat with pre-#15623 playbooks, matching downstream
- Forward aap_token through the controller_api lookup and controller
  inventory plugins via short_params, and add the missing
  CONTROLLER_OAUTH_TOKEN/TOWER_OAUTH_TOKEN env sources to the plugin
  doc fragment (plugins resolve env vars from doc fragments, not
  env_fallback); AAP_TOKEN is no longer marked deprecated there
- Support tokens in the awxkit-based export/import modules
- Add unit tests covering the Bearer header for both token forms, the
  aliases, the bad-dict failure, and the basic-auth fallback

Verified end-to-end against a live gateway-fronted AAP 2.7 deployment:
modules, the lookup plugin, both aliases, all env sources, dict-form
tokens, job launch/wait, and a clean HTTP 401 on an invalid token.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 18:18:25 -04:00
Seth Foster
d5e5ea3670 Lazy-load plugin registries, move DB sync to dispatcher (#16483)
Move plugin loading to lazy-on-first-access, DB sync to dispatcher

Remove credential type and inventory plugin loading from Django's
app.ready() path. In-memory registries (ManagedCredentialType.registry
and InventorySourceOptions.injectors) are now populated lazily on first
access via LazyLoadDict, a dict subclass that calls a loader function
on the first read operation. This ensures web workers, dispatcher
workers, and management commands all get the registries populated
exactly when needed, without eager loading at startup.

The DB sync (CredentialType.setup_tower_managed_defaults) is moved to
the dispatcher's startup task, where it only needs to run once per
deployment rather than in every Django process.

Co-Authored-By: Alan Rominger <arominge@redhat.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-09 12:45:50 -04:00
Bryan Havenstein
d566f71ae0 AAP-65883: Refactor clear_setting_cache to use DAB utility (#16469)
* AAP-65883: Refactor clear_setting_cache to use DAB shared utility

Delegate cache invalidation logic to ansible_base.lib.cache.tasks.clear_cache,
passing AWX-specific dependent key resolution (settings_registry) and
post-invalidation hook (LOG_AGGREGATOR_LEVEL reconfiguration) as callbacks.

Requires: ansible/django-ansible-base AAP-65883/dab-cache-invalidation-job

Assisted-by: Claude Code / Opus 4.6 (Anthropic)

* AAP-65883: Extract helper functions to module level

Move _resolve_setting_dependents and _post_setting_invalidation out of
clear_setting_cache for better stack traces and independent testability
per review feedback (John Westcott).

Assisted-by: Claude Code / Opus 4.6 (Anthropic)
2026-06-09 11:56:58 -04:00
Peter Braun
c8cb465fde fix: do not allow exec instances to be added to the control plane (#16477)
Co-authored-by: Stevenson Michel <iamstevensonmichel@outlook.com>
2026-06-09 09:04:10 -04:00
Alan Rominger
49e21d7c1c Move PG version check to awx-manage check_db & migrate commands (#15463)
* Move PG version check to check_db command

Move to utils, check in pre_migrate signal

* Add back in environment var skip

* Add tests for compliance

tests Assisted-By: claude
2026-06-08 10:57:55 -04:00
Ryan Williams
b531151931 chore: bump ATF pipeline bundle sha (#16480)
Signed-off-by: Ryan Williams <3375653+ryankwilliams@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-06-04 14:24:39 -04:00
Stevenson Michel
54857c7a82 [Devel] Bump kubernetes client to >=36.0.0 to fix NO_PROXY handling (#16478)
Bump kubernetes client to >=36.0.0 to fix NO_PROXY handling
2026-06-04 13:21:55 -04:00
Hamzah Yousuf
e03899b581 FIX: Add Organization Parameter to Demo Credential (#16474)
Fix demo credential organization in create_preload_data
2026-06-03 18:32:16 +00:00
Lila Yasin
b4f27de4a2 [AAP-53283] Use getproxies() for analytics API proxy configuration (#16475)
Revert "[AAP-53283] Fix analytics API requests to respect proxy environment variables (#16451)"

This reverts commit 45480941f8.
2026-06-02 15:38:18 -04:00
Seth Foster
5cc467d4cf [AAP-74497] Reset orphaned waiting jobs when controller node is deprovisioned (#16467)
Reset orphaned waiting jobs when controller node is deprovisioned

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-02 10:46:52 -04:00
Stevenson Michel
b14b9e1771 [Devel] Reject RRULE with INTERVAL=0 to Prevent Scheduler Hang (#16464)
* added interval null rrule check and updated tests

* Added secondly to the expected errors
2026-06-02 10:22:36 -04:00
Ryan Williams
c4c2779976 fix: invalid renovate config to bump tekton pipeline bundles (#16472)
Signed-off-by: Ryan Williams <3375653+ryankwilliams@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-02 15:34:55 +02:00
Tomas Z
4bdb11c2a6 [AAP-65882] feat: switch to DAB Redis cache backend (#16471)
[AAP-65882] Switch to DAB Redis cache backend

Replace awx.main.cache.AWXRedisCache with the equivalent
ansible_base.lib.cache.redis_cache.DABRedisCache and remove the
now-redundant AWX driver. The DAB driver was migrated from this
exact code and provides the same fault-tolerant cache behavior.

Signed-off-by: Tomas Z <tznamena@redhat.com>
2026-06-01 17:15:20 -04:00
Alan Rominger
80f8ee1dec Fix skip of push operations which are still failing in mirrors (#16468)
* Fix bash operator precedence in repo ownership check

The condition had || operators outside proper test block grouping,
which could cause the check to fail with a shell error. Wrap the
OR conditions in parentheses with explicit [[ ]] tests.

Assisted-by: Claude Haiku 4.5 <noreply@anthropic.com>

* Replace reusable workflow with direct if conditions for repo ownership check

The reusable workflow with job dependencies had a timing/evaluation issue
where jobs would still execute even when should_run=false. Using direct
if conditions with github context variables (repository, ref_name) is more
reliable and ensures jobs are properly skipped on fork pushes.

Assisted-by: Claude Haiku 4.5 <noreply@anthropic.com>

* Remove unused repo-owns-branch reusable workflow

No longer needed after replacing with direct if conditions.

Assisted-by: Claude Haiku 4.5 <noreply@anthropic.com>
2026-06-01 13:25:27 -04:00
Chris Meyers
f22df56e44 Fix TypeError when update_fields=None in model save methods (#16466)
Django allows passing update_fields=None to model.save() to mean 'update all fields'.
However, kwargs.get('update_fields', []) returns None when the key exists with a None value,
rather than the default empty list.

This caused crashes in mark_field_for_save() and other code that assumes update_fields is
a list when doing membership tests ('field' not in update_fields) or append operations.

Changed pattern from:
  update_fields = kwargs.get('update_fields', [])

To:
  update_fields = kwargs.get('update_fields') or []

This correctly handles:
- Key missing: get() returns None → or [] gives []
- Key present with None: get() returns None → or [] gives []
- Key present with list: get() returns list → or [] keeps the list

Fixed in 12 locations across 8 model files:
- awx/main/models/base.py (3 instances)
- awx/main/models/unified_jobs.py (2)
- awx/main/models/jobs.py (2)
- awx/main/models/ad_hoc_commands.py
- awx/main/models/inventory.py
- awx/main/models/notifications.py
- awx/main/models/projects.py
- awx/main/models/workflow.py

Fixes task manager crashes with:
TypeError: argument of type 'NoneType' is not iterable
2026-05-29 14:14:16 -04:00
Alan Rominger
fccb6744f9 Address even more pytest warnings, and migrate smart inventory tests (#16330)
* Address even more pytest warnings, co-authored with Opus 4.6

* Upgrade pyparsing

* Attempt to update smart inventory logic

* Move smart inventory tests here

* Fix some failing dev env tests

Assisted-by: claude

* Use shared fixture for teardown

* Fix test goof

assisted-by: claude Opus 4.6
2026-05-28 16:17:13 -04:00
Alan Rominger
200a68aefa [AAP-57274] Fix creator permissions for models without old-style roles (#16457)
* [AAP-57274] Fix creator permissions for models without old-style roles

NotificationTemplate has no old-style ImplicitRoleField (like admin_role)
because notification permissions were historically org-level only.
When a non-admin user creates a notification template,
give_creator_permissions tries to sync the DAB RBAC assignment back
to the old role system and hits an AttributeError.

Catch the AttributeError so the DAB RBAC assignment still succeeds.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-28 16:06:03 -04:00
Alan Rominger
9b922f70ed Make dispatcherd min_workers literally 4 (#16421)
assisted-by: claude
2026-05-28 09:58:13 -04:00
59 changed files with 1774 additions and 643 deletions

View File

@@ -1,55 +0,0 @@
---
name: Repo Owns Branch
# Reusable workflow that determines whether the current repository
# owns the current branch for push operations.
#
# Ownership rules:
# - ansible/awx owns: devel, feature_*
# - ansible/tower owns: stable-*, release_*
# - workflow_dispatch is always allowed
#
# All other repo/branch combinations are skipped.
on:
workflow_call:
outputs:
should_run:
description: Whether this repo owns the current branch
value: ${{ jobs.check.outputs.should_run }}
jobs:
check:
runs-on: ubuntu-latest
outputs:
should_run: ${{ steps.check.outputs.should_run }}
steps:
- name: Check branch ownership
id: check
run: |
REPO="${{ github.repository }}"
BRANCH="${{ github.ref_name }}"
EVENT="${{ github.event_name }}"
if [[ "$EVENT" == "workflow_dispatch" ]]; then
echo "should_run=true" >> $GITHUB_OUTPUT
echo "Manual trigger — allowed"
exit 0
fi
# ansible/awx owns devel and feature_* branches
if [[ "$REPO" == "ansible/awx" ]] && [[ "$BRANCH" == "devel" || "$BRANCH" == feature_* ]]; then
echo "should_run=true" >> $GITHUB_OUTPUT
echo "Repository '$REPO' owns branch '$BRANCH'"
exit 0
fi
# ansible/tower owns stable-* and release_* branches
if [[ "$REPO" == "ansible/tower" ]] && [[ "$BRANCH" == stable-* || "$BRANCH" == release_* ]]; then
echo "should_run=true" >> $GITHUB_OUTPUT
echo "Repository '$REPO' owns branch '$BRANCH'"
exit 0
fi
echo "should_run=false" >> $GITHUB_OUTPUT
echo "Repository '$REPO' does not own branch '$BRANCH' — skipping"

View File

@@ -12,12 +12,11 @@ on:
- feature_*
- stable-*
jobs:
check-ownership:
uses: ./.github/workflows/_repo-owns-branch.yml
push-development-images:
needs: check-ownership
if: needs.check-ownership.outputs.should_run == 'true'
if: |
github.event_name == 'workflow_dispatch' ||
(github.repository == 'ansible/awx' && (github.ref_name == 'devel' || startsWith(github.ref_name, 'feature_'))) ||
(github.repository == 'ansible/tower' && (startsWith(github.ref_name, 'stable-') || startsWith(github.ref_name, 'release_')))
runs-on: ubuntu-latest
timeout-minutes: 120
permissions:

View File

@@ -20,12 +20,11 @@ on:
- 'stable-2.[1-9][0-9]'
workflow_dispatch: # Allow manual triggering for testing
jobs:
check-ownership:
uses: ./.github/workflows/_repo-owns-branch.yml
sync-openapi-spec:
needs: check-ownership
if: needs.check-ownership.outputs.should_run == 'true'
if: |
github.event_name == 'workflow_dispatch' ||
(github.repository == 'ansible/awx' && (github.ref_name == 'devel' || startsWith(github.ref_name, 'feature_'))) ||
(github.repository == 'ansible/tower' && (startsWith(github.ref_name, 'stable-') || startsWith(github.ref_name, 'release_')))
name: Sync OpenAPI spec to central repo
runs-on: ubuntu-latest
permissions:

View File

@@ -13,12 +13,11 @@ on:
- feature_**
- stable-**
jobs:
check-ownership:
uses: ./.github/workflows/_repo-owns-branch.yml
push:
needs: check-ownership
if: needs.check-ownership.outputs.should_run == 'true'
if: |
github.event_name == 'workflow_dispatch' ||
(github.repository == 'ansible/awx' && (github.ref_name == 'devel' || startsWith(github.ref_name, 'feature_'))) ||
(github.repository == 'ansible/tower' && (startsWith(github.ref_name, 'stable-') || startsWith(github.ref_name, 'release_')))
runs-on: ubuntu-latest
timeout-minutes: 60
permissions:

View File

@@ -27,7 +27,7 @@ spec:
- name: name
value: aap-api-tests
- name: bundle
value: quay.io/aap-ci/tekton-catalog/pipeline/test/aap-api-tests:0.1@sha256:54d9e941748bae94b2154b3b253a985e628751dfa4508a138d9b05f74a3c1ddf
value: quay.io/aap-ci/tekton-catalog/pipeline/test/aap-api-tests:0.1@sha256:50aadd6725a239ab53247deb7cf601d1163ceb1792792fd239a3f37d21a490d7
- name: kind
value: pipeline
- name: secret

View File

@@ -52,14 +52,6 @@ except ImportError: # pragma: no cover
MODE = 'production'
try:
import django # noqa: F401
except ImportError:
pass
else:
from django.db import connection
def prepare_env():
# Update the default settings environment variable based on current mode.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'awx.settings')
@@ -79,14 +71,6 @@ def manage():
from django.conf import settings
from django.core.management import execute_from_command_line
# enforce the postgres version is a minimum of 12 (we need this for partitioning); if not, then terminate program with exit code of 1
# In the future if we require a feature of a version of postgres > 12 this should be updated to reflect that.
# The return of connection.pg_version is something like 12013
if not os.getenv('SKIP_PG_VERSION_CHECK', False) and not MODE == 'development':
if (connection.pg_version // 10000) < 12:
sys.stderr.write("At a minimum, postgres version 12 is required\n")
sys.exit(1)
if len(sys.argv) >= 2 and sys.argv[1] in ('version', '--version'): # pragma: no cover
sys.stdout.write('%s\n' % __version__)
# If running as a user without permission to read settings, display an

View File

@@ -120,7 +120,7 @@ from awx.main.utils.named_url_graph import reset_counters
from awx.main.utils.inventory_vars import update_group_variables
from awx.main.scheduler.task_manager_models import TaskManagerModels
from awx.main.redact import UriCleaner, REPLACE_STR
from awx.main.signals import update_inventory_computed_fields
from awx.main.tasks.system import update_inventory_computed_fields
from awx.main.validators import vars_validate_or_raise
@@ -961,32 +961,14 @@ class UnifiedJobSerializer(BaseSerializer):
class UnifiedJobListSerializer(UnifiedJobSerializer):
# these fields can be included optionally in the response
OPTIONAL_INCLUDE_FIELDS = frozenset({'artifacts', 'extra_vars'})
# these fields are stripped from the response
_STRIPPED_FIELDS = frozenset({'job_args', 'job_cwd', 'job_env', 'result_traceback', 'event_processing_finished', 'artifacts', 'extra_vars'})
class Meta:
fields = ('*', '-job_args', '-job_cwd', '-job_env', '-result_traceback', '-event_processing_finished', '-artifacts', '-extra_vars')
# processes the include query param if present
def _requested_includes(self):
request = self.context.get('request')
if request is None:
return frozenset()
raw = request.query_params.get('include', '')
requested = {name.strip() for name in raw.split(',') if name.strip()}
# only allow the fields listed in OPTIONAL_INCLUDE_FIELDS
return frozenset(requested) & self.OPTIONAL_INCLUDE_FIELDS
fields = ('*', '-job_args', '-job_cwd', '-job_env', '-result_traceback', '-event_processing_finished', '-artifacts')
def get_field_names(self, declared_fields, info):
field_names = super(UnifiedJobListSerializer, self).get_field_names(declared_fields, info)
# Meta multiple inheritance and -field_name options don't seem to be
# taking effect above, so remove the undesired fields here.
strip = self._STRIPPED_FIELDS - self._requested_includes()
return tuple(x for x in field_names if x not in strip)
return tuple(x for x in field_names if x not in ('job_args', 'job_cwd', 'job_env', 'result_traceback', 'event_processing_finished', 'artifacts'))
def get_types(self):
if type(self) is UnifiedJobListSerializer:
@@ -5468,7 +5450,11 @@ class SchedulePreviewSerializer(BaseSerializer):
for a_rule in match_multiple_rrule:
if 'interval' not in a_rule.lower():
errors.append("{0}: {1}".format(_('INTERVAL required in rrule'), a_rule))
elif 'secondly' in a_rule.lower():
else:
match_interval = re.match(r".*?INTERVAL=([0-9]+)", a_rule)
if match_interval and int(match_interval.group(1)) < 1:
errors.append("{0}: {1}".format(_("INTERVAL must be a positive integer"), a_rule))
if 'secondly' in a_rule.lower():
errors.append("{0}: {1}".format(_('SECONDLY is not supported'), a_rule))
if re.match(by_day_with_numeric_prefix, a_rule):
errors.append("{0}: {1}".format(_("BYDAY with numeric prefix not supported"), a_rule))

View File

@@ -127,7 +127,6 @@ from awx.api.views.mixin import (
RelatedJobsPreventDeleteMixin,
UnifiedJobDeletionMixin,
NoTruncateMixin,
UnifiedJobIncludeMixin,
)
from awx.api.pagination import UnifiedJobEventPagination
from awx.main.utils import set_environ
@@ -3851,7 +3850,7 @@ class SystemJobTemplateNotificationTemplatesSuccessList(SystemJobTemplateNotific
resource_purpose = 'notification templates triggered on system job success'
class JobList(UnifiedJobIncludeMixin, ListAPIView):
class JobList(ListAPIView):
model = models.Job
serializer_class = serializers.JobListSerializer
resource_purpose = 'jobs'
@@ -4568,7 +4567,7 @@ class UnifiedJobTemplateList(ListAPIView):
resource_purpose = 'unified job templates'
class UnifiedJobList(UnifiedJobIncludeMixin, ListAPIView):
class UnifiedJobList(ListAPIView):
model = models.UnifiedJob
serializer_class = serializers.UnifiedJobListSerializer
search_fields = ('description', 'name', 'job__playbook')

View File

@@ -9,7 +9,7 @@ from django.utils import translation
from awx.api.generics import APIView, Response
from awx.api.permissions import AnalyticsPermission
from awx.api.versioning import reverse
from awx.main.utils import get_awx_version, set_environ
from awx.main.utils import get_awx_version
from awx.main.utils.analytics_proxy import OIDCClient
from rest_framework import status
@@ -210,32 +210,31 @@ class AnalyticsGenericView(APIView):
return self._error_response(ERROR_UNSUPPORTED_METHOD, method, remote=False, status_code=status.HTTP_500_INTERNAL_SERVER_ERROR)
url = self._get_analytics_url(request.path)
using_subscriptions_credentials = False
with set_environ(**settings.AWX_TASK_ENV):
try:
rh_user = getattr(settings, 'REDHAT_USERNAME', None)
rh_password = getattr(settings, 'REDHAT_PASSWORD', None)
if not (rh_user and rh_password):
rh_user = self._get_setting('SUBSCRIPTIONS_CLIENT_ID', None, ERROR_MISSING_USER)
rh_password = self._get_setting('SUBSCRIPTIONS_CLIENT_SECRET', None, ERROR_MISSING_PASSWORD)
using_subscriptions_credentials = True
try:
rh_user = getattr(settings, 'REDHAT_USERNAME', None)
rh_password = getattr(settings, 'REDHAT_PASSWORD', None)
if not (rh_user and rh_password):
rh_user = self._get_setting('SUBSCRIPTIONS_CLIENT_ID', None, ERROR_MISSING_USER)
rh_password = self._get_setting('SUBSCRIPTIONS_CLIENT_SECRET', None, ERROR_MISSING_PASSWORD)
using_subscriptions_credentials = True
client = OIDCClient(rh_user, rh_password)
response = client.make_request(
method,
url,
headers=headers,
verify=settings.INSIGHTS_CERT_PATH,
params=getattr(request, 'query_params', {}),
json=getattr(request, 'data', {}),
timeout=(31, 31),
)
except requests.RequestException:
# subscriptions credentials are not valid for basic auth, so just return 401
if using_subscriptions_credentials:
response = Response(status=status.HTTP_401_UNAUTHORIZED)
else:
logger.error("Automation Analytics API request failed, trying base auth method")
response = self._base_auth_request(request, method, url, rh_user, rh_password, headers)
client = OIDCClient(rh_user, rh_password)
response = client.make_request(
method,
url,
headers=headers,
verify=settings.INSIGHTS_CERT_PATH,
params=getattr(request, 'query_params', {}),
json=getattr(request, 'data', {}),
timeout=(31, 31),
)
except requests.RequestException:
# subscriptions credentials are not valid for basic auth, so just return 401
if using_subscriptions_credentials:
response = Response(status=status.HTTP_401_UNAUTHORIZED)
else:
logger.error("Automation Analytics API request failed, trying base auth method")
response = self._base_auth_request(request, method, url, rh_user, rh_password, headers)
#
# Missing or wrong user/pass
#

View File

@@ -212,9 +212,3 @@ class NoTruncateMixin(object):
if self.request.query_params.get('no_truncate'):
context.update(no_truncate=True)
return context
class UnifiedJobIncludeMixin(object):
# Reserve the name 'include' so we can use it as a query param. Otherwise, the rest-filters backend
# would treat it as a model field lookup.
rest_filters_reserved_names = ('include',)

View File

@@ -1,22 +1,25 @@
import os
from dispatcherd.config import setup as dispatcher_setup
from django.apps import AppConfig
from django.db import connection
from django.utils.translation import gettext_lazy as _
from awx.main.utils.common import bypass_in_test, load_all_entry_points_for
from awx.main.utils.migration import is_database_synchronized
from awx.main.utils.named_url_graph import _customize_graph, generate_graph
from awx.conf import register, fields
from django.core.management.base import CommandError
from django.db.models.signals import pre_migrate
from awx_plugins.interfaces._temporary_private_licensing_api import detect_server_product_name
from awx.main.utils.named_url_graph import _customize_graph, generate_graph
from awx.main.utils.db import db_requirement_violations
from awx.conf import register, fields
class MainConfig(AppConfig):
name = 'awx.main'
verbose_name = _('Main')
def check_db_requirement(self, *args, **kwargs):
violations = db_requirement_violations()
if violations:
raise CommandError(violations)
def load_named_url_feature(self):
models = [m for m in self.get_models() if hasattr(m, 'get_absolute_url')]
generate_graph(models)
@@ -43,42 +46,6 @@ class MainConfig(AppConfig):
category_slug='named-url',
)
def _load_credential_types_feature(self):
"""
Create CredentialType records for any discovered credentials.
Note that Django docs advise _against_ interacting with the database using
the ORM models in the ready() path. Specifically, during testing.
However, we explicitly use the @bypass_in_test decorator to avoid calling this
method during testing.
Django also advises against running pattern because it runs everywhere i.e.
every management command. We use an advisory lock to ensure correctness and
we will deal performance if it becomes an issue.
"""
from awx.main.models.credential import CredentialType
if is_database_synchronized():
CredentialType.setup_tower_managed_defaults(app_config=self)
@bypass_in_test
def load_credential_types_feature(self):
from awx.main.models.credential import load_credentials
load_credentials()
return self._load_credential_types_feature()
def load_inventory_plugins(self):
from awx.main.models.inventory import InventorySourceOptions
is_awx = detect_server_product_name() == 'AWX'
extra_entry_point_groups = () if is_awx else ('inventory.supported',)
entry_points = load_all_entry_points_for(['inventory', *extra_entry_point_groups])
for entry_point_name, entry_point in entry_points.items():
cls = entry_point.load()
InventorySourceOptions.injectors[entry_point_name] = cls
def configure_dispatcherd(self):
"""This implements the default configuration for dispatcherd
@@ -100,13 +67,5 @@ class MainConfig(AppConfig):
super().ready()
self.configure_dispatcherd()
"""
Credential loading triggers database operations. There are cases we want to call
awx-manage collectstatic without a database. All management commands invoke the ready() code
path. Using settings.AWX_SKIP_CREDENTIAL_TYPES_DISCOVER _could_ invoke a database operation.
"""
if not os.environ.get('AWX_SKIP_CREDENTIAL_TYPES_DISCOVER', None):
self.load_credential_types_feature()
self.load_named_url_feature()
self.load_inventory_plugins()
pre_migrate.connect(self.check_db_requirement, sender=self)

View File

@@ -1,87 +0,0 @@
import functools
from django.conf import settings
from django.core.cache.backends.base import DEFAULT_TIMEOUT
from django.core.cache.backends.redis import RedisCache
from redis.exceptions import ConnectionError, ResponseError, TimeoutError
import socket
# This list comes from what django-redis ignores and the behavior we are trying
# to retain while dropping the dependency on django-redis.
IGNORED_EXCEPTIONS = (TimeoutError, ResponseError, ConnectionError, socket.timeout)
CONNECTION_INTERRUPTED_SENTINEL = object()
def optionally_ignore_exceptions(func=None, return_value=None):
if func is None:
return functools.partial(optionally_ignore_exceptions, return_value=return_value)
@functools.wraps(func)
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except IGNORED_EXCEPTIONS as e:
if settings.DJANGO_REDIS_IGNORE_EXCEPTIONS:
return return_value
raise e.__cause__ or e
return wrapper
class AWXRedisCache(RedisCache):
"""
We just want to wrap the upstream RedisCache class so that we can ignore
the exceptions that it raises when the cache is unavailable.
"""
@optionally_ignore_exceptions
def add(self, key, value, timeout=DEFAULT_TIMEOUT, version=None):
return super().add(key, value, timeout, version)
@optionally_ignore_exceptions(return_value=CONNECTION_INTERRUPTED_SENTINEL)
def _get(self, key, default=None, version=None):
return super().get(key, default, version)
def get(self, key, default=None, version=None):
value = self._get(key, default, version)
if value is CONNECTION_INTERRUPTED_SENTINEL:
return default
return value
@optionally_ignore_exceptions
def set(self, key, value, timeout=DEFAULT_TIMEOUT, version=None):
return super().set(key, value, timeout, version)
@optionally_ignore_exceptions
def touch(self, key, timeout=DEFAULT_TIMEOUT, version=None):
return super().touch(key, timeout, version)
@optionally_ignore_exceptions
def delete(self, key, version=None):
return super().delete(key, version)
@optionally_ignore_exceptions
def get_many(self, keys, version=None):
return super().get_many(keys, version)
@optionally_ignore_exceptions
def has_key(self, key, version=None):
return super().has_key(key, version)
@optionally_ignore_exceptions
def incr(self, key, delta=1, version=None):
return super().incr(key, delta, version)
@optionally_ignore_exceptions
def set_many(self, data, timeout=DEFAULT_TIMEOUT, version=None):
return super().set_many(data, timeout, version)
@optionally_ignore_exceptions
def delete_many(self, keys, version=None):
return super().delete_many(keys, version)
@optionally_ignore_exceptions
def clear(self):
return super().clear()

View File

@@ -25,7 +25,7 @@ def get_dispatcherd_config(for_service: bool = False, mock_publish: bool = False
"version": 2,
"service": {
"pool_kwargs": {
"min_workers": settings.JOB_EVENT_WORKERS,
"min_workers": settings.DISPATCHER_MIN_WORKERS,
"max_workers": max_workers,
# This must be less than max_workers to make sense, which is usually 4
# With reserve of 1, after a burst of tasks, load needs to down to 4-1=3

View File

@@ -1,9 +1,11 @@
# Copyright (c) 2015 Ansible, Inc.
# All Rights Reserved
from django.core.management.base import BaseCommand
from django.core.management.base import BaseCommand, CommandError
from django.db import connection
from awx.main.utils.db import db_requirement_violations
class Command(BaseCommand):
"""Checks connection to the database, and prints out connection info if not connected"""
@@ -13,4 +15,8 @@ class Command(BaseCommand):
cursor.execute("SELECT version()")
version = str(cursor.fetchone()[0])
violations = db_requirement_violations()
if violations:
raise CommandError(violations)
return "Database Version: {}".format(version)

View File

@@ -52,7 +52,11 @@ class Command(BaseCommand):
ssh_type = CredentialType.objects.filter(namespace='ssh').first()
c, _ = Credential.objects.get_or_create(
credential_type=ssh_type, name='Demo Credential', inputs={'username': getattr(superuser, 'username', 'null')}, created_by=superuser
credential_type=ssh_type,
name='Demo Credential',
inputs={'username': getattr(superuser, 'username', 'null')},
created_by=superuser,
organization=o,
)
if superuser:

View File

@@ -211,7 +211,7 @@ class AdHocCommand(UnifiedJob, JobNotificationMixin):
return AdHocCommand.objects.create(**data)
def save(self, *args, **kwargs):
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
def add_to_update_fields(name):
if name not in update_fields:

View File

@@ -177,7 +177,7 @@ class CreatedModifiedModel(BaseModel):
)
def save(self, *args, **kwargs):
update_fields = list(kwargs.get('update_fields', []))
update_fields = list(kwargs.get('update_fields') or [])
# Manually perform auto_now_add and auto_now logic.
if not self.pk and not self.created:
self.created = now()
@@ -207,7 +207,7 @@ class PasswordFieldsModel(BaseModel):
new_instance = not bool(self.pk)
# If update_fields has been specified, add our field names to it,
# if it hasn't been specified, then we're just doing a normal save.
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
# When first saving to the database, don't store any password field
# values, but instead save them until after the instance is created.
# Otherwise, store encrypted values to the database.
@@ -322,7 +322,7 @@ class PrimordialModel(HasEditsMixin, CreatedModifiedModel):
self._prior_values_store = {}
def save(self, *args, **kwargs):
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
user = get_current_user()
if user and not user.id:
user = None

View File

@@ -47,6 +47,7 @@ from awx.main.models.rbac import (
)
from awx.main.models import Team, Organization
from awx.main.utils import encrypt_field
from awx.main.utils.lazy_registry import LazyLoadDict
from awx_plugins.interfaces._temporary_private_licensing_api import detect_server_product_name
__all__ = ['Credential', 'CredentialType', 'CredentialInputSource', 'build_safe_env']
@@ -569,7 +570,7 @@ class CredentialTypeHelper:
class ManagedCredentialType(SimpleNamespace):
registry = {}
registry = None # initialized as LazyLoadDict after load_credentials is defined
class CredentialInputSource(PrimordialModel):
@@ -661,6 +662,8 @@ def _is_oidc_namespace_disabled(ns):
def load_credentials():
ManagedCredentialType.registry.clear()
awx_entry_points = {ep.name: ep for ep in entry_points(group='awx_plugins.managed_credentials')}
supported_entry_points = {ep.name: ep for ep in entry_points(group='awx_plugins.managed_credentials.supported')}
plugin_entry_points = awx_entry_points if detect_server_product_name() == 'AWX' else {**awx_entry_points, **supported_entry_points}
@@ -692,3 +695,8 @@ def load_credentials():
plugin = ep.load()
CredentialType.load_plugin(ns, plugin)
# load_credentials writes directly into this dict via registry[ns] = ...,
# LazyLoadDict just ensures it runs once before the first read access
ManagedCredentialType.registry = LazyLoadDict(load_credentials)

View File

@@ -27,7 +27,10 @@ from ansible_base.lib.utils.models import prevent_search
# AWX
from awx.api.versioning import reverse
from awx.main.utils.common import load_all_entry_points_for
from awx.main.utils.lazy_registry import LazyLoadDict
from awx.main.utils.plugins import discover_available_cloud_provider_plugin_names, compute_cloud_inventory_sources
from awx_plugins.interfaces._temporary_private_licensing_api import detect_server_product_name
from awx.main.consumers import emit_channel_notification
from awx.main.fields import (
ImplicitRoleField,
@@ -926,12 +929,22 @@ class HostMetricSummaryMonthly(models.Model):
indirectly_managed_hosts = models.IntegerField(default=0, help_text=("Manually entered number indirectly managed hosts for a certain month"))
def _load_inventory_plugins():
is_awx = detect_server_product_name() == 'AWX'
extra_entry_point_groups = () if is_awx else ('inventory.supported',)
all_entry_points = load_all_entry_points_for(['inventory', *extra_entry_point_groups])
for entry_point_name, entry_point in all_entry_points.items():
cls = entry_point.load()
InventorySourceOptions.injectors[entry_point_name] = cls
class InventorySourceOptions(BaseModel):
"""
Common fields for InventorySource and InventoryUpdate.
"""
injectors = dict()
injectors = LazyLoadDict(_load_inventory_plugins)
# From the options of the Django management base command
INVENTORY_UPDATE_VERBOSITY_CHOICES = [
@@ -1149,7 +1162,7 @@ class InventorySource(UnifiedJobTemplate, InventorySourceOptions, CustomVirtualE
# If update_fields has been specified, add our field names to it,
# if it hasn't been specified, then we're just doing a normal save.
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
is_new_instance = not bool(self.pk)
# Set name automatically. Include PK (or placeholder) to make sure the names are always unique.

View File

@@ -347,7 +347,7 @@ class JobTemplate(
return actual_slice_count
def save(self, *args, **kwargs):
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
# if project is deleted for some reason, then keep the old organization
# to retain ownership for organization admins
if self.project and self.project.organization_id != self.organization_id:
@@ -1165,7 +1165,7 @@ class JobHostSummary(CreatedModifiedModel):
# if it hasn't been specified, then we're just doing a normal save.
if self.host is not None:
self.host_name = self.host.name
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
self.failed = bool(self.dark or self.failures)
update_fields.append('failed')
super(JobHostSummary, self).save(*args, **kwargs)

View File

@@ -99,7 +99,7 @@ class NotificationTemplate(CommonModelNameNotUnique):
def save(self, *args, **kwargs):
new_instance = not bool(self.pk)
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
# preserve existing notification messages if not overwritten by new messages
if not new_instance:

View File

@@ -367,7 +367,7 @@ class Project(UnifiedJobTemplate, ProjectOptions, ResourceMixin, CustomVirtualEn
pre_save_vals = getattr(self, '_prior_values_store', {})
# If update_fields has been specified, add our field names to it,
# if it hasn't been specified, then we're just doing a normal save.
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
self._skip_update = bool(kwargs.pop('skip_update', False))
# Create auto-generated local path if project uses SCM.
if self.pk and self.scm_type and not self.local_path.startswith('_'):

View File

@@ -613,7 +613,7 @@ def get_role_from_object_role(object_role):
model_name, role_name = rd.name.split()
role_name = role_name.lower()
role_name += '_role'
return getattr(object_role.content_object, role_name)
return getattr(object_role.content_object, role_name, None)
def give_or_remove_permission(role, actor, giving=True, rd=None):
@@ -649,6 +649,8 @@ def give_creator_permissions(user, obj):
if assignment:
with disable_rbac_sync():
old_role = get_role_from_object_role(assignment.object_role)
if old_role is None:
return
old_role.members.add(user)

View File

@@ -305,7 +305,7 @@ class UnifiedJobTemplate(PolymorphicModel, CommonModelNameNotUnique, ExecutionEn
def save(self, *args, **kwargs):
# If update_fields has been specified, add our field names to it,
# if it hasn't been specified, then we're just doing a normal save.
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
# Update status and last_updated fields.
if not getattr(_inventory_updates, 'is_updating', False):
updated_fields = self._set_status_and_last_job_run(save=False)
@@ -877,7 +877,7 @@ class UnifiedJob(
"""
# If update_fields has been specified, add our field names to it,
# if it hasn't been specified, then we're just doing a normal save.
update_fields = kwargs.get('update_fields', [])
update_fields = kwargs.get('update_fields') or []
# Get status before save...
status_before = self.status or 'new'

View File

@@ -900,7 +900,7 @@ class WorkflowApproval(UnifiedJob, JobNotificationMixin):
return 'workflow_approval_template'
def save(self, *args, **kwargs):
update_fields = list(kwargs.get('update_fields', []))
update_fields = list(kwargs.get('update_fields') or [])
if self.timeout != 0 and ((not self.pk) or (not update_fields) or ('timeout' in update_fields)):
if not self.created: # on creation, created will be set by parent class, so we fudge it here
created = now()

View File

@@ -688,6 +688,17 @@ class TaskManager(TaskBase):
logger.error(f'{j.execution_node} is not a registered instance; reaping {j.log_format}')
reap_job(j, 'failed')
# Reset waiting jobs whose controller_node was deprovisioned (e.g. K8s pod replaced).
# These jobs will never be picked up because no live node is listening for them.
registered_control_nodes = Instance.objects.filter(node_type__in=('control', 'hybrid')).values_list('hostname', flat=True)
orphaned_waiting = UnifiedJob.objects.filter(status='waiting').exclude(controller_node__in=registered_control_nodes)
for j in orphaned_waiting:
logger.warning(f'{j.controller_node} is not a registered instance; resetting {j.log_format} to pending')
j.status = 'pending'
j.controller_node = ''
j.execution_node = ''
j.save(update_fields=['status', 'controller_node', 'execution_node'])
def process_tasks(self):
# maintain a list of jobs that went to an early failure state,
# meaning the dispatcher never got these jobs,

View File

@@ -19,6 +19,7 @@ from dispatcherd.publish import task
# Runner
import ansible_runner.cleanup
import psycopg
from ansible_base.lib.cache.tasks import clear_cache as dab_clear_cache
from ansible_base.lib.utils.db import advisory_lock
# django-ansible-base
@@ -68,10 +69,12 @@ from awx.main.models import (
UnifiedJob,
convert_jsonfields,
)
from awx.main.models.credential import CredentialType
from awx.main.tasks.helpers import is_run_threshold_reached
from awx.main.tasks.host_indirect import save_indirect_host_entries
from awx.main.tasks.receptor import administrative_workunit_reaper, get_receptor_ctl, worker_cleanup, worker_info, write_receptor_config
from awx.main.utils.common import ignore_inventory_computed_fields, ignore_inventory_group_removal
from awx.main.utils.migration import is_database_synchronized
from awx.main.utils.reload import stop_local_services
logger = logging.getLogger('awx.main.tasks.system')
@@ -83,6 +86,16 @@ Try upgrading OpenSSH or providing your private key in an different format. \
'''
def _sync_credential_types_to_db():
"""Ensure CredentialType DB rows match the installed plugins.
The in-memory registry is populated lazily on first access via LazyLoadDict.
This function only handles the DB sync step.
"""
if is_database_synchronized():
CredentialType.setup_tower_managed_defaults()
def _run_dispatch_startup_common():
"""
Execute the common startup initialization steps.
@@ -98,6 +111,11 @@ def _run_dispatch_startup_common():
except Exception:
logger.exception("Failed to write receptor config, skipping.")
try:
_sync_credential_types_to_db()
except Exception:
logger.exception("Failed to sync credential types to DB, skipping.")
try:
convert_jsonfields()
except Exception:
@@ -240,12 +258,17 @@ def apply_cluster_membership_policies():
# Process policy instance list first, these will represent manually managed memberships
instance_hostnames_map = {inst.hostname: inst for inst in all_instances}
for ig in all_groups:
# we don't want to allow execution nodes in the control plane
exclude_type = 'execution' if ig.name == settings.DEFAULT_CONTROL_PLANE_QUEUE_NAME else 'control'
group_actual = Group(obj=ig, instances=[], prior_instances=[instance.pk for instance in ig.instances.all()]) # obtained in prefetch
for hostname in ig.policy_instance_list:
if hostname not in instance_hostnames_map:
logger.info("Unknown instance {} in {} policy list".format(hostname, ig.name))
continue
inst = instance_hostnames_map[hostname]
if inst.node_type == exclude_type:
logger.info("Instance {} is excluded in {} policy list".format(hostname, ig.name))
continue
group_actual.instances.append(inst.id)
# NOTE: arguable behavior: policy-list-group is not added to
# instance's group count for consideration in minimum-policy rules
@@ -326,24 +349,22 @@ def apply_cluster_membership_policies():
logger.debug('Cluster policy computation finished in {} seconds'.format(time.time() - started_compute))
@task(queue='tower_settings_change', timeout=600)
def clear_setting_cache(setting_keys):
# log that cache is being cleared
logger.info(f"clear_setting_cache of keys {setting_keys}")
orig_len = len(setting_keys)
for i in range(orig_len):
for dependent_key in settings_registry.get_dependent_settings(setting_keys[i]):
setting_keys.append(dependent_key)
cache_keys = set(setting_keys)
logger.debug('cache delete_many(%r)', cache_keys)
cache.delete_many(cache_keys)
def _resolve_setting_dependents(key):
return settings_registry.get_dependent_settings(key)
if 'LOG_AGGREGATOR_LEVEL' in setting_keys:
def _post_setting_invalidation(invalidated_keys):
if 'LOG_AGGREGATOR_LEVEL' in invalidated_keys:
ctl = get_control_from_settings()
ctl.queuename = get_task_queuename()
ctl.control('set_log_level', data={'level': settings.LOG_AGGREGATOR_LEVEL})
@task(queue='tower_settings_change', timeout=600)
def clear_setting_cache(setting_keys):
dab_clear_cache(setting_keys, _resolve_setting_dependents, _post_setting_invalidation)
@task(queue='tower_broadcast_all', timeout=600)
def delete_project_files(project_path):
# TODO: possibly implement some retry logic

View File

@@ -1,4 +1,3 @@
import os
import pytest
import requests
from unittest import mock
@@ -258,92 +257,3 @@ class TestAnalyticsGenericView:
else:
# assert mock_base_auth_request not called
mock_base_auth_request.assert_not_called()
@pytest.mark.django_db
def test__send_to_analytics_respects_proxy_env_oidc(self):
settings_map = {
'INSIGHTS_TRACKING_STATE': True,
'AUTOMATION_ANALYTICS_URL': 'https://example.com',
'REDHAT_USERNAME': 'redhat_user',
'REDHAT_PASSWORD': 'redhat_pass',
'SUBSCRIPTIONS_CLIENT_ID': '',
'SUBSCRIPTIONS_CLIENT_SECRET': '',
'AWX_TASK_ENV': {'HTTPS_PROXY': '192.168.50.100:1234', 'HTTP_PROXY': '192.168.50.100:5678'},
}
with override_settings(**settings_map):
request = RequestFactory().post('/some/path')
view = AnalyticsGenericView()
with mock.patch('awx.api.views.analytics.OIDCClient') as mock_oidc_client:
mock_client_instance = mock.Mock()
mock_oidc_client.return_value = mock_client_instance
def _check_env_and_respond(*args, **kwargs):
assert os.environ.get('HTTPS_PROXY') == '192.168.50.100:1234'
assert os.environ.get('HTTP_PROXY') == '192.168.50.100:5678'
return mock.Mock(status_code=200)
mock_client_instance.make_request.side_effect = _check_env_and_respond
response = view._send_to_analytics(request, 'POST')
assert response.status_code == 200
mock_client_instance.make_request.assert_called_once()
@pytest.mark.django_db
def test__send_to_analytics_respects_proxy_env_basic_auth(self):
settings_map = {
'INSIGHTS_TRACKING_STATE': True,
'AUTOMATION_ANALYTICS_URL': 'https://example.com',
'REDHAT_USERNAME': 'redhat_user',
'REDHAT_PASSWORD': 'redhat_pass',
'SUBSCRIPTIONS_CLIENT_ID': '',
'SUBSCRIPTIONS_CLIENT_SECRET': '',
'AWX_TASK_ENV': {'HTTPS_PROXY': '192.168.50.100:1234'},
}
with override_settings(**settings_map):
request = RequestFactory().post('/some/path')
view = AnalyticsGenericView()
with mock.patch('awx.api.views.analytics.OIDCClient') as mock_oidc_client, mock.patch(
'awx.api.views.analytics.AnalyticsGenericView._base_auth_request'
) as mock_base_auth:
mock_client_instance = mock.Mock()
mock_oidc_client.return_value = mock_client_instance
mock_client_instance.make_request.side_effect = requests.RequestException("OIDC failed")
def _check_env_and_respond(*args, **kwargs):
assert os.environ.get('HTTPS_PROXY') == '192.168.50.100:1234'
return mock.Mock(status_code=200)
mock_base_auth.side_effect = _check_env_and_respond
response = view._send_to_analytics(request, 'POST')
assert response.status_code == 200
mock_base_auth.assert_called_once()
@pytest.mark.django_db
def test__send_to_analytics_restores_env_after_request(self):
original_value = os.environ.pop('HTTPS_PROXY', None)
settings_map = {
'INSIGHTS_TRACKING_STATE': True,
'AUTOMATION_ANALYTICS_URL': 'https://example.com',
'REDHAT_USERNAME': 'redhat_user',
'REDHAT_PASSWORD': 'redhat_pass',
'SUBSCRIPTIONS_CLIENT_ID': '',
'SUBSCRIPTIONS_CLIENT_SECRET': '',
'AWX_TASK_ENV': {'HTTPS_PROXY': '192.168.50.100:1234'},
}
try:
with override_settings(**settings_map):
request = RequestFactory().post('/some/path')
view = AnalyticsGenericView()
with mock.patch('awx.api.views.analytics.OIDCClient') as mock_oidc_client:
mock_client_instance = mock.Mock()
mock_oidc_client.return_value = mock_client_instance
mock_client_instance.make_request.return_value = mock.Mock(status_code=200)
view._send_to_analytics(request, 'POST')
assert 'HTTPS_PROXY' not in os.environ
finally:
if original_value is not None:
os.environ['HTTPS_PROXY'] = original_value

View File

@@ -1,5 +1,3 @@
# TODO: As of writing this our only concern is ensuring that the fact feature is reflected in the Host endpoint.
# Other host tests should live here to make this test suite more complete.
import pytest
import urllib.parse
@@ -20,6 +18,48 @@ def inventory_structure():
Group.objects.create(name="g3", inventory=inv)
@pytest.fixture
def host_filter_inventory():
"""Inventory with hosts and groups matching the tower-qa test_host_filter structure.
Groups: groupA (contains groupAA as child), groupAA, groupB
Hosts: hostA (in groupA), hostAA (in groupAA), hostB (in groupB), hostDup (in all 3 groups)
"""
org = Organization.objects.create(name="hf-org")
inv = Inventory.objects.create(name="hf-inv", organization=org)
groupA = Group.objects.create(name="groupA", inventory=inv)
groupAA = Group.objects.create(name="groupAA", inventory=inv)
groupB = Group.objects.create(name="groupB", inventory=inv)
hostA = Host.objects.create(name="hostA", inventory=inv)
hostAA = Host.objects.create(name="hostAA", inventory=inv)
hostB = Host.objects.create(name="hostB", inventory=inv)
hostDup = Host.objects.create(name="hostDup", inventory=inv)
groupA.hosts.add(hostA, hostDup)
groupAA.hosts.add(hostAA, hostDup)
groupB.hosts.add(hostB, hostDup)
groupA.children.add(groupAA)
return {
'org': org,
'inv': inv,
'hosts': {'hostA': hostA, 'hostAA': hostAA, 'hostB': hostB, 'hostDup': hostDup},
'groups': {'groupA': groupA, 'groupAA': groupAA, 'groupB': groupB},
}
def get_host_names(response):
return sorted(h['name'] for h in response.data['results'])
def host_filter_get(get, user, host_filter):
url = reverse('api:host_list')
params = "?host_filter=%s" % urllib.parse.quote(host_filter, safe='')
return get(url + params, user)
@pytest.mark.django_db
def test_q1(inventory_structure, get, user):
def evaluate_query(query, expected_hosts):
@@ -50,3 +90,184 @@ def test_q1(inventory_structure, get, user):
# The following test verifies if the search in host_filter is case insensitive.
query = 'search="HOST1"'
evaluate_query(query, [hosts[0]])
# --- Host filter query tests (migrated from tower-qa test_host_filter.py) ---
@pytest.mark.django_db
@pytest.mark.parametrize(
"host_filter, expected",
[
("name=hostA", ["hostA"]),
("name=not_found", []),
("name=hostDup", ["hostDup"]),
],
)
def test_basic_host_name_search(host_filter_inventory, get, admin_user, host_filter, expected):
response = host_filter_get(get, admin_user, host_filter)
assert response.status_code == 200
assert get_host_names(response) == sorted(expected)
@pytest.mark.django_db
@pytest.mark.parametrize(
"host_filter, expected",
[
("name=hostA or name=hostB", ["hostA", "hostB"]),
("name=hostA or name=not_found", ["hostA"]),
("name=not_found or name=not_found", []),
("name=hostA or name=hostA", ["hostA"]),
("name=hostDup or name=hostDup", ["hostDup"]),
("name=hostA or name=hostAA or name=not_found", ["hostA", "hostAA"]),
],
)
def test_host_name_search_with_or(host_filter_inventory, get, admin_user, host_filter, expected):
response = host_filter_get(get, admin_user, host_filter)
assert response.status_code == 200
assert get_host_names(response) == sorted(expected)
@pytest.mark.django_db
@pytest.mark.parametrize(
"host_filter, expected",
[
("name=hostA and name=hostB", []),
("name=hostA and name=hostA", ["hostA"]),
("name=not_found and name=not_found", []),
("name=hostDup and name=hostDup", ["hostDup"]),
("name=hostA and name=hostB and name=not_found", []),
],
)
def test_host_name_search_with_and(host_filter_inventory, get, admin_user, host_filter, expected):
response = host_filter_get(get, admin_user, host_filter)
assert response.status_code == 200
assert get_host_names(response) == sorted(expected)
@pytest.mark.django_db
@pytest.mark.parametrize(
"host_filter, expected",
[
("groups__name=groupA", ["hostA", "hostDup"]),
("groups__name=groupAA", ["hostAA", "hostDup"]),
("groups__name=not_found", []),
],
)
def test_basic_group_search(host_filter_inventory, get, admin_user, host_filter, expected):
response = host_filter_get(get, admin_user, host_filter)
assert response.status_code == 200
assert get_host_names(response) == sorted(expected)
@pytest.mark.django_db
@pytest.mark.parametrize(
"host_filter, expected",
[
("groups__name=groupA or groups__name=groupB", ["hostA", "hostB", "hostDup"]),
("groups__name=groupA or groups__name=not_found", ["hostA", "hostDup"]),
("groups__name=not_found or groups__name=not_found", []),
("groups__name=groupA or groups__name=groupA", ["hostA", "hostDup"]),
(
"groups__name=groupA or groups__name=groupAA or groups__name=not_found",
["hostA", "hostAA", "hostDup"],
),
],
)
def test_group_search_with_or(host_filter_inventory, get, admin_user, host_filter, expected):
response = host_filter_get(get, admin_user, host_filter)
assert response.status_code == 200
assert get_host_names(response) == sorted(expected)
@pytest.mark.django_db
@pytest.mark.parametrize(
"host_filter, expected",
[
("groups__name=groupA and groups__name=groupB", ["hostDup"]),
("groups__name=groupA and groups__name=groupA", ["hostA", "hostDup"]),
("groups__name=not_found and groups__name=not_found", []),
("groups__name=groupA and groups__name=groupB and groups__name=not_found", []),
],
)
def test_group_search_with_and(host_filter_inventory, get, admin_user, host_filter, expected):
response = host_filter_get(get, admin_user, host_filter)
assert response.status_code == 200
assert get_host_names(response) == sorted(expected)
@pytest.mark.django_db
@pytest.mark.parametrize(
"host_filter, expected",
[
("name=hostA or groups__name=groupB", ["hostA", "hostB", "hostDup"]),
("name=hostA and groups__name=groupA", ["hostA"]),
("name=hostA and groups__name=not_found", []),
("name=not_found and groups__name=not_found", []),
("name=hostDup and groups__name=groupA", ["hostDup"]),
("name=hostDup and groups__name=groupB", ["hostDup"]),
],
)
def test_basic_hybrid_search(host_filter_inventory, get, admin_user, host_filter, expected):
response = host_filter_get(get, admin_user, host_filter)
assert response.status_code == 200
assert get_host_names(response) == sorted(expected)
@pytest.mark.django_db
def test_smart_search(get, admin_user):
org = Organization.objects.create(name="search-org")
inv = Inventory.objects.create(name="search-inv", organization=org)
host = Host.objects.create(name="unique_search_target", description="findme_description", inventory=inv)
for search_term in ["unique_search_target", "findme_description"]:
response = host_filter_get(get, admin_user, "search=%s" % search_term)
assert response.status_code == 200
names = get_host_names(response)
assert host.name in names
@pytest.mark.django_db
def test_password_field_filter_blocked(get, admin_user):
url = reverse('api:host_list')
filters = [
"created_by__password__icontains=pas3w3rd",
"search=foo or created_by__password__icontains=pas3w3rd",
"created_by__password__icontains=passw3rd or search=foo",
]
for f in filters:
params = "?host_filter=%s" % urllib.parse.quote(f, safe='')
response = get(url + params, admin_user)
assert response.status_code == 400, f"Expected 400 for filter: {f}"
@pytest.mark.django_db
def test_unicode_host_filter(get, admin_user):
org = Organization.objects.create(name="unicode-org")
inv = Inventory.objects.create(name="unicode-inv", organization=org)
host = Host.objects.create(name="ホスト", inventory=inv)
group = Group.objects.create(name="グループ", inventory=inv)
group.hosts.add(host)
response = host_filter_get(get, admin_user, "name=ホスト")
assert response.status_code == 200
assert len(response.data['results']) == 1
assert response.data['results'][0]['id'] == host.id
response = host_filter_get(get, admin_user, "groups__name=グループ")
assert response.status_code == 200
assert len(response.data['results']) == 1
assert response.data['results'][0]['id'] == host.id
@pytest.mark.django_db
@pytest.mark.parametrize(
"invalid_filter",
["string_without_equals", "1", "1.0", "true"],
ids=["bare_string", "integer", "float", "bool"],
)
def test_invalid_host_filter(get, admin_user, invalid_filter):
url = reverse('api:host_list')
params = "?host_filter=%s" % urllib.parse.quote(invalid_filter, safe='')
response = get(url + params, admin_user)
assert response.status_code == 400

View File

@@ -139,6 +139,7 @@ def test_survey_password_default(post, patch, admin_user, project, inventory, su
("DTSTART:20300308T050000Z", "One or more rule required in rrule"),
("DTSTART:20300308T050000Z RRULE:FREQ=MONTHLY;INTERVAL=1; EXDATE:20220401", "EXDATE not allowed in rrule"),
("DTSTART:20300308T050000Z RRULE:FREQ=MONTHLY;INTERVAL=1; RDATE:20220401", "RDATE not allowed in rrule"),
("DTSTART:20300308T050000Z RRULE:FREQ=YEARLY;INTERVAL=0;BYDAY=MO", "INTERVAL must be a positive integer"),
("DTSTART:20300308T050000Z RRULE:FREQ=SECONDLY;INTERVAL=5;COUNT=6", "SECONDLY is not supported"),
# Individual rule test
("DTSTART:20300308T050000Z RRULE:NONSENSE", "INTERVAL required in rrule"),
@@ -202,6 +203,7 @@ def test_multiple_invalid_rrules(post, admin_user, project, inventory):
"rrule": [
"Multiple DTSTART is not supported.",
"INTERVAL required in rrule: RULE:FREQ=SECONDLY",
"SECONDLY is not supported: RULE:FREQ=SECONDLY",
"RRULE may not contain both COUNT and UNTIL: RULE:FREQ=MINUTELY;INTERVAL=10;COUNT=5;UNTIL=20220101",
"rrule parsing failed validation: 'NoneType' object has no attribute 'group'",
]

View File

@@ -0,0 +1,191 @@
import json
import pytest
from awx.api.versioning import reverse
from awx.main.models import Organization, Host, Group, Inventory
@pytest.fixture
def smart_inv_org():
return Organization.objects.create(name="smart-org")
@pytest.fixture
def smart_inv_source(smart_inv_org):
inv = Inventory.objects.create(name="smart-source-inv", organization=smart_inv_org)
Host.objects.create(name="hostA", inventory=inv)
Host.objects.create(name="hostB", inventory=inv)
Host.objects.create(name="hostDup", inventory=inv)
groupA = Group.objects.create(name="groupA", inventory=inv)
groupB = Group.objects.create(name="groupB", inventory=inv)
groupA.hosts.add(*inv.hosts.filter(name__in=["hostA", "hostDup"]))
groupB.hosts.add(*inv.hosts.filter(name__in=["hostB", "hostDup"]))
return inv
@pytest.mark.django_db
def test_create_smart_inventory(post, admin_user, smart_inv_org):
resp = post(
reverse('api:inventory_list'),
{
'name': 'my-smart-inv',
'kind': 'smart',
'organization': smart_inv_org.pk,
'host_filter': 'name=hostA',
},
admin_user,
expect=201,
)
assert resp.data['kind'] == 'smart'
assert resp.data['host_filter'] == 'name=hostA'
@pytest.mark.django_db
def test_create_smart_inventory_requires_host_filter(post, admin_user, smart_inv_org):
resp = post(
reverse('api:inventory_list'),
{
'name': 'no-filter-smart',
'kind': 'smart',
'organization': smart_inv_org.pk,
},
admin_user,
expect=400,
)
assert 'host_filter' in json.dumps(resp.data)
@pytest.mark.django_db
def test_unable_to_create_host_in_smart_inventory(post, admin_user, smart_inv_org):
smart_inv = Inventory.objects.create(
name="no-host-create",
kind="smart",
host_filter="name=hostA",
organization=smart_inv_org,
)
url = reverse('api:inventory_hosts_list', kwargs={'pk': smart_inv.pk})
resp = post(url, {'name': 'new-host'}, admin_user, expect=400)
assert 'Cannot create' in json.dumps(resp.data)
@pytest.mark.django_db
def test_unable_to_create_group_in_smart_inventory(post, admin_user, smart_inv_org):
smart_inv = Inventory.objects.create(
name="no-group-create",
kind="smart",
host_filter="name=hostA",
organization=smart_inv_org,
)
url = reverse('api:inventory_groups_list', kwargs={'pk': smart_inv.pk})
resp = post(url, {'name': 'new-group'}, admin_user, expect=400)
assert 'Cannot create' in json.dumps(resp.data)
@pytest.mark.django_db
def test_unable_to_create_inventory_source_in_smart_inventory(post, admin_user, smart_inv_org):
smart_inv = Inventory.objects.create(
name="no-src-create",
kind="smart",
host_filter="name=hostA",
organization=smart_inv_org,
)
url = reverse('api:inventory_inventory_sources_list', kwargs={'pk': smart_inv.pk})
resp = post(url, {'name': 'new-src', 'source': 'ec2'}, admin_user, expect=400)
assert 'Cannot create' in json.dumps(resp.data)
@pytest.mark.django_db
def test_convert_smart_to_regular_inventory(admin_user, smart_inv_org):
smart_inv = Inventory.objects.create(
name="convert-to-regular",
kind="smart",
host_filter="name=anything",
organization=smart_inv_org,
)
assert smart_inv.kind == 'smart'
smart_inv.host_filter = ''
smart_inv.kind = ''
smart_inv.save()
smart_inv.refresh_from_db()
assert smart_inv.kind == ''
assert not smart_inv.host_filter
@pytest.mark.django_db
def test_smart_inventory_deletion_does_not_cascade(admin_user, smart_inv_source, smart_inv_org):
host = smart_inv_source.hosts.first()
smart_inv = Inventory.objects.create(
name="delete-no-cascade",
kind="smart",
host_filter="name=%s" % host.name,
organization=smart_inv_org,
)
smart_inv.delete()
assert Host.objects.filter(pk=host.pk).exists()
@pytest.mark.django_db
def test_urlencode_host_filter(post, admin_user, smart_inv_org):
post(
reverse('api:inventory_list'),
data={
'name': 'url-encoded-smart',
'kind': 'smart',
'organization': smart_inv_org.pk,
'host_filter': 'ansible_facts__ansible_distribution_version=%227.4%22',
},
user=admin_user,
expect=201,
)
si = Inventory.objects.get(name='url-encoded-smart')
assert si.host_filter == 'ansible_facts__ansible_distribution_version="7.4"'
@pytest.mark.django_db
def test_host_filter_unicode(post, admin_user, smart_inv_org):
post(
reverse('api:inventory_list'),
data={
'name': 'unicode-smart',
'kind': 'smart',
'organization': smart_inv_org.pk,
'host_filter': u'ansible_facts__ansible_distribution=レッドハット',
},
user=admin_user,
expect=201,
)
si = Inventory.objects.get(name='unicode-smart')
assert si.host_filter == u'ansible_facts__ansible_distribution=レッドハット'
@pytest.mark.django_db
@pytest.mark.parametrize("lookup", ['icontains', 'has_keys'])
def test_host_filter_invalid_ansible_facts_lookup(post, admin_user, smart_inv_org, lookup):
resp = post(
reverse('api:inventory_list'),
data={
'name': 'invalid-lookup-smart',
'kind': 'smart',
'organization': smart_inv_org.pk,
'host_filter': u'ansible_facts__ansible_distribution__{}=cent'.format(lookup),
},
user=admin_user,
expect=400,
)
assert 'ansible_facts does not support searching with __{}'.format(lookup) in json.dumps(resp.data)
@pytest.mark.django_db
def test_host_filter_ansible_facts_exact(post, admin_user, smart_inv_org):
post(
reverse('api:inventory_list'),
data={
'name': 'exact-smart',
'kind': 'smart',
'organization': smart_inv_org.pk,
'host_filter': 'ansible_facts__ansible_distribution__exact="CentOS"',
},
user=admin_user,
expect=201,
)

View File

@@ -145,124 +145,3 @@ def test_delete_ad_hoc_command_in_active_state(ad_hoc_command_factory, delete, a
adhoc = ad_hoc_command_factory(initial_state=status)
url = reverse('api:ad_hoc_command_detail', kwargs={'pk': adhoc.pk})
delete(url, None, admin, expect=403)
@pytest.fixture
def job_with_heavy_fields(job_factory):
job = job_factory()
job.extra_vars = '{"some_var": "some_value"}'
job.artifacts = {"some_artifact": "some_value"}
job.save()
return job
def _job_result(response, job_id):
for row in response.data['results']:
if row['id'] == job_id:
return row
raise AssertionError('job {} not found in {}'.format(job_id, [r['id'] for r in response.data['results']]))
@pytest.mark.django_db
def test_unified_jobs_list_strips_heavy_fields_by_default(get, admin, job_with_heavy_fields):
response = get(reverse('api:unified_job_list') + '?id={}'.format(job_with_heavy_fields.id), admin, expect=200)
row = _job_result(response, job_with_heavy_fields.id)
assert 'artifacts' not in row
assert 'extra_vars' not in row
@pytest.mark.django_db
def test_unified_jobs_list_include_artifacts(get, admin, job_with_heavy_fields):
response = get(
reverse('api:unified_job_list') + '?id={}&include=artifacts'.format(job_with_heavy_fields.id),
admin,
expect=200,
)
row = _job_result(response, job_with_heavy_fields.id)
assert 'artifacts' in row
assert 'extra_vars' not in row
@pytest.mark.django_db
def test_unified_jobs_list_include_extra_vars(get, admin, job_with_heavy_fields):
response = get(
reverse('api:unified_job_list') + '?id={}&include=extra_vars'.format(job_with_heavy_fields.id),
admin,
expect=200,
)
row = _job_result(response, job_with_heavy_fields.id)
assert 'extra_vars' in row
assert 'artifacts' not in row
@pytest.mark.django_db
def test_unified_jobs_list_include_both(get, admin, job_with_heavy_fields):
response = get(
reverse('api:unified_job_list') + '?id={}&include=artifacts,extra_vars'.format(job_with_heavy_fields.id),
admin,
expect=200,
)
row = _job_result(response, job_with_heavy_fields.id)
assert 'artifacts' in row
assert 'extra_vars' in row
@pytest.mark.django_db
def test_unified_jobs_list_include_tolerates_whitespace(get, admin, job_with_heavy_fields):
response = get(
reverse('api:unified_job_list') + '?id={}&include=%20artifacts%20,%20extra_vars%20'.format(job_with_heavy_fields.id),
admin,
expect=200,
)
row = _job_result(response, job_with_heavy_fields.id)
assert 'artifacts' in row
assert 'extra_vars' in row
@pytest.mark.django_db
def test_unified_jobs_list_include_ignores_unknown(get, admin, job_with_heavy_fields):
response = get(
reverse('api:unified_job_list') + '?id={}&include=does_not_exist'.format(job_with_heavy_fields.id),
admin,
expect=200,
)
row = _job_result(response, job_with_heavy_fields.id)
assert 'artifacts' not in row
assert 'extra_vars' not in row
@pytest.mark.django_db
def test_unified_jobs_list_include_does_not_honor_disallowed(get, admin, job_with_heavy_fields):
# event_processing_finished triggers a count(*) on main_jobevent and must
# not be re-enabled via the public ?include= param.
response = get(
reverse('api:unified_job_list') + '?id={}&include=event_processing_finished,job_args,result_traceback'.format(job_with_heavy_fields.id),
admin,
expect=200,
)
row = _job_result(response, job_with_heavy_fields.id)
assert 'event_processing_finished' not in row
assert 'job_args' not in row
assert 'result_traceback' not in row
assert 'artifacts' not in row
assert 'extra_vars' not in row
@pytest.mark.django_db
def test_jobs_list_strips_heavy_fields_by_default(get, admin, job_with_heavy_fields):
response = get(reverse('api:job_list') + '?id={}'.format(job_with_heavy_fields.id), admin, expect=200)
row = _job_result(response, job_with_heavy_fields.id)
assert 'artifacts' not in row
assert 'extra_vars' not in row
@pytest.mark.django_db
def test_jobs_list_include_extra_vars(get, admin, job_with_heavy_fields):
response = get(
reverse('api:job_list') + '?id={}&include=extra_vars'.format(job_with_heavy_fields.id),
admin,
expect=200,
)
row = _job_result(response, job_with_heavy_fields.id)
assert 'extra_vars' in row
assert 'artifacts' not in row

View File

@@ -0,0 +1,240 @@
import pytest
from awx.api.versioning import reverse
from awx.main.models import NotificationTemplate, Organization
from ansible_base.rbac.models import RoleDefinition
from ansible_base.rbac import permission_registry
NT_DATA = {
'notification_type': 'webhook',
'notification_configuration': {
'url': 'http://localhost',
'username': '',
'password': '',
'headers': {},
},
}
def nt_url(pk):
return reverse('api:notification_template_detail', kwargs={'pk': pk})
@pytest.fixture
def nt_add_role(setup_managed_roles):
"""A custom role with only add_notificationtemplate and view_organization.
This is intentionally narrower than Organization NotificationTemplate Admin
so that give_creator_permissions actually creates creator permissions."""
rd, _ = RoleDefinition.objects.get_or_create(
name='nt-add-only',
permissions=['add_notificationtemplate', 'view_organization'],
content_type=permission_registry.content_type_model.objects.get_for_model(Organization),
)
return rd
@pytest.mark.django_db
def test_create_with_add_only_role_gets_creator_permissions(rando, organization, post, get, patch, nt_add_role):
"""User with only add permission creates a notification template and gets
creator permissions (change, delete, view) via give_creator_permissions.
This exercises the fix for models without old-style roles (AAP-57274)."""
nt_add_role.give_permission(rando, organization)
r = post(
reverse('api:notification_template_list'),
dict(name='rando-nt', organization=organization.id, **NT_DATA),
user=rando,
expect=201,
)
nt = NotificationTemplate.objects.get(pk=r.data['id'])
assert rando.has_obj_perm(nt, 'change')
assert rando.has_obj_perm(nt, 'view')
# Creator permissions survive revocation of the org-level add role
nt_add_role.remove_permission(rando, organization)
get(nt_url(nt.pk), user=rando, expect=200)
patch(nt_url(nt.pk), data={'description': 'updated'}, user=rando, expect=200)
@pytest.mark.django_db
def test_org_admin_can_crud(rando, organization, post, get, patch, delete, setup_managed_roles):
"""User with org-level notification admin can create, view, edit, and delete"""
rd = RoleDefinition.objects.get(name='Organization NotificationTemplate Admin')
rd.give_permission(rando, organization)
r = post(
reverse('api:notification_template_list'),
dict(name='rando-nt', organization=organization.id, **NT_DATA),
user=rando,
expect=201,
)
pk = r.data['id']
url = nt_url(pk)
get(url, user=rando, expect=200)
patch(url, data={'description': 'updated'}, user=rando, expect=200)
delete(url, user=rando, expect=204)
assert not NotificationTemplate.objects.filter(pk=pk).exists()
@pytest.mark.django_db
def test_unpermissioned_user_cannot_access(rando, notification_template, get, patch, delete, setup_managed_roles):
"""User without any permissions cannot view, edit, or delete a notification template"""
url = nt_url(notification_template.pk)
get(url, user=rando, expect=403)
patch(url, data={'description': 'nope'}, user=rando, expect=403)
delete(url, user=rando, expect=403)
@pytest.mark.django_db
def test_grant_and_revoke_object_role(rando, notification_template, get, patch, setup_managed_roles):
"""Granting and revoking NotificationTemplate Admin role controls access"""
rd = RoleDefinition.objects.get(name='NotificationTemplate Admin')
url = nt_url(notification_template.pk)
get(url, user=rando, expect=403)
rd.give_permission(rando, notification_template)
get(url, user=rando, expect=200)
patch(url, data={'description': 'changed'}, user=rando, expect=200)
rd.remove_permission(rando, notification_template)
get(url, user=rando, expect=403)
patch(url, data={'description': 'nope'}, user=rando, expect=403)
@pytest.mark.django_db
def test_creator_can_access_sub_endpoints(rando, organization, post, get, nt_add_role):
"""Creator can access notification list sub-endpoint"""
nt_add_role.give_permission(rando, organization)
r = post(
reverse('api:notification_template_list'),
dict(name='rando-nt', organization=organization.id, **NT_DATA),
user=rando,
expect=201,
)
pk = r.data['id']
# Revoke org-level role so only creator permissions remain
nt_add_role.remove_permission(rando, organization)
get(
reverse('api:notification_template_notification_list', kwargs={'pk': pk}),
user=rando,
expect=200,
)
@pytest.mark.django_db
def test_list_filtered_by_permissions(rando, admin_user, organization, post, get, nt_add_role):
"""Notification template list only shows templates the user has access to"""
nt_add_role.give_permission(rando, organization)
post(
reverse('api:notification_template_list'),
dict(name='admin-nt', organization=organization.id, **NT_DATA),
user=admin_user,
expect=201,
)
post(
reverse('api:notification_template_list'),
dict(name='rando-nt', organization=organization.id, **NT_DATA),
user=rando,
expect=201,
)
# rando has org-level add, but admin-nt was created by admin → rando shouldn't see it
# unless org admin role also gives view. With add-only role, rando has view_organization
# but not view_notificationtemplate at the org level, so they only see their own (via creator perms)
nt_add_role.remove_permission(rando, organization)
r = get(reverse('api:notification_template_list'), user=rando, expect=200)
visible_names = {item['name'] for item in r.data['results']}
assert 'rando-nt' in visible_names
assert 'admin-nt' not in visible_names
@pytest.mark.django_db
def test_creator_access_list_with_add_only_role(rando, organization, post, get, nt_add_role):
"""User with add_only role creates a notification template and can access its access_list endpoint"""
from ansible_base.rbac.models import DABContentType
nt_add_role.give_permission(rando, organization)
r = post(
reverse('api:notification_template_list'),
dict(name='rando-nt', organization=organization.id, **NT_DATA),
user=rando,
expect=201,
)
nt = NotificationTemplate.objects.get(pk=r.data['id'])
# Revoke org-level role so only creator permissions remain
nt_add_role.remove_permission(rando, organization)
# Creator should be able to access the access_list endpoint for their own notification template
# Use the DAB access_list endpoint pattern: /api/v2/role_user_access/{model_name}/{pk}/
ct = DABContentType.objects.get_for_model(NotificationTemplate)
access_list_url = f'/api/v2/role_user_access/{ct.api_slug}/{nt.pk}/?order_by=id'
r = get(access_list_url, user=rando, expect=200)
# The creator should be listed in the access list
usernames = {user['username'] for user in r.data['results']}
assert rando.username in usernames
@pytest.mark.django_db
def test_unpermissioned_user_cannot_access_access_list(rando, organization, post, admin_user, get, setup_managed_roles):
"""User without view permission cannot access the access_list endpoint"""
from ansible_base.rbac.models import DABContentType
# Create a notification template as admin
r = post(
reverse('api:notification_template_list'),
dict(name='admin-nt', organization=organization.id, **NT_DATA),
user=admin_user,
expect=201,
)
nt = NotificationTemplate.objects.get(pk=r.data['id'])
ct = DABContentType.objects.get_for_model(NotificationTemplate)
access_list_url = f'/api/v2/role_user_access/{ct.api_slug}/{nt.pk}/?order_by=id'
# rando has no permissions on this notification template, so they can't see it or its access list
# The endpoint returns 404 (not found) instead of 403 when user can't view the resource
get(access_list_url, user=rando, expect=404)
@pytest.mark.django_db
def test_access_list_shows_creator(rando, organization, post, get, nt_add_role, setup_managed_roles):
"""Access list shows the creator with direct permissions"""
from ansible_base.rbac.models import DABContentType
from ansible_base.rbac.models import RoleDefinition
nt_add_role.give_permission(rando, organization)
# rando creates a notification template
r = post(
reverse('api:notification_template_list'),
dict(name='rando-nt', organization=organization.id, **NT_DATA),
user=rando,
expect=201,
)
nt = NotificationTemplate.objects.get(pk=r.data['id'])
# Now assign them the object admin role directly too
rd = RoleDefinition.objects.get(name='NotificationTemplate Admin')
rd.give_permission(rando, nt)
ct = DABContentType.objects.get_for_model(NotificationTemplate)
access_list_url = f'/api/v2/role_user_access/{ct.api_slug}/{nt.pk}/?order_by=id'
r = get(access_list_url, user=rando, expect=200)
# rando should be listed with direct permissions from both creator and object role assignment
user_data = {item['username']: item for item in r.data['results']}
assert rando.username in user_data
# Verify they have direct role assignments
assert len(user_data[rando.username]['object_role_assignments']) > 0
assert any(assign.get('type') == 'direct' for assign in user_data[rando.username]['object_role_assignments'])

View File

@@ -173,6 +173,22 @@ def test_creator_permission(rando, admin_user, inventory, setup_managed_roles):
assert rando in inventory.admin_role.members.all()
@pytest.mark.django_db
def test_creator_permission_notification_template(rando, organization, setup_managed_roles):
"""NotificationTemplate has no old-style roles, give_creator_permissions should not error"""
from awx.main.models import NotificationTemplate
nt = NotificationTemplate.objects.create(
name='test-nt',
organization=organization,
notification_type='slack',
notification_configuration={'token': 'x', 'channels': ['#test']},
)
give_creator_permissions(rando, nt)
assignment = RoleUserAssignment.objects.filter(user=rando, object_id=nt.pk).first()
assert assignment is not None
@pytest.mark.django_db
def test_implicit_parents_no_assignments(organization):
"""Through the normal course of creating models, we should not be changing DAB RBAC permissions"""

View File

@@ -8,7 +8,7 @@ from awx.main.management.commands.dispatcherd import _hash_config
def test_dispatcherd_config_hash_is_stable(settings, monkeypatch):
monkeypatch.setenv('AWX_COMPONENT', 'dispatcher')
settings.CLUSTER_HOST_ID = 'test-node'
settings.JOB_EVENT_WORKERS = 1
settings.DISPATCHER_MIN_WORKERS = 1
settings.DISPATCHER_SCHEDULE = {}
config_one = get_dispatcherd_config(for_service=True)

View File

@@ -1,7 +1,8 @@
import pytest
# AWX context managers for testing
from awx.main.signals import disable_activity_stream, disable_computed_fields, update_inventory_computed_fields
from awx.main.signals import disable_activity_stream, disable_computed_fields
from awx.main.tasks.system import update_inventory_computed_fields
# AWX models
from awx.main.models.organization import Organization

View File

@@ -1,7 +1,13 @@
import urllib.parse
import pytest
from awx.api.versioning import reverse
from awx.main.models import (
Group,
Host,
Inventory,
Organization,
Schedule,
)
from awx.main.access import (
@@ -128,3 +134,94 @@ class TestSmartInventory:
assert InventoryAccess(org_admin).can_admin(smart_inventory, {'host_filter': 'search=foo'})
smart_inventory.admin_role.members.add(rando)
assert not InventoryAccess(rando).can_admin(smart_inventory, {'host_filter': 'search=foo'})
def test_host_filter_edit_unprivileged(self, smart_inventory, user):
unprivileged = user('unprivileged', False)
assert not InventoryAccess(unprivileged).can_change(smart_inventory, None)
assert not InventoryAccess(unprivileged).can_admin(smart_inventory, {'host_filter': 'search=bar'})
def test_host_filter_edit_inventory_admin_role(self, smart_inventory, user):
inv_admin = user('inv_admin', False)
smart_inventory.admin_role.members.add(inv_admin)
assert InventoryAccess(inv_admin).can_change(smart_inventory, None)
assert not InventoryAccess(inv_admin).can_admin(smart_inventory, {'host_filter': 'search=bar'})
def test_host_filter_edit_org_admin_via_api(self, smart_inventory, patch, user):
oa = user('smart_oa', False)
smart_inventory.organization.admin_role.members.add(oa)
url = reverse('api:inventory_detail', kwargs={'pk': smart_inventory.pk})
resp = patch(url, {'host_filter': 'search=bar'}, oa, expect=200)
assert resp.data['host_filter'] == 'search=bar'
@pytest.mark.parametrize("role_field", ['admin_role', 'use_role', 'adhoc_role', 'read_role'])
def test_inventory_role_cannot_edit_host_filter(self, smart_inventory, patch, user, role_field):
u = user('role_test_user', False)
getattr(smart_inventory, role_field).members.add(u)
url = reverse('api:inventory_detail', kwargs={'pk': smart_inventory.pk})
patch(url, {'host_filter': 'search=bar'}, u, expect=403)
@pytest.mark.django_db
class TestHostFilterRBAC:
@pytest.fixture
def two_org_inventories(self):
orgA = Organization.objects.create(name="rbac-orgA")
orgB = Organization.objects.create(name="rbac-orgB")
invA = Inventory.objects.create(name="rbac-invA", organization=orgA)
invB = Inventory.objects.create(name="rbac-invB", organization=orgB)
hostA = Host.objects.create(name="shared_name", inventory=invA)
hostB = Host.objects.create(name="shared_name", inventory=invB)
groupA = Group.objects.create(name="shared_group", inventory=invA)
groupB = Group.objects.create(name="shared_group", inventory=invB)
groupA.hosts.add(hostA)
groupB.hosts.add(hostB)
return {
'orgA': orgA,
'orgB': orgB,
'invA': invA,
'invB': invB,
'hostA': hostA,
'hostB': hostB,
}
@pytest.mark.parametrize("host_filter", ["name=shared_name", "groups__name=shared_group"])
def test_host_filter_scoped_to_inventory_read_role(self, two_org_inventories, get, user, host_filter):
data = two_org_inventories
userA = user('rbac_userA', False)
userB = user('rbac_userB', False)
data['invA'].read_role.members.add(userA)
data['invB'].read_role.members.add(userB)
url = reverse('api:host_list')
params = "?host_filter=%s" % urllib.parse.quote(host_filter, safe='')
respA = get(url + params, userA)
idsA = [h['id'] for h in respA.data['results']]
assert data['hostA'].id in idsA
assert data['hostB'].id not in idsA
respB = get(url + params, userB)
idsB = [h['id'] for h in respB.data['results']]
assert data['hostB'].id in idsB
assert data['hostA'].id not in idsB
@pytest.mark.parametrize("host_filter", ["name=shared_name", "groups__name=shared_group"])
def test_host_filter_scoped_to_org_admin(self, two_org_inventories, get, user, host_filter):
data = two_org_inventories
adminA = user('rbac_adminA', False)
adminB = user('rbac_adminB', False)
data['orgA'].admin_role.members.add(adminA)
data['orgB'].admin_role.members.add(adminB)
url = reverse('api:host_list')
params = "?host_filter=%s" % urllib.parse.quote(host_filter, safe='')
respA = get(url + params, adminA)
idsA = [h['id'] for h in respA.data['results']]
assert data['hostA'].id in idsA
assert data['hostB'].id not in idsA
respB = get(url + params, adminB)
idsB = [h['id'] for h in respB.data['results']]
assert data['hostB'].id in idsB
assert data['hostA'].id not in idsB

View File

@@ -1,6 +1,9 @@
import pytest
from django.apps import apps
from django.core.management.base import CommandError
from awx.main.tasks.system import _sync_credential_types_to_db
@pytest.fixture
@@ -9,18 +12,38 @@ def mock_setup_tower_managed_defaults(mocker):
@pytest.mark.django_db
def test_load_credential_types_feature_migrations_ran(mocker, mock_setup_tower_managed_defaults):
mocker.patch('awx.main.apps.is_database_synchronized', return_value=True)
def test_sync_credential_types_migrations_ran(mocker, mock_setup_tower_managed_defaults):
mocker.patch('awx.main.tasks.system.is_database_synchronized', return_value=True)
apps.get_app_config('main')._load_credential_types_feature()
_sync_credential_types_to_db()
mock_setup_tower_managed_defaults.assert_called_once()
@pytest.mark.django_db
def test_load_credential_types_feature_migrations_not_ran(mocker, mock_setup_tower_managed_defaults):
mocker.patch('awx.main.apps.is_database_synchronized', return_value=False)
def test_sync_credential_types_migrations_not_ran(mocker, mock_setup_tower_managed_defaults):
mocker.patch('awx.main.tasks.system.is_database_synchronized', return_value=False)
apps.get_app_config('main')._load_credential_types_feature()
_sync_credential_types_to_db()
mock_setup_tower_managed_defaults.assert_not_called()
def test_check_db_requirement_no_violations(mocker):
mocker.patch('awx.main.apps.db_requirement_violations', return_value=None)
main_config = apps.get_app_config('main')
result = main_config.check_db_requirement()
assert result is None
def test_check_db_requirement_with_violations(mocker):
violation_msg = "Database version check failed"
mocker.patch('awx.main.apps.db_requirement_violations', return_value=violation_msg)
main_config = apps.get_app_config('main')
with pytest.raises(CommandError) as exc_info:
main_config.check_db_requirement()
assert str(exc_info.value) == violation_msg

View File

@@ -160,3 +160,38 @@ class TestJobReaper(object):
assert job.started > ref_time
assert job.status == 'running'
assert job.job_explanation == ''
def test_waiting_job_reset_when_controller_node_deprovisioned(self):
"""When a controller pod is replaced (e.g. K8s rollout), waiting jobs
assigned to the now-gone controller_node should be reset to pending
by the task manager so they can be re-dispatched."""
from awx.main.scheduler import TaskManager
live_inst = Instance(hostname='awx-task-live', node_type='control')
live_inst.save()
# No instance record for 'awx-task-dead' — it was already deprovisioned
job = Job.objects.create(status='waiting', controller_node='awx-task-dead', execution_node='')
tm = TaskManager()
tm.reap_jobs_from_orphaned_instances()
job.refresh_from_db()
assert job.status == 'pending'
assert job.controller_node == ''
assert job.execution_node == ''
@pytest.mark.parametrize('node_type', ['control', 'hybrid'])
def test_waiting_job_not_reset_when_controller_node_alive(self, node_type):
"""Waiting jobs on a live control or hybrid node should not be touched."""
from awx.main.scheduler import TaskManager
live_inst = Instance(hostname='awx-task-live', node_type=node_type)
live_inst.save()
job = Job.objects.create(status='waiting', controller_node='awx-task-live', execution_node='')
tm = TaskManager()
tm.reap_jobs_from_orphaned_instances()
job.refresh_from_db()
assert job.status == 'waiting'
assert job.controller_node == 'awx-task-live'

View File

@@ -287,6 +287,20 @@ def test_control_plane_policy_exception(controlplane_instance_group):
assert 'foo-1' not in [inst.hostname for inst in controlplane_instance_group.instances.all()]
@pytest.mark.django_db
def test_policy_instance_list_controlplane_excludes_execution_node(controlplane_instance_group):
controlplane_instance_group.policy_instance_percentage = 100
controlplane_instance_group.save()
exec_inst = Instance.objects.create(hostname='exec-1', node_type='execution')
control_inst = Instance.objects.create(hostname='control-1', node_type='control')
controlplane_instance_group.policy_instance_list = [exec_inst.hostname]
controlplane_instance_group.save()
apply_cluster_membership_policies()
members = list(controlplane_instance_group.instances.all())
assert exec_inst not in members
assert control_inst in members
@pytest.mark.django_db
def test_normal_instance_group_policy_exception():
ig = InstanceGroup.objects.create(name='bar', policy_instance_percentage=100, policy_instance_minimum=2)

View File

@@ -0,0 +1,320 @@
"""Smart inventory tests that require PostgreSQL.
These tests exercise SmartFilter and smart inventory host resolution against
a real PostgreSQL database. Most are unit-style tests that set ansible_facts
directly on Host objects rather than running playbooks.
The smart inventory HostManager uses DISTINCT ON which requires PostgreSQL,
so any test that reads smart inventory hosts must run here (not in functional/).
"""
import pytest
from awx.main.models import Organization, Inventory, Host, Group
from awx.main.utils.filters import SmartFilter
@pytest.fixture
def fact_org():
org, _ = Organization.objects.get_or_create(name='smart-inv-fact-test-org')
return org
@pytest.fixture
def fact_inventory(fact_org):
inv, created = Inventory.objects.get_or_create(name='smart-inv-fact-test-inv', organization=fact_org)
if not created:
inv.hosts.all().delete()
inv.groups.all().delete()
groupA = Group.objects.create(name='factGroupA', inventory=inv)
groupB = Group.objects.create(name='factGroupB', inventory=inv)
hostA = Host.objects.create(
name='factHostA',
inventory=inv,
ansible_facts={
'ansible_system': 'Linux',
'ansible_distribution': 'CentOS',
'ansible_python': {
'version': {'major': 3, 'minor': 9, 'micro': 7},
'version_info': [3, 9, 7, 'final', 0],
},
'ansible_env': {'HOME': '/root'},
},
)
hostB = Host.objects.create(
name='factHostB',
inventory=inv,
ansible_facts={
'ansible_system': 'Linux',
'ansible_distribution': 'Ubuntu',
'ansible_python': {
'version': {'major': 3, 'minor': 11, 'micro': 2},
'version_info': [3, 11, 2, 'final', 0],
},
'ansible_env': {'HOME': '/home/user'},
},
)
hostC = Host.objects.create(
name='factHostC',
inventory=inv,
ansible_facts={
'ansible_system': 'Darwin',
'ansible_distribution': 'MacOSX',
'ansible_python': {
'version': {'major': 3, 'minor': 10, 'micro': 0},
'version_info': [3, 10, 0, 'final', 0],
},
'ansible_env': {'HOME': '/Users/test'},
},
)
groupA.hosts.add(hostA, hostC)
groupB.hosts.add(hostB, hostC)
yield {
'org': fact_org,
'inv': inv,
'hosts': {'hostA': hostA, 'hostB': hostB, 'hostC': hostC},
'groups': {'groupA': groupA, 'groupB': groupB},
}
hostA.delete()
hostB.delete()
hostC.delete()
groupA.delete()
groupB.delete()
@pytest.fixture
def smart_inventory_factory():
created = []
def _factory(name, host_filter, organization):
inv = Inventory.objects.create(name=name, kind='smart', host_filter=host_filter, organization=organization)
created.append(inv)
return inv
yield _factory
for inv in reversed(created):
inv.delete()
@pytest.fixture
def host_factory():
created = []
def _factory(**kwargs):
host = Host.objects.create(**kwargs)
created.append(host)
return host
yield _factory
for host in reversed(created):
if host.pk is not None:
host.delete()
@pytest.fixture
def group_factory():
created = []
def _factory(**kwargs):
group = Group.objects.create(**kwargs)
created.append(group)
return group
yield _factory
for group in reversed(created):
group.delete()
def query_names(filter_string):
return sorted(SmartFilter.query_from_string(filter_string).distinct().values_list('name', flat=True))
# --- Fact-based filter tests (require PostgreSQL for JSONField __contains) ---
def test_fact_based_host_filter(fact_inventory):
assert query_names('ansible_facts__ansible_system=Linux') == ['factHostA', 'factHostB']
assert query_names('ansible_facts__ansible_distribution=CentOS') == ['factHostA']
assert query_names('ansible_facts__ansible_distribution=Ubuntu') == ['factHostB']
assert query_names('ansible_facts__ansible_system=Darwin') == ['factHostC']
assert query_names('ansible_facts__ansible_system=Windows') == []
def test_nested_fact_search(fact_inventory):
assert query_names('ansible_facts__ansible_python__version__major=3') == ['factHostA', 'factHostB', 'factHostC']
assert query_names('ansible_facts__ansible_python__version__minor=9') == ['factHostA']
assert query_names('ansible_facts__ansible_python__version__minor=11') == ['factHostB']
assert query_names('ansible_facts__ansible_env__HOME=/root') == ['factHostA']
def test_list_fact_search(fact_inventory):
assert query_names('ansible_facts__ansible_python__version_info[]=9') == ['factHostA']
assert query_names('ansible_facts__ansible_python__version_info[]=11') == ['factHostB']
assert query_names('ansible_facts__ansible_python__version_info[]=3') == ['factHostA', 'factHostB', 'factHostC']
def test_fact_search_with_or(fact_inventory):
assert query_names('ansible_facts__ansible_system=Linux or ansible_facts__ansible_system=Linux') == ['factHostA', 'factHostB']
assert query_names('ansible_facts__ansible_system=Linux or ansible_facts__ansible_system=not_found') == ['factHostA', 'factHostB']
assert query_names('ansible_facts__ansible_system=not_found or ansible_facts__ansible_system=not_found') == []
assert query_names('ansible_facts__ansible_system=Linux or ansible_facts__ansible_system=Darwin') == ['factHostA', 'factHostB', 'factHostC']
def test_fact_search_with_and(fact_inventory):
assert query_names('ansible_facts__ansible_system=Linux and ansible_facts__ansible_system=Linux') == ['factHostA', 'factHostB']
assert query_names('ansible_facts__ansible_system=Linux and ansible_facts__ansible_system=not_found') == []
assert query_names('ansible_facts__ansible_system=Linux and ansible_facts__ansible_distribution=CentOS') == ['factHostA']
def test_hybrid_fact_name_group_search(fact_inventory):
assert query_names('name=factHostA or groups__name=factGroupB or ansible_facts__ansible_system=Linux') == ['factHostA', 'factHostB', 'factHostC']
assert query_names('name=factHostA or groups__name=factGroupA or ansible_facts__ansible_system=not_found') == ['factHostA', 'factHostC']
assert query_names('name=factHostA and groups__name=factGroupA and ansible_facts__ansible_system=not_found') == []
assert query_names('name=factHostA and groups__name=factGroupA and ansible_facts__ansible_system=Linux') == ['factHostA']
def test_advanced_hybrid_with_parentheses(fact_inventory):
assert query_names('name=factHostA or (groups__name=factGroupB and ansible_facts__ansible_system=not_found)') == ['factHostA']
assert query_names('name=not_found or (groups__name=factGroupB and ansible_facts__ansible_system=Linux)') == ['factHostB']
assert query_names('(name=factHostA or groups__name=factGroupB) and ansible_facts__ansible_system=not_found') == []
assert query_names('(name=factHostA or groups__name=factGroupB) and ansible_facts__ansible_system=Linux') == ['factHostA', 'factHostB']
assert query_names('(name=factHostC or groups__name=factGroupA) and ansible_facts__ansible_system=Darwin') == ['factHostC']
# --- Smart inventory host resolution tests (require PostgreSQL for DISTINCT ON) ---
def test_smart_inventory_hosts_by_name(fact_inventory, smart_inventory_factory):
org = fact_inventory['org']
smart_inv = smart_inventory_factory('smart-by-name', 'name=factHostA', org)
hosts = sorted(smart_inv.hosts.values_list('name', flat=True))
assert hosts == ['factHostA']
def test_smart_inventory_hosts_by_group(fact_inventory, smart_inventory_factory):
org = fact_inventory['org']
smart_inv = smart_inventory_factory('smart-by-group', 'groups__name=factGroupA', org)
hosts = sorted(smart_inv.hosts.values_list('name', flat=True))
assert hosts == ['factHostA', 'factHostC']
def test_smart_inventory_with_facts(fact_inventory, smart_inventory_factory):
org = fact_inventory['org']
smart_inv = smart_inventory_factory('fact-smart-inv', 'ansible_facts__ansible_system=Linux', org)
hosts = sorted(smart_inv.hosts.values_list('name', flat=True))
assert hosts == ['factHostA', 'factHostB']
assert smart_inv.total_hosts == 2
def test_smart_inventory_with_nested_facts(fact_inventory, smart_inventory_factory):
org = fact_inventory['org']
smart_inv = smart_inventory_factory(
'nested-fact-smart-inv',
'ansible_facts__ansible_distribution=CentOS and ansible_facts__ansible_python__version__minor=9',
org,
)
hosts = list(smart_inv.hosts.values_list('name', flat=True))
assert hosts == ['factHostA']
def test_host_filter_is_organization_scoped(fact_inventory, smart_inventory_factory, host_factory):
"""Smart inventory only includes hosts from its own organization."""
org1 = fact_inventory['org']
org2, _ = Organization.objects.get_or_create(name='smart-inv-other-org')
inv2, _ = Inventory.objects.get_or_create(name='other-org-inv', organization=org2)
Host.objects.filter(name='factHostA', inventory=inv2).delete()
host_factory(name='factHostA', inventory=inv2)
smart_inv = smart_inventory_factory('scoped-smart', 'name=factHostA', org1)
hosts = list(smart_inv.hosts.all())
assert len(hosts) == 1
assert hosts[0].inventory_id == fact_inventory['inv'].id
def test_duplicate_hosts_deduplicated(smart_inventory_factory, host_factory):
"""Same-name hosts across inventories in the same org yield only one smart inventory entry."""
org, _ = Organization.objects.get_or_create(name='smart-inv-dedup-org')
inv1, _ = Inventory.objects.get_or_create(name='dedup-inv1', organization=org)
inv2, _ = Inventory.objects.get_or_create(name='dedup-inv2', organization=org)
Host.objects.filter(name='dedup_host', inventory__in=[inv1, inv2]).delete()
host1 = host_factory(name='dedup_host', inventory=inv1)
host2 = host_factory(name='dedup_host', inventory=inv2)
smart_inv = smart_inventory_factory('dedup-smart', 'name=dedup_host', org)
hosts = list(smart_inv.hosts.all())
assert len(hosts) == 1
assert hosts[0].id == min(host1.id, host2.id)
def test_host_sources_original_inventory(fact_inventory, smart_inventory_factory):
"""Hosts in a smart inventory still reference their source inventory."""
org = fact_inventory['org']
source_inv = fact_inventory['inv']
smart_inv = smart_inventory_factory('sources-original', 'name=factHostA', org)
host = smart_inv.hosts.first()
assert host.inventory_id == source_inv.id
def test_host_updates_reflected_in_smart_inventory(fact_inventory, smart_inventory_factory, host_factory):
"""Editing or deleting a host is immediately reflected in a smart inventory."""
org = fact_inventory['org']
inv = fact_inventory['inv']
host = host_factory(name='mutable_host', inventory=inv)
smart_inv = smart_inventory_factory('updates-reflected', 'name=mutable_host', org)
assert smart_inv.hosts.count() == 1
host.description = 'updated'
host.save()
assert smart_inv.hosts.first().description == 'updated'
host.delete()
assert smart_inv.hosts.count() == 0
def test_smart_inventory_duplicate_hosts_matching_group_names(fact_inventory, smart_inventory_factory, host_factory, group_factory):
"""A host in multiple groups whose names match an icontains filter appears only once."""
org = fact_inventory['org']
inv = fact_inventory['inv']
g1 = group_factory(name='dedup_another_group', inventory=inv)
g2 = group_factory(name='dedup_yet_another_group', inventory=inv)
host = host_factory(name='dedup_grouped_host', inventory=inv)
g1.hosts.add(host)
g2.hosts.add(host)
smart_inv = smart_inventory_factory('group-dedup-smart', 'groups__name__icontains=dedup_another', org)
assert smart_inv.hosts.count() == 1
def test_smart_inventory_computed_fields(fact_inventory, smart_inventory_factory):
"""Smart inventory total_hosts and related computed fields are accurate."""
org = fact_inventory['org']
smart_inv = smart_inventory_factory('computed-fields', 'name=factHostA or name=factHostB', org)
assert smart_inv.total_hosts == 2
assert smart_inv.total_groups == 0
assert smart_inv.total_inventory_sources == 0
assert smart_inv.has_inventory_sources is False
def test_smart_inventory_matches_host_filter(fact_inventory, smart_inventory_factory):
"""Smart inventory hosts should match the equivalent SmartFilter query."""
org = fact_inventory['org']
host_filter = 'groups__name=factGroupA or groups__name=factGroupB'
smart_inv = smart_inventory_factory('match-filter', host_filter, org)
smart_names = sorted(smart_inv.hosts.values_list('name', flat=True))
filter_names = sorted(SmartFilter.query_from_string(host_filter).distinct().values_list('name', flat=True))
assert smart_names == filter_names

View File

@@ -39,7 +39,7 @@ def test_unified_job_detail_exclusive_fields():
For each type, assert that the only fields allowed to be exclusive to
detail view are the allowed types
"""
allowed_detail_fields = frozenset(('result_traceback', 'job_args', 'job_cwd', 'job_env', 'event_processing_finished', 'artifacts', 'extra_vars'))
allowed_detail_fields = frozenset(('result_traceback', 'job_args', 'job_cwd', 'job_env', 'event_processing_finished', 'artifacts'))
for cls in UnifiedJob.__subclasses__():
list_serializer = getattr(serializers, '{}ListSerializer'.format(cls.__name__))
detail_serializer = getattr(serializers, '{}Serializer'.format(cls.__name__))

View File

@@ -0,0 +1,35 @@
import pytest
from django.core.management.base import CommandError
from awx.main.management.commands.check_db import Command
def test_check_db_command_success(mocker):
mock_cursor = mocker.MagicMock()
mock_cursor.fetchone.return_value = ['PostgreSQL 12.8 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 9.3.0, 64-bit']
mock_connection = mocker.MagicMock()
mock_connection.cursor.return_value.__enter__.return_value = mock_cursor
mocker.patch('awx.main.management.commands.check_db.connection', mock_connection)
mocker.patch('awx.main.management.commands.check_db.db_requirement_violations', return_value=None)
command = Command()
result = command.handle()
assert 'Database Version:' in result
mock_cursor.execute.assert_called_once_with('SELECT version()')
def test_check_db_command_version_violations(mocker):
mock_cursor = mocker.MagicMock()
mock_cursor.fetchone.return_value = ['PostgreSQL 11.0 on x86_64-pc-linux-gnu']
mock_connection = mocker.MagicMock()
mock_connection.cursor.return_value.__enter__.return_value = mock_cursor
mocker.patch('awx.main.management.commands.check_db.connection', mock_connection)
violation_msg = "At a minimum, postgres version 12 is required, found 11\n"
mocker.patch('awx.main.management.commands.check_db.db_requirement_violations', return_value=violation_msg)
command = Command()
with pytest.raises(CommandError) as exc_info:
command.handle()
assert str(exc_info.value) == violation_msg

View File

@@ -10,8 +10,8 @@ def test_send_messages():
with mock.patch('awx.main.notifications.grafana_backend.requests') as requests_mock:
requests_mock.post.return_value.status_code = 200
m = {}
m['started'] = dt.datetime.utcfromtimestamp(60).isoformat()
m['finished'] = dt.datetime.utcfromtimestamp(120).isoformat()
m['started'] = dt.datetime.fromtimestamp(60, tz=dt.timezone.utc).isoformat()
m['finished'] = dt.datetime.fromtimestamp(120, tz=dt.timezone.utc).isoformat()
m['subject'] = "test subject"
backend = grafana_backend.GrafanaBackend("testapikey", dashboardId='', panelId='')
message = EmailMessage(
@@ -40,8 +40,8 @@ def test_send_messages_with_no_verify_ssl():
with mock.patch('awx.main.notifications.grafana_backend.requests') as requests_mock:
requests_mock.post.return_value.status_code = 200
m = {}
m['started'] = dt.datetime.utcfromtimestamp(60).isoformat()
m['finished'] = dt.datetime.utcfromtimestamp(120).isoformat()
m['started'] = dt.datetime.fromtimestamp(60, tz=dt.timezone.utc).isoformat()
m['finished'] = dt.datetime.fromtimestamp(120, tz=dt.timezone.utc).isoformat()
m['subject'] = "test subject"
backend = grafana_backend.GrafanaBackend("testapikey", dashboardId='', panelId='', grafana_no_verify_ssl=True)
message = EmailMessage(
@@ -71,8 +71,8 @@ def test_send_messages_with_dashboardid(dashboardId):
with mock.patch('awx.main.notifications.grafana_backend.requests') as requests_mock:
requests_mock.post.return_value.status_code = 200
m = {}
m['started'] = dt.datetime.utcfromtimestamp(60).isoformat()
m['finished'] = dt.datetime.utcfromtimestamp(120).isoformat()
m['started'] = dt.datetime.fromtimestamp(60, tz=dt.timezone.utc).isoformat()
m['finished'] = dt.datetime.fromtimestamp(120, tz=dt.timezone.utc).isoformat()
m['subject'] = "test subject"
backend = grafana_backend.GrafanaBackend("testapikey", dashboardId=dashboardId, panelId='')
message = EmailMessage(
@@ -102,8 +102,8 @@ def test_send_messages_with_panelid(panelId):
with mock.patch('awx.main.notifications.grafana_backend.requests') as requests_mock:
requests_mock.post.return_value.status_code = 200
m = {}
m['started'] = dt.datetime.utcfromtimestamp(60).isoformat()
m['finished'] = dt.datetime.utcfromtimestamp(120).isoformat()
m['started'] = dt.datetime.fromtimestamp(60, tz=dt.timezone.utc).isoformat()
m['finished'] = dt.datetime.fromtimestamp(120, tz=dt.timezone.utc).isoformat()
m['subject'] = "test subject"
backend = grafana_backend.GrafanaBackend("testapikey", dashboardId='', panelId=panelId)
message = EmailMessage(
@@ -132,8 +132,8 @@ def test_send_messages_with_bothids():
with mock.patch('awx.main.notifications.grafana_backend.requests') as requests_mock:
requests_mock.post.return_value.status_code = 200
m = {}
m['started'] = dt.datetime.utcfromtimestamp(60).isoformat()
m['finished'] = dt.datetime.utcfromtimestamp(120).isoformat()
m['started'] = dt.datetime.fromtimestamp(60, tz=dt.timezone.utc).isoformat()
m['finished'] = dt.datetime.fromtimestamp(120, tz=dt.timezone.utc).isoformat()
m['subject'] = "test subject"
backend = grafana_backend.GrafanaBackend("testapikey", dashboardId='42', panelId='42')
message = EmailMessage(
@@ -162,8 +162,8 @@ def test_send_messages_with_emptyids():
with mock.patch('awx.main.notifications.grafana_backend.requests') as requests_mock:
requests_mock.post.return_value.status_code = 200
m = {}
m['started'] = dt.datetime.utcfromtimestamp(60).isoformat()
m['finished'] = dt.datetime.utcfromtimestamp(120).isoformat()
m['started'] = dt.datetime.fromtimestamp(60, tz=dt.timezone.utc).isoformat()
m['finished'] = dt.datetime.fromtimestamp(120, tz=dt.timezone.utc).isoformat()
m['subject'] = "test subject"
backend = grafana_backend.GrafanaBackend("testapikey", dashboardId='', panelId='')
message = EmailMessage(
@@ -192,8 +192,8 @@ def test_send_messages_with_tags():
with mock.patch('awx.main.notifications.grafana_backend.requests') as requests_mock:
requests_mock.post.return_value.status_code = 200
m = {}
m['started'] = dt.datetime.utcfromtimestamp(60).isoformat()
m['finished'] = dt.datetime.utcfromtimestamp(120).isoformat()
m['started'] = dt.datetime.fromtimestamp(60, tz=dt.timezone.utc).isoformat()
m['finished'] = dt.datetime.fromtimestamp(120, tz=dt.timezone.utc).isoformat()
m['subject'] = "test subject"
backend = grafana_backend.GrafanaBackend("testapikey", dashboardId='', panelId='', annotation_tags=["ansible"])
message = EmailMessage(

View File

@@ -8,6 +8,7 @@ import pytest
import awx
from awx.main.db.profiled_pg.base import RecordedQueryLog
from awx.main.utils.db import db_requirement_violations
QUERY = {'sql': 'SELECT * FROM main_job', 'time': '.01'}
EXPLAIN = 'Seq Scan on public.main_job (cost=0.00..1.18 rows=18 width=86)'
@@ -145,3 +146,71 @@ def test_sql_above_threshold(tmpdir):
assert q['sql'] == QUERY['sql']
assert EXPLAIN in q['explain']
assert 'test_sql_above_threshold' in q['bt']
def test_db_requirement_violations_skip_env_var(mocker):
mocker.patch.dict(os.environ, {'SKIP_PG_VERSION_CHECK': 'true'})
result = db_requirement_violations()
assert result is None
def test_db_requirement_violations_postgresql_sufficient_version(mocker):
mock_connection = mocker.MagicMock()
mock_connection.vendor = 'postgresql'
mock_connection.pg_version = 120000 # Version 12.0
mocker.patch('awx.main.utils.db.connection', mock_connection)
mocker.patch.dict(os.environ, {}, clear=True)
result = db_requirement_violations()
assert result is None
def test_db_requirement_violations_postgresql_insufficient_version(mocker):
mock_connection = mocker.MagicMock()
mock_connection.vendor = 'postgresql'
mock_connection.pg_version = 110000 # Version 11.0
mocker.patch('awx.main.utils.db.connection', mock_connection)
mocker.patch.dict(os.environ, {}, clear=True)
result = db_requirement_violations()
assert result is not None
assert "At a minimum, postgres version 12 is required, found 11" in result
def test_db_requirement_violations_non_postgresql_production(mocker):
mock_connection = mocker.MagicMock()
mock_connection.vendor = 'sqlite'
mocker.patch('awx.main.utils.db.connection', mock_connection)
mocker.patch('awx.main.utils.db.MODE', 'production')
mocker.patch.dict(os.environ, {}, clear=True)
result = db_requirement_violations()
assert result is not None
assert "Running server with 'sqlite' type database is not supported" in result
def test_db_requirement_violations_non_postgresql_development(mocker):
mock_connection = mocker.MagicMock()
mock_connection.vendor = 'sqlite'
mocker.patch('awx.main.utils.db.connection', mock_connection)
mocker.patch('awx.main.utils.db.MODE', 'development')
mocker.patch.dict(os.environ, {}, clear=True)
result = db_requirement_violations()
assert result is None
def test_db_requirement_violations_postgresql_edge_case_version(mocker):
mock_connection = mocker.MagicMock()
mock_connection.vendor = 'postgresql'
mock_connection.pg_version = 129999 # Version 12.9999
mocker.patch('awx.main.utils.db.connection', mock_connection)
mocker.patch.dict(os.environ, {}, clear=True)
result = db_requirement_violations()
assert result is None

View File

@@ -151,14 +151,6 @@ def is_testing(argv=None):
return False
def bypass_in_test(func):
def fn(*args, **kwargs):
if not is_testing():
return func(*args, **kwargs)
return fn
class RequireDebugTrueOrTest(logging.Filter):
"""
Logging filter to output when in DEBUG mode or running tests.

View File

@@ -1,9 +1,14 @@
# Copyright (c) 2017 Ansible by Red Hat
# All Rights Reserved.
from typing import Optional
import os
from awx.settings.application_name import set_application_name
from awx import MODE
from django.conf import settings
from django.db import connection
def set_connection_name(function):
@@ -32,3 +37,25 @@ def bulk_update_sorted_by_id(model, objects, fields, batch_size=1000):
sorted_objects = sorted(objects, key=lambda obj: obj.id)
return model.objects.bulk_update(sorted_objects, fields, batch_size=batch_size)
MIN_PG_VERSION = 12
def db_requirement_violations() -> Optional[str]:
if os.getenv('SKIP_PG_VERSION_CHECK', False):
return None
if connection.vendor == 'postgresql':
# enforce the postgres version is a minimum of 12 (we need this for partitioning); if not, then terminate program with exit code of 1
# In the future if we require a feature of a version of postgres > 12 this should be updated to reflect that.
# The return of connection.pg_version is something like 12013
major_version = connection.pg_version // 10000
if major_version < MIN_PG_VERSION:
return f"At a minimum, postgres version {MIN_PG_VERSION} is required, found {major_version}\n"
return None
else:
if MODE == 'production':
return f"Running server with '{connection.vendor}' type database is not supported\n"
return None

View File

@@ -2,14 +2,7 @@ import re
from functools import reduce
from django.core.exceptions import FieldDoesNotExist
from pyparsing import (
infixNotation,
opAssoc,
Optional,
Literal,
CharsNotIn,
ParseException,
)
import pyparsing as pp
import logging
from logging import Filter
@@ -247,32 +240,19 @@ class SmartFilter(object):
return (assembled_k, assembled_v)
def _extract_key_value(self, t):
t_len = len(t)
k = t[0]
v = t[1] if len(t) > 1 else u""
k = None
v = None
# Strip quotes from key
if isinstance(k, str) and k.startswith('"') and k.endswith('"'):
k = k[1:-1]
# key
# "something"=
v_offset = 2
if t_len >= 2 and t[0] == "\"" and t[2] == "\"":
k = t[1]
v_offset = 4
# something=
# For quoted values, keep the quotes (strip_quotes_* will handle them later).
# For unquoted values, convert to the appropriate Python type.
if isinstance(v, str) and v.startswith('"') and v.endswith('"'):
pass # keep as-is, e.g. '"true"', '""', '"null"'
else:
k = t[0]
# value
# ="something"
if t_len > (v_offset + 2) and t[v_offset] == "\"" and t[v_offset + 2] == "\"":
v = u'"' + str(t[v_offset + 1]) + u'"'
# v = t[v_offset + 1]
# empty ""
elif t_len > (v_offset + 1):
v = u""
# no ""
else:
v = string_to_type(t[v_offset])
v = string_to_type(v)
return (k, v)
@@ -288,7 +268,7 @@ class SmartFilter(object):
try:
model = get_model(relation)
except LookupError:
raise ParseException('No related field named %s' % relation)
raise pp.ParseException('No related field named %s' % relation)
search_kwargs = {}
if model is not None:
@@ -328,34 +308,31 @@ class SmartFilter(object):
def query_from_string(cls, filter_string):
"""
TODO:
* handle values with " via: a.b.c.d="hello\"world"
* handle keys with " via: a.\"b.c="yeah"
* handle key with __ in it
"""
filter_string_raw = filter_string
filter_string = str(filter_string)
unicode_spaces = list(set(str(c) for c in filter_string if c.isspace()))
unicode_spaces_other = unicode_spaces + [u'(', u')', u'=', u'"']
atom = CharsNotIn(unicode_spaces_other)
atom_inside_quotes = CharsNotIn(u'"')
atom_quoted = Literal('"') + Optional(atom_inside_quotes) + Literal('"')
EQUAL = Literal('=')
unquoted = pp.CharsNotIn('()= \t\r\n"')
unquoted.skipWhitespace = True
quoted = pp.QuotedString('"', esc_char='\\', unquote_results=False)
token = quoted | unquoted
grammar = (atom_quoted | atom) + EQUAL + Optional((atom_quoted | atom))
grammar.setParseAction(cls.BoolOperand)
operand = token + pp.Suppress("=") + pp.Optional(token, default="")
operand.set_parse_action(cls.BoolOperand)
boolExpr = infixNotation(
grammar,
bool_expr = pp.infix_notation(
operand,
[
("and", 2, opAssoc.LEFT, cls.BoolAnd),
("or", 2, opAssoc.LEFT, cls.BoolOr),
(pp.Keyword("and"), 2, pp.OpAssoc.LEFT, cls.BoolAnd),
(pp.Keyword("or"), 2, pp.OpAssoc.LEFT, cls.BoolOr),
],
)
try:
res = boolExpr.parseString('(' + filter_string + ')')
except (ParseException, FieldDoesNotExist):
res = bool_expr.parse_string(filter_string, parse_all=True)
except (pp.ParseException, FieldDoesNotExist):
raise RuntimeError(u"Invalid query %s" % filter_string_raw)
if len(res) > 0:

View File

@@ -0,0 +1,64 @@
class LazyLoadDict(dict):
"""A dict subclass that calls a loader function on first read access.
Writes (e.g. during the loading process itself) go straight through
without triggering the loader.
"""
def __init__(self, loader):
super().__init__()
self._loader = loader
self._loaded = False
def _ensure_loaded(self):
if not self._loaded:
self._loaded = True
self._loader()
def __getitem__(self, key):
self._ensure_loaded()
return super().__getitem__(key)
def get(self, key, default=None):
self._ensure_loaded()
return super().get(key, default)
def __contains__(self, key):
self._ensure_loaded()
return super().__contains__(key)
def __iter__(self):
self._ensure_loaded()
return super().__iter__()
def __len__(self):
self._ensure_loaded()
return super().__len__()
def keys(self):
self._ensure_loaded()
return super().keys()
def values(self):
self._ensure_loaded()
return super().values()
def items(self):
self._ensure_loaded()
return super().items()
def __bool__(self):
self._ensure_loaded()
return super().__bool__()
def __repr__(self):
self._ensure_loaded()
return super().__repr__()
def copy(self):
self._ensure_loaded()
return super().copy()
def clear(self):
super().clear()
self._loaded = True

View File

@@ -215,6 +215,9 @@ LOCAL_STDOUT_EXPIRE_TIME = 2592000
# events into the database
JOB_EVENT_WORKERS = 4
# Minimum number of workers for the dispatcher (dispatcherd) process pool
DISPATCHER_MIN_WORKERS = 4
# The number of seconds to buffer callback receiver bulk
# writes in memory before flushing via JobEvent.objects.bulk_create()
JOB_EVENT_BUFFER_SECONDS = 1
@@ -445,7 +448,7 @@ DISPATCHER_SCHEDULE = {
# Django Caching Configuration
DJANGO_REDIS_IGNORE_EXCEPTIONS = True
CACHES = {'default': {'BACKEND': 'awx.main.cache.AWXRedisCache', 'LOCATION': 'unix:///var/run/redis/redis.sock?db=1'}}
CACHES = {'default': {'BACKEND': 'ansible_base.lib.cache.redis_cache.DABRedisCache', 'LOCATION': 'unix:///var/run/redis/redis.sock?db=1'}}
ROLE_SINGLETON_USER_RELATIONSHIP = ''
ROLE_SINGLETON_TEAM_RELATIONSHIP = ''

View File

@@ -34,13 +34,15 @@ options:
aliases: [ tower_password , aap_password ]
aap_token:
description:
- The OAuth token to use.
- The OAuth token to use, sent as a Bearer token in the Authorization header.
- When connecting through the AAP gateway, use a token issued by the gateway.
- This value can be in one of two formats.
- A string which is the token itself. (i.e. bqV5txm97wqJqtkxlMkhQz0pKhRMMX)
- A dictionary structure as returned by the token module.
- A dictionary structure as set as a fact by the M(ansible.platform.token) module.
- If value not set, will try environment variable C(CONTROLLER_OAUTH_TOKEN) and then config files
type: raw
version_added: "3.7.0"
aliases: [ oauth_token, controller_oauthtoken, tower_oauthtoken ]
validate_certs:
description:
- Whether to allow insecure connections to AWX.

View File

@@ -42,13 +42,23 @@ options:
alternatives: 'TOWER_PASSWORD, AAP_PASSWORD'
aap_token:
description:
- The OAuth token to use.
- The OAuth token to use, sent as a Bearer token in the Authorization header.
- When connecting through the AAP gateway, use a token issued by the gateway.
env:
- name: AAP_TOKEN
- name: CONTROLLER_OAUTH_TOKEN
deprecated:
collection_name: 'awx.awx'
version: '4.0.0'
why: Collection name change
alternatives: 'AAP_TOKEN'
- name: TOWER_OAUTH_TOKEN
deprecated:
collection_name: 'awx.awx'
version: '4.0.0'
why: Collection name change
alternatives: 'AAP_TOKEN'
- name: AAP_TOKEN
aliases: [ oauth_token, controller_oauthtoken, tower_oauthtoken ]
verify_ssl:
description:
- Specify whether Ansible should verify the SSL certificate of the controller host.

View File

@@ -34,7 +34,10 @@ class ControllerAWXKitModule(ControllerModule):
def authenticate(self):
try:
self.connection.login(username=self.username, password=self.password)
if self.aap_token:
self.connection.session.headers['Authorization'] = 'Bearer {0}'.format(self.aap_token)
else:
self.connection.login(username=self.username, password=self.password)
self.authenticated = True
except Exception:
self.fail_json("Failed to authenticate")

View File

@@ -99,6 +99,7 @@ class ControllerModule(AnsibleModule):
aap_token=dict(
type='raw',
no_log=True,
aliases=['oauth_token', 'controller_oauthtoken', 'tower_oauthtoken'],
required=False,
fallback=(env_fallback, ['CONTROLLER_OAUTH_TOKEN', 'TOWER_OAUTH_TOKEN', 'AAP_TOKEN'])
),
@@ -118,10 +119,12 @@ class ControllerModule(AnsibleModule):
'request_timeout': 'request_timeout',
'max_retries': 'max_retries',
'retry_backoff_factor': 'retry_backoff_factor',
'aap_token': 'aap_token',
}
host = '127.0.0.1'
username = None
password = None
aap_token = None
verify_ssl = True
request_timeout = 10
max_retries = 5
@@ -160,6 +163,8 @@ class ControllerModule(AnsibleModule):
if direct_value is not None:
setattr(self, short_param, direct_value)
self._parse_aap_token()
# Perform some basic validation
if not self.host.startswith(("https://", "http://")): # NOSONAR
self.host = "https://{0}".format(self.host)
@@ -186,6 +191,15 @@ class ControllerModule(AnsibleModule):
except Exception as e:
self.fail_json(msg="Unable to resolve controller_host ({1}): {0}".format(self.url.hostname, e))
def _parse_aap_token(self):
# aap_token can be the token string itself, or the dict that the
# ansible.platform.token module sets as the aap_token fact
if isinstance(self.aap_token, dict):
if 'token' in self.aap_token:
self.aap_token = self.aap_token['token']
else:
self.fail_json(msg="The provided dict in aap_token did not properly contain the token entry")
def build_url(self, endpoint, query_params=None, app_key=None):
# Make sure we start with /api/vX
if not endpoint.startswith("/"):
@@ -284,7 +298,8 @@ class ControllerModule(AnsibleModule):
# If we made it here then we have values from reading the ini file, so let's pull them out into a dict
config_data = {}
for honorred_setting in self.short_params:
# 'oauth_token' is the legacy (pre-aap_token) config file key, kept for backward compatibility
for honorred_setting in list(self.short_params) + ['oauth_token']:
try:
config_data[honorred_setting] = config.get('general', honorred_setting)
except NoOptionError:
@@ -296,6 +311,12 @@ class ControllerModule(AnsibleModule):
except Exception as e:
raise_from(ConfigFileException("An unknown exception occured trying to load config file: {0}".format(e)), e)
# Backward compatibility: config files written for older collection
# releases used the oauth_token key; map it to aap_token.
# If both keys are present, the new aap_token key wins.
if 'oauth_token' in config_data and 'aap_token' not in config_data:
config_data['aap_token'] = config_data['oauth_token']
# If we made it here, we have a dict which has values in it from our config, any final settings logic can be performed here
for honorred_setting in self.short_params:
if honorred_setting in config_data:
@@ -572,12 +593,7 @@ class ControllerAPIModule(ControllerModule):
# Extract the headers, this will be used in a couple of places
headers = kwargs.get('headers', {})
# Authenticate to AWX (if not already done so)
if not self.authenticated:
# This method will set a cookie in the cookie jar for us
self.authenticate(**kwargs)
headers['Authorization'] = self._get_basic_authorization_header()
headers['Authorization'] = self._get_authorization_header(**kwargs)
if method in ['POST', 'PUT', 'PATCH']:
headers.setdefault('Content-Type', 'application/json')
@@ -761,6 +777,19 @@ class ControllerAPIModule(ControllerModule):
return prefix
def _get_authorization_header(self, **kwargs):
if self.aap_token:
# A token (e.g. issued by the AAP gateway) is validated by the server on
# every request, so no login round-trip is needed
return 'Bearer {0}'.format(self.aap_token)
# Authenticate to AWX (if not already done so)
if not self.authenticated:
# This method will set a cookie in the cookie jar for us
self.authenticate(**kwargs)
return self._get_basic_authorization_header()
def _get_basic_authorization_header(self):
basic_credentials = b64encode("{0}:{1}".format(self.username, self.password).encode()).decode()
return "Basic {0}".format(basic_credentials)

View File

@@ -0,0 +1,138 @@
from __future__ import absolute_import, division, print_function
__metaclass__ = type
import json
import pytest
from ansible.module_utils import basic
from ansible.module_utils.common.text.converters import to_bytes
from requests.models import Response
from unittest import mock
def getheader(self, header_name, default):
return default
def read(self):
return json.dumps({})
def status(self):
return 200
def make_recorder():
"""Build a mock for Request.open that records every call made through it."""
calls = []
def opener(self, method, url, **kwargs):
calls.append({'method': method, 'url': url, 'headers': kwargs.get('headers') or {}})
r = Response()
r.getheader = getheader.__get__(r)
r.read = read.__get__(r)
r.status = status.__get__(r)
return r
return opener, calls
def make_module(collection_import, module_args, **kwargs):
ControllerAPIModule = collection_import('plugins.module_utils.controller_api').ControllerAPIModule
cli_data = {'ANSIBLE_MODULE_ARGS': module_args}
# patch the cached args directly: AnsibleModule caches sys.argv parsing in
# basic._ANSIBLE_ARGS, so patching sys.argv would leak args between tests
with mock.patch.object(basic, '_ANSIBLE_ARGS', to_bytes(json.dumps(cli_data))):
# ansible-core 2.21+ also requires a serialization profile alongside the args
if hasattr(basic, '_ANSIBLE_PROFILE'):
with mock.patch.object(basic, '_ANSIBLE_PROFILE', 'legacy'):
return ControllerAPIModule(argument_spec=dict(), **kwargs)
return ControllerAPIModule(argument_spec=dict(), **kwargs)
@pytest.mark.parametrize(
'token_value',
[
'a-token-string',
{'token': 'a-token-string', 'id': 1}, # the aap_token fact set by ansible.platform.token
],
ids=['string', 'dict'],
)
def test_aap_token_sends_bearer_header(collection_import, token_value):
module = make_module(collection_import, {'aap_token': token_value})
assert module.aap_token == 'a-token-string'
opener, calls = make_recorder()
with mock.patch('ansible.module_utils.urls.Request.open', new=opener):
module.get_endpoint('ping')
assert len(calls) == 1, calls
assert calls[0]['headers']['Authorization'] == 'Bearer a-token-string'
# a token needs no login round-trip
assert module.authenticated is False
@pytest.mark.parametrize('param', ['oauth_token', 'controller_oauthtoken', 'tower_oauthtoken'])
def test_aap_token_legacy_aliases(collection_import, param):
module = make_module(collection_import, {param: 'legacy-token'})
assert module.aap_token == 'legacy-token'
def test_lookup_oauth_token_option_maps_to_aap_token(collection_import):
# Older lookup/inventory plugin releases pass options through as direct
# params keyed by the plugin option name; oauth_token must resolve to
# aap_token via the argspec alias.
module = make_module(collection_import, {'oauth_token': 'plugin-token'})
assert module.aap_token == 'plugin-token'
opener, calls = make_recorder()
with mock.patch('ansible.module_utils.urls.Request.open', new=opener):
module.get_endpoint('ping')
assert calls[0]['headers']['Authorization'] == 'Bearer plugin-token'
def test_config_file_legacy_oauth_token_key(collection_import, tmp_path):
# tower_cli.cfg-style config files from older releases used the oauth_token key
config_file = tmp_path / 'tower_cli.cfg'
config_file.write_text('[general]\nhost = https://127.0.0.1\noauth_token = ini-legacy-token\n')
module = make_module(collection_import, {'controller_config_file': str(config_file)})
assert module.aap_token == 'ini-legacy-token'
def test_config_file_aap_token_wins_over_legacy_key(collection_import, tmp_path):
config_file = tmp_path / 'tower_cli.cfg'
config_file.write_text('[general]\nhost = https://127.0.0.1\noauth_token = ini-legacy-token\naap_token = ini-new-token\n')
module = make_module(collection_import, {'controller_config_file': str(config_file)})
assert module.aap_token == 'ini-new-token'
def test_aap_token_dict_without_token_entry_fails(collection_import):
errors = []
def error_callback(**kwargs):
errors.append(kwargs)
raise SystemExit(1)
with pytest.raises(SystemExit):
make_module(collection_import, {'aap_token': {'id': 1}}, error_callback=error_callback)
assert 'did not properly contain the token entry' in errors[0]['msg']
def test_no_token_falls_back_to_basic_auth(collection_import):
module = make_module(collection_import, {'controller_username': 'admin', 'controller_password': 'secret'})
opener, calls = make_recorder()
with mock.patch('ansible.module_utils.urls.Request.open', new=opener):
module.get_endpoint('ping')
# first call is the authentication probe, second is the actual request
assert len(calls) == 2, calls
for call in calls:
assert call['headers']['Authorization'].startswith('Basic '), call
assert module.authenticated is True

View File

@@ -15,19 +15,6 @@ markers =
filterwarnings =
error
# FIXME: Upgrade python-dateutil https://github.com/dateutil/dateutil/issues/1340
once:datetime.datetime.utcfromtimestamp\(\) is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC:DeprecationWarning
# NOTE: the following are present using python 3.11
# FIXME: Delete this entry once `pyparsing` is updated.
once:module 'sre_constants' is deprecated:DeprecationWarning:_pytest.assertion.rewrite
# FIXME: Delete this entry once `polymorphic` is updated.
once:pkg_resources is deprecated as an API.
# FIXME: Delete this entry once `zope` is updated.
once:Deprecated call to `pkg_resources.declare_namespace.'zope'.`.\nImplementing implicit namespace packages .as specified in PEP 420. is preferred to `pkg_resources.declare_namespace`. See https.//setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages:DeprecationWarning:
# FIXME: Delete this entry once the deprecation is acted upon.
# Note: RemovedInDjango51Warning may not exist in newer Django versions
ignore:'index_together' is deprecated. Use 'Meta.indexes' in 'main.\w+' instead.

View File

@@ -1,11 +1,15 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"packageRules": [
{
"description": "Update aap-ci tekton-catalog pipeline bundles",
"matchPackageNames": ["/^quay\\.io\\/aap-ci\\/tekton-catalog\\/pipeline\\//"],
"matchManagers": ["tekton"],
"automerge": true
}
]
}
"enabledManagers": ["tekton"],
"tekton": {
"schedule": ["0 * * * *"],
"packageRules": [
{
"description": "Update aap-ci tekton-catalog pipeline bundles",
"matchPackageNames": ["/^quay\\.io\\/aap-ci\\/tekton-catalog\\/pipeline\\/"],
"matchManagers": ["tekton"],
"automerge": true
}
]
}
}

View File

@@ -36,7 +36,7 @@ maturin # pydantic-core build dep
msgpack
msrestazure
OPA-python-client==2.0.2 # upgrading requires urllib3 2.5.0+ which is blocked by other deps
kubernetes>=35.0.0
kubernetes>=36.0.0 # fixes NO_PROXY silently being reset to None
openshift
opentelemetry-api~=1.37 # new y streams can be drastically different, in a good way
opentelemetry-sdk~=1.37
@@ -50,7 +50,7 @@ pyasn1>=0.6.2 # CVE-2026-2349
pygerduty
PyGithub
pyopenssl
pyparsing==2.4.7 # Upgrading to v3 of pyparsing introduce errors on smart host filtering: Expected 'or' term, found 'or' (at char 15), (line:1, col:16)
pyparsing>3.0 # Upgraded to v3 and changed import patterns
python-daemon
python-dsv-sdk>=1.0.4
python-tss-sdk>=1.2.1

View File

@@ -10,6 +10,7 @@ aiohttp[speedups]==3.13.0
# via
# -r /awx_devel/requirements/requirements.in
# aiohttp-retry
# kubernetes
# opa-python-client
# twilio
aiohttp-retry==2.9.1
@@ -251,7 +252,7 @@ jsonschema==4.25.1
# drf-spectacular
jsonschema-specifications==2025.9.1
# via jsonschema
kubernetes==35.0.0
kubernetes==36.0.0
# via
# -r /awx_devel/requirements/requirements.in
# openshift
@@ -392,7 +393,7 @@ pyopenssl==25.3.0
# via
# -r /awx_devel/requirements/requirements.in
# twisted
pyparsing==2.4.7
pyparsing==3.3.2
# via -r /awx_devel/requirements/requirements.in
python-daemon==3.1.2
# via