AAP-76460 Prioritize REDHAT_CANDLEPIN_HOST over rhsm.conf for subscription validation (#16543)

* AAP-76460 — Prioritize REDHAT_CANDLEPIN_HOST over rhsm.conf for subscription validation

When REDHAT_CANDLEPIN_HOST is explicitly configured (Satellite/disconnected
environments), use it as the subscription host instead of falling back to
rhsm.conf. This fixes subscription loading failures in containerized
deployments where rhsm.conf defaults to subscription.rhsm.redhat.com but
the system cannot reach it.

Also guard against double port-appending in get_satellite_subs when the
host URL already includes a port.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Liam Allen <lallen@redhat.com>

* Optimize HostList API: conditional DISTINCT + composite index on JobHostSummary (#16530)

AAP-81517 — Optimize HostList API: conditional DISTINCT + composite index

Make .distinct() conditional on host_filter being set — without it the
RBAC IN subquery on a direct FK cannot produce duplicates, so DISTINCT
is pure overhead. Add composite index (host_id, id DESC) on
main_jobhostsummary so the with_latest_summary_id() correlated subquery
can use an index-only top-1 scan instead of scanning and sorting.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Fix: Increase awx-operator molecule timeout to 20m (#16534)

The awx-operator molecule kind test intermittently times out after
15 minutes on GitHub Actions runners, causing flaky CI failures.
Bump the bash-level timeout from 15m to 20m (step-level
timeout-minutes: 60 is unchanged).

Closes: AAP-81583

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* AAP-81860 — Eliminate LEFT OUTER JOINs in ActivityStream RBAC query (#16533)

* AAP-81860 — Eliminate LEFT OUTER JOINs in ActivityStream RBAC query

ActivityStreamAccess.filtered_queryset() used field-traversal Q objects
(e.g. Q(host__inventory__in=...)) across 18 M2M relationships, which
Django translates into LEFT OUTER JOINs. This forces PostgreSQL to join
all intermediary tables before filtering, making it the #2 DB consumer
at 483s in a 10-minute Scale Lab window.

Replace all M2M field traversals with pk__in subqueries using .through
intermediary tables, following the same pattern proven in AAP-81082
(28% improvement for unified job RBAC). This changes the SQL from
LEFT OUTER JOIN to IN (SELECT ...) semi-joins, allowing PostgreSQL to
skip the unconditional joins. Also:

- Remove .distinct() (no longer needed without M2M JOINs)
- Remove unnecessary if-truthy checks that evaluated subqueries as
  boolean before including them (5 extra DB roundtrips per request)
- Migrate accessible_pk_qs(user, 'read_role') to access_ids_qs(user,
  'view') for direct DAB RBAC API usage

* Use .exists() instead of queryset truth-test for auditing_orgs guard

Avoids evaluating the full queryset when checking whether the user has
auditing orgs — .exists() issues a cheaper SELECT 1 … LIMIT 1 instead.

* Restore .exists() guards for resource-type Q branches

Re-add conditional guards around inventory, credential, project,
job template, workflow job template, and team Q branches. Uses
.exists() (cheap SELECT 1 LIMIT 1) instead of the old queryset
truth-test to avoid unnecessary query evaluation while still
preventing generation of an overly complex query when the user
has no access to a given resource type.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* chore: Upgrade kubernetes package to 36.0.2

* feat: remove extra_vars from jobs and unified_jobs list endpoint. Add… (#16461)

feat: add exclude query parameter to unified_jobs and jobs list endpoints

Allow clients to exclude heavy fields (artifacts, extra_vars) from list
responses via ?exclude=artifacts,extra_vars. These fields are now included
by default instead of being stripped.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* AAP-80457 - Fix credential_input_source to handle state: absent when target crede… (#16523)

Fix credential_input_source to handle state: absent when target credential is missing

The credential_input_source module would error when trying to delete a
credential input source (state: absent) if the target credential didn't
exist. This breaks idempotent playbook runs where cleanup tasks assume
resources may already be gone.

The fix only applies to state: absent; state: present still correctly
fails when the target credential doesn't exist.

Co-authored-by: Liam Allen <lallen@redhat.com>

* Add additional test coverage

* get_satellite_subs() reads verify from rhsm.conf, never from REDHAT_CANDLEPIN_VERIFY.

* Address coderabbit feedback

---------

Signed-off-by: Liam Allen <lallen@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Dirk Julich <djulich@redhat.com>
Co-authored-by: Rodrigo Toshiaki Horie <rodrigo.horie@hotmail.com>
Co-authored-by: Adrià Sala <22398818+adrisala@users.noreply.github.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
Co-authored-by: Nick Meyer <nick.a.meyer@icloud.com>

awx/main/tests/unit/utils/test_validate_rh.py

@@ -1,3 +1,6 @@
+import configparser
+
+import pytest
 from unittest.mock import patch
 from awx.main.utils.licensing import Licenser
 
@@ -6,21 +9,25 @@ def test_validate_rh_basic_auth_rhsm():
     """
     Assert get_rhsm_subs is called when
     - basic_auth=True
+    - REDHAT_CANDLEPIN_HOST is not set
     - host is subscription.rhsm.redhat.com
     """
     licenser = Licenser()
 
-    with patch.object(licenser, 'get_host_from_rhsm_config', return_value='https://subscription.rhsm.redhat.com') as mock_get_host, patch.object(
+    with patch('awx.main.utils.licensing.settings') as mock_settings, patch.object(
-        licenser, 'get_rhsm_subs', return_value=[]
+        licenser, 'get_host_from_rhsm_config', return_value='https://subscription.rhsm.redhat.com'
-    ) as mock_get_rhsm, patch.object(licenser, 'get_satellite_subs') as mock_get_satellite, patch.object(
+    ) as mock_get_host, patch.object(licenser, 'get_rhsm_subs', return_value=[]) as mock_get_rhsm, patch.object(
+        licenser, 'get_satellite_subs'
+    ) as mock_get_satellite, patch.object(
         licenser, 'get_crc_subs'
     ) as mock_get_crc, patch.object(
         licenser, 'generate_license_options_from_entitlements'
     ) as mock_generate:
 
+        mock_settings.REDHAT_CANDLEPIN_HOST = None
+
         licenser.validate_rh('testuser', 'testpass', basic_auth=True)
 
-        # Assert the correct methods were called
         mock_get_host.assert_called_once()
         mock_get_rhsm.assert_called_once_with('https://subscription.rhsm.redhat.com', 'testuser', 'testpass')
         mock_get_satellite.assert_not_called()
@@ -32,21 +39,25 @@ def test_validate_rh_basic_auth_satellite():
     """
     Assert get_satellite_subs is called when
     - basic_auth=True
-    - custom satellite host
+    - REDHAT_CANDLEPIN_HOST is not set
+    - rhsm.conf points to a non-RHSM host
     """
     licenser = Licenser()
 
-    with patch.object(licenser, 'get_host_from_rhsm_config', return_value='https://satellite.example.com') as mock_get_host, patch.object(
+    with patch('awx.main.utils.licensing.settings') as mock_settings, patch.object(
-        licenser, 'get_rhsm_subs'
+        licenser, 'get_host_from_rhsm_config', return_value='https://satellite.example.com'
-    ) as mock_get_rhsm, patch.object(licenser, 'get_satellite_subs', return_value=[]) as mock_get_satellite, patch.object(
+    ) as mock_get_host, patch.object(licenser, 'get_rhsm_subs') as mock_get_rhsm, patch.object(
+        licenser, 'get_satellite_subs', return_value=[]
+    ) as mock_get_satellite, patch.object(
         licenser, 'get_crc_subs'
     ) as mock_get_crc, patch.object(
         licenser, 'generate_license_options_from_entitlements'
     ) as mock_generate:
 
+        mock_settings.REDHAT_CANDLEPIN_HOST = None
+
         licenser.validate_rh('testuser', 'testpass', basic_auth=True)
 
-        # Assert the correct methods were called
         mock_get_host.assert_called_once()
         mock_get_rhsm.assert_not_called()
         mock_get_satellite.assert_called_once_with('https://satellite.example.com', 'testuser', 'testpass')
@@ -73,7 +84,6 @@ def test_validate_rh_service_account_crc():
 
         licenser.validate_rh('client_id', 'client_secret', basic_auth=False)
 
-        # Assert the correct methods were called
         mock_get_host.assert_not_called()
         mock_get_rhsm.assert_not_called()
         mock_get_satellite.assert_not_called()
@@ -81,74 +91,127 @@ def test_validate_rh_service_account_crc():
         mock_generate.assert_called_once_with([], is_candlepin=False)
 
 
+def test_validate_rh_candlepin_host_prioritized_over_rhsm_config():
+    """Test REDHAT_CANDLEPIN_HOST takes priority over rhsm.conf
+    - basic_auth=True
+    - REDHAT_CANDLEPIN_HOST is set
+    - rhsm.conf should NOT be consulted
+    """
+    licenser = Licenser()
+
+    with patch('awx.main.utils.licensing.settings') as mock_settings, patch.object(licenser, 'get_host_from_rhsm_config') as mock_get_host, patch.object(
+        licenser, 'get_rhsm_subs'
+    ) as mock_get_rhsm, patch.object(licenser, 'get_satellite_subs', return_value=[]) as mock_get_satellite, patch.object(
+        licenser, 'get_crc_subs'
+    ) as mock_get_crc, patch.object(
+        licenser, 'generate_license_options_from_entitlements'
+    ) as mock_generate:
+
+        mock_settings.REDHAT_CANDLEPIN_HOST = 'https://satellite.example.com'
+        licenser.validate_rh('testuser', 'testpass', basic_auth=True)
+
+        mock_get_host.assert_not_called()
+        mock_get_rhsm.assert_not_called()
+        mock_get_satellite.assert_called_once_with('https://satellite.example.com', 'testuser', 'testpass')
+        mock_get_crc.assert_not_called()
+        mock_generate.assert_called_once_with([], is_candlepin=True)
+
+
+def test_validate_rh_prepends_scheme_when_missing():
+    """REDHAT_CANDLEPIN_HOST without a scheme gets https:// prepended"""
+    licenser = Licenser()
+
+    with patch('awx.main.utils.licensing.settings') as mock_settings, patch.object(licenser, 'get_host_from_rhsm_config'), patch.object(
+        licenser, 'get_rhsm_subs'
+    ) as mock_get_rhsm, patch.object(licenser, 'get_satellite_subs', return_value=[]) as mock_get_satellite, patch.object(
+        licenser, 'get_crc_subs'
+    ), patch.object(
+        licenser, 'generate_license_options_from_entitlements'
+    ):
+
+        mock_settings.REDHAT_CANDLEPIN_HOST = 'satellite.example.com'
+        licenser.validate_rh('testuser', 'testpass', basic_auth=True)
+
+        mock_get_satellite.assert_called_once_with('https://satellite.example.com', 'testuser', 'testpass')
+        mock_get_rhsm.assert_not_called()
+
+
 def test_validate_rh_missing_user_raises_error():
     """Test validate_rh raises ValueError when user is missing"""
     licenser = Licenser()
 
-    with patch.object(licenser, 'get_host_from_rhsm_config', return_value='https://subscription.rhsm.redhat.com'):
+    with patch('awx.main.utils.licensing.settings') as mock_settings, patch.object(
-        try:
+        licenser, 'get_host_from_rhsm_config', return_value='https://subscription.rhsm.redhat.com'
+    ):
+        mock_settings.REDHAT_CANDLEPIN_HOST = None
+        with pytest.raises(ValueError, match='subscriptions_client_id or subscriptions_username is required'):
             licenser.validate_rh(None, 'testpass', basic_auth=True)
-            assert False, "Expected ValueError to be raised"
-        except ValueError as e:
-            assert 'subscriptions_client_id or subscriptions_username is required' in str(e)
 
 
 def test_validate_rh_missing_password_raises_error():
     """Test validate_rh raises ValueError when password is missing"""
     licenser = Licenser()
 
-    with patch.object(licenser, 'get_host_from_rhsm_config', return_value='https://subscription.rhsm.redhat.com'):
+    with patch('awx.main.utils.licensing.settings') as mock_settings, patch.object(
-        try:
+        licenser, 'get_host_from_rhsm_config', return_value='https://subscription.rhsm.redhat.com'
+    ):
+        mock_settings.REDHAT_CANDLEPIN_HOST = None
+        with pytest.raises(ValueError, match='subscriptions_client_secret or subscriptions_password is required'):
             licenser.validate_rh('testuser', None, basic_auth=True)
-            assert False, "Expected ValueError to be raised"
-        except ValueError as e:
-            assert 'subscriptions_client_secret or subscriptions_password is required' in str(e)
 
 
-def test_validate_rh_no_host_fallback_to_candlepin():
+@pytest.mark.parametrize(
-    """Test validate_rh falls back to REDHAT_CANDLEPIN_HOST when no host from config
+    'host_input, rhsm_port, expected_in_url, not_expected_in_url',
+    [
+        ('https://satellite.example.com:8443', '443', ':8443', ':8443:443'),
+        ('https://satellite.example.com', '8443', ':8443', None),
+        ('https://satellite.example.com/', '8443', ':8443', '/:'),
+    ],
+    ids=['skip-port-when-present', 'append-port-when-missing', 'strip-trailing-slash'],
+)
+def test_get_satellite_subs_port_handling(host_input, rhsm_port, expected_in_url, not_expected_in_url):
+    licenser = Licenser()
+    licenser.config = configparser.ConfigParser()
+    licenser.config.read_string(f"[server]\nhostname=satellite.example.com\nport={rhsm_port}\n[rhsm]\nrepo_ca_cert=/etc/rhsm/ca/redhat-uep.pem\n")
+
+    with patch('awx.main.utils.licensing.settings') as mock_settings, patch('awx.main.utils.licensing.requests') as mock_requests:
+        mock_settings.REDHAT_CANDLEPIN_VERIFY = None
+        mock_orgs = mock_requests.get.return_value
+        mock_orgs.json.return_value = {'results': []}
+
+        licenser.get_satellite_subs(host_input, 'user', 'pw')
+
+        called_url = mock_requests.get.call_args[0][0]
+        assert expected_in_url in called_url
+        if not_expected_in_url:
+            assert not_expected_in_url not in called_url
+
+
+def test_get_satellite_subs_uses_candlepin_verify_setting():
+    """REDHAT_CANDLEPIN_VERIFY should take priority over rhsm.conf ca_cert"""
+    licenser = Licenser()
+    licenser.config = configparser.ConfigParser()
+    licenser.config.read_string("[server]\nhostname=satellite.example.com\n[rhsm]\nrepo_ca_cert=/etc/rhsm/ca/redhat-uep.pem\n")
+
+    with patch('awx.main.utils.licensing.settings') as mock_settings, patch('awx.main.utils.licensing.requests') as mock_requests:
+        mock_settings.REDHAT_CANDLEPIN_VERIFY = False
+        mock_orgs = mock_requests.get.return_value
+        mock_orgs.json.return_value = {'results': []}
+
+        licenser.get_satellite_subs('https://satellite.example.com', 'user', 'pw')
+
+        assert mock_requests.get.call_args[1]['verify'] is False
+
+
+def test_validate_rh_no_host_raises_error():
+    """Test validate_rh raises ValueError when no host is available
     - basic_auth=True
-    - no host from config
+    - REDHAT_CANDLEPIN_HOST is not set
-    - REDHAT_CANDLEPIN_HOST is set
+    - rhsm.conf returns None
     """
     licenser = Licenser()
 
-    with patch('awx.main.utils.licensing.settings') as mock_settings, patch.object(
+    with patch('awx.main.utils.licensing.settings') as mock_settings, patch.object(licenser, 'get_host_from_rhsm_config', return_value=None):
-        licenser, 'get_host_from_rhsm_config', return_value=None
+        mock_settings.REDHAT_CANDLEPIN_HOST = None
-    ) as mock_get_host, patch.object(licenser, 'get_rhsm_subs', return_value=[]) as mock_get_rhsm, patch.object(
+        with pytest.raises(ValueError, match='Could not get host url for subscriptions'):
-        licenser, 'get_satellite_subs', return_value=[]
-    ) as mock_get_satellite, patch.object(
-        licenser, 'get_crc_subs'
-    ) as mock_get_crc, patch.object(
-        licenser, 'generate_license_options_from_entitlements'
-    ) as mock_generate:
-
-        mock_settings.REDHAT_CANDLEPIN_HOST = 'https://candlepin.example.com'
             licenser.validate_rh('testuser', 'testpass', basic_auth=True)
-
-        # Assert the correct methods were called
-        mock_get_host.assert_called_once()
-        mock_get_rhsm.assert_not_called()
-        mock_get_satellite.assert_called_once_with('https://candlepin.example.com', 'testuser', 'testpass')
-        mock_get_crc.assert_not_called()
-        mock_generate.assert_called_once_with([], is_candlepin=True)
-
-
-def test_validate_rh_empty_credentials_basic_auth():
-    """Test validate_rh with empty string credentials raises ValueError"""
-    licenser = Licenser()
-
-    with patch.object(licenser, 'get_host_from_rhsm_config', return_value='https://subscription.rhsm.redhat.com'):
-        # Test empty user
-        try:
-            licenser.validate_rh(None, 'testpass', basic_auth=True)
-            assert False, "Expected ValueError to be raised"
-        except ValueError as e:
-            assert 'subscriptions_client_id or subscriptions_username is required' in str(e)
-
-        # Test empty password
-        try:
-            licenser.validate_rh('testuser', None, basic_auth=True)
-            assert False, "Expected ValueError to be raised"
-        except ValueError as e:
-            assert 'subscriptions_client_secret or subscriptions_password is required' in str(e)

awx/main/utils/licensing.py

@@ -19,6 +19,7 @@ import json
 import logging
 import re
 import requests
+from urllib.parse import urlparse
 import time
 import zipfile
 
@@ -228,18 +229,16 @@ class Licenser(object):
         return host
 
     def validate_rh(self, user, pw, basic_auth):
-        # if basic auth is True, host is read from rhsm.conf (subscription.rhsm.redhat.com)
-        # if basic auth is False, host is settings.SUBSCRIPTIONS_RHSM_URL (console.redhat.com)
-        # if rhsm.conf is not found, host is settings.REDHAT_CANDLEPIN_HOST (satellite server)
         if basic_auth:
+            if not (host := getattr(settings, 'REDHAT_CANDLEPIN_HOST', None)):
                 host = self.get_host_from_rhsm_config()
-            if not host:
-                host = getattr(settings, 'REDHAT_CANDLEPIN_HOST', None)
         else:
             host = settings.SUBSCRIPTIONS_RHSM_URL
 
         if not host:
             raise ValueError('Could not get host url for subscriptions')
+        if not host.startswith(('https://', 'http://')):
+            host = 'https://' + host
 
         if not user:
             raise ValueError('subscriptions_client_id or subscriptions_username is required')
@@ -308,13 +307,20 @@ class Licenser(object):
 
     def get_satellite_subs(self, host, user, pw):
         port = None
+        if (verify := getattr(settings, 'REDHAT_CANDLEPIN_VERIFY', None)) is None:
             try:
                 verify = str(self.config.get("rhsm", "repo_ca_cert"))
-            port = str(self.config.get("server", "port"))
             except Exception as e:
-            logger.exception('Unable to read rhsm config to get ca_cert location. {}'.format(str(e)))
+                logger.exception(f'Unable to read rhsm config to get ca_cert location. {e}')
                 verify = True
-        if port:
+        try:
+            port = str(self.config.get("server", "port"))
+        except Exception:
+            port = None
+        host = host.rstrip('/')
+        # Append port from rhsm.conf only if the host URL doesn't already include one
+        # (REDHAT_CANDLEPIN_HOST may already contain a port)
+        if port and not urlparse(host).port:
             host = ':'.join([host, port])
         json = []
         try: