* Enable new fancy asyncio metrics for dispatcherd
Remove old dispatcher metrics and patch in new data from local whatever
Update test fixture to new dispatcherd version
* Update dispatcherd again
* Handle node filter in URL, and catch more errors
* Add test for metric filter
* Split module for dispatcherd metrics
The OpenAPI schema incorrectly showed all 12 credential type kinds as
valid for POST/PUT/PATCH operations, when only 'cloud' and 'net' are
allowed for custom credential types. This caused API clients and LLM
agents to receive HTTP 400 errors when attempting to create credential
types with invalid kind values.
Add postprocessing hook to filter CredentialTypeRequest and
PatchedCredentialTypeRequest schemas to only show 'cloud', 'net',
and null as valid enum values, matching the existing validation logic.
No API behavior changes - this is purely a documentation fix.
Co-authored-by: Claude <noreply@anthropic.com>
Strip leading and trailing whitespace from SSH keys in validate_ssh_private_key()
to handle common copy-paste scenarios where hidden newlines cause base64 decoding
failures.
Changes:
- Added data.strip() in validate_ssh_private_key() before calling validate_pem()
- Added test_ssh_key_with_whitespace() to verify keys with leading/trailing
newlines are properly sanitized and validated
This prevents the confusing "HTTP 500: Internal Server Error" and
"binascii.Error: Incorrect padding" errors when users paste SSH keys with
accidental whitespace.
Fixes#14219
Signed-off-by: Joey Washburn <joey@joeywashburn.com>
* Add dispatcherctl command
* Add tests for dispatcherctl command
* Exit early if sqlite3
* Switch to dispatcherd mgmt cmd
* Move unwanted command options to run_dispatcher
* Add test for new stuff
* Update the SOS report status command
* make docs always reference new command
* Consistently error if given config file
* Additional dispatcher removal simplifications and waiting repear updates
* Fix double call and logging message
* Implement bugbot comment, should reap running on lost instances
* Add test case for new pending behavior
* WIP First pass
* started removing feature flags and adjusting logic
* Add decorator
* moved to dispatcher decorator
* updated as many as I could find
* Keep callback receiver working
* remove any code that is not used by the call back receiver
* add back auto_max_workers
* added back get_auto_max_workers into common utils
* Remove control and hazmat (squash this not done)
* moved status out and deleted control as no longer needed
* removed unused imports
* adjusted test import to pull correct method
* fixed imports and addressed clusternode heartbeat test
* Update function comments
* Add back hazmat for config and remove baseworker
* added back hazmat per @alancoding feedback around config
* removed baseworker completely and refactored it into the callback
worker
* Fix dispatcher run call and remove dispatch setting
* remove dispatcher mock publish setting
* Adjust heartbeat arg and more formatting
* fixed the call to cluster_node_heartbeat missing binder
* Fix attribute error in server logs
Refactored code to use Python's built-in datetime.timezone and zoneinfo instead of pytz for timezone handling. This modernizes the codebase and removes the dependency on pytz, aligning with current best practices for timezone-aware datetime objects.
* update to Python 3.12
* remove use of utcnow
* switch to timezone.utc
datetime.UTC is an alias of datetime.timezone.utc. if we're doing the double import for datetime it's more straightforward to just import timezone as well and get it directly
* debug python env version issue
* change python version
* pin to SHA and remove debug portion
* Remove the dynamic filter on dispatcher startup
Configure the dynamic logging level only on startup
* Special case for log level on settings change
* Add unit test for new behavior
* Add test for initial config
* Mark test django DB
* Do necessary requirement bump
* Delete cache in live test fixture
* Add test to recreate the error
* Also begin to add detection for empty event
* Remove breakpoint
* fix: ignore events with missing event types
* run linter and apply changes
---------
Co-authored-by: AlanCoding <arominge@redhat.com>
Co-authored-by: Peter Braun <pbraun@redhat.com>
* AAP-57817 Add Redis connection retry using redis-py 7.0+ built-in mechanism
* Refactor Redis client helpers to use settings and eliminate code duplication
* Create awx/main/utils/redis.py and move Redis client functions to avoid circular imports
* Fix subsystem_metrics to share Redis connection pool between
client and pipeline
* Cache Redis clients in RelayConsumer and RelayWebsocketStatsManager to avoid creating new connection pools on every call
* Add cap and base config
* Add Redis retry logic with exponential backoff to handle connection failures during long-running operations
* Add REDIS_BACKOFF_CAP and REDIS_BACKOFF_BASE settings to allow
adjustment of retry timing in worst-case scenarios without code changes
* Simplify Redis retry tests by removing unnecessary reload logic
* add force flag to refspec
* Development of git --amend test
* Update awx/main/tests/live/tests/conftest.py
Co-authored-by: Alan Rominger <arominge@redhat.com>
---------
Co-authored-by: AlanCoding <arominge@redhat.com>
* Change Swagger UI endpoint from /api/swagger/ to /api/docs/
- Update URL pattern to use /docs/ instead of /swagger/
- Update API root response to show 'docs' key instead of 'swagger'
- Add authentication requirement for schema documentation endpoints
- Update contact email to controller-eng@redhat.com
The schema endpoints (/api/docs/, /api/schema/, /api/redoc/) now
require authentication to prevent unauthorized access to API
documentation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Require authentication for all schema endpoints including /api/schema/
Create custom view classes that enforce authentication for all schema
endpoints to prevent inconsistent access control where UI views required
authentication but the raw schema endpoint remained publicly accessible.
This ensures all schema endpoints (/api/schema/, /api/docs/, /api/redoc/)
consistently require authentication.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add unit tests for authenticated schema view classes
Add test coverage for the new AuthenticatedSpectacular* view classes
to ensure they properly require authentication.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* remove unused import
---------
Co-authored-by: Claude <noreply@anthropic.com>
* AAP-45927 Add drf-spectacular
- Remove drf-yasg
- Add drf-spectacular
* move SPECTACULAR_SETTINGS from development_defaults.py to defaults.py
* move SPECTACULAR_SETTINGS from development_defaults.py to defaults.py
* Fix swagger tests: enable schema endpoints in all modes
Schema endpoints were restricted to development mode, causing
test_swagger_generation.py to fail. Made schema URLs available in
all modes and fixed deprecated Django warning filters in pytest.ini.
* remove swagger from Makefile
* remove swagger from Makefile
* change docker-compose-build-swagger to docker-compose-build-schema
* remove MODE
* remove unused import
* Update genschema to use drf-spectacular with awx-link dependency
- Add awx-link as dependency for genschema targets to ensure package metadata exists
- Remove --validate --fail-on-warn flags (schema needs improvements first)
- Add genschema-yaml target for YAML output
- Add schema.yaml to .gitignore
* Fix detect-schema-change to not fail on schema differences
Add '-' prefix to diff command so Make ignores its exit status.
diff returns exit code 1 when files differ, which is expected behavior
for schema change detection, not an error.
* Truncate schema diff summary to stay under GitHub's 1MB limit
Limit schema diff output in job summary to first 1000 lines to avoid
exceeding GitHub's 1MB step summary size limit. Add message indicating
when diff is truncated and direct users to job logs or artifacts for
full output.
* readd MODE
* add drf-spectacular to requirements.in and the requirements.txt generated from the script
* Add drf-spectacular BSD license file
Required for test_python_licenses test to pass now that drf-spectacular
is in requirements.txt.
* add licenses
* Add comprehensive unit tests for CustomAutoSchema
Adds 15 unit tests for awx/api/schema.py to improve SonarCloud test
coverage. Tests cover all code paths in CustomAutoSchema including:
- get_tags() method with various scenarios (swagger_topic, serializer
Meta.model, view.model, exception handling, fallbacks, warnings)
- is_deprecated() method with different view configurations
- Edge cases and priority ordering
All tests passing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* remove unused imports
---------
Co-authored-by: Claude <noreply@anthropic.com>
* prometheus-client returns an additional value as of v.0.22.0
* add license, remove outdated ones, add new embedded sources
* update requirements and UPGRADE BLOCKERs in README
settings.SUBSCRIPTIONS_USERNAME and
settings.SUBSCRIPTIONS_CLIENT_ID
should be mutually exclusive. This is because
the POST to api/v2/config/attach/ accepts only
a subscription_id, and infers which credentials to
use based on settings. If both are set, it is ambiguous
and can lead to unexpected 400s when attempting
to attach a license.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* We had race conditions with the system_administrator role being
created just-in-time. Instead of fixing the race condition(s), dodge
them by ensuring the role always exists
* Disconnect logic to fill in role parents
Get tests passing hopefully
Whatever SonarCloud
* remove role parents/children endpoints and related views
* remove duplicate get_queryset method from RoleTeamsList
---------
Co-authored-by: Peter Braun <pbraun@redhat.com>
Allow users to do subscription management using
Red Hat username and password.
In basic auth case, the candlepin API
at subscriptions.rhsm.redhat.com will be used instead
of console.redhat.com.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* Added tests for cross org sharing of credentials
* added negative testing for sharing of credentials
* added conditions and tests for roleteamslist regarding cross org credentials
* removed redundant codes
* made error message more articulated and specific
Bump migrations and delete some files
Resolve remaining conflicts
Fix requirements
Flake8 fixes
Prefer devel changes for schema
Use correct versions
Remove sso connected stuff
Update to modern actions and collection fixes
Remove unwated alias
Version problems in actions
Fix more versioning problems
Update warning string
Messed it up again
Shorten exception
More removals
Remove pbr license
Remove tests deleted in devel
Remove unexpected files
Remove some content missed in the rebase
Use sleep_task from devel
Restore devel live conftest file
Add in settings that got missed
Prefer devel version of collection test
Finish repairing .github path
Remove unintended test file duplication
Undo more unintended file additions
* Allow creating galaxy credential types without an organization (#16077)
* remove requirement for galaxy credentials to belong to an organization
* remove organization check for galaxy credential type
* add functional test
* fix: do not create multiple mappers for lists of emails or usernames
* fix: create multiple matchers, don't rely on matches_or
* fix tests
* truncate mapper names to a max of 128 chars
* better naming scheme for matchers
* Add LDAP support to gateway_mapping and expand test coverage
- Add new process_ldap_user_list function for LDAP group processing
- Add auth_type parameter to org_map_to_gateway_format and team_map_to_gateway_format
- Support both 'sso' and 'ldap' authentication types in mapping functions
- Fix syntax error and logic bug in existing code
- Add comprehensive unit tests for process_ldap_user_list function (13 test cases)
- Add unit tests for auth_type parameter functionality
- Update helper functions to support new auth_type parameter
- All tests pass and maintain backward compatibility
Technical changes:
- process_ldap_user_list handles None, boolean, string, and list inputs
- Proper type hints with mypy compatibility
- LDAP groups use 'has_or' trigger format vs SSO attribute matching
- Boolean True/False create Always/Never Allow triggers for LDAP
- Maintains proper ordering and mapper structure
Co-authored-by: Claude (Anthropic AI Assistant) <claude@anthropic.com>
* Fix empty list bug in process_ldap_user_list and add comprehensive tests
- Fix process_ldap_user_list to return empty list for empty input instead of creating invalid trigger
- Empty list [] now correctly returns no triggers instead of trigger with empty has_or array
- Add test case for empty list behavior in both LDAP and SSO functions
- Update existing test_empty_list to expect correct behavior (0 triggers)
- Maintain backward compatibility for all other input types
- Comprehensive testing confirms no regression in existing functionality
Bug Details:
- Before: process_ldap_user_list([]) returned [{'name': 'Match User Groups', 'trigger': {'groups': {'has_or': []}}}]
- After: process_ldap_user_list([]) returns [] (correct behavior)
- SSO function already handled this correctly
This prevents potential Gateway issues with empty has_or arrays and ensures logical consistency.
Co-authored-by: Claude (Anthropic AI Assistant) <claude@anthropic.com>
* Add comprehensive LDAP migrator tests and fix category handling
- Add comprehensive unit test suite for LDAPMigrator class (26 tests)
- Test LDAP configuration scenarios including multiple instances, mappings, and edge cases
- Add tests for mixed boolean/group mappings, special characters in org names, and empty configs
- Fix LDAP authenticator category to always be 'ldap' (not 'ldap<suffix>')
- Add auth_type='ldap' parameter to org_map_to_gateway_format and team_map_to_gateway_format calls
- Include AAP-51531 reference comments for specific test cases
- All tests passing (26/26)
Co-authored-by: Claude <claude@anthropic.com>
---------
Co-authored-by: Claude (Anthropic AI Assistant) <claude@anthropic.com>
This commit completely refactors how SSO organization and team mappings are processed
and exported for Gateway authentication, moving from a group-based approach to a more
flexible attribute-based system.
Key Changes:
- Introduced new process_sso_user_list() function for centralized user processing
- Enhanced boolean handling to support both native booleans and string representations
- Added email detection and regex pattern support for flexible user matching
- Refactored trigger generation from groups-based to attributes-based system
Gateway Mapping Enhancements (awx/main/utils/gateway_mapping.py):
- Added email regex detection for automatic email vs username classification
- Added pattern_to_slash_format() for regex pattern conversion
- Enhanced process_sso_user_list() with support for:
- Boolean values: True/False and ["true"]/["false"]
- String usernames and email addresses with automatic detection
- Regex patterns with both username and email matching
- Custom email_attr and username_attr parameters
- Refactored team_map_to_gateway_format() to use new processing system
- Refactored org_map_to_gateway_format() to use new processing system
- Changed trigger structure from {"groups": {"has_or": [...]}} to attribute-based triggers
- Improved naming convention to include trigger type in mapping names
Comprehensive Test Coverage (awx/main/tests/unit/utils/test_auth_migration.py):
- Added complete TestProcessSSOUserList class with 8 comprehensive test methods
- Enhanced TestOrgMapToGatewayFormat with string boolean and new functionality tests
- Enhanced TestTeamMapToGatewayFormat with string boolean and new functionality tests
- Added tests for email detection, regex patterns, and custom attributes
- Verified backward compatibility and integration functionality
- All existing tests updated to work with new attribute-based trigger system
Breaking Changes:
- Trigger structure changed from group-based to attribute-based
- Mapping names now include trigger descriptions for better clarity
- Function signatures updated to include email_attr and username_attr parameters
Co-Authored with Claude-4 via Cursor
Co-authored-by: Peter Braun <pbraun@redhat.com>
* Added better error handling and messaging when the service token authentication is broken. Allowed for GATEWAY_BASE_URL to override the service token's base url if it is set in the environment variables.
Co-Authored-By: Cursor (claude-4-sonnet)
* Removed GATEWAY_BASE_URL override for service token auth.
* Working branch for testing DAB RBAC changes
* AAP-48392 Handle DAB RBAC either before or after new type model (for merge) (#16045)
* Handle DAB RBAC either before or after new type model
* Translate CT to DAB CT
* Fix for rearrangement of post_migration methods
* Directly include RBAC service URLs
* Add a run before remote permission additions
* Sync old rbac to remote rbac (#7025)
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* Set DAB requirement back to devel
---------
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>
* migrate settings using the existing authenticator framework
* add method to get settings value to gateway client
* add transformer functions for settings
* Switched back to PUT for settings updates
* Started wiring in testing changes
* Added settings_* aggregation results. Added skip-github option. Added tests.
Assisted-by: Cursor
* Added --skip-all-authenticators command line argument. Added GoogleOAuth testing. Added tests for skipping all authenticators.
Assisted-by: Cursor
* wip: migrate other missing settings
* update login_redirect_override in google_oauth2
* impement login redirect for azuread
* implement login redirect for github
* implement login redirect for saml
* set LOGIN_REDIRECT_OVERRIDE even if no authenticator matched
* extract logic for login redirect override to base class
* use urlparse to compare valid redirect urls
* Preserve the original query parameters
* Fix flake8 issues
* Preserve the query parameter in sso_login_url
Gateway sets the sso_login_url to
/api/gateway/social/login/aap-saml-keycloak/?idp=IdP
The idp needs to be preserved when creating the redirect
* Update awx/main/utils/gateway_client.py
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* Update awx/main/management/commands/import_auth_config_to_gateway.py
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* list of settings updated
* Update awx/main/utils/gateway_client.py
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* Update awx/sso/utils/base_migrator.py
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
* fix tests
---------
Co-authored-by: Andrew Potozniak <potozniak@redhat.com>
Co-authored-by: Madhu Kanoor <mkanoor@redhat.com>
Co-authored-by: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
