Compare commits

...

142 Commits

Author SHA1 Message Date
Jake Jackson
8e0c09f015 update editable_dependencies readme
fixed typo/project naming to match example.
2024-08-27 15:20:39 -04:00
Alan Rominger
9cf66de454 Pin DAB to devel again (#15467) 2024-08-27 11:18:09 -04:00
Alan Rominger
f5760b149d Fix 500 error when ordinary user viewed system JTs (#15465) 2024-08-26 11:51:16 -04:00
Seth Foster
7ed0eee60c Make controller specific team and org roles (#15445)
Adds the following managed Role Definitions

Controller Team Admin
Controller Team Member
Controller Organization Admin
Controller Organization Member

These have the same permission set as the
platform roles (without the Controller prefix)

Adding members to teams and orgs via the legacy RBAC system
will use these role definitions.

Other changes:
- Bump DAB to 2024.08.22
- Set ALLOW_LOCAL_ASSIGNING_JWT_ROLES to False in defaults.py.
This setting prevents assignments to the platform roles (e.g. Team Member).

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-08-22 15:41:54 -04:00
Hao Liu
78f345c486 Remove old UI (#15414)
* Remove source code for old UI
* Rename ui-next to ui
* Remove license scan for javascript dependencies
2024-08-22 13:48:56 -04:00
Peter Braun
3f8274d371 fix: avoid calling undefined method for anonymous users (#15440) 2024-08-22 18:01:31 +02:00
Peter Braun
c6223c076f fix: catch correct exception when parsing filter (#15458) 2024-08-22 16:12:54 +02:00
jessicamack
1b5cdf6bef Replace ansiconv with ansi2html (#15328)
* replace ansiconv with ansi2html

The ansiconv package is archived so I'm replacing it with a similar package that's still actively being worked on.

* remove minimum version

The version minimum was used to get the latest version while running the upgrader

* set minimum version for ansi2html

* provide usage info
2024-08-22 09:38:57 -04:00
Alan Rominger
5a8429deed Update django-ansible-base version to 2024.8.19 (#15454)
Update django-ansible-base version to 2024.8.9

Co-authored-by: chrismeyersfsu <722880+chrismeyersfsu@users.noreply.github.com>
2024-08-21 14:16:49 -04:00
Alan Rominger
af537b5261 Rewrite more access logic in terms of permissions instead of roles (#15453)
* Rewrite more access logic in terms of permissions instead of roles

* Cut down supported logic because that would not work anyway

* Remove methods not needed anymore

* Create managed roles in test before delegating permissions
2024-08-21 13:14:40 -04:00
Seth Foster
500b1c47ba SSO login should redirect to new UI index (#15456)
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-08-21 16:31:45 +00:00
Elijah DeLee
c5c617b178 Guard around race condition (#15452)
I had the luck of running into this race condition that broke my deployment. No instance was ever able to register because on running "awx-manage" in some check of a setting, it would end up failing here with

```
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/conf/license.py", line 10, in _get_validated_license_data
    return get_licenser().validate()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/utils/licensing.py", line 453, in validate
    automated_since = int(Instance.objects.order_by('id').first().created.timestamp())
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'created'
```
2024-08-20 16:24:56 -04:00
Jake Jackson
39d1922b80 Update editable deps docs (#15451)
update editable deps docs
2024-08-20 14:21:27 -04:00
jessicamack
6b462cdfdb Unpin django-guid and update license (#15381)
unpin django-guid and update license

there's no reason listed for the pin and the changelog doesn't describe any changes that should block a full upgrade. they changed licenses to MIT
2024-08-16 18:54:59 -04:00
jessicamack
ca3e899c2c Unpin django-split-settings (#15379)
* unpin django-split-settings

blocker is 2 years old. upgrading to see if the previous issue is still present. upgrading to a version with Python 3.11 support

* remove UPGRADE BLOCKER in README
2024-08-16 15:07:54 -04:00
Chris Meyers
43a3d4a394 Fixes pytest CI error
```
  /var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/python.py:163:
  PytestReturnNotNoneWarning: Expected None, but
  awx/main/tests/unit/test_tasks.py::TestJobCredentials::test_custom_environment_injectors_with_boolean_extra_vars
  returned ['successful', 0], which will be an error in a future version
  of pytest.  Did you mean to use `assert` instead of `return`?
```

* Dug into the git blame for this one
  060585434a is the commit for any
  historians. It was wrongfully carried over from a mock pexpect
  implementation. Our new tests are nice. They don't go as far as trying
  to run the task so they do not need to mock pexpect. That is why it is
  safe to remove this code without finding it a new home.
2024-08-10 10:29:34 -04:00
Alan Rominger
af02ab46e3 Bump DAB version manually because bot is on vacation (#15434) 2024-08-09 15:42:18 -04:00
jessicamack
1ef77abdc3 Remove 'AWX' from setting endpoint (#15432)
* Remove AWX from display text

Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
2024-08-09 11:44:40 -04:00
Rick Elrod
cb2ad41d8d Fix a test in preparation for syncing description
Refs ansible/django-ansible-base#447

Signed-off-by: Rick Elrod <rick@elrod.me>
2024-08-08 16:35:17 +02:00
Seth Foster
73b1536356 Only refresh session if updating own password (#15426)
Fixes bug where creating a new user will
request a new awx_sessionid cookie, invalidating
the previous session.

Do not refresh session if updating or
creating a password for a different user.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-08-07 17:04:19 -04:00
jessicamack
37b7a69303 Unpin channels-redis (#15329)
* unpin channels-redis

The bug that initially caused the upgrade block has been resolved https://github.com/django/channels_redis/issues/332

* replace aioredis Exception with a redis Exception

Version 4.0.0 of channel-redis migrated the underlying Redis library from aioredis to redis-py. The Exception has been changed to an equivalent

* remove unused license

* remove UPGRADE BLOCKER in README

* remove hiredis

it was an indirect dependency from aioredis which was removed

* remove unused license

* add back hiredis

it's potentially providing a performance boost. install explicitly as a part of redis. upgrade to more recent version

* remove UPGRADE BLOCKER for hiredis

it was also addressed as a part of this PR
2024-08-07 13:44:24 +00:00
TVo
6d0c47fdd0 Re-do PR #14685 for alt-text inventories. (#15394) 2024-07-31 09:40:28 +01:00
TVo
54b4acbdfc Added docs for OTel - awx integration (#15408) 2024-07-29 14:36:22 -06:00
Seth Foster
a41766090e Make ui_next the default UI (#15405)
Change django url dispatcher to serve up ui_next files instead of old ui files

Old UI will not be served with this change

Github CI still runs old ui tests (to be removed in another PR)

Remove the Github workflows that build old UI

---------

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-07-29 15:13:09 -04:00
github-actions[bot]
34fa897dda Bump django-ansible-base to 2024.7.17 (#15373)
Update django-ansible-base version to devel

Co-authored-by: chrismeyersfsu <722880+chrismeyersfsu@users.noreply.github.com>
2024-07-26 09:11:12 -04:00
Hao Liu
32df114e41 Improve asyncio debugging (#15398)
- use asyncio.get_running_loop() instead of passing around event_loops
- add name to all of the asyncio tasks for easier debugging

we are trying to figure out which task is
```
Task was destroyed but it is pending!
task: <Task pending name='Task-<id>' coro=<RedisConnection._read_data() done, defined at /var/lib/awx/venv/awx/lib64/python3.9/site-packages/aioredis/connection.py:180> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fba77bf1700>()]> cb=[RedisConnection.__init__.<locals>.<lambda>() at /var/lib/awx/venv/awx/lib64/python3.9/site-packages/aioredis/connection.py:168]>
```
is referring to
2024-07-24 12:05:02 -04:00
TVo
018f235a64 Replaced all references of downstream docs to upstream docs (#15388)
* Replaced all references of downstream docs to upstream docs.

* Update README.md

Co-authored-by: Don Naro <dnaro@redhat.com>

* Update README.md.j2

Co-authored-by: Don Naro <dnaro@redhat.com>

* Update README.md.j2

Co-authored-by: Don Naro <dnaro@redhat.com>

* Incorpor'd review feedback from @oraNod and @samccann

* Updated with agreed link (for now) until further change is needed.

---------

Co-authored-by: Don Naro <dnaro@redhat.com>
2024-07-24 07:54:43 -06:00
Hao Liu
7e77235d5e Add UI for RECEPTOR_KEEP_WORK_ON_ERROR
In Troubleshooting settings
2024-07-22 17:02:37 -04:00
Hao Liu
139d8f0ae2 Add RECEPTOR_KEEP_WORK_ON_ERROR setting
If RECEPTOR_KEEP_WORK_ON_ERROR is set to true receptor work unit will not be automatically released

Co-Authored-By: Chris Meyers <chrismeyersfsu@users.noreply.github.com>
2024-07-22 17:02:37 -04:00
Hao Liu
7691365aea Fix depends_on for awx devel when editable dependencies is enabled (#15393)
Fix depends_on for awx devel...

when editable dependencies is on

bug introduced by https://github.com/ansible/awx/pull/15386
2024-07-22 16:48:53 -04:00
Alan Rominger
59f61517d4 Loosen up team EE restrictions (#15384)
* Try to loosen up team EE restrictions

* Fix missed permission case of nulling EE org
2024-07-22 14:51:32 -04:00
Alan Rominger
fa670e2d7f Upgrade to v4 checkout, hide output (#15322) 2024-07-19 15:33:11 -04:00
Jake Jackson
a87a044d64 Update test to conform with new DAB change (#15385)
* update tests to not fail with new version of DAB

* comment out conditional for now and add TODOs to fix it
2024-07-19 13:58:09 -04:00
Alan Rominger
381ade1148 Fix create_preload_data to allow running without an admin user created (#15356)
* Allow create_preload_data without having superuser created first

* Temporarily change the DAB requirement

* Put DAB branch back to devel
2024-07-19 10:35:33 -04:00
Sandra McCann
864a30e3d4 Remove remnants of controller terms from quickstart docs (#15350)
Remove remnants of controller terms from quickstart

Signed-off-by: Sandra McCann <samccann@redhat.com>
2024-07-18 22:42:35 -06:00
Sandra McCann
5f42db67e6 Remove references to translated versions of the docs (#15354)
remove references to translated versions of the docs

Signed-off-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: TVo <thavo@redhat.com>
2024-07-19 01:53:06 +00:00
Hao Liu
ddf4f288d4 Remove links from docker-compose template (#15386)
Links are use to indicate network connectivity and optionally provide alias

it is not needed for communication since all the container are on the awx network

in the prometheus container case since awx_ container now have valid hostname it's no longer required (also i think the link is missing a `-` anyway...)

links also implicitly imply dependency between services in this i see awx container depends on redis and postgres so i switch to depends_on to retain that

Making this change to be podman compatible
because i get
```
Error response from daemon: bad parameter: link is not supported
```
2024-07-18 21:19:50 -04:00
Lila Yasin
e75bc8bc1e Fix test_url_base_defaults_to_request to reference local host instead… (#15367)
* Update all references to towerhost to platformhost

* Run prettier on failing ui files
2024-07-18 15:28:54 -04:00
Roberto Chaud
bb533287b8 Create receptor group if missing (#15276) 2024-07-18 17:59:23 +00:00
Sandra McCann
9979fc659e Update docs replacements to AWX (#15349)
Update replacements to AWX

Signed-off-by: Sandra McCann <samccann@redhat.com>
2024-07-18 13:36:00 -04:00
Hao Liu
9e5babc093 Fix ui-next build for release staging GHA (#15383) 2024-07-18 13:26:07 -04:00
Hao Liu
c71e2524ed Disable dab-release GHA on fork unless explicitly triggered (#15382)
Disable dab-release GHA on fork Unless explicitly triggered
2024-07-18 14:55:52 +00:00
Hao Liu
48b4c62186 Update DAB update automation PR template (#15376)
To pass PR Check github action
2024-07-18 10:24:47 -04:00
Seth Foster
853730acb9 Allow deleting org of a running workflow job (#15374)
Old RBAC system hits DOESNOTEXIST query errors
if a user deletes an org while a workflow job is active.

The error is triggered by
1. starting workflow job
2. delete the org that the workflow job is a part of
3. The workflow changes status (e.g. pending to waiting)

This error message would surface
awx.main.models.rbac.Role.DoesNotExist: Role matching
query does not exist.

The fix is wrap the query in a try catch, and skip
over some logic if the roles don't exist.

---------

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-07-18 09:40:58 -04:00
Don Naro
f1448fced1 update terminology (#15357)
* update terminology

Replace some instances of Tower with AWX and remove some references to
enterprise left over from the migration of RST content from the
Automation Controller docs.

* Update docs/docsite/rst/userguide/overview.rst

Co-authored-by: TVo <thavo@redhat.com>

---------

Co-authored-by: TVo <thavo@redhat.com>
2024-07-18 10:29:21 +01:00
Hao Liu
7697b6a69b Pin 3rd party action at SHA
For safety
2024-07-17 22:58:15 +02:00
Chris Meyers
22a491c32c Put DAB version in the PR title 2024-07-17 15:10:25 -04:00
Chris Meyers
cbd9dce940 Run at 6 am every day 2024-07-17 15:10:25 -04:00
Chris Meyers
a4fdcc1cca Update .github/workflows/dab-release.yml
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
2024-07-17 15:10:25 -04:00
Chris Meyers
df95439008 Update .github/workflows/dab-release.yml
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
2024-07-17 15:10:25 -04:00
Chris Meyers
acd834df8b Check and update django-ansible-base
* Check upstream django-ansible-base releases. If the version upstream
  does not match the version we are pinned to then submit a PR with the
  upstream version.
2024-07-17 15:10:25 -04:00
TVo
587f0ecf98 Updated the api file to reflect 2024 date (#15369) 2024-07-16 19:58:55 +00:00
Hao Liu
5a2091f7bf Build new/old UI with different nodejs version (#15368) 2024-07-16 13:18:47 -04:00
Hao Liu
fa7423819a Fix minor docker build warning (#15362)
Fix docker build warning

Fix
```
WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 8)
```
2024-07-15 13:34:46 +00:00
Alan Rominger
fde8af9f11 Fix task ending in error due to bad iterator (#15355) 2024-07-12 13:20:39 -04:00
Seth Foster
209e7e27b1 Check member of org when granting cred (#15353)
A user needs to be a member of the org
in order to use a credential in that org.

We were incorrectly checking for "change"
permission of the org, instead of "member".

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-07-10 21:46:26 -04:00
Hao Liu
6c7d29a982 Fix command to set db session timeout for locks (#15352)
Fix command to set db session timeout

Add quote around the value of the setting

Example failures
```
2024-07-10 13:33:29,237 ERROR    [a7e55a64e6744a0e920bb1fd78615e5f] awx.main.dispatch Worker failed to run task awx.main.tasks.system.awx_periodic_scheduler(*[], **{}
Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/backends/utils.py", line 87, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
psycopg.errors.SyntaxError: trailing junk after numeric literal at or near "1d"
LINE 1: SET idle_in_transaction_session_timeout = 1d
                                                  ^
```
2024-07-10 11:11:12 -04:00
Alan Rominger
282ba36839 Fix EE admin not being able to PATCH/PUT object while providing organization (#15348)
* Fix bug where EE object-level admin could not set organization

* Finish polishing up test
2024-07-09 16:55:09 -04:00
Alan Rominger
b727d2c3b3 Log conflicts and created items by the periodic resource sync (#15337)
* Initial lazy logging of periodic sync results

* Add desired polish to log

* Add debug log
2024-07-09 15:13:08 -04:00
Hao Liu
7fc3d5c7c7 Update ActivityStream UI query to order by id (#15346)
Timestamp for activity stream is not indexed result in slow query. switching to ID (which effectively will is order by created time) to improve performance
2024-07-09 14:38:07 -04:00
TVo
4e055f46c4 Added note to API guide for filtering exact matches (#15332) 2024-07-09 10:55:19 -06:00
Seth Foster
f595985b7c Callback for role assignment (#15339)
Validate role assignment if org defined

Check that organization is defined on credential
before running queries.

Fixes a "None type does not have attribute id" error.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-07-09 09:44:27 -04:00
Alan Rominger
ea232315bf Do not reference self.messages when it does not exist (#15331) 2024-07-03 20:07:46 +00:00
Alan Rominger
ee251812b5 Add complete test that we have analogs to old versions of roles, fix some mismatches (#15321)
* Add test that we got all permissions right for every role

* Fix missing Org execute role and missing adhoc role permission

* Add in missing Organization Approval Role as well

* Remove Role from role names
2024-07-03 15:40:55 -04:00
Alan Rominger
00ba1ea569 Suppress docker pull output in checks (#15323)
Supress docker pull output in checks
2024-07-03 15:04:59 -04:00
Alan Rominger
d91af132c1 Fix server error assigning teams EE object roles (#15320) 2024-07-03 14:07:03 -04:00
Seth Foster
94e5795dfc Prevent assigning credential to user of other org (#15296)
Utilizes the `validate_role_assignment` callback
from dab (see dab PR #490) to prevent granting credential
access to a user of another organization.

This logic will work for role_user_assignments
and role_team_assignments endpoints.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-07-02 21:05:22 +00:00
Alan Rominger
c4688d6298 Add in missing read permissions for organization audit role (#15318)
* Add in missing read permissions for organization audit role

* Add missing audit permission, special case name handling
2024-07-02 15:20:40 -04:00
TVo
6763badea3 Added new OpenShift Virtualization inventory source to docs. (#15299)
* Added new OpenShift Virtualization inventory source to docs.

* Incorporated review feedback from @fosterseth and @TheRealHaoLiu.

* Fixed link to correct kubevirt.core.kubevirt documentation.
2024-07-01 11:47:39 -06:00
Hao Liu
2c4ad6ef0f Add better 403 error message for Job template create (#15307)
* Add better 403 error message for Job template create

To create Job template u need access to projects and inventory

---------

Co-authored-by: Chris Meyers <chris.meyers.fsu@gmail.com>
2024-07-01 15:02:07 +00:00
Hao Liu
37f44d7214 Add better error message for wfjt create 403 (#15309) 2024-07-01 10:50:49 -04:00
Alan Rominger
98bbc836a6 Fix server error from DAB ValidationError with strings (#15312) 2024-07-01 10:11:22 -04:00
Alan Rominger
b59aff50dc Update ExecutionEnvironment model so object-level roles work with DAB RBAC system (#15289)
* Add initial test for deletion of stale permission

* Delete existing EE view permission

* Hypothetically complete update of EE model permissions setup

* Tests passing locally

* Issue with user_capabilities was a test bug, fixed
2024-06-28 16:09:42 -04:00
Alan Rominger
a70b0c1ddc Do not use cache in github image build action (#15308)
* Do not use cache in actual image build action

* Add cache args to kube prod builds
2024-06-28 09:52:59 -04:00
Alan Rominger
db72c9d5b8 Fix permissions that come from an external auditor role (#15291)
* Add tests for external auditor

* Add assertion for unified JTs which fails

* Fix UJT listing bug

* Add test for ad hoc commands just to be sure
2024-06-27 15:57:39 -04:00
jamesmarshall24
4e0d19914f LISTENER_DATABASES clobbers DATABASES OPTIONS (#15306)
Do not overwrite DATABASES OPTIONS with LISTENER_DATABASES
2024-06-27 13:26:30 -04:00
Hao Liu
6f2307f50e Add TASK_MANAGER_LOCK_TIMEOUT (#15300)
* Add TASK_MANAGER_LOCK_TIMEOUT

`TASK_MANAGER_LOCK_TIMEOUT` controls the `idle_in_transaction_session_timeout` and `idle_session_timeout` configuration for task manager connections and lock in database

hope to prevent the situation that the task instance that holds the lock becomes unresponsive and preventing other instance to be able to run task manager

* Add session timeout to periodic scheduler and all sub task manager locks
2024-06-27 09:42:41 -04:00
Alan Rominger
dbc2215bb6 Make attached user models adhere to new API assignments (#15298) 2024-06-26 23:00:25 -04:00
Hao Liu
7c08b29827 Temporary workaround for CI failure (#15305)
Workaround
```
ERROR awx/main/tests/functional/test_licenses.py - pip._vendor.distlib.DistlibException: Unable to locate finder for 'pip._vendor.distlib'
```
2024-06-26 15:29:22 -04:00
TVo
407194d320 Added troubleshooting and tips tricks content (#15212)
* Added troubleshooting and tips tricks content

* Added troubleshooting and tips tricks content

* Moved DNS host entry override info to customize pod spec section of CG chapter.

* Added troubleshooting and tips tricks content

* Moved DNS host entry override info to customize pod spec section of CG chapter.

* Update docs/docsite/rst/administration/containers_instance_groups.rst

Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>

* Update docs/docsite/rst/administration/containers_instance_groups.rst

Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>

* Update docs/docsite/rst/administration/containers_instance_groups.rst

Co-authored-by: Sandra McCann <samccann@redhat.com>

* Incorp'd review feedback from @fosterseth and @samccann

* Update docs/docsite/rst/administration/containers_instance_groups.rst

Co-authored-by: Sandra McCann <samccann@redhat.com>

* Final revisions based on @fosterseth's inputs.

---------

Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
2024-06-24 12:17:31 -06:00
Alan Rominger
853af295d9 Various RBAC fixes related to managed RoleDefinitions (#15287)
* Add migration testing for certain managed roles

* Fix managed role bugs

* Add more tests

* Fix another bug with org workflow admin role reference

* Add test because another issue is fixed

* Mark reason for test

* Remove internal markers

* Reword failure message

Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>

---------

Co-authored-by: Seth Foster <fosterseth@users.noreply.github.com>
2024-06-21 09:29:34 -04:00
Alan Rominger
4738c8333a Fix object-level permission bugs with DAB RBAC system (#15284)
* Fix object-level permission bugs with DAB RBAC system

* Fix NT organization change regression

* Mark tests to AAP number
2024-06-20 16:34:34 -04:00
Seth Foster
13dcea0afd Check for admin_role in role_check.py (#15283)
Script was falsely identifying cross-linked
parents. It needs to check if parent roles if
content type is Team and role_field is
member_role OR admin_role.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-06-20 14:04:04 -04:00
Chris Meyers
bc2d339981 Clarify the search for a proxy 2024-06-18 16:41:45 -04:00
Chris Meyers
bef9ef10bb Rename delete
* Include a bit of context into the name of the delete function. The
  HTTP_ added prepended string may be unexpected if Django's header
  transformation isn't top of mind.
2024-06-18 16:41:45 -04:00
Chris Meyers
8645fe5c57 Add support for x-trusted-proxy
* Increase the surface area of the set of headers that the proxy list
  feature looks at for the remote proxy IF x-trusted-proxy is valid.
2024-06-18 16:41:45 -04:00
Chris Meyers
b93aa20362 Revert "Trust proxy headers for host provision callback"
This reverts commit 49e3971cd577127705fc0fd1d3b4ab7e3a3c3c2b.
2024-06-18 16:41:45 -04:00
Chris Meyers
4bbfc8a946 Tests for trust proxy and existing explicit proxy
* Integration tests to ensure the integration of the two features.
2024-06-18 16:41:45 -04:00
Chris Meyers
2c8eef413b Trust proxy headers for host provision callback
* Do not remove special header list if request is from a trusted proxy.
* Continue to remove headers if request if from a non-trusted proxy.
2024-06-18 16:41:45 -04:00
Alan Rominger
d5bad1a533 Pass the Makefile python exe to ansible-playbook (#15282) 2024-06-18 13:03:01 -04:00
Alan Rominger
f6c0effcb2 Use public methods to reference registered models (#15277) 2024-06-17 11:45:44 -04:00
Chad Ferman
31a086b11a Add OpenShift Virtualization Inventory source option (#15047)
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
2024-06-14 13:38:37 -04:00
a_nackov
d94f766fcb Fix notification name search (#15231)
Signed-off-by: Adrian Nackov <adrian.nackov@mail.schwarz>
2024-06-13 14:49:54 +00:00
Viktor Varga
a7113549eb Add 'Terraform State' inventory source support for collection (#15258) 2024-06-12 19:22:21 +00:00
Jake Jackson
bfd811f408 Upgrade aiohttp for cve 2024-23829 (#15257) 2024-06-12 19:20:40 +00:00
Jeff Bradberry
030704a9e1 Change all uses of ImplicitRoleField to do on_delete=SET_NULL
This will mitigate the problem where if any Role gets deleted for some
weird reason it could previously cascade delete important objects.
2024-06-12 13:08:03 -04:00
Seth Foster
c312d9bce3 Rename setting to allow local resource management (#15269)
rename AWX_DIRECT_SHARED_RESOURCE_MANAGEMENT_ENABLED
to
ALLOW_LOCAL_RESOURCE_MANAGEMENT

- clearer meaning
- drop prefix so the same setting is used across the platform

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-06-11 12:50:18 -04:00
Jeff Bradberry
aadcc217eb This should deal correctly with the ancestor list mismatches 2024-06-10 16:36:22 -04:00
Jeff Bradberry
345c1c11e9 Guard against the role field not being populated
when doing the final reset of Role.implicit_parents.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
2c3a7fafc5 Add a new test scenario
to trigger the implicit parent not being in the parents and ancestors lists.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
dbcd32a1d9 Mark and rebuild the implicit_parents field for all affected roles 2024-06-10 16:36:22 -04:00
Jeff Bradberry
d45e258a78 Wait until the end of the fix script to clean up orphaned roles 2024-06-10 16:36:22 -04:00
Jeff Bradberry
d16b69a102 Add output of the update and deletion counts to fix.py 2024-06-10 16:36:22 -04:00
Jeff Bradberry
8b4efbc973 Do not throw away the container of cross-linked parents
Since we use it twice, the second time to get the id field of each.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
4cb061e7db Add a readme file with instructions 2024-06-10 16:36:22 -04:00
Jeff Bradberry
31db6a1447 Fix another instance where a bad resource->Role fk could throw a traceback 2024-06-10 16:36:22 -04:00
Jeff Bradberry
ad9d5904d8 Adjusted foreignkeys.sql for correctness
Some relationships known to be handled by the special mapping sql file
were being caught as false positives.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
b837d549ff Split the foreign key sql script into an 'into' and 'from' portion
Also, make use of up-front defined arrays of the tables involved, for
ease of editing in the future.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
9e22865d2e Filter out the relations within the known topology tables 2024-06-10 16:36:22 -04:00
Jeff Bradberry
ee3e3e1516 First cut at detecting which foreign keys enter and exit the topology tables 2024-06-10 16:36:22 -04:00
Jeff Bradberry
4a8f6e45f8 Move the "test" files into their own directory 2024-06-10 16:36:22 -04:00
Jeff Bradberry
6a317cca1b Remove the role_chain.py module
it wound up being unworkable, and I think ultimately we only need to
check the immediate parentage of each role.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
d67af79451 Attempt to correct any crosslinked parents
I think that rebuild_role_ancestor_list() will then correctly update
all of the affected Role.ancestors.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
fe77fda7b2 Exclude more files in the .gitignore 2024-06-10 16:36:22 -04:00
Jeff Bradberry
f613b76baa Modify the role parent check logic to stay in the roles as much as possible
since the foreign keys to the roles from the resources can make us go
wrong almost immediately.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
054cbe69d7 Exclude the team grant false positives
The results in my test now look correct.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
87e9dcb6d7 Attempt to more thoroughly check the parents of each Role
This version, however, has false positives because Roles become
children of Team.member_role when a Role is granted to a Team.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
c8829b057e First cut at checking the role hierarchy
Checking if parents and implicit_parents are consistent with ancestors.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
a0b376a6ca Set up a scenario where IG.use_role_id points to something no longer there
This is actually happening for one customer, though it seems like it
shouldn't be if the foreign key constraint is set back up properly.
In order to recreate it, I had to add the constraint back with 'NOT
VALID' added on to prevent the check.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
d675207f99 Handle the case where a resource points to a Role which isn't in the db 2024-06-10 16:36:22 -04:00
Jeff Bradberry
20504042c9 Graph out only the parent/child chains from a given Role
Doing the entire graph is too much on any system with real amounts of Roles.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
0e87e97820 Check for a broken ContentType -> model and log and skip
Apparently this has happened to a customer, per Nate Becker.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
1f154742df Make the role_chain.py script emit a Graphviz file
of the Role relationships.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
85fc81aab1 Start a new script that can be used to examine a Role's ancestry 2024-06-10 16:36:22 -04:00
Jeff Bradberry
5cfeeb3e87 Treat resources with null role fks differently
The underlying role should be re-linked, instead of treated as orphaned.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
a8c07b06d8 Set up an enhanced version of Seth's bad role scenario 2024-06-10 16:36:22 -04:00
Jeff Bradberry
53c5feaf6b Set up Seth's bad role scenario 2024-06-10 16:36:22 -04:00
Jeff Bradberry
6f57aaa8f5 When checking reverse links, treat duplicate Roles different from bad ones
Also, null out the generic foreign key on orphaned roles before deleting.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
bea74a401d Attempt to be more efficient about grouping the content types
Also, attempt to rebuild the role ancestors in the fixup script.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
54e85813c8 First full check script
This version emits the first fix-up script as its output.
2024-06-10 16:36:22 -04:00
Jeff Bradberry
b69ed08fe5 Specifically examine the InstanceGroup roles 2024-06-10 16:36:22 -04:00
Jeff Bradberry
de25408a23 Print out details of all of the crosslinked roles 2024-06-10 16:36:22 -04:00
Jeff Bradberry
b17f0a188b Initial check 2024-06-10 16:36:22 -04:00
Hao Liu
fb860d76ce Add receptor work list command to sosreport (#15207) 2024-06-10 19:39:24 +00:00
Artsiom Musin
451f20ce0f Use patch to update users in awx cli 2024-06-10 14:54:10 -04:00
Hao Liu
c1dc0c7b86 Periodically sync from share resource provider (#15264)
* Periodically sync from share resource provider

- add periodic task `periodic_resource_sync` run once every 15 min
- if `RESOURCE_SERVER` is not configured sync will not run
- only 1 node

example RESOURCE_SERVER configuration
```
RESOURCE_SERVER = {
    "URL": "<resource server url>",
    "SECRET_KEY": "<resource server auth token>",
    "VALIDATE_HTTPS": <True/False>,
}
RESOURCE_SERVICE_PATH = <resource_service_path>
```
2024-06-10 18:10:57 +00:00
Seth Foster
d65ea2a3d5 Fix race condition when deleting schedules (#15259)
If more than one schedule for a unified job template
is removed at once, a race condition can arise.

example scenario:  delete schedules with ids 7 and 8
- unified job template next_schedule is currently 7
- on delete of schedule 7, update_computed_fields will try to set
next_schedule to 8
- but while this logic is occurring, another transaction
is deleting 8

This leads to a db IntegrityError

The solution here is to call select_for_update() on the
next schedule, so that 8 cannot be deleted until
the transaction for deleting 7 is completed.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-06-09 20:39:18 -04:00
Dave
8827ae7554 Replace REMOTE_ADDR with ansible_base.lib.utils.requests.get_remote_host (#15175) 2024-06-06 14:47:04 +01:00
Jeff Bradberry
4915262af1 Do each batch of the HostMetric updates in a transaction
It looks like we can't do upserts currently without dropping to raw
SQL, but if we wrap each batch in a transaction, that should insure
that each is updated with the correct count.
2024-06-05 14:15:21 -04:00
Seth Foster
d43c91e1a5 Option for dev env to enable ssl for postgres (#15151)
PG_TLS=true make docker-compose

This will add some extra startup commands
for the postgres container to generate a key and
cert to use for postgres connections.
It will also mount in pgssl.conf which has ssl configuration.

This can be useful for debugging issues that only surface
when using ssl postgres connections.
2024-06-05 12:48:08 -04:00
Seth Foster
b470ca32af Prevent modifying shared resources when using platform ingress (#15234)
* Prevent modifying shared resources

Adds a class decorator to prevent modifying shared resources
when gateway is being used.

AWX_DIRECT_SHARED_RESOURCE_MANAGEMENT_ENABLED is the setting
to enable/disable this feature.

Works by overriding these view methods:
- create
- delete
- perform_update

create and delete are overridden to raise a
PermissionDenied exception.

perform_update is overridden to check if any shared
fields are being modified, and raise a PermissionDenied
exception if so.

Additional changes:

Prevent sso conf from registering external authentication related settings if
AWX_DIRECT_SHARED_RESOURCE_MANAGEMENT_ENABLED is False

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
2024-06-05 12:44:01 -04:00
Satoe Imaishi
793777bec7 Add cython to VENV_BOOTSTRAP for grpcio (#15256) 2024-06-05 11:04:15 -04:00
2007 changed files with 4604 additions and 344996 deletions

View File

@@ -24,7 +24,7 @@ runs:
- name: Pre-pull latest devel image to warm cache
shell: bash
run: docker pull ghcr.io/${OWNER_LC}/awx_devel:${{ github.base_ref }}
run: docker pull -q ghcr.io/${OWNER_LC}/awx_devel:${{ github.base_ref }}
- name: Build image for current source checkout
shell: bash

View File

@@ -57,16 +57,6 @@ runs:
awx-manage update_password --username=admin --password=password
EOSH
- name: Build UI
# This must be a string comparison in composite actions:
# https://github.com/actions/runner/issues/2238
if: ${{ inputs.build-ui == 'true' }}
shell: bash
run: |
docker exec -i tools_awx_1 sh <<-EOSH
make ui-devel
EOSH
- name: Get instance data
id: data
shell: bash

View File

@@ -6,8 +6,6 @@ needs_triage:
- "Feature Summary"
"component:ui":
- "\\[X\\] UI"
"component:ui_next":
- "\\[X\\] UI \\(tech preview\\)"
"component:api":
- "\\[X\\] API"
"component:docs":

View File

@@ -1,8 +1,5 @@
"component:api":
- any: ["awx/**/*", "!awx/ui/**"]
"component:ui":
- any: ["awx/ui/**/*"]
- any: ["awx/**/*"]
"component:docs":
- any: ["docs/**/*"]
@@ -14,5 +11,4 @@
- any: ["awx_collection/**/*"]
"dependencies":
- any: ["awx/ui/package.json"]
- any: ["requirements/*"]

View File

@@ -1,7 +1,7 @@
## General
- For the roundup of all the different mailing lists available from AWX, Ansible, and beyond visit: https://docs.ansible.com/ansible/latest/community/communication.html
- Hello, we think your question is answered in our FAQ. Does this: https://www.ansible.com/products/awx-project/faq cover your question?
- You can find the latest documentation here: https://docs.ansible.com/automation-controller/latest/html/userguide/index.html
- You can find the latest documentation here: https://ansible.readthedocs.io/projects/awx/en/latest/userguide/index.html

View File

@@ -31,14 +31,11 @@ jobs:
command: /start_tests.sh test_collection_all
- name: api-schema
command: /start_tests.sh detect-schema-change SCHEMA_DIFF_BASE_BRANCH=${{ github.event.pull_request.base.ref }}
- name: ui-lint
command: make ui-lint
- name: ui-test-screens
command: make ui-test-screens
- name: ui-test-general
command: make ui-test-general
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
- name: Build awx_devel image for running checks
uses: ./.github/actions/awx_devel_image
@@ -52,7 +49,9 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
- uses: ./.github/actions/run_awx_devel
id: awx
@@ -70,13 +69,15 @@ jobs:
DEBUG_OUTPUT_DIR: /tmp/awx_operator_molecule_test
steps:
- name: Checkout awx
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
show-progress: false
path: awx
- name: Checkout awx-operator
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
show-progress: false\
repository: ansible/awx-operator
path: awx-operator
@@ -130,7 +131,9 @@ jobs:
strategy:
fail-fast: false
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
# The containers that GitHub Actions use have Ansible installed, so upgrade to make sure we have the latest version.
- name: Upgrade ansible-core
@@ -154,7 +157,9 @@ jobs:
- name: r-z0-9
regex: ^[r-z0-9]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
- uses: ./.github/actions/run_awx_devel
id: awx
@@ -200,7 +205,9 @@ jobs:
strategy:
fail-fast: false
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
- name: Upgrade ansible-core
run: python3 -m pip install --upgrade ansible-core

57
.github/workflows/dab-release.yml vendored Normal file
View File

@@ -0,0 +1,57 @@
---
name: django-ansible-base requirements update
on:
workflow_dispatch:
schedule:
- cron: '0 6 * * *' # once an day @ 6 AM
permissions:
pull-requests: write
contents: write
jobs:
dab-pin-newest:
if: (github.repository_owner == 'ansible' && endsWith(github.repository, 'awx')) || github.event_name != 'schedule'
runs-on: ubuntu-latest
steps:
- id: dab-release
name: Get current django-ansible-base release version
uses: pozetroninc/github-action-get-latest-release@2a61c339ea7ef0a336d1daa35ef0cb1418e7676c # v0.8.0
with:
owner: ansible
repo: django-ansible-base
excludes: prerelease, draft
- name: Check out respository code
uses: actions/checkout@v4
- id: dab-pinned
name: Get current django-ansible-base pinned version
run:
echo "version=$(requirements/django-ansible-base-pinned-version.sh)" >> "$GITHUB_OUTPUT"
- name: Update django-ansible-base pinned version to upstream release
run:
requirements/django-ansible-base-pinned-version.sh -s ${{ steps.dab-release.outputs.release }}
- name: Create Pull Request
uses: peter-evans/create-pull-request@c5a7806660adbe173f04e3e038b0ccdcd758773c # v6
with:
base: devel
branch: bump-django-ansible-base
title: Bump django-ansible-base to ${{ steps.dab-release.outputs.release }}
body: |
##### SUMMARY
Automated .github/workflows/dab-release.yml
django-ansible-base upstream released version == ${{ steps.dab-release.outputs.release }}
requirements_git.txt django-ansible-base pinned version == ${{ steps.dab-pinned.outputs.version }}
##### ISSUE TYPE
- Bug, Docs Fix or other nominal change
##### COMPONENT NAME
- API
commit-message: |
Update django-ansible-base version to ${{ steps.dab-pinned.outputs.version }}
add-paths:
requirements/requirements_git.txt

View File

@@ -2,6 +2,7 @@
name: Build/Push Development Images
env:
LC_ALL: "C.UTF-8" # prevent ERROR: Ansible could not initialize the preferred locale: unsupported locale setting
DOCKER_CACHE: "--no-cache" # using the cache will not rebuild git requirements and other things
on:
workflow_dispatch:
push:
@@ -34,7 +35,9 @@ jobs:
exit 0
if: matrix.build-targets.image-name == 'awx' && !endsWith(github.repository, '/awx')
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -59,17 +62,15 @@ jobs:
run: |
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
- name: Setup node and npm
- name: Setup node and npm for the new UI build
uses: actions/setup-node@v2
with:
node-version: '16.13.1'
node-version: '18'
if: matrix.build-targets.image-name == 'awx'
- name: Prebuild UI for awx image (to speed up build process)
- name: Prebuild new UI for awx image (to speed up build process)
run: |
sudo apt-get install gettext
make ui-release
make ui-next
make ui
if: matrix.build-targets.image-name == 'awx'
- name: Build and push AWX devel images

View File

@@ -8,7 +8,9 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
- name: install tox
run: pip install tox

View File

@@ -30,7 +30,10 @@ jobs:
timeout-minutes: 20
name: Label Issue - Community
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
- uses: actions/setup-python@v4
- name: Install python requests
run: pip install requests

View File

@@ -29,7 +29,10 @@ jobs:
timeout-minutes: 20
name: Label PR - Community
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
- uses: actions/setup-python@v4
- name: Install python requests
run: pip install requests

View File

@@ -32,7 +32,9 @@ jobs:
echo "TAG_NAME=${{ github.event.release.tag_name }}" >> $GITHUB_ENV
- name: Checkout awx
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
show-progress: false
- name: Get python version from Makefile
run: echo py_version=`make PYTHON_VERSION` >> $GITHUB_ENV

View File

@@ -45,19 +45,22 @@ jobs:
exit 0
- name: Checkout awx
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
show-progress: false
path: awx
- name: Checkout awx-operator
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
show-progress: false
repository: ${{ github.repository_owner }}/awx-operator
path: awx-operator
- name: Checkout awx-logos
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
show-progress: false
repository: ansible/awx-logos
path: awx-logos
@@ -86,17 +89,14 @@ jobs:
run: |
cp ../awx-logos/awx/ui/client/assets/* awx/ui/public/static/media/
- name: Setup node and npm
- name: Setup node and npm for new UI build
uses: actions/setup-node@v2
with:
node-version: '16.13.1'
node-version: '18'
- name: Prebuild UI for awx image (to speed up build process)
- name: Prebuild new UI for awx image (to speed up build process)
working-directory: awx
run: |
sudo apt-get install gettext
make ui-release
make ui-next
run: make ui
- name: Set build env variables
run: |
@@ -136,9 +136,9 @@ jobs:
- name: Pulling images for test deployment with awx-operator
# awx operator molecue test expect to kind load image and buildx exports image to registry and not local
run: |
docker pull ${AWX_OPERATOR_TEST_IMAGE}
docker pull ${AWX_EE_TEST_IMAGE}
docker pull ${AWX_TEST_IMAGE}:${AWX_TEST_VERSION}
docker pull -q ${AWX_OPERATOR_TEST_IMAGE}
docker pull -q ${AWX_EE_TEST_IMAGE}
docker pull -q ${AWX_TEST_IMAGE}:${AWX_TEST_VERSION}
- name: Run test deployment with awx-operator
working-directory: awx-operator

View File

@@ -13,7 +13,9 @@ jobs:
steps:
- name: Checkout branch
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
show-progress: false
- name: Update PR Body
env:

View File

@@ -18,7 +18,9 @@ jobs:
packages: write
contents: read
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
show-progress: false
- name: Get python version from Makefile
run: echo py_version=`make PYTHON_VERSION` >> $GITHUB_ENV
@@ -34,7 +36,7 @@ jobs:
- name: Pre-pull image to warm build cache
run: |
docker pull ghcr.io/${{ github.repository_owner }}/awx_devel:${GITHUB_REF##*/} || :
docker pull -q ghcr.io/${{ github.repository_owner }}/awx_devel:${GITHUB_REF##*/} || :
- name: Build image
run: |

23
.gitignore vendored
View File

@@ -20,23 +20,10 @@ awx/projects
awx/job_output
awx/public/media
awx/public/static
awx/ui/tests/test-results.xml
awx/ui/client/src/local_settings.json
awx/main/fixtures
awx/*.log
tower/tower_warnings.log
celerybeat-schedule
awx/ui/static
awx/ui/build_test
awx/ui/client/languages
awx/ui/templates/ui/index.html
awx/ui/templates/ui/installing.html
awx/ui/node_modules/
awx/ui/src/locales/*/messages.js
awx/ui/coverage/
awx/ui/build
awx/ui/.env.local
awx/ui/instrumented
rsyslog.pid
tools/docker-compose/ansible/awx_dump.sql
tools/docker-compose/Dockerfile
@@ -79,11 +66,6 @@ __pycache__
/tmp
**/npm-debug.log*
# UI build flag files
awx/ui/.deps_built
awx/ui/.release_built
awx/ui/.release_deps_built
# Testing
.cache
.coverage
@@ -161,15 +143,14 @@ use_dev_supervisor.txt
.idea/*
*.unison.tmp
*.#
/awx/ui/.ui-built
/_build/
/_build_kube_dev/
/Dockerfile
/Dockerfile.dev
/Dockerfile.kube-dev
awx/ui_next/src
awx/ui_next/build
awx/ui/src
awx/ui/build
# Docs build stuff
docs/docsite/build/

View File

@@ -5,8 +5,6 @@ ignore: |
awx/main/tests/data/inventory/plugins/**
# vault files
awx/main/tests/data/ansible_utils/playbooks/valid/vault.yml
awx/ui/test/e2e/tests/smoke-vars.yml
awx/ui/node_modules
tools/docker-compose/_sources
# django template files
awx/api/templates/instance_install_bundle/**

View File

@@ -67,7 +67,7 @@ If you're not using Docker for Mac, or Docker for Windows, you may need, or choo
#### Frontend Development
See [the ui development documentation](awx/ui/CONTRIBUTING.md).
See [the ansible-ui development documentation](https://github.com/ansible/ansible-ui/blob/main/CONTRIBUTING.md).
#### Fork and clone the AWX repo
@@ -121,18 +121,18 @@ If it has someone assigned to it then that person is the person responsible for
**NOTES**
> Issue assignment will only be done for maintainers of the project. If you decide to work on an issue, please feel free to add a comment in the issue to let others know that you are working on it; but know that we will accept the first pull request from whomever is able to fix an issue. Once your PR is accepted we can add you as an assignee to an issue upon request.
> Issue assignment will only be done for maintainers of the project. If you decide to work on an issue, please feel free to add a comment in the issue to let others know that you are working on it; but know that we will accept the first pull request from whomever is able to fix an issue. Once your PR is accepted we can add you as an assignee to an issue upon request.
> If you work in a part of the codebase that is going through active development, your changes may be rejected, or you may be asked to `rebase`. A good idea before starting work is to have a discussion with us in the `#ansible-awx` channel on irc.libera.chat, or on the [mailing list](https://groups.google.com/forum/#!forum/awx-project).
> If you're planning to develop features or fixes for the UI, please review the [UI Developer doc](./awx/ui/README.md).
> If you're planning to develop features or fixes for the UI, please review the [UI Developer doc](https://github.com/ansible/ansible-ui/blob/main/CONTRIBUTING.md).
### Translations
At this time we do not accept PRs for adding additional language translations as we have an automated process for generating our translations. This is because translations require constant care as new strings are added and changed in the code base. Because of this the .po files are overwritten during every translation release cycle. We also can't support a lot of translations on AWX as its an open source project and each language adds time and cost to maintain. If you would like to see AWX translated into a new language please create an issue and ask others you know to upvote the issue. Our translation team will review the needs of the community and see what they can do around supporting additional language.
If you find an issue with an existing translation, please see the [Reporting Issues](#reporting-issues) section to open an issue and our translation team will work with you on a resolution.
If you find an issue with an existing translation, please see the [Reporting Issues](#reporting-issues) section to open an issue and our translation team will work with you on a resolution.
## Submitting Pull Requests
@@ -143,10 +143,8 @@ Here are a few things you can do to help the visibility of your change, and incr
- No issues when running linters/code checkers
- Python: black: `(container)/awx_devel$ make black`
- Javascript: `(container)/awx_devel$ make ui-lint`
- No issues from unit tests
- Python: py.test: `(container)/awx_devel$ make test`
- JavaScript: `(container)/awx_devel$ make ui-test`
- Write tests for new functionality, update/add tests for bug fixes
- Make the smallest change possible
- Write good commit messages. See [How to write a Git commit message](https://chris.beams.io/posts/git-commit/).
@@ -161,7 +159,7 @@ Sometimes it might take us a while to fully review your PR. We try to keep the `
When your PR is initially submitted the checks will not be run until a maintainer allows them to be. Once a maintainer has done a quick review of your work the PR will have the linter and unit tests run against them via GitHub Actions, and the status reported in the PR.
## Reporting Issues
We welcome your feedback, and encourage you to file an issue when you run into a problem. But before opening a new issues, we ask that you please view our [Issues guide](./ISSUES.md).
## Getting Help

View File

@@ -4,9 +4,7 @@ recursive-include awx *.mo
recursive-include awx/static *
recursive-include awx/templates *.html
recursive-include awx/api/templates *.md *.html *.yml
recursive-include awx/ui/build *.html
recursive-include awx/ui/build *
recursive-include awx/ui_next/build *
recursive-include awx/playbooks *.yml
recursive-include awx/lib/site-packages *
recursive-include awx/plugins *.ps1
@@ -17,7 +15,6 @@ recursive-include licenses *
recursive-exclude awx devonly.py*
recursive-exclude awx/api/tests *
recursive-exclude awx/main/tests *
recursive-exclude awx/ui/client *
recursive-exclude awx/settings local_settings.py*
include tools/scripts/request_tower_configuration.sh
include tools/scripts/request_tower_configuration.ps1

147
Makefile
View File

@@ -1,4 +1,4 @@
-include awx/ui_next/Makefile
-include awx/ui/Makefile
PYTHON := $(notdir $(shell for i in python3.11 python3; do command -v $$i; done|sed 1q))
SHELL := bash
@@ -53,6 +53,8 @@ OTEL ?= false
LOKI ?= false
# If set to true docker-compose will install editable dependencies
EDITABLE_DEPENDENCIES ?= false
# If set to true, use tls for postgres connection
PG_TLS ?= false
VENV_BASE ?= /var/lib/awx/venv
@@ -61,6 +63,11 @@ DEV_DOCKER_OWNER ?= ansible
DEV_DOCKER_OWNER_LOWER = $(shell echo $(DEV_DOCKER_OWNER) | tr A-Z a-z)
DEV_DOCKER_TAG_BASE ?= ghcr.io/$(DEV_DOCKER_OWNER_LOWER)
DEVEL_IMAGE_NAME ?= $(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG)
IMAGE_KUBE_DEV=$(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG)
IMAGE_KUBE=$(DEV_DOCKER_TAG_BASE)/awx:$(COMPOSE_TAG)
# Common command to use for running ansible-playbook
ANSIBLE_PLAYBOOK ?= ansible-playbook -e ansible_python_interpreter=$(PYTHON)
RECEPTOR_IMAGE ?= quay.io/ansible/receptor:devel
@@ -69,7 +76,7 @@ RECEPTOR_IMAGE ?= quay.io/ansible/receptor:devel
SRC_ONLY_PKGS ?= cffi,pycparser,psycopg,twilio
# These should be upgraded in the AWX and Ansible venv before attempting
# to install the actual requirements
VENV_BOOTSTRAP ?= pip==21.2.4 setuptools==69.0.2 setuptools_scm[toml]==8.0.4 wheel==0.42.0
VENV_BOOTSTRAP ?= pip==21.2.4 setuptools==69.0.2 setuptools_scm[toml]==8.0.4 wheel==0.42.0 cython==0.29.37
NAME ?= awx
@@ -84,11 +91,22 @@ I18N_FLAG_FILE = .i18n_built
## PLATFORMS defines the target platforms for the manager image be build to provide support to multiple
PLATFORMS ?= linux/amd64,linux/arm64 # linux/ppc64le,linux/s390x
# Set up cache variables for image builds, allowing to control whether cache is used or not, ex:
# DOCKER_CACHE=--no-cache make docker-compose-build
ifeq ($(DOCKER_CACHE),)
DOCKER_DEVEL_CACHE_FLAG=--cache-from=$(DEVEL_IMAGE_NAME)
DOCKER_KUBE_DEV_CACHE_FLAG=--cache-from=$(IMAGE_KUBE_DEV)
DOCKER_KUBE_CACHE_FLAG=--cache-from=$(IMAGE_KUBE)
else
DOCKER_DEVEL_CACHE_FLAG=$(DOCKER_CACHE)
DOCKER_KUBE_DEV_CACHE_FLAG=$(DOCKER_CACHE)
DOCKER_KUBE_CACHE_FLAG=$(DOCKER_CACHE)
endif
.PHONY: awx-link clean clean-tmp clean-venv requirements requirements_dev \
develop refresh adduser migrate dbchange \
receiver test test_unit test_coverage coverage_html \
sdist \
ui-release ui-devel \
VERSION PYTHON_VERSION docker-compose-sources \
.git/hooks/pre-commit
@@ -111,7 +129,7 @@ clean-languages:
find ./awx/locale/ -type f -regex '.*\.mo$$' -delete
## Remove temporary build files, compiled Python files.
clean: clean-ui clean-api clean-awxkit clean-dist
clean: clean-api clean-awxkit clean-dist
rm -rf awx/public
rm -rf awx/lib/site-packages
rm -rf awx/job_status
@@ -366,7 +384,7 @@ symlink_collection:
ln -s $(shell pwd)/awx_collection $(COLLECTION_INSTALL)
awx_collection_build: $(shell find awx_collection -type f)
ansible-playbook -i localhost, awx_collection/tools/template_galaxy.yml \
$(ANSIBLE_PLAYBOOK) -i localhost, awx_collection/tools/template_galaxy.yml \
-e collection_package=$(COLLECTION_PACKAGE) \
-e collection_namespace=$(COLLECTION_NAMESPACE) \
-e collection_version=$(COLLECTION_VERSION) \
@@ -420,76 +438,7 @@ bulk_data:
fi; \
$(PYTHON) tools/data_generators/rbac_dummy_data_generator.py --preset=$(DATA_GEN_PRESET)
# UI TASKS
# --------------------------------------
UI_BUILD_FLAG_FILE = awx/ui/.ui-built
clean-ui:
rm -rf node_modules
rm -rf awx/ui/node_modules
rm -rf awx/ui/build
rm -rf awx/ui/src/locales/_build
rm -rf $(UI_BUILD_FLAG_FILE)
# the collectstatic command doesn't like it if this dir doesn't exist.
mkdir -p awx/ui/build/static
awx/ui/node_modules:
NODE_OPTIONS=--max-old-space-size=6144 $(NPM_BIN) --prefix awx/ui --loglevel warn --force ci
$(UI_BUILD_FLAG_FILE):
$(MAKE) awx/ui/node_modules
$(PYTHON) tools/scripts/compilemessages.py
$(NPM_BIN) --prefix awx/ui --loglevel warn run compile-strings
$(NPM_BIN) --prefix awx/ui --loglevel warn run build
touch $@
ui-release: $(UI_BUILD_FLAG_FILE)
ui-devel: awx/ui/node_modules
@$(MAKE) -B $(UI_BUILD_FLAG_FILE)
@if [ -d "/var/lib/awx" ] ; then \
mkdir -p /var/lib/awx/public/static/css; \
mkdir -p /var/lib/awx/public/static/js; \
mkdir -p /var/lib/awx/public/static/media; \
cp -r awx/ui/build/static/css/* /var/lib/awx/public/static/css; \
cp -r awx/ui/build/static/js/* /var/lib/awx/public/static/js; \
cp -r awx/ui/build/static/media/* /var/lib/awx/public/static/media; \
fi
ui-devel-instrumented: awx/ui/node_modules
$(NPM_BIN) --prefix awx/ui --loglevel warn run start-instrumented
ui-devel-test: awx/ui/node_modules
$(NPM_BIN) --prefix awx/ui --loglevel warn run start
ui-lint:
$(NPM_BIN) --prefix awx/ui install
$(NPM_BIN) run --prefix awx/ui lint
$(NPM_BIN) run --prefix awx/ui prettier-check
ui-test:
$(NPM_BIN) --prefix awx/ui install
$(NPM_BIN) run --prefix awx/ui test
ui-test-screens:
$(NPM_BIN) --prefix awx/ui install
$(NPM_BIN) run --prefix awx/ui pretest
$(NPM_BIN) run --prefix awx/ui test-screens --runInBand
ui-test-general:
$(NPM_BIN) --prefix awx/ui install
$(NPM_BIN) run --prefix awx/ui pretest
$(NPM_BIN) run --prefix awx/ui/ test-general --runInBand
# NOTE: The make target ui-next is imported from awx/ui_next/Makefile
HEADLESS ?= no
ifeq ($(HEADLESS), yes)
dist/$(SDIST_TAR_FILE):
else
dist/$(SDIST_TAR_FILE): $(UI_BUILD_FLAG_FILE) ui-next
endif
$(PYTHON) -m build -s
ln -sf $(SDIST_TAR_FILE) dist/awx.tar.gz
@@ -520,10 +469,10 @@ endif
docker-compose-sources: .git/hooks/pre-commit
@if [ $(MINIKUBE_CONTAINER_GROUP) = true ]; then\
ansible-playbook -i tools/docker-compose/inventory -e minikube_setup=$(MINIKUBE_SETUP) tools/docker-compose-minikube/deploy.yml; \
$(ANSIBLE_PLAYBOOK) -i tools/docker-compose/inventory -e minikube_setup=$(MINIKUBE_SETUP) tools/docker-compose-minikube/deploy.yml; \
fi;
ansible-playbook -i tools/docker-compose/inventory tools/docker-compose/ansible/sources.yml \
$(ANSIBLE_PLAYBOOK) -i tools/docker-compose/inventory tools/docker-compose/ansible/sources.yml \
-e awx_image=$(DEV_DOCKER_TAG_BASE)/awx_devel \
-e awx_image_tag=$(COMPOSE_TAG) \
-e receptor_image=$(RECEPTOR_IMAGE) \
@@ -542,11 +491,12 @@ docker-compose-sources: .git/hooks/pre-commit
-e enable_otel=$(OTEL) \
-e enable_loki=$(LOKI) \
-e install_editable_dependencies=$(EDITABLE_DEPENDENCIES) \
-e pg_tls=$(PG_TLS) \
$(EXTRA_SOURCES_ANSIBLE_OPTS)
docker-compose: awx/projects docker-compose-sources
ansible-galaxy install --ignore-certs -r tools/docker-compose/ansible/requirements.yml;
ansible-playbook -i tools/docker-compose/inventory tools/docker-compose/ansible/initialize_containers.yml \
$(ANSIBLE_PLAYBOOK) -i tools/docker-compose/inventory tools/docker-compose/ansible/initialize_containers.yml \
-e enable_vault=$(VAULT) \
-e vault_tls=$(VAULT_TLS) \
-e enable_ldap=$(LDAP); \
@@ -589,7 +539,7 @@ docker-compose-container-group-clean:
.PHONY: Dockerfile.dev
## Generate Dockerfile.dev for awx_devel image
Dockerfile.dev: tools/ansible/roles/dockerfile/templates/Dockerfile.j2
ansible-playbook tools/ansible/dockerfile.yml \
$(ANSIBLE_PLAYBOOK) tools/ansible/dockerfile.yml \
-e dockerfile_name=Dockerfile.dev \
-e build_dev=True \
-e receptor_image=$(RECEPTOR_IMAGE)
@@ -600,8 +550,7 @@ docker-compose-build: Dockerfile.dev
-f Dockerfile.dev \
-t $(DEVEL_IMAGE_NAME) \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG) .
$(DOCKER_DEVEL_CACHE_FLAG) .
.PHONY: docker-compose-buildx
## Build awx_devel image for docker compose development environment for multiple architectures
@@ -611,7 +560,7 @@ docker-compose-buildx: Dockerfile.dev
- docker buildx build \
--push \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG) \
$(DOCKER_DEVEL_CACHE_FLAG) \
--platform=$(PLATFORMS) \
--tag $(DEVEL_IMAGE_NAME) \
-f Dockerfile.dev .
@@ -664,7 +613,7 @@ version-for-buildyml:
.PHONY: Dockerfile
## Generate Dockerfile for awx image
Dockerfile: tools/ansible/roles/dockerfile/templates/Dockerfile.j2
ansible-playbook tools/ansible/dockerfile.yml \
$(ANSIBLE_PLAYBOOK) tools/ansible/dockerfile.yml \
-e receptor_image=$(RECEPTOR_IMAGE) \
-e headless=$(HEADLESS)
@@ -674,7 +623,8 @@ awx-kube-build: Dockerfile
--build-arg VERSION=$(VERSION) \
--build-arg SETUPTOOLS_SCM_PRETEND_VERSION=$(VERSION) \
--build-arg HEADLESS=$(HEADLESS) \
-t $(DEV_DOCKER_TAG_BASE)/awx:$(COMPOSE_TAG) .
$(DOCKER_KUBE_CACHE_FLAG) \
-t $(IMAGE_KUBE) .
## Build multi-arch awx image for deployment on Kubernetes environment.
awx-kube-buildx: Dockerfile
@@ -686,7 +636,8 @@ awx-kube-buildx: Dockerfile
--build-arg SETUPTOOLS_SCM_PRETEND_VERSION=$(VERSION) \
--build-arg HEADLESS=$(HEADLESS) \
--platform=$(PLATFORMS) \
--tag $(DEV_DOCKER_TAG_BASE)/awx:$(COMPOSE_TAG) \
$(DOCKER_KUBE_CACHE_FLAG) \
--tag $(IMAGE_KUBE) \
-f Dockerfile .
- docker buildx rm awx-kube-buildx
@@ -694,7 +645,7 @@ awx-kube-buildx: Dockerfile
.PHONY: Dockerfile.kube-dev
## Generate Docker.kube-dev for awx_kube_devel image
Dockerfile.kube-dev: tools/ansible/roles/dockerfile/templates/Dockerfile.j2
ansible-playbook tools/ansible/dockerfile.yml \
$(ANSIBLE_PLAYBOOK) tools/ansible/dockerfile.yml \
-e dockerfile_name=Dockerfile.kube-dev \
-e kube_dev=True \
-e template_dest=_build_kube_dev \
@@ -704,8 +655,8 @@ Dockerfile.kube-dev: tools/ansible/roles/dockerfile/templates/Dockerfile.j2
awx-kube-dev-build: Dockerfile.kube-dev
DOCKER_BUILDKIT=1 docker build -f Dockerfile.kube-dev \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG) \
-t $(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG) .
$(DOCKER_KUBE_DEV_CACHE_FLAG) \
-t $(IMAGE_KUBE_DEV) .
## Build and push multi-arch awx_kube_devel image for development on local Kubernetes environment.
awx-kube-dev-buildx: Dockerfile.kube-dev
@@ -714,28 +665,18 @@ awx-kube-dev-buildx: Dockerfile.kube-dev
- docker buildx build \
--push \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--cache-from=$(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG) \
$(DOCKER_KUBE_DEV_CACHE_FLAG) \
--platform=$(PLATFORMS) \
--tag $(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG) \
--tag $(IMAGE_KUBE_DEV) \
-f Dockerfile.kube-dev .
- docker buildx rm awx-kube-dev-buildx
kind-dev-load: awx-kube-dev-build
$(KIND_BIN) load docker-image $(DEV_DOCKER_TAG_BASE)/awx_kube_devel:$(COMPOSE_TAG)
$(KIND_BIN) load docker-image $(IMAGE_KUBE_DEV)
# Translation TASKS
# --------------------------------------
## generate UI .pot file, an empty template of strings yet to be translated
pot: $(UI_BUILD_FLAG_FILE)
$(NPM_BIN) --prefix awx/ui --loglevel warn run extract-template --clean
$(NPM_BIN) --prefix awx/ui_next --loglevel warn run extract-template --clean
## generate UI .po files for each locale (will update translated strings for `en`)
po: $(UI_BUILD_FLAG_FILE)
$(NPM_BIN) --prefix awx/ui --loglevel warn run extract-strings -- --clean
$(NPM_BIN) --prefix awx/ui_next --loglevel warn run extract-strings -- --clean
## generate API django .pot .po
messages:
@if [ "$(VENV_BASE)" ]; then \
@@ -782,6 +723,6 @@ help/generate:
{ lastLine = $$0 }' $(MAKEFILE_LIST) | sort -u
@printf "\n"
## Display help for ui-next targets
help/ui-next:
@$(MAKE) -s help MAKEFILE_LIST="awx/ui_next/Makefile"
## Display help for ui targets
help/ui:
@$(MAKE) -s help MAKEFILE_LIST="awx/ui/Makefile"

View File

@@ -33,8 +33,10 @@ from rest_framework.negotiation import DefaultContentNegotiation
# django-ansible-base
from ansible_base.rest_filters.rest_framework.field_lookup_backend import FieldLookupBackend
from ansible_base.lib.utils.models import get_all_field_names
from ansible_base.lib.utils.requests import get_remote_host
from ansible_base.rbac.models import RoleEvaluation, RoleDefinition
from ansible_base.rbac.permission_registry import permission_registry
from ansible_base.jwt_consumer.common.util import validate_x_trusted_proxy_header
# AWX
from awx.main.models import UnifiedJob, UnifiedJobTemplate, User, Role, Credential, WorkflowJobTemplateNode, WorkflowApprovalTemplate
@@ -42,6 +44,7 @@ from awx.main.models.rbac import give_creator_permissions
from awx.main.access import optimize_queryset
from awx.main.utils import camelcase_to_underscore, get_search_fields, getattrd, get_object_or_400, decrypt_field, get_awx_version
from awx.main.utils.licensing import server_product_name
from awx.main.utils.proxy import is_proxy_in_headers, delete_headers_starting_with_http
from awx.main.views import ApiErrorView
from awx.api.serializers import ResourceAccessListElementSerializer, CopySerializer
from awx.api.versioning import URLPathVersioning
@@ -93,8 +96,9 @@ class LoggedLoginView(auth_views.LoginView):
def post(self, request, *args, **kwargs):
ret = super(LoggedLoginView, self).post(request, *args, **kwargs)
ip = get_remote_host(request) # request.META.get('REMOTE_ADDR', None)
if request.user.is_authenticated:
logger.info(smart_str(u"User {} logged in from {}".format(self.request.user.username, request.META.get('REMOTE_ADDR', None))))
logger.info(smart_str(u"User {} logged in from {}".format(self.request.user.username, ip)))
ret.set_cookie(
'userLoggedIn', 'true', secure=getattr(settings, 'SESSION_COOKIE_SECURE', False), samesite=getattr(settings, 'USER_COOKIE_SAMESITE', 'Lax')
)
@@ -103,7 +107,7 @@ class LoggedLoginView(auth_views.LoginView):
return ret
else:
if 'username' in self.request.POST:
logger.warning(smart_str(u"Login failed for user {} from {}".format(self.request.POST.get('username'), request.META.get('REMOTE_ADDR', None))))
logger.warning(smart_str(u"Login failed for user {} from {}".format(self.request.POST.get('username'), ip)))
ret.status_code = 401
return ret
@@ -151,22 +155,23 @@ class APIView(views.APIView):
Store the Django REST Framework Request object as an attribute on the
normal Django request, store time the request started.
"""
remote_headers = ['REMOTE_ADDR', 'REMOTE_HOST']
self.time_started = time.time()
if getattr(settings, 'SQL_DEBUG', False):
self.queries_before = len(connection.queries)
if 'HTTP_X_TRUSTED_PROXY' in request.environ:
if validate_x_trusted_proxy_header(request.environ['HTTP_X_TRUSTED_PROXY']):
remote_headers = settings.REMOTE_HOST_HEADERS
else:
logger.warning("Request appeared to be a trusted upstream proxy but failed to provide a matching shared secret.")
# If there are any custom headers in REMOTE_HOST_HEADERS, make sure
# they respect the allowed proxy list
if all(
[
settings.PROXY_IP_ALLOWED_LIST,
request.environ.get('REMOTE_ADDR') not in settings.PROXY_IP_ALLOWED_LIST,
request.environ.get('REMOTE_HOST') not in settings.PROXY_IP_ALLOWED_LIST,
]
):
for custom_header in settings.REMOTE_HOST_HEADERS:
if custom_header.startswith('HTTP_'):
request.environ.pop(custom_header, None)
if settings.PROXY_IP_ALLOWED_LIST:
if not is_proxy_in_headers(self.request, settings.PROXY_IP_ALLOWED_LIST, remote_headers):
delete_headers_starting_with_http(request, settings.REMOTE_HOST_HEADERS)
drf_request = super(APIView, self).initialize_request(request, *args, **kwargs)
request.drf_request = drf_request
@@ -211,17 +216,21 @@ class APIView(views.APIView):
return response
if response.status_code >= 400:
ip = get_remote_host(request) # request.META.get('REMOTE_ADDR', None)
msg_data = {
'status_code': response.status_code,
'user_name': request.user,
'url_path': request.path,
'remote_addr': request.META.get('REMOTE_ADDR', None),
'remote_addr': ip,
}
if type(response.data) is dict:
msg_data['error'] = response.data.get('error', response.status_text)
elif type(response.data) is list:
msg_data['error'] = ", ".join(list(map(lambda x: x.get('error', response.status_text), response.data)))
if len(response.data) > 0 and isinstance(response.data[0], str):
msg_data['error'] = str(response.data[0])
else:
msg_data['error'] = ", ".join(list(map(lambda x: x.get('error', response.status_text), response.data)))
else:
msg_data['error'] = response.status_text

View File

@@ -103,7 +103,7 @@ class Metadata(metadata.SimpleMetadata):
default = field.get_default()
if type(default) is UUID:
default = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
if field.field_name == 'TOWER_URL_BASE' and default == 'https://towerhost':
if field.field_name == 'TOWER_URL_BASE' and default == 'https://platformhost':
default = '{}://{}'.format(self.request.scheme, self.request.get_host())
field_info['default'] = default
except serializers.SkipField:

View File

@@ -1038,7 +1038,9 @@ class UserSerializer(BaseSerializer):
# as the modified user then inject a session key derived from
# the updated user to prevent logout. This is the logic used by
# the Django admin's own user_change_password view.
update_session_auth_hash(self.context['request'], obj)
if self.instance and self.context['request'].user.username == obj.username:
update_session_auth_hash(self.context['request'], obj)
elif not obj.password:
obj.set_unusable_password()
obj.save(update_fields=['password'])

View File

@@ -2,6 +2,12 @@
- hosts: all
become: yes
tasks:
- name: Create the receptor group
group:
{% verbatim %}
name: "{{ receptor_group }}"
{% endverbatim %}
state: present
- name: Create the receptor user
user:
{% verbatim %}

View File

@@ -33,7 +33,6 @@ from django.http import HttpResponse, HttpResponseRedirect
from django.contrib.contenttypes.models import ContentType
from django.utils.translation import gettext_lazy as _
# Django REST Framework
from rest_framework.exceptions import APIException, PermissionDenied, ParseError, NotFound
from rest_framework.parsers import FormParser
@@ -48,8 +47,8 @@ from rest_framework import status
from rest_framework_yaml.parsers import YAMLParser
from rest_framework_yaml.renderers import YAMLRenderer
# ANSIConv
import ansiconv
# ansi2html
from ansi2html import Ansi2HTMLConverter
# Python Social Auth
from social_core.backends.utils import load_backends
@@ -61,7 +60,9 @@ import pytz
from wsgiref.util import FileWrapper
# django-ansible-base
from ansible_base.lib.utils.requests import get_remote_hosts
from ansible_base.rbac.models import RoleEvaluation, ObjectRole
from ansible_base.resource_registry.shared_types import OrganizationType, TeamType, UserType
# AWX
from awx.main.tasks.system import send_notifications, update_inventory_computed_fields
@@ -710,16 +711,81 @@ class AuthView(APIView):
return Response(data)
def immutablesharedfields(cls):
'''
Class decorator to prevent modifying shared resources when ALLOW_LOCAL_RESOURCE_MANAGEMENT setting is set to False.
Works by overriding these view methods:
- create
- delete
- perform_update
create and delete are overridden to raise a PermissionDenied exception.
perform_update is overridden to check if any shared fields are being modified,
and raise a PermissionDenied exception if so.
'''
# create instead of perform_create because some of our views
# override create instead of perform_create
if hasattr(cls, 'create'):
cls.original_create = cls.create
@functools.wraps(cls.create)
def create_wrapper(*args, **kwargs):
if settings.ALLOW_LOCAL_RESOURCE_MANAGEMENT:
return cls.original_create(*args, **kwargs)
raise PermissionDenied({'detail': _('Creation of this resource is not allowed. Create this resource via the platform ingress.')})
cls.create = create_wrapper
if hasattr(cls, 'delete'):
cls.original_delete = cls.delete
@functools.wraps(cls.delete)
def delete_wrapper(*args, **kwargs):
if settings.ALLOW_LOCAL_RESOURCE_MANAGEMENT:
return cls.original_delete(*args, **kwargs)
raise PermissionDenied({'detail': _('Deletion of this resource is not allowed. Delete this resource via the platform ingress.')})
cls.delete = delete_wrapper
if hasattr(cls, 'perform_update'):
cls.original_perform_update = cls.perform_update
@functools.wraps(cls.perform_update)
def update_wrapper(*args, **kwargs):
if not settings.ALLOW_LOCAL_RESOURCE_MANAGEMENT:
view, serializer = args
instance = view.get_object()
if instance:
if isinstance(instance, models.Organization):
shared_fields = OrganizationType._declared_fields.keys()
elif isinstance(instance, models.User):
shared_fields = UserType._declared_fields.keys()
elif isinstance(instance, models.Team):
shared_fields = TeamType._declared_fields.keys()
attrs = serializer.validated_data
for field in shared_fields:
if field in attrs and getattr(instance, field) != attrs[field]:
raise PermissionDenied({field: _(f"Cannot change shared field '{field}'. Alter this field via the platform ingress.")})
return cls.original_perform_update(*args, **kwargs)
cls.perform_update = update_wrapper
return cls
@immutablesharedfields
class TeamList(ListCreateAPIView):
model = models.Team
serializer_class = serializers.TeamSerializer
@immutablesharedfields
class TeamDetail(RetrieveUpdateDestroyAPIView):
model = models.Team
serializer_class = serializers.TeamSerializer
@immutablesharedfields
class TeamUsersList(BaseUsersList):
model = models.User
serializer_class = serializers.UserSerializer
@@ -1101,6 +1167,7 @@ class ProjectCopy(CopyAPIView):
copy_return_serializer_class = serializers.ProjectSerializer
@immutablesharedfields
class UserList(ListCreateAPIView):
model = models.User
serializer_class = serializers.UserSerializer
@@ -1271,7 +1338,16 @@ class UserRolesList(SubListAttachDetachAPIView):
user = get_object_or_400(models.User, pk=self.kwargs['pk'])
role = get_object_or_400(models.Role, pk=sub_id)
credential_content_type = ContentType.objects.get_for_model(models.Credential)
content_types = ContentType.objects.get_for_models(models.Organization, models.Team, models.Credential) # dict of {model: content_type}
# Prevent user to be associated with team/org when ALLOW_LOCAL_RESOURCE_MANAGEMENT is False
if not settings.ALLOW_LOCAL_RESOURCE_MANAGEMENT:
for model in [models.Organization, models.Team]:
ct = content_types[model]
if role.content_type == ct and role.role_field in ['member_role', 'admin_role']:
data = dict(msg=_(f"Cannot directly modify user membership to {ct.model}. Direct shared resource management disabled"))
return Response(data, status=status.HTTP_403_FORBIDDEN)
credential_content_type = content_types[models.Credential]
if role.content_type == credential_content_type:
if 'disassociate' not in request.data and role.content_object.organization and user not in role.content_object.organization.member_role:
data = dict(msg=_("You cannot grant credential access to a user not in the credentials' organization"))
@@ -1343,6 +1419,7 @@ class UserActivityStreamList(SubListAPIView):
return qs.filter(Q(actor=parent) | Q(user__in=[parent]))
@immutablesharedfields
class UserDetail(RetrieveUpdateDestroyAPIView):
model = models.User
serializer_class = serializers.UserSerializer
@@ -2313,6 +2390,17 @@ class JobTemplateList(ListCreateAPIView):
serializer_class = serializers.JobTemplateSerializer
always_allow_superuser = False
def check_permissions(self, request):
if request.method == 'POST':
if request.user.is_anonymous:
self.permission_denied(request)
else:
can_access, messages = request.user.can_access_with_errors(self.model, 'add', request.data)
if not can_access:
self.permission_denied(request, message=messages)
super(JobTemplateList, self).check_permissions(request)
class JobTemplateDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIView):
model = models.JobTemplate
@@ -2692,12 +2780,7 @@ class JobTemplateCallback(GenericAPIView):
host for the current request.
"""
# Find the list of remote host names/IPs to check.
remote_hosts = set()
for header in settings.REMOTE_HOST_HEADERS:
for value in self.request.META.get(header, '').split(','):
value = value.strip()
if value:
remote_hosts.add(value)
remote_hosts = set(get_remote_hosts(self.request))
# Add the reverse lookup of IP addresses.
for rh in list(remote_hosts):
try:
@@ -3037,6 +3120,17 @@ class WorkflowJobTemplateList(ListCreateAPIView):
serializer_class = serializers.WorkflowJobTemplateSerializer
always_allow_superuser = False
def check_permissions(self, request):
if request.method == 'POST':
if request.user.is_anonymous:
self.permission_denied(request)
else:
can_access, messages = request.user.can_access_with_errors(self.model, 'add', request.data)
if not can_access:
self.permission_denied(request, message=messages)
super(WorkflowJobTemplateList, self).check_permissions(request)
class WorkflowJobTemplateDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIView):
model = models.WorkflowJobTemplate
@@ -4115,7 +4209,8 @@ class UnifiedJobStdout(RetrieveAPIView):
# Remove any ANSI escape sequences containing job event data.
content = re.sub(r'\x1b\[K(?:[A-Za-z0-9+/=]+\x1b\[\d+D)+\x1b\[K', '', content)
body = ansiconv.to_html(html.escape(content))
conv = Ansi2HTMLConverter()
body = conv.convert(html.escape(content))
context = {'title': get_view_name(self.__class__), 'body': mark_safe(body), 'dark': dark_bg, 'content_only': content_only}
data = render_to_string('api/stdout.html', context).strip()
@@ -4295,7 +4390,15 @@ class RoleUsersList(SubListAttachDetachAPIView):
user = get_object_or_400(models.User, pk=sub_id)
role = self.get_parent_object()
credential_content_type = ContentType.objects.get_for_model(models.Credential)
content_types = ContentType.objects.get_for_models(models.Organization, models.Team, models.Credential) # dict of {model: content_type}
if not settings.ALLOW_LOCAL_RESOURCE_MANAGEMENT:
for model in [models.Organization, models.Team]:
ct = content_types[model]
if role.content_type == ct and role.role_field in ['member_role', 'admin_role']:
data = dict(msg=_(f"Cannot directly modify user membership to {ct.model}. Direct shared resource management disabled"))
return Response(data, status=status.HTTP_403_FORBIDDEN)
credential_content_type = content_types[models.Credential]
if role.content_type == credential_content_type:
if 'disassociate' not in request.data and role.content_object.organization and user not in role.content_object.organization.member_role:
data = dict(msg=_("You cannot grant credential access to a user not in the credentials' organization"))

View File

@@ -53,15 +53,18 @@ from awx.api.serializers import (
CredentialSerializer,
)
from awx.api.views.mixin import RelatedJobsPreventDeleteMixin, OrganizationCountsMixin
from awx.api.views import immutablesharedfields
logger = logging.getLogger('awx.api.views.organization')
@immutablesharedfields
class OrganizationList(OrganizationCountsMixin, ListCreateAPIView):
model = Organization
serializer_class = OrganizationSerializer
@immutablesharedfields
class OrganizationDetail(RelatedJobsPreventDeleteMixin, RetrieveUpdateDestroyAPIView):
model = Organization
serializer_class = OrganizationSerializer
@@ -104,6 +107,7 @@ class OrganizationInventoriesList(SubListAPIView):
relationship = 'inventories'
@immutablesharedfields
class OrganizationUsersList(BaseUsersList):
model = User
serializer_class = UserSerializer
@@ -112,6 +116,7 @@ class OrganizationUsersList(BaseUsersList):
ordering = ('username',)
@immutablesharedfields
class OrganizationAdminsList(BaseUsersList):
model = User
serializer_class = UserSerializer
@@ -150,6 +155,7 @@ class OrganizationWorkflowJobTemplatesList(SubListCreateAPIView):
parent_key = 'organization'
@immutablesharedfields
class OrganizationTeamsList(SubListCreateAttachDetachAPIView):
model = Team
serializer_class = TeamSerializer

View File

@@ -285,9 +285,6 @@ class ApiV2ConfigView(APIView):
pendo_state = settings.PENDO_TRACKING_STATE if settings.PENDO_TRACKING_STATE in ('off', 'anonymous', 'detailed') else 'off'
# Guarding against settings.UI_NEXT being set to a non-boolean value
ui_next_state = settings.UI_NEXT if settings.UI_NEXT in (True, False) else False
data = dict(
time_zone=settings.TIME_ZONE,
license_info=license_data,
@@ -296,7 +293,6 @@ class ApiV2ConfigView(APIView):
analytics_status=pendo_state,
analytics_collectors=all_collectors(),
become_methods=PRIVILEGE_ESCALATION_METHODS,
ui_next=ui_next_state,
)
# If LDAP is enabled, user_ldap_fields will return a list of field

View File

@@ -242,9 +242,10 @@ class BaseAccess(object):
return qs
def filtered_queryset(self):
# Override in subclasses
# filter objects according to user's read access
return self.model.objects.none()
if permission_registry.is_registered(self.model):
return self.model.access_qs(self.user, 'view')
else:
raise NotImplementedError('Filtered queryset for model is not written')
def can_read(self, obj):
return bool(obj and self.get_queryset().filter(pk=obj.pk).exists())
@@ -598,7 +599,7 @@ class InstanceGroupAccess(BaseAccess):
- a superuser
- admin role on the Instance group
I can add/delete Instance Groups:
- a superuser(system administrator)
- a superuser(system administrator), because these are not org-scoped
I can use Instance Groups when I have:
- use_role on the instance group
"""
@@ -606,9 +607,6 @@ class InstanceGroupAccess(BaseAccess):
model = InstanceGroup
prefetch_related = ('instances',)
def filtered_queryset(self):
return self.model.accessible_objects(self.user, 'read_role')
@check_superuser
def can_use(self, obj):
return self.user in obj.use_role
@@ -627,7 +625,7 @@ class InstanceGroupAccess(BaseAccess):
def can_delete(self, obj):
if obj.name in [settings.DEFAULT_EXECUTION_QUEUE_NAME, settings.DEFAULT_CONTROL_PLANE_QUEUE_NAME]:
return False
return self.user.is_superuser
return self.user.has_obj_perm(obj, 'delete')
class UserAccess(BaseAccess):
@@ -654,7 +652,7 @@ class UserAccess(BaseAccess):
qs = User.objects.all()
else:
qs = (
User.objects.filter(pk__in=Organization.accessible_objects(self.user, 'read_role').values('member_role__members'))
User.objects.filter(pk__in=Organization.access_qs(self.user, 'view').values('member_role__members'))
| User.objects.filter(pk=self.user.id)
| User.objects.filter(is_superuser=True)
).distinct()
@@ -671,7 +669,7 @@ class UserAccess(BaseAccess):
return True
if not settings.MANAGE_ORGANIZATION_AUTH:
return False
return Organization.accessible_objects(self.user, 'admin_role').exists()
return Organization.access_qs(self.user, 'change').exists()
def can_change(self, obj, data):
if data is not None and ('is_superuser' in data or 'is_system_auditor' in data):
@@ -691,7 +689,7 @@ class UserAccess(BaseAccess):
"""
Returns all organizations that count `u` as a member
"""
return Organization.accessible_objects(u, 'member_role')
return Organization.access_qs(u, 'member')
def is_all_org_admin(self, u):
"""
@@ -774,7 +772,7 @@ class OAuth2ApplicationAccess(BaseAccess):
prefetch_related = ('organization', 'oauth2accesstoken_set')
def filtered_queryset(self):
org_access_qs = Organization.accessible_objects(self.user, 'member_role')
org_access_qs = Organization.access_qs(self.user, 'member')
return self.model.objects.filter(organization__in=org_access_qs)
def can_change(self, obj, data):
@@ -787,7 +785,7 @@ class OAuth2ApplicationAccess(BaseAccess):
if self.user.is_superuser:
return True
if not data:
return Organization.accessible_objects(self.user, 'admin_role').exists()
return Organization.access_qs(self.user, 'change').exists()
return self.check_related('organization', Organization, data, role_field='admin_role', mandatory=True)
@@ -855,9 +853,6 @@ class OrganizationAccess(NotificationAttachMixin, BaseAccess):
# organization admin_role is not a parent of organization auditor_role
notification_attach_roles = ['admin_role', 'auditor_role']
def filtered_queryset(self):
return self.model.accessible_objects(self.user, 'read_role')
@check_superuser
def can_change(self, obj, data):
if data and data.get('default_environment'):
@@ -925,9 +920,6 @@ class InventoryAccess(BaseAccess):
Prefetch('labels', queryset=Label.objects.all().order_by('name')),
)
def filtered_queryset(self, allowed=None, ad_hoc=None):
return self.model.accessible_objects(self.user, 'read_role')
@check_superuser
def can_use(self, obj):
return self.user in obj.use_role
@@ -936,7 +928,7 @@ class InventoryAccess(BaseAccess):
def can_add(self, data):
# If no data is specified, just checking for generic add permission?
if not data:
return Organization.accessible_objects(self.user, 'inventory_admin_role').exists()
return Organization.access_qs(self.user, 'add_inventory').exists()
return self.check_related('organization', Organization, data, role_field='inventory_admin_role')
@check_superuser
@@ -998,7 +990,7 @@ class HostAccess(BaseAccess):
def can_add(self, data):
if not data: # So the browseable API will work
return Inventory.accessible_objects(self.user, 'admin_role').exists()
return Inventory.access_qs(self.user, 'change').exists()
# Checks for admin or change permission on inventory.
if not self.check_related('inventory', Inventory, data):
@@ -1060,7 +1052,7 @@ class GroupAccess(BaseAccess):
def can_add(self, data):
if not data: # So the browseable API will work
return Inventory.accessible_objects(self.user, 'admin_role').exists()
return Inventory.access_qs(self.user, 'change').exists()
if 'inventory' not in data:
return False
# Checks for admin or change permission on inventory.
@@ -1102,7 +1094,7 @@ class InventorySourceAccess(NotificationAttachMixin, UnifiedCredentialsMixin, Ba
def can_add(self, data):
if not data or 'inventory' not in data:
return Inventory.accessible_objects(self.user, 'admin_role').exists()
return Inventory.access_qs(self.user, 'change').exists()
if not self.check_related('source_project', Project, data, role_field='use_role'):
return False
@@ -1216,9 +1208,6 @@ class CredentialAccess(BaseAccess):
)
prefetch_related = ('admin_role', 'use_role', 'read_role', 'admin_role__parents', 'admin_role__members', 'credential_type', 'organization')
def filtered_queryset(self):
return self.model.accessible_objects(self.user, 'read_role')
@check_superuser
def can_add(self, data):
if not data: # So the browseable API will work
@@ -1329,7 +1318,7 @@ class TeamAccess(BaseAccess):
@check_superuser
def can_add(self, data):
if not data: # So the browseable API will work
return Organization.accessible_objects(self.user, 'admin_role').exists()
return Organization.access_qs(self.user, 'view').exists()
if not settings.MANAGE_ORGANIZATION_AUTH:
return False
return self.check_related('organization', Organization, data)
@@ -1387,12 +1376,11 @@ class TeamAccess(BaseAccess):
class ExecutionEnvironmentAccess(BaseAccess):
"""
I can see an execution environment when:
- I'm a superuser
- I'm a member of the same organization
- it is a global ExecutionEnvironment
- I can see its organization
- It is a global ExecutionEnvironment
I can create/change an execution environment when:
- I'm a superuser
- I'm an admin for the organization(s)
- I have an organization or object role that gives access
"""
model = ExecutionEnvironment
@@ -1401,13 +1389,15 @@ class ExecutionEnvironmentAccess(BaseAccess):
def filtered_queryset(self):
return ExecutionEnvironment.objects.filter(
Q(organization__in=Organization.accessible_pk_qs(self.user, 'read_role')) | Q(organization__isnull=True)
Q(organization__in=Organization.access_ids_qs(self.user, 'view'))
| Q(organization__isnull=True)
| Q(id__in=ExecutionEnvironment.access_ids_qs(self.user, 'change'))
).distinct()
@check_superuser
def can_add(self, data):
if not data: # So the browseable API will work
return Organization.accessible_objects(self.user, 'execution_environment_admin_role').exists()
return Organization.access_qs(self.user, 'add_executionenvironment').exists()
return self.check_related('organization', Organization, data, mandatory=True, role_field='execution_environment_admin_role')
@check_superuser
@@ -1416,15 +1406,17 @@ class ExecutionEnvironmentAccess(BaseAccess):
raise PermissionDenied
if settings.ANSIBLE_BASE_ROLE_SYSTEM_ACTIVATED:
if not self.user.has_obj_perm(obj, 'change'):
raise PermissionDenied
return False
else:
if self.user not in obj.organization.execution_environment_admin_role:
raise PermissionDenied
if data and 'organization' in data:
new_org = get_object_from_data('organization', Organization, data, obj=obj)
if not new_org or self.user not in new_org.execution_environment_admin_role:
if not self.check_related('organization', Organization, data, obj=obj, role_field='execution_environment_admin_role'):
return False
# Special case that check_related does not catch, org users can not remove the organization from the EE
if data and ('organization' in data or 'organization_id' in data):
if (not data.get('organization')) and (not data.get('organization_id')):
return False
return self.check_related('organization', Organization, data, obj=obj, mandatory=True, role_field='execution_environment_admin_role')
return True
def can_delete(self, obj):
if obj.managed:
@@ -1454,13 +1446,10 @@ class ProjectAccess(NotificationAttachMixin, BaseAccess):
prefetch_related = ('modified_by', 'created_by', 'organization', 'last_job', 'current_job')
notification_attach_roles = ['admin_role']
def filtered_queryset(self):
return self.model.accessible_objects(self.user, 'read_role')
@check_superuser
def can_add(self, data):
if not data: # So the browseable API will work
return Organization.accessible_objects(self.user, 'project_admin_role').exists()
return Organization.access_qs(self.user, 'add_project').exists()
if data.get('default_environment'):
ee = get_object_from_data('default_environment', ExecutionEnvironment, data)
@@ -1556,9 +1545,6 @@ class JobTemplateAccess(NotificationAttachMixin, UnifiedCredentialsMixin, BaseAc
Prefetch('last_job', queryset=UnifiedJob.objects.non_polymorphic()),
)
def filtered_queryset(self):
return self.model.accessible_objects(self.user, 'read_role')
def can_add(self, data):
"""
a user can create a job template if
@@ -1571,7 +1557,7 @@ class JobTemplateAccess(NotificationAttachMixin, UnifiedCredentialsMixin, BaseAc
Users who are able to create deploy jobs can also run normal and check (dry run) jobs.
"""
if not data: # So the browseable API will work
return Project.accessible_objects(self.user, 'use_role').exists()
return Project.access_qs(self.user, 'use_project').exists()
# if reference_obj is provided, determine if it can be copied
reference_obj = data.get('reference_obj', None)
@@ -1596,6 +1582,8 @@ class JobTemplateAccess(NotificationAttachMixin, UnifiedCredentialsMixin, BaseAc
inventory = get_value(Inventory, 'inventory')
if inventory:
if self.user not in inventory.use_role:
if self.save_messages:
self.messages['inventory'] = [_('You do not have use permission on Inventory')]
return False
if not self.check_related('execution_environment', ExecutionEnvironment, data, role_field='read_role'):
@@ -1604,11 +1592,16 @@ class JobTemplateAccess(NotificationAttachMixin, UnifiedCredentialsMixin, BaseAc
project = get_value(Project, 'project')
# If the user has admin access to the project (as an org admin), should
# be able to proceed without additional checks.
if project:
return self.user in project.use_role
else:
if not project:
return False
if self.user not in project.use_role:
if self.save_messages:
self.messages['project'] = [_('You do not have use permission on Project')]
return False
return True
@check_superuser
def can_copy_related(self, obj):
"""
@@ -1755,13 +1748,13 @@ class JobAccess(BaseAccess):
def filtered_queryset(self):
qs = self.model.objects
qs_jt = qs.filter(job_template__in=JobTemplate.accessible_objects(self.user, 'read_role'))
qs_jt = qs.filter(job_template__in=JobTemplate.access_qs(self.user, 'view'))
org_access_qs = Organization.objects.filter(Q(admin_role__members=self.user) | Q(auditor_role__members=self.user))
if not org_access_qs.exists():
return qs_jt
return qs.filter(Q(job_template__in=JobTemplate.accessible_objects(self.user, 'read_role')) | Q(organization__in=org_access_qs)).distinct()
return qs.filter(Q(job_template__in=JobTemplate.access_qs(self.user, 'view')) | Q(organization__in=org_access_qs)).distinct()
def can_add(self, data, validate_license=True):
raise NotImplementedError('Direct job creation not possible in v2 API')
@@ -1850,6 +1843,11 @@ class SystemJobTemplateAccess(BaseAccess):
model = SystemJobTemplate
def filtered_queryset(self):
if self.user.is_superuser or self.user.is_system_auditor:
return self.model.objects.all()
return self.model.objects.none()
@check_superuser
def can_start(self, obj, validate_license=True):
'''Only a superuser can start a job from a SystemJobTemplate'''
@@ -1962,7 +1960,7 @@ class WorkflowJobTemplateNodeAccess(UnifiedCredentialsMixin, BaseAccess):
prefetch_related = ('success_nodes', 'failure_nodes', 'always_nodes', 'unified_job_template', 'workflow_job_template')
def filtered_queryset(self):
return self.model.objects.filter(workflow_job_template__in=WorkflowJobTemplate.accessible_objects(self.user, 'read_role'))
return self.model.objects.filter(workflow_job_template__in=WorkflowJobTemplate.access_qs(self.user, 'view'))
@check_superuser
def can_add(self, data):
@@ -2077,9 +2075,6 @@ class WorkflowJobTemplateAccess(NotificationAttachMixin, BaseAccess):
'read_role',
)
def filtered_queryset(self):
return self.model.accessible_objects(self.user, 'read_role')
@check_superuser
def can_add(self, data):
"""
@@ -2090,13 +2085,25 @@ class WorkflowJobTemplateAccess(NotificationAttachMixin, BaseAccess):
Users who are able to create deploy jobs can also run normal and check (dry run) jobs.
"""
if not data: # So the browseable API will work
return Organization.accessible_objects(self.user, 'workflow_admin_role').exists()
return Organization.access_qs(self.user, 'add_workflowjobtemplate').exists()
return bool(
self.check_related('organization', Organization, data, role_field='workflow_admin_role', mandatory=True)
and self.check_related('inventory', Inventory, data, role_field='use_role')
and self.check_related('execution_environment', ExecutionEnvironment, data, role_field='read_role')
)
if not self.check_related('organization', Organization, data, role_field='workflow_admin_role', mandatory=True):
if data.get('organization', None) is None:
if self.save_messages:
self.messages['organization'] = [_('An organization is required to create a workflow job template for normal user')]
return False
if not self.check_related('inventory', Inventory, data, role_field='use_role'):
if self.save_messages:
self.messages['inventory'] = [_('You do not have use_role to the inventory')]
return False
if not self.check_related('execution_environment', ExecutionEnvironment, data, role_field='read_role'):
if self.save_messages:
self.messages['execution_environment'] = [_('You do not have read_role to the execution environment')]
return False
return True
def can_copy(self, obj):
if self.save_messages:
@@ -2628,7 +2635,7 @@ class ScheduleAccess(UnifiedCredentialsMixin, BaseAccess):
class NotificationTemplateAccess(BaseAccess):
"""
I can see/use a notification_template if I have permission to
Run standard logic from DAB RBAC
"""
model = NotificationTemplate
@@ -2638,21 +2645,18 @@ class NotificationTemplateAccess(BaseAccess):
if settings.ANSIBLE_BASE_ROLE_SYSTEM_ACTIVATED:
return self.model.access_qs(self.user, 'view')
return self.model.objects.filter(
Q(organization__in=Organization.accessible_objects(self.user, 'notification_admin_role')) | Q(organization__in=self.user.auditor_of_organizations)
Q(organization__in=Organization.access_qs(self.user, 'add_notificationtemplate')) | Q(organization__in=self.user.auditor_of_organizations)
).distinct()
@check_superuser
def can_add(self, data):
if not data:
return Organization.accessible_objects(self.user, 'notification_admin_role').exists()
return Organization.access_qs(self.user, 'add_notificationtemplate').exists()
return self.check_related('organization', Organization, data, role_field='notification_admin_role', mandatory=True)
@check_superuser
def can_change(self, obj, data):
if obj.organization is None:
# only superusers are allowed to edit orphan notification templates
return False
return self.check_related('organization', Organization, data, obj=obj, role_field='notification_admin_role', mandatory=True)
return self.user.has_obj_perm(obj, 'change') and self.check_related('organization', Organization, data, obj=obj, role_field='notification_admin_role')
def can_admin(self, obj, data):
return self.can_change(obj, data)
@@ -2662,9 +2666,7 @@ class NotificationTemplateAccess(BaseAccess):
@check_superuser
def can_start(self, obj, validate_license=True):
if obj.organization is None:
return False
return self.user in obj.organization.notification_admin_role
return self.can_change(obj, None)
class NotificationAccess(BaseAccess):
@@ -2677,7 +2679,7 @@ class NotificationAccess(BaseAccess):
def filtered_queryset(self):
return self.model.objects.filter(
Q(notification_template__organization__in=Organization.accessible_objects(self.user, 'notification_admin_role'))
Q(notification_template__organization__in=Organization.access_qs(self.user, 'add_notificationtemplate'))
| Q(notification_template__organization__in=self.user.auditor_of_organizations)
).distinct()
@@ -2793,11 +2795,7 @@ class ActivityStreamAccess(BaseAccess):
if credential_set:
q |= Q(credential__in=credential_set)
auditing_orgs = (
(Organization.accessible_objects(self.user, 'admin_role') | Organization.accessible_objects(self.user, 'auditor_role'))
.distinct()
.values_list('id', flat=True)
)
auditing_orgs = (Organization.access_qs(self.user, 'change') | Organization.access_qs(self.user, 'audit')).distinct().values_list('id', flat=True)
if auditing_orgs:
q |= (
Q(user__in=auditing_orgs.values('member_role__members'))

View File

@@ -66,10 +66,8 @@ class FixedSlidingWindow:
class RelayWebsocketStatsManager:
def __init__(self, event_loop, local_hostname):
def __init__(self, local_hostname):
self._local_hostname = local_hostname
self._event_loop = event_loop
self._stats = dict()
self._redis_key = BROADCAST_WEBSOCKET_REDIS_KEY_NAME
@@ -94,7 +92,10 @@ class RelayWebsocketStatsManager:
self.start()
def start(self):
self.async_task = self._event_loop.create_task(self.run_loop())
self.async_task = asyncio.get_running_loop().create_task(
self.run_loop(),
name='RelayWebsocketStatsManager.run_loop',
)
return self.async_task
@classmethod

View File

@@ -843,22 +843,12 @@ register(
hidden=True,
)
register(
'UI_NEXT',
field_class=fields.BooleanField,
default=False,
label=_('Enable Preview of New User Interface'),
help_text=_('Enable preview of new user interface.'),
category=_('System'),
category_slug='system',
hidden=True,
)
register(
'SUBSCRIPTION_USAGE_MODEL',
field_class=fields.ChoiceField,
choices=[
('', _('Default model for AWX - no subscription. Deletion of host_metrics will not be considered for purposes of managed host counting')),
('', _('No subscription. Deletion of host_metrics will not be considered for purposes of managed host counting')),
(
SUBSCRIPTION_USAGE_MODEL_UNIQUE_HOSTS,
_('Usage based on unique managed nodes in a large historical time frame and delete functionality for no longer used managed nodes'),
@@ -929,6 +919,16 @@ register(
category_slug='debug',
)
register(
'RECEPTOR_KEEP_WORK_ON_ERROR',
field_class=fields.BooleanField,
label=_('Keep receptor work on error'),
default=False,
help_text=_('Prevent receptor work from being released on when error is detected'),
category=('Debug'),
category_slug='debug',
)
def logging_validate(serializer, attrs):
if not serializer.instance or not hasattr(serializer.instance, 'LOG_AGGREGATOR_HOST') or not hasattr(serializer.instance, 'LOG_AGGREGATOR_TYPE'):

View File

@@ -14,7 +14,7 @@ __all__ = [
'STANDARD_INVENTORY_UPDATE_ENV',
]
CLOUD_PROVIDERS = ('azure_rm', 'ec2', 'gce', 'vmware', 'openstack', 'rhv', 'satellite6', 'controller', 'insights', 'terraform')
CLOUD_PROVIDERS = ('azure_rm', 'ec2', 'gce', 'vmware', 'openstack', 'rhv', 'satellite6', 'controller', 'insights', 'terraform', 'openshift_virtualization')
PRIVILEGE_ESCALATION_METHODS = [
('sudo', _('Sudo')),
('su', _('Su')),
@@ -43,6 +43,7 @@ STANDARD_INVENTORY_UPDATE_ENV = {
}
CAN_CANCEL = ('new', 'pending', 'waiting', 'running')
ACTIVE_STATES = CAN_CANCEL
ERROR_STATES = ('error',)
MINIMAL_EVENTS = set(['playbook_on_play_start', 'playbook_on_task_start', 'playbook_on_stats', 'EOF'])
CENSOR_VALUE = '************'
ENV_BLOCKLIST = frozenset(

View File

@@ -102,7 +102,8 @@ def create_listener_connection():
# Apply overrides specifically for the listener connection
for k, v in settings.LISTENER_DATABASES.get('default', {}).items():
conf[k] = v
if k != 'OPTIONS':
conf[k] = v
for k, v in settings.LISTENER_DATABASES.get('default', {}).get('OPTIONS', {}).items():
conf['OPTIONS'][k] = v

View File

@@ -252,7 +252,7 @@ class ImplicitRoleField(models.ForeignKey):
kwargs.setdefault('related_name', '+')
kwargs.setdefault('null', 'True')
kwargs.setdefault('editable', False)
kwargs.setdefault('on_delete', models.CASCADE)
kwargs.setdefault('on_delete', models.SET_NULL)
super(ImplicitRoleField, self).__init__(*args, **kwargs)
def deconstruct(self):

View File

@@ -2,6 +2,7 @@
# All Rights Reserved
from django.core.management.base import BaseCommand
from django.db import transaction
from crum import impersonate
from awx.main.models import User, Organization, Project, Inventory, CredentialType, Credential, Host, JobTemplate
from awx.main.signals import disable_computed_fields
@@ -13,6 +14,12 @@ class Command(BaseCommand):
help = 'Creates a preload tower data if there is none.'
def handle(self, *args, **kwargs):
# Wrap the operation in an atomic block, so we do not on accident
# create the organization but not create the project, etc.
with transaction.atomic():
self._handle()
def _handle(self):
changed = False
# Create a default organization as the first superuser found.
@@ -43,10 +50,11 @@ class Command(BaseCommand):
ssh_type = CredentialType.objects.filter(namespace='ssh').first()
c, _ = Credential.objects.get_or_create(
credential_type=ssh_type, name='Demo Credential', inputs={'username': superuser.username}, created_by=superuser
credential_type=ssh_type, name='Demo Credential', inputs={'username': getattr(superuser, 'username', 'null')}, created_by=superuser
)
c.admin_role.members.add(superuser)
if superuser:
c.admin_role.members.add(superuser)
public_galaxy_credential, _ = Credential.objects.get_or_create(
name='Ansible Galaxy',

View File

@@ -17,49 +17,49 @@ class Migration(migrations.Migration):
model_name='organization',
name='execute_role',
field=awx.main.fields.ImplicitRoleField(
null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='admin_role', related_name='+', to='main.Role'
null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='admin_role', related_name='+', to='main.Role'
),
),
migrations.AddField(
model_name='organization',
name='job_template_admin_role',
field=awx.main.fields.ImplicitRoleField(
editable=False, null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='admin_role', related_name='+', to='main.Role'
editable=False, null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='admin_role', related_name='+', to='main.Role'
),
),
migrations.AddField(
model_name='organization',
name='credential_admin_role',
field=awx.main.fields.ImplicitRoleField(
null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='admin_role', related_name='+', to='main.Role'
null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='admin_role', related_name='+', to='main.Role'
),
),
migrations.AddField(
model_name='organization',
name='inventory_admin_role',
field=awx.main.fields.ImplicitRoleField(
null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='admin_role', related_name='+', to='main.Role'
null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='admin_role', related_name='+', to='main.Role'
),
),
migrations.AddField(
model_name='organization',
name='project_admin_role',
field=awx.main.fields.ImplicitRoleField(
null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='admin_role', related_name='+', to='main.Role'
null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='admin_role', related_name='+', to='main.Role'
),
),
migrations.AddField(
model_name='organization',
name='workflow_admin_role',
field=awx.main.fields.ImplicitRoleField(
null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='admin_role', related_name='+', to='main.Role'
null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='admin_role', related_name='+', to='main.Role'
),
),
migrations.AddField(
model_name='organization',
name='notification_admin_role',
field=awx.main.fields.ImplicitRoleField(
null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='admin_role', related_name='+', to='main.Role'
null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='admin_role', related_name='+', to='main.Role'
),
),
migrations.AlterField(
@@ -67,7 +67,7 @@ class Migration(migrations.Migration):
name='admin_role',
field=awx.main.fields.ImplicitRoleField(
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['singleton:system_administrator', 'organization.credential_admin_role'],
related_name='+',
to='main.Role',
@@ -77,7 +77,7 @@ class Migration(migrations.Migration):
model_name='inventory',
name='admin_role',
field=awx.main.fields.ImplicitRoleField(
null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='organization.inventory_admin_role', related_name='+', to='main.Role'
null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='organization.inventory_admin_role', related_name='+', to='main.Role'
),
),
migrations.AlterField(
@@ -85,7 +85,7 @@ class Migration(migrations.Migration):
name='admin_role',
field=awx.main.fields.ImplicitRoleField(
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['organization.project_admin_role', 'singleton:system_administrator'],
related_name='+',
to='main.Role',
@@ -96,7 +96,7 @@ class Migration(migrations.Migration):
name='admin_role',
field=awx.main.fields.ImplicitRoleField(
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['singleton:system_administrator', 'organization.workflow_admin_role'],
related_name='+',
to='main.Role',
@@ -107,7 +107,7 @@ class Migration(migrations.Migration):
name='execute_role',
field=awx.main.fields.ImplicitRoleField(
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['admin_role', 'organization.execute_role'],
related_name='+',
to='main.Role',
@@ -119,7 +119,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['project.organization.job_template_admin_role', 'inventory.organization.job_template_admin_role'],
related_name='+',
to='main.Role',
@@ -130,7 +130,7 @@ class Migration(migrations.Migration):
name='execute_role',
field=awx.main.fields.ImplicitRoleField(
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['admin_role', 'project.organization.execute_role', 'inventory.organization.execute_role'],
related_name='+',
to='main.Role',
@@ -142,7 +142,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=[
'admin_role',
'execute_role',

View File

@@ -18,7 +18,7 @@ class Migration(migrations.Migration):
model_name='organization',
name='member_role',
field=awx.main.fields.ImplicitRoleField(
editable=False, null='True', on_delete=django.db.models.deletion.CASCADE, parent_role=['admin_role'], related_name='+', to='main.Role'
editable=False, null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role=['admin_role'], related_name='+', to='main.Role'
),
),
migrations.AlterField(
@@ -27,7 +27,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=[
'member_role',
'auditor_role',

View File

@@ -36,7 +36,7 @@ class Migration(migrations.Migration):
model_name='organization',
name='approval_role',
field=awx.main.fields.ImplicitRoleField(
editable=False, null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='admin_role', related_name='+', to='main.Role'
editable=False, null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='admin_role', related_name='+', to='main.Role'
),
preserve_default='True',
),
@@ -46,7 +46,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['organization.approval_role', 'admin_role'],
related_name='+',
to='main.Role',
@@ -116,7 +116,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=[
'member_role',
'auditor_role',
@@ -139,7 +139,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['singleton:system_auditor', 'organization.auditor_role', 'execute_role', 'admin_role', 'approval_role'],
related_name='+',
to='main.Role',

View File

@@ -80,7 +80,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['organization.job_template_admin_role'],
related_name='+',
to='main.Role',
@@ -92,7 +92,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['admin_role', 'organization.execute_role'],
related_name='+',
to='main.Role',
@@ -104,7 +104,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['organization.auditor_role', 'inventory.organization.auditor_role', 'execute_role', 'admin_role'],
related_name='+',
to='main.Role',

View File

@@ -26,7 +26,7 @@ class Migration(migrations.Migration):
model_name='organization',
name='execution_environment_admin_role',
field=awx.main.fields.ImplicitRoleField(
editable=False, null='True', on_delete=django.db.models.deletion.CASCADE, parent_role='admin_role', related_name='+', to='main.Role'
editable=False, null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role='admin_role', related_name='+', to='main.Role'
),
preserve_default='True',
),

View File

@@ -17,7 +17,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=[
'member_role',
'auditor_role',

View File

@@ -17,7 +17,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['singleton:system_administrator'],
related_name='+',
to='main.role',
@@ -30,7 +30,7 @@ class Migration(migrations.Migration):
field=awx.main.fields.ImplicitRoleField(
editable=False,
null='True',
on_delete=django.db.models.deletion.CASCADE,
on_delete=django.db.models.deletion.SET_NULL,
parent_role=['singleton:system_auditor', 'use_role', 'admin_role'],
related_name='+',
to='main.role',
@@ -41,7 +41,7 @@ class Migration(migrations.Migration):
model_name='instancegroup',
name='use_role',
field=awx.main.fields.ImplicitRoleField(
editable=False, null='True', on_delete=django.db.models.deletion.CASCADE, parent_role=['admin_role'], related_name='+', to='main.role'
editable=False, null='True', on_delete=django.db.models.deletion.SET_NULL, parent_role=['admin_role'], related_name='+', to='main.role'
),
preserve_default='True',
),

View File

@@ -0,0 +1,61 @@
# Generated by Django 4.2.10 on 2024-06-12 19:59
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('main', '0193_alter_notification_notification_type_and_more'),
]
operations = [
migrations.AlterField(
model_name='inventorysource',
name='source',
field=models.CharField(
choices=[
('file', 'File, Directory or Script'),
('constructed', 'Template additional groups and hostvars at runtime'),
('scm', 'Sourced from a Project'),
('ec2', 'Amazon EC2'),
('gce', 'Google Compute Engine'),
('azure_rm', 'Microsoft Azure Resource Manager'),
('vmware', 'VMware vCenter'),
('satellite6', 'Red Hat Satellite 6'),
('openstack', 'OpenStack'),
('rhv', 'Red Hat Virtualization'),
('controller', 'Red Hat Ansible Automation Platform'),
('insights', 'Red Hat Insights'),
('terraform', 'Terraform State'),
('openshift_virtualization', 'OpenShift Virtualization'),
],
default=None,
max_length=32,
),
),
migrations.AlterField(
model_name='inventoryupdate',
name='source',
field=models.CharField(
choices=[
('file', 'File, Directory or Script'),
('constructed', 'Template additional groups and hostvars at runtime'),
('scm', 'Sourced from a Project'),
('ec2', 'Amazon EC2'),
('gce', 'Google Compute Engine'),
('azure_rm', 'Microsoft Azure Resource Manager'),
('vmware', 'VMware vCenter'),
('satellite6', 'Red Hat Satellite 6'),
('openstack', 'OpenStack'),
('rhv', 'Red Hat Virtualization'),
('controller', 'Red Hat Ansible Automation Platform'),
('insights', 'Red Hat Insights'),
('terraform', 'Terraform State'),
('openshift_virtualization', 'OpenShift Virtualization'),
],
default=None,
max_length=32,
),
),
]

View File

@@ -0,0 +1,26 @@
# Generated by Django 4.2.6 on 2024-06-20 15:55
from django.db import migrations
def delete_execution_environment_read_role(apps, schema_editor):
permission_classes = [apps.get_model('auth', 'Permission'), apps.get_model('dab_rbac', 'DABPermission')]
for permission_cls in permission_classes:
ee_read_perm = permission_cls.objects.filter(codename='view_executionenvironment').first()
if ee_read_perm:
ee_read_perm.delete()
class Migration(migrations.Migration):
dependencies = [
('main', '0194_alter_inventorysource_source_and_more'),
]
operations = [
migrations.AlterModelOptions(
name='executionenvironment',
options={'default_permissions': ('add', 'change', 'delete'), 'ordering': ('-created',)},
),
migrations.RunPython(delete_execution_environment_read_role, migrations.RunPython.noop),
]

View File

@@ -134,8 +134,7 @@ def get_permissions_for_role(role_field, children_map, apps):
# more special cases for those same above special org-level roles
if role_field.name == 'auditor_role':
for codename in ('view_notificationtemplate', 'view_executionenvironment'):
perm_list.append(Permission.objects.get(codename=codename))
perm_list.append(Permission.objects.get(codename='view_notificationtemplate'))
return perm_list
@@ -168,7 +167,7 @@ def migrate_to_new_rbac(apps, schema_editor):
perm.delete()
managed_definitions = dict()
for role_definition in RoleDefinition.objects.filter(managed=True):
for role_definition in RoleDefinition.objects.filter(managed=True).exclude(name__in=(settings.ANSIBLE_BASE_JWT_MANAGED_ROLES)):
permissions = frozenset(role_definition.permissions.values_list('id', flat=True))
managed_definitions[permissions] = role_definition
@@ -290,14 +289,15 @@ def setup_managed_role_definitions(apps, schema_editor):
managed_role_definitions = []
org_perms = set()
for cls in permission_registry._registry:
for cls in permission_registry.all_registered_models:
ct = ContentType.objects.get_for_model(cls)
cls_name = cls._meta.model_name
object_perms = set(Permission.objects.filter(content_type=ct))
# Special case for InstanceGroup which has an organiation field, but is not an organization child object
if cls._meta.model_name != 'instancegroup':
if cls_name != 'instancegroup':
org_perms.update(object_perms)
if 'object_admin' in to_create and cls != Organization:
if 'object_admin' in to_create and cls_name != 'organization':
indiv_perms = object_perms.copy()
add_perms = [perm for perm in indiv_perms if perm.codename.startswith('add_')]
if add_perms:
@@ -309,8 +309,18 @@ def setup_managed_role_definitions(apps, schema_editor):
to_create['object_admin'].format(cls=cls), f'Has all permissions to a single {cls._meta.verbose_name}', ct, indiv_perms, RoleDefinition
)
)
if cls_name == 'team':
managed_role_definitions.append(
get_or_create_managed(
'Controller Team Admin',
f'Has all permissions to a single {cls._meta.verbose_name}',
ct,
indiv_perms,
RoleDefinition,
)
)
if 'org_children' in to_create and cls != Organization:
if 'org_children' in to_create and (cls_name not in ('organization', 'instancegroup', 'team')):
org_child_perms = object_perms.copy()
org_child_perms.add(Permission.objects.get(codename='view_organization'))
@@ -327,20 +337,40 @@ def setup_managed_role_definitions(apps, schema_editor):
if 'special' in to_create:
special_perms = []
for perm in object_perms:
if perm.codename.split('_')[0] not in ('add', 'change', 'update', 'delete', 'view'):
# Organization auditor is handled separately
if perm.codename.split('_')[0] not in ('add', 'change', 'delete', 'view', 'audit'):
special_perms.append(perm)
for perm in special_perms:
action = perm.codename.split('_')[0]
view_perm = Permission.objects.get(content_type=ct, codename__startswith='view_')
perm_list = [perm, view_perm]
# Handle special-case where adhoc role also listed use permission
if action == 'adhoc':
for other_perm in object_perms:
if other_perm.codename == 'use_inventory':
perm_list.append(other_perm)
break
managed_role_definitions.append(
get_or_create_managed(
to_create['special'].format(cls=cls, action=action.title()),
f'Has {action} permissions to a single {cls._meta.verbose_name}',
ct,
[perm, view_perm],
perm_list,
RoleDefinition,
)
)
if action == 'member' and cls_name in ('organization', 'team'):
suffix = to_create['special'].format(cls=cls, action=action.title())
rd_name = f'Controller {suffix}'
managed_role_definitions.append(
get_or_create_managed(
rd_name,
f'Has {action} permissions to a single {cls._meta.verbose_name}',
ct,
perm_list,
RoleDefinition,
)
)
if 'org_admin' in to_create:
managed_role_definitions.append(
@@ -352,6 +382,50 @@ def setup_managed_role_definitions(apps, schema_editor):
RoleDefinition,
)
)
managed_role_definitions.append(
get_or_create_managed(
'Controller Organization Admin',
'Has all permissions to a single organization and all objects inside of it',
org_ct,
org_perms,
RoleDefinition,
)
)
# Special "organization action" roles
audit_permissions = [perm for perm in org_perms if perm.codename.startswith('view_')]
audit_permissions.append(Permission.objects.get(codename='audit_organization'))
managed_role_definitions.append(
get_or_create_managed(
'Organization Audit',
'Has permission to view all objects inside of a single organization',
org_ct,
audit_permissions,
RoleDefinition,
)
)
org_execute_permissions = {'view_jobtemplate', 'execute_jobtemplate', 'view_workflowjobtemplate', 'execute_workflowjobtemplate', 'view_organization'}
managed_role_definitions.append(
get_or_create_managed(
'Organization Execute',
'Has permission to execute all runnable objects in the organization',
org_ct,
[perm for perm in org_perms if perm.codename in org_execute_permissions],
RoleDefinition,
)
)
org_approval_permissions = {'view_organization', 'view_workflowjobtemplate', 'approve_workflowjobtemplate'}
managed_role_definitions.append(
get_or_create_managed(
'Organization Approval',
'Has permission to approve any workflow steps within a single organization',
org_ct,
[perm for perm in org_perms if perm.codename in org_approval_permissions],
RoleDefinition,
)
)
unexpected_role_definitions = RoleDefinition.objects.filter(managed=True).exclude(pk__in=[rd.pk for rd in managed_role_definitions])
for role_definition in unexpected_role_definitions:

View File

@@ -176,17 +176,17 @@ pre_delete.connect(cleanup_created_modified_by, sender=User)
@property
def user_get_organizations(user):
return Organization.objects.filter(member_role__members=user)
return Organization.access_qs(user, 'member')
@property
def user_get_admin_of_organizations(user):
return Organization.objects.filter(admin_role__members=user)
return Organization.access_qs(user, 'change')
@property
def user_get_auditor_of_organizations(user):
return Organization.objects.filter(auditor_role__members=user)
return Organization.access_qs(user, 'audit')
@property

View File

@@ -21,6 +21,10 @@ from django.conf import settings
from django.utils.encoding import force_str
from django.utils.functional import cached_property
from django.utils.timezone import now
from django.contrib.auth.models import User
# DRF
from rest_framework.serializers import ValidationError as DRFValidationError
# AWX
from awx.api.versioning import reverse
@@ -41,6 +45,7 @@ from awx.main.models.rbac import (
ROLE_SINGLETON_SYSTEM_ADMINISTRATOR,
ROLE_SINGLETON_SYSTEM_AUDITOR,
)
from awx.main.models import Team, Organization
from awx.main.utils import encrypt_field
from . import injectors as builtin_injectors
@@ -315,6 +320,16 @@ class Credential(PasswordFieldsModel, CommonModelNameNotUnique, ResourceMixin):
else:
raise ValueError('{} is not a dynamic input field'.format(field_name))
def validate_role_assignment(self, actor, role_definition):
if self.organization:
if isinstance(actor, User):
if actor.is_superuser or Organization.access_qs(actor, 'member').filter(id=self.organization.id).exists():
return
if isinstance(actor, Team):
if actor.organization == self.organization:
return
raise DRFValidationError({'detail': _(f"You cannot grant credential access to a {actor._meta.object_name} not in the credentials' organization")})
class CredentialType(CommonModelNameNotUnique):
"""

View File

@@ -4,11 +4,12 @@ import datetime
from datetime import timezone
import logging
from collections import defaultdict
import itertools
import time
from django.conf import settings
from django.core.exceptions import ObjectDoesNotExist
from django.db import models, DatabaseError
from django.db import models, DatabaseError, transaction
from django.db.models.functions import Cast
from django.utils.dateparse import parse_datetime
from django.utils.text import Truncator
@@ -605,19 +606,23 @@ class JobEvent(BasePlaybookEvent):
def _update_host_metrics(updated_hosts_list):
from awx.main.models import HostMetric # circular import
# bulk-create
current_time = now()
HostMetric.objects.bulk_create(
[HostMetric(hostname=hostname, last_automation=current_time) for hostname in updated_hosts_list], ignore_conflicts=True, batch_size=100
)
# bulk-update
batch_start, batch_size = 0, 1000
while batch_start <= len(updated_hosts_list):
batched_host_list = updated_hosts_list[batch_start : (batch_start + batch_size)]
HostMetric.objects.filter(hostname__in=batched_host_list).update(
last_automation=current_time, automated_counter=models.F('automated_counter') + 1, deleted=False
)
batch_start += batch_size
# FUTURE:
# - Hand-rolled implementation of itertools.batched(), introduced in Python 3.12. Replace.
# - Ability to do ORM upserts *may* have been introduced in Django 5.0.
# See the entry about `create_defaults` in https://docs.djangoproject.com/en/5.0/releases/5.0/#models.
# Hopefully this will be fully ready for batch use by 5.2 LTS.
args = [iter(updated_hosts_list)] * 500
for hosts in itertools.zip_longest(*args):
with transaction.atomic():
HostMetric.objects.bulk_create(
[HostMetric(hostname=hostname, last_automation=current_time) for hostname in hosts if hostname is not None], ignore_conflicts=True
)
HostMetric.objects.filter(hostname__in=hosts).update(
last_automation=current_time, automated_counter=models.F('automated_counter') + 1, deleted=False
)
@property
def job_verbosity(self):

View File

@@ -1,6 +1,8 @@
from django.db import models
from django.utils.translation import gettext_lazy as _
from rest_framework.exceptions import ValidationError
from awx.api.versioning import reverse
from awx.main.models.base import CommonModel
from awx.main.validators import validate_container_image_name
@@ -12,6 +14,8 @@ __all__ = ['ExecutionEnvironment']
class ExecutionEnvironment(CommonModel):
class Meta:
ordering = ('-created',)
# Remove view permission, as a temporary solution, defer to organization read permission
default_permissions = ('add', 'change', 'delete')
PULL_CHOICES = [
('always', _("Always pull container before running.")),
@@ -53,3 +57,12 @@ class ExecutionEnvironment(CommonModel):
def get_absolute_url(self, request=None):
return reverse('api:execution_environment_detail', kwargs={'pk': self.pk}, request=request)
def validate_role_assignment(self, actor, role_definition):
if self.managed:
raise ValidationError({'object_id': _('Can not assign object roles to managed Execution Environments')})
if self.organization_id is None:
raise ValidationError({'object_id': _('Can not assign object roles to global Execution Environments')})
if actor._meta.model_name == 'user' and (not actor.has_obj_perm(self.organization, 'view')):
raise ValidationError({'user': _('User must have view permission to Execution Environment organization')})

View File

@@ -933,6 +933,7 @@ class InventorySourceOptions(BaseModel):
('controller', _('Red Hat Ansible Automation Platform')),
('insights', _('Red Hat Insights')),
('terraform', _('Terraform State')),
('openshift_virtualization', _('OpenShift Virtualization')),
]
# From the options of the Django management base command
@@ -1042,7 +1043,7 @@ class InventorySourceOptions(BaseModel):
def cloud_credential_validation(source, cred):
if not source:
return None
if cred and source not in ('custom', 'scm'):
if cred and source not in ('custom', 'scm', 'openshift_virtualization'):
# If a credential was provided, it's important that it matches
# the actual inventory source being used (Amazon requires Amazon
# credentials; Rackspace requires Rackspace credentials; etc...)
@@ -1051,12 +1052,14 @@ class InventorySourceOptions(BaseModel):
# Allow an EC2 source to omit the credential. If Tower is running on
# an EC2 instance with an IAM Role assigned, boto will use credentials
# from the instance metadata instead of those explicitly provided.
elif source in CLOUD_PROVIDERS and source != 'ec2':
elif source in CLOUD_PROVIDERS and source not in ['ec2', 'openshift_virtualization']:
return _('Credential is required for a cloud source.')
elif source == 'custom' and cred and cred.credential_type.kind in ('scm', 'ssh', 'insights', 'vault'):
return _('Credentials of type machine, source control, insights and vault are disallowed for custom inventory sources.')
elif source == 'scm' and cred and cred.credential_type.kind in ('insights', 'vault'):
return _('Credentials of type insights and vault are disallowed for scm inventory sources.')
elif source == 'openshift_virtualization' and cred and cred.credential_type.kind != 'kubernetes':
return _('Credentials of type kubernetes is requred for openshift_virtualization inventory sources.')
return None
def get_cloud_credential(self):
@@ -1693,6 +1696,16 @@ class insights(PluginFileInjector):
use_fqcn = True
class openshift_virtualization(PluginFileInjector):
plugin_name = 'kubevirt'
base_injector = 'template'
namespace = 'kubevirt'
collection = 'core'
downstream_namespace = 'redhat'
downstream_collection = 'openshift_virtualization'
use_fqcn = True
class constructed(PluginFileInjector):
plugin_name = 'constructed'
namespace = 'ansible'

View File

@@ -396,11 +396,11 @@ class JobNotificationMixin(object):
'verbosity': 0,
},
'job_friendly_name': 'Job',
'url': 'https://towerhost/#/jobs/playbook/1010',
'url': 'https://platformhost/#/jobs/playbook/1010',
'approval_status': 'approved',
'approval_node_name': 'Approve Me',
'workflow_url': 'https://towerhost/#/jobs/workflow/1010',
'job_metadata': """{'url': 'https://towerhost/$/jobs/playbook/13',
'workflow_url': 'https://platformhost/#/jobs/workflow/1010',
'job_metadata': """{'url': 'https://platformhost/$/jobs/playbook/13',
'traceback': '',
'status': 'running',
'started': '2019-08-07T21:46:38.362630+00:00',

View File

@@ -557,12 +557,25 @@ def get_role_definition(role):
f = obj._meta.get_field(role.role_field)
action_name = f.name.rsplit("_", 1)[0]
model_print = type(obj).__name__
rd_name = f'{model_print} {action_name.title()} Compat'
perm_list = get_role_codenames(role)
defaults = {
'content_type_id': role.content_type_id,
'description': f'Has {action_name.title()} permission to {model_print} for backwards API compatibility',
}
# use Controller-specific role definitions for Team/Organization and member/admin
# instead of platform role definitions
# these should exist in the system already, so just do a lookup by role definition name
if model_print in ['Team', 'Organization'] and action_name in ['member', 'admin']:
rd_name = f'Controller {model_print} {action_name.title()}'
rd = RoleDefinition.objects.filter(name=rd_name).first()
if rd:
return rd
else:
return RoleDefinition.objects.create_from_permissions(permissions=perm_list, name=rd_name, managed=True, **defaults)
else:
rd_name = f'{model_print} {action_name.title()} Compat'
with impersonate(None):
try:
rd, created = RoleDefinition.objects.get_or_create(name=rd_name, permissions=perm_list, defaults=defaults)
@@ -585,20 +598,32 @@ def get_role_from_object_role(object_role):
model_name, role_name, _ = rd.name.split()
role_name = role_name.lower()
role_name += '_role'
elif rd.name.startswith('Controller') and rd.name.endswith(' Admin'):
# Controller Organization Admin and Controller Team Admin
role_name = 'admin_role'
elif rd.name.startswith('Controller') and rd.name.endswith(' Member'):
# Controller Organization Member and Controller Team Member
role_name = 'member_role'
elif rd.name.endswith(' Admin') and rd.name.count(' ') == 2:
# cases like "Organization Project Admin"
model_name, target_model_name, role_name = rd.name.split()
role_name = role_name.lower()
model_cls = apps.get_model('main', target_model_name)
target_model_name = get_type_for_model(model_cls)
# exception cases completely specific to one model naming convention
if target_model_name == 'notification_template':
target_model_name = 'notification' # total exception
target_model_name = 'notification'
elif target_model_name == 'workflow_job_template':
target_model_name = 'workflow'
role_name = f'{target_model_name}_admin_role'
elif rd.name.endswith(' Admin'):
# cases like "project-admin"
role_name = 'admin_role'
elif rd.name == 'Organization Audit':
role_name = 'auditor_role'
else:
print(rd.name)
model_name, role_name = rd.name.split()
role_name = role_name.lower()
role_name += '_role'
@@ -683,9 +708,15 @@ def sync_parents_to_new_rbac(instance, action, model, pk_set, reverse, **kwargs)
for role_id in pk_set:
if reverse:
child_role = Role.objects.get(id=role_id)
try:
child_role = Role.objects.get(id=role_id)
except Role.DoesNotExist:
continue
else:
parent_role = Role.objects.get(id=role_id)
try:
parent_role = Role.objects.get(id=role_id)
except Role.DoesNotExist:
continue
# To a fault, we want to avoid running this if triggered from implicit_parents management
# we only want to do anything if we know for sure this is a non-implicit team role

View File

@@ -17,7 +17,7 @@ from collections import OrderedDict
# Django
from django.conf import settings
from django.db import models, connection
from django.db import models, connection, transaction
from django.core.exceptions import NON_FIELD_ERRORS
from django.utils.translation import gettext_lazy as _
from django.utils.timezone import now
@@ -31,6 +31,7 @@ from rest_framework.exceptions import ParseError
from polymorphic.models import PolymorphicModel
from ansible_base.lib.utils.models import prevent_search, get_type_for_model
from ansible_base.rbac import permission_registry
# AWX
from awx.main.models.base import CommonModelNameNotUnique, PasswordFieldsModel, NotificationFieldsModel
@@ -197,9 +198,7 @@ class UnifiedJobTemplate(PolymorphicModel, CommonModelNameNotUnique, ExecutionEn
@classmethod
def _submodels_with_roles(cls):
ujt_classes = [c for c in cls.__subclasses__() if c._meta.model_name not in ['inventorysource', 'systemjobtemplate']]
ct_dict = ContentType.objects.get_for_models(*ujt_classes)
return [ct.id for ct in ct_dict.values()]
return [c for c in cls.__subclasses__() if permission_registry.is_registered(c)]
@classmethod
def accessible_pk_qs(cls, accessor, role_field):
@@ -215,8 +214,16 @@ class UnifiedJobTemplate(PolymorphicModel, CommonModelNameNotUnique, ExecutionEn
action = to_permissions[role_field]
# Special condition for super auditor
role_subclasses = cls._submodels_with_roles()
role_cts = ContentType.objects.get_for_models(*role_subclasses).values()
all_codenames = {f'{action}_{cls._meta.model_name}' for cls in role_subclasses}
if not (all_codenames - accessor.singleton_permissions()):
qs = cls.objects.filter(polymorphic_ctype__in=role_cts)
return qs.values_list('id', flat=True)
return (
RoleEvaluation.objects.filter(role__in=accessor.has_roles.all(), codename__startswith=action, content_type_id__in=cls._submodels_with_roles())
RoleEvaluation.objects.filter(role__in=accessor.has_roles.all(), codename__in=all_codenames, content_type_id__in=[ct.id for ct in role_cts])
.values_list('object_id')
.distinct()
)
@@ -273,7 +280,14 @@ class UnifiedJobTemplate(PolymorphicModel, CommonModelNameNotUnique, ExecutionEn
if new_next_schedule:
if new_next_schedule.pk == self.next_schedule_id and new_next_schedule.next_run == self.next_job_run:
return # no-op, common for infrequent schedules
self.next_schedule = new_next_schedule
# If in a transaction, use select_for_update to lock the next schedule row, which
# prevents a race condition if new_next_schedule is deleted elsewhere during this transaction
if transaction.get_autocommit():
self.next_schedule = related_schedules.first()
else:
self.next_schedule = related_schedules.select_for_update().first()
self.next_job_run = new_next_schedule.next_run
self.save(update_fields=['next_schedule', 'next_job_run'])

View File

@@ -138,7 +138,8 @@ class TaskBase:
# Lock
with task_manager_bulk_reschedule():
with advisory_lock(f"{self.prefix}_lock", wait=False) as acquired:
lock_session_timeout_milliseconds = settings.TASK_MANAGER_LOCK_TIMEOUT * 1000 # convert to milliseconds
with advisory_lock(f"{self.prefix}_lock", lock_session_timeout_milliseconds=lock_session_timeout_milliseconds, wait=False) as acquired:
with transaction.atomic():
if acquired is False:
logger.debug(f"Not running {self.prefix} scheduler, another task holds lock")

View File

@@ -405,10 +405,11 @@ class AWXReceptorJob:
finally:
# Make sure to always release the work unit if we established it
if self.unit_id is not None and settings.RECEPTOR_RELEASE_WORK:
try:
receptor_ctl.simple_command(f"work release {self.unit_id}")
except Exception:
logger.exception(f"Error releasing work unit {self.unit_id}.")
if settings.RECPETOR_KEEP_WORK_ON_ERROR and getattr(res, 'status', 'error') == 'error':
try:
receptor_ctl.simple_command(f"work release {self.unit_id}")
except Exception:
logger.exception(f"Error releasing work unit {self.unit_id}.")
def _run_internal(self, receptor_ctl):
# Create a socketpair. Where the left side will be used for writing our payload

View File

@@ -36,6 +36,9 @@ import ansible_runner.cleanup
# dateutil
from dateutil.parser import parse as parse_date
# django-ansible-base
from ansible_base.resource_registry.tasks.sync import SyncExecutor
# AWX
from awx import __version__ as awx_application_version
from awx.main.access import access_registry
@@ -51,7 +54,7 @@ from awx.main.models import (
Job,
convert_jsonfields,
)
from awx.main.constants import ACTIVE_STATES
from awx.main.constants import ACTIVE_STATES, ERROR_STATES
from awx.main.dispatch.publish import task
from awx.main.dispatch import get_task_queuename, reaper
from awx.main.utils.common import ignore_inventory_computed_fields, ignore_inventory_group_removal
@@ -682,6 +685,8 @@ def awx_receptor_workunit_reaper():
unit_ids = [id for id in receptor_work_list]
jobs_with_unreleased_receptor_units = UnifiedJob.objects.filter(work_unit_id__in=unit_ids).exclude(status__in=ACTIVE_STATES)
if settings.RECEPTOR_KEEP_WORK_ON_ERROR:
jobs_with_unreleased_receptor_units = jobs_with_unreleased_receptor_units.exclude(status__in=ERROR_STATES)
for job in jobs_with_unreleased_receptor_units:
logger.debug(f"{job.log_format} is not active, reaping receptor work unit {job.work_unit_id}")
receptor_ctl.simple_command(f"work cancel {job.work_unit_id}")
@@ -701,7 +706,10 @@ def awx_k8s_reaper():
logger.debug("Checking for orphaned k8s pods for {}.".format(group))
pods = PodManager.list_active_jobs(group)
time_cutoff = now() - timedelta(seconds=settings.K8S_POD_REAPER_GRACE_PERIOD)
for job in UnifiedJob.objects.filter(pk__in=pods.keys(), finished__lte=time_cutoff).exclude(status__in=ACTIVE_STATES):
reap_job_candidates = UnifiedJob.objects.filter(pk__in=pods.keys(), finished__lte=time_cutoff).exclude(status__in=ACTIVE_STATES)
if settings.RECEPTOR_KEEP_WORK_ON_ERROR:
reap_job_candidates = reap_job_candidates.exclude(status__in=ERROR_STATES)
for job in reap_job_candidates:
logger.debug('{} is no longer active, reaping orphaned k8s pod'.format(job.log_format))
try:
pm = PodManager(job)
@@ -712,7 +720,8 @@ def awx_k8s_reaper():
@task(queue=get_task_queuename)
def awx_periodic_scheduler():
with advisory_lock('awx_periodic_scheduler_lock', wait=False) as acquired:
lock_session_timeout_milliseconds = settings.TASK_MANAGER_LOCK_TIMEOUT * 1000
with advisory_lock('awx_periodic_scheduler_lock', lock_session_timeout_milliseconds=lock_session_timeout_milliseconds, wait=False) as acquired:
if acquired is False:
logger.debug("Not running periodic scheduler, another task holds lock")
return
@@ -964,3 +973,27 @@ def deep_copy_model_obj(model_module, model_name, obj_pk, new_obj_pk, user_pk, p
permission_check_func(creater, copy_mapping.values())
if isinstance(new_obj, Inventory):
update_inventory_computed_fields.delay(new_obj.id)
@task(queue=get_task_queuename)
def periodic_resource_sync():
if not getattr(settings, 'RESOURCE_SERVER', None):
logger.debug("Skipping periodic resource_sync, RESOURCE_SERVER not configured")
return
with advisory_lock('periodic_resource_sync', wait=False) as acquired:
if acquired is False:
logger.debug("Not running periodic_resource_sync, another task holds lock")
return
logger.debug("Running periodic resource sync")
executor = SyncExecutor()
executor.run()
for key, item_list in executor.results.items():
if not item_list or key == 'noop':
continue
# Log creations and conflicts
if len(item_list) > 10 and settings.LOG_AGGREGATOR_LEVEL != 'DEBUG':
logger.info(f'Periodic resource sync {key}, first 10 items:\n{item_list[:10]}')
else:
logger.info(f'Periodic resource sync {key}:\n{item_list}')

View File

@@ -0,0 +1,5 @@
{
"K8S_AUTH_HOST": "https://foo.invalid",
"K8S_AUTH_API_KEY": "fooo",
"K8S_AUTH_VERIFY_SSL": "False"
}

View File

@@ -1,22 +1,30 @@
import pytest
from unittest import mock
from awx.api.versioning import reverse
from django.test.utils import override_settings
from ansible_base.jwt_consumer.common.util import generate_x_trusted_proxy_header
from ansible_base.lib.testing.fixtures import rsa_keypair_factory, rsa_keypair # noqa: F401; pylint: disable=unused-import
class HeaderTrackingMiddleware(object):
def __init__(self):
self.environ = {}
def process_request(self, request):
pass
def process_response(self, request, response):
self.environ = request.environ
@pytest.mark.django_db
def test_proxy_ip_allowed(get, patch, admin):
url = reverse('api:setting_singleton_detail', kwargs={'category_slug': 'system'})
patch(url, user=admin, data={'REMOTE_HOST_HEADERS': ['HTTP_X_FROM_THE_LOAD_BALANCER', 'REMOTE_ADDR', 'REMOTE_HOST']})
class HeaderTrackingMiddleware(object):
environ = {}
def process_request(self, request):
pass
def process_response(self, request, response):
self.environ = request.environ
# By default, `PROXY_IP_ALLOWED_LIST` is disabled, so custom `REMOTE_HOST_HEADERS`
# should just pass through
middleware = HeaderTrackingMiddleware()
@@ -45,6 +53,51 @@ def test_proxy_ip_allowed(get, patch, admin):
assert middleware.environ['HTTP_X_FROM_THE_LOAD_BALANCER'] == 'some-actual-ip'
@pytest.mark.django_db
class TestTrustedProxyAllowListIntegration:
@pytest.fixture
def url(self, patch, admin):
url = reverse('api:setting_singleton_detail', kwargs={'category_slug': 'system'})
patch(url, user=admin, data={'REMOTE_HOST_HEADERS': ['HTTP_X_FROM_THE_LOAD_BALANCER', 'REMOTE_ADDR', 'REMOTE_HOST']})
patch(url, user=admin, data={'PROXY_IP_ALLOWED_LIST': ['my.proxy.example.org']})
return url
@pytest.fixture
def middleware(self):
return HeaderTrackingMiddleware()
def test_x_trusted_proxy_valid_signature(self, get, admin, rsa_keypair, url, middleware): # noqa: F811
# Headers should NOT get deleted
headers = {
'HTTP_X_TRUSTED_PROXY': generate_x_trusted_proxy_header(rsa_keypair.private),
'HTTP_X_FROM_THE_LOAD_BALANCER': 'some-actual-ip',
}
with mock.patch('ansible_base.jwt_consumer.common.cache.JWTCache.get_key_from_cache', lambda self: None):
with override_settings(ANSIBLE_BASE_JWT_KEY=rsa_keypair.public, PROXY_IP_ALLOWED_LIST=[]):
get(url, user=admin, middleware=middleware, **headers)
assert middleware.environ['HTTP_X_FROM_THE_LOAD_BALANCER'] == 'some-actual-ip'
def test_x_trusted_proxy_invalid_signature(self, get, admin, url, patch, middleware):
# Headers should NOT get deleted
headers = {
'HTTP_X_TRUSTED_PROXY': 'DEAD-BEEF',
'HTTP_X_FROM_THE_LOAD_BALANCER': 'some-actual-ip',
}
with override_settings(PROXY_IP_ALLOWED_LIST=[]):
get(url, user=admin, middleware=middleware, **headers)
assert middleware.environ['HTTP_X_FROM_THE_LOAD_BALANCER'] == 'some-actual-ip'
def test_x_trusted_proxy_invalid_signature_valid_proxy(self, get, admin, url, middleware):
# A valid explicit proxy SHOULD result in sensitive headers NOT being deleted, regardless of the trusted proxy signature results
headers = {
'HTTP_X_TRUSTED_PROXY': 'DEAD-BEEF',
'REMOTE_ADDR': 'my.proxy.example.org',
'HTTP_X_FROM_THE_LOAD_BALANCER': 'some-actual-ip',
}
get(url, user=admin, middleware=middleware, **headers)
assert middleware.environ['HTTP_X_FROM_THE_LOAD_BALANCER'] == 'some-actual-ip'
@pytest.mark.django_db
class TestDeleteViews:
def test_sublist_delete_permission_check(self, inventory_source, host, rando, delete):

View File

@@ -0,0 +1,64 @@
import pytest
from awx.api.versioning import reverse
from awx.main.models import Organization
@pytest.mark.django_db
class TestImmutableSharedFields:
@pytest.fixture(autouse=True)
def configure_settings(self, settings):
settings.ALLOW_LOCAL_RESOURCE_MANAGEMENT = False
def test_create_raises_permission_denied(self, admin_user, post):
orgA = Organization.objects.create(name='orgA')
resp = post(
url=reverse('api:team_list'),
data={'name': 'teamA', 'organization': orgA.id},
user=admin_user,
expect=403,
)
assert "Creation of this resource is not allowed" in resp.data['detail']
def test_perform_delete_raises_permission_denied(self, admin_user, delete):
orgA = Organization.objects.create(name='orgA')
team = orgA.teams.create(name='teamA')
resp = delete(
url=reverse('api:team_detail', kwargs={'pk': team.id}),
user=admin_user,
expect=403,
)
assert "Deletion of this resource is not allowed" in resp.data['detail']
def test_perform_update(self, admin_user, patch):
orgA = Organization.objects.create(name='orgA')
# allow patching non-shared fields
patch(
url=reverse('api:organization_detail', kwargs={'pk': orgA.id}),
data={"max_hosts": 76},
user=admin_user,
expect=200,
)
# prevent patching shared fields
resp = patch(url=reverse('api:organization_detail', kwargs={'pk': orgA.id}), data={"name": "orgB"}, user=admin_user, expect=403)
assert "Cannot change shared field" in resp.data['name']
@pytest.mark.parametrize(
'role',
['admin_role', 'member_role'],
)
@pytest.mark.parametrize('resource', ['organization', 'team'])
def test_prevent_assigning_member_to_organization_or_team(self, admin_user, post, resource, role):
orgA = Organization.objects.create(name='orgA')
if resource == 'organization':
role = getattr(orgA, role)
elif resource == 'team':
teamA = orgA.teams.create(name='teamA')
role = getattr(teamA, role)
resp = post(
url=reverse('api:user_roles_list', kwargs={'pk': admin_user.id}),
data={'id': role.id},
user=admin_user,
expect=403,
)
assert f"Cannot directly modify user membership to {resource}." in resp.data['msg']

View File

@@ -32,13 +32,6 @@ def node_type_instance():
return fn
@pytest.fixture
def instance_group(job_factory):
ig = InstanceGroup(name="east")
ig.save()
return ig
@pytest.fixture
def containerized_instance_group(instance_group, kube_credential):
ig = InstanceGroup(name="container")

View File

@@ -1,4 +1,5 @@
import pytest
from unittest import mock
# AWX
from awx.api.serializers import JobTemplateSerializer
@@ -8,10 +9,15 @@ from awx.main.migrations import _save_password_keys as save_password_keys
# Django
from django.apps import apps
from django.test.utils import override_settings
# DRF
from rest_framework.exceptions import ValidationError
# DAB
from ansible_base.jwt_consumer.common.util import generate_x_trusted_proxy_header
from ansible_base.lib.testing.fixtures import rsa_keypair_factory, rsa_keypair # noqa: F401; pylint: disable=unused-import
@pytest.mark.django_db
@pytest.mark.parametrize(
@@ -369,3 +375,113 @@ def test_job_template_missing_inventory(project, inventory, admin_user, post):
)
assert r.status_code == 400
assert "Cannot start automatically, an inventory is required." in str(r.data)
@pytest.mark.django_db
class TestJobTemplateCallbackProxyIntegration:
"""
Test the interaction of provision job template callback feature and:
settings.PROXY_IP_ALLOWED_LIST
x-trusted-proxy http header
"""
@pytest.fixture
def job_template(self, inventory, project):
jt = JobTemplate.objects.create(name='test-jt', inventory=inventory, project=project, playbook='helloworld.yml', host_config_key='abcd')
return jt
@override_settings(REMOTE_HOST_HEADERS=['HTTP_X_FROM_THE_LOAD_BALANCER', 'REMOTE_ADDR', 'REMOTE_HOST'], PROXY_IP_ALLOWED_LIST=['my.proxy.example.org'])
def test_host_not_found(self, job_template, admin_user, post, rsa_keypair): # noqa: F811
job_template.inventory.hosts.create(name='foobar')
headers = {
'HTTP_X_FROM_THE_LOAD_BALANCER': 'baz',
'REMOTE_HOST': 'baz',
'REMOTE_ADDR': 'baz',
}
r = post(
url=reverse('api:job_template_callback', kwargs={'pk': job_template.pk}), data={'host_config_key': 'abcd'}, user=admin_user, expect=400, **headers
)
assert r.data['msg'] == 'No matching host could be found!'
@pytest.mark.parametrize(
'headers, expected',
(
pytest.param(
{
'HTTP_X_FROM_THE_LOAD_BALANCER': 'foobar',
'REMOTE_HOST': 'my.proxy.example.org',
},
201,
),
pytest.param(
{
'HTTP_X_FROM_THE_LOAD_BALANCER': 'foobar',
'REMOTE_HOST': 'not-my-proxy.org',
},
400,
),
),
)
@override_settings(REMOTE_HOST_HEADERS=['HTTP_X_FROM_THE_LOAD_BALANCER', 'REMOTE_ADDR', 'REMOTE_HOST'], PROXY_IP_ALLOWED_LIST=['my.proxy.example.org'])
def test_proxy_ip_allowed_list(self, job_template, admin_user, post, headers, expected): # noqa: F811
job_template.inventory.hosts.create(name='my.proxy.example.org')
post(
url=reverse('api:job_template_callback', kwargs={'pk': job_template.pk}),
data={'host_config_key': 'abcd'},
user=admin_user,
expect=expected,
**headers
)
@override_settings(REMOTE_HOST_HEADERS=['HTTP_X_FROM_THE_LOAD_BALANCER', 'REMOTE_ADDR', 'REMOTE_HOST'], PROXY_IP_ALLOWED_LIST=[])
def test_no_proxy_trust_all_headers(self, job_template, admin_user, post):
job_template.inventory.hosts.create(name='foobar')
headers = {
'HTTP_X_FROM_THE_LOAD_BALANCER': 'foobar',
'REMOTE_ADDR': 'bar',
'REMOTE_HOST': 'baz',
}
post(url=reverse('api:job_template_callback', kwargs={'pk': job_template.pk}), data={'host_config_key': 'abcd'}, user=admin_user, expect=201, **headers)
@override_settings(REMOTE_HOST_HEADERS=['HTTP_X_FROM_THE_LOAD_BALANCER', 'REMOTE_ADDR', 'REMOTE_HOST'], PROXY_IP_ALLOWED_LIST=['my.proxy.example.org'])
def test_trusted_proxy(self, job_template, admin_user, post, rsa_keypair): # noqa: F811
job_template.inventory.hosts.create(name='foobar')
headers = {
'HTTP_X_TRUSTED_PROXY': generate_x_trusted_proxy_header(rsa_keypair.private),
'HTTP_X_FROM_THE_LOAD_BALANCER': 'foobar, my.proxy.example.org',
}
with mock.patch('ansible_base.jwt_consumer.common.cache.JWTCache.get_key_from_cache', lambda self: None):
with override_settings(ANSIBLE_BASE_JWT_KEY=rsa_keypair.public):
post(
url=reverse('api:job_template_callback', kwargs={'pk': job_template.pk}),
data={'host_config_key': 'abcd'},
user=admin_user,
expect=201,
**headers
)
@override_settings(REMOTE_HOST_HEADERS=['HTTP_X_FROM_THE_LOAD_BALANCER', 'REMOTE_ADDR', 'REMOTE_HOST'], PROXY_IP_ALLOWED_LIST=['my.proxy.example.org'])
def test_trusted_proxy_host_not_found(self, job_template, admin_user, post, rsa_keypair): # noqa: F811
job_template.inventory.hosts.create(name='foobar')
headers = {
'HTTP_X_TRUSTED_PROXY': generate_x_trusted_proxy_header(rsa_keypair.private),
'HTTP_X_FROM_THE_LOAD_BALANCER': 'baz, my.proxy.example.org',
'REMOTE_ADDR': 'bar',
'REMOTE_HOST': 'baz',
}
with mock.patch('ansible_base.jwt_consumer.common.cache.JWTCache.get_key_from_cache', lambda self: None):
with override_settings(ANSIBLE_BASE_JWT_KEY=rsa_keypair.public):
post(
url=reverse('api:job_template_callback', kwargs={'pk': job_template.pk}),
data={'host_config_key': 'abcd'},
user=admin_user,
expect=400,
**headers
)

File diff suppressed because one or more lines are too long

View File

@@ -33,6 +33,27 @@ def test_fail_double_create_user(post, admin):
assert response.status_code == 400
@pytest.mark.django_db
def test_creating_user_retains_session(post, admin):
'''
Creating a new user should not refresh a new session id for the current user.
'''
with mock.patch('awx.api.serializers.update_session_auth_hash') as update_session_auth_hash:
response = post(reverse('api:user_list'), EXAMPLE_USER_DATA, admin)
assert response.status_code == 201
assert not update_session_auth_hash.called
@pytest.mark.django_db
def test_updating_own_password_refreshes_session(patch, admin):
'''
Updating your own password should refresh the session id.
'''
with mock.patch('awx.api.serializers.update_session_auth_hash') as update_session_auth_hash:
patch(reverse('api:user_detail', kwargs={'pk': admin.pk}), {'password': 'newpassword'}, admin, middleware=SessionMiddleware(mock.Mock()))
assert update_session_auth_hash.called
@pytest.mark.django_db
def test_create_delete_create_user(post, delete, admin):
response = post(reverse('api:user_list'), EXAMPLE_USER_DATA, admin, middleware=SessionMiddleware(mock.Mock()))

View File

@@ -20,7 +20,7 @@ from awx.main.migrations._dab_rbac import setup_managed_role_definitions
# AWX
from awx.main.models.projects import Project
from awx.main.models.ha import Instance
from awx.main.models.ha import Instance, InstanceGroup
from rest_framework.test import (
APIRequestFactory,
@@ -92,6 +92,11 @@ def deploy_jobtemplate(project, inventory, credential):
return jt
@pytest.fixture()
def execution_environment():
return ExecutionEnvironment.objects.create(name="test-ee", description="test-ee", managed=True)
@pytest.fixture
def setup_managed_roles():
"Run the migration script to pre-create managed role definitions"
@@ -730,6 +735,11 @@ def jt_linked(organization, project, inventory, machine_credential, credential,
return jt
@pytest.fixture
def instance_group():
return InstanceGroup.objects.create(name="east")
@pytest.fixture
def workflow_job_template(organization):
wjt = WorkflowJobTemplate.objects.create(name='test-workflow_job_template', organization=organization)

View File

@@ -109,3 +109,17 @@ def test_team_indirect_access(get, team, admin_user, inventory):
assert len(by_username['u1']['summary_fields']['indirect_access']) == 0
access_entry = by_username['u1']['summary_fields']['direct_access'][0]
assert sorted(access_entry['descendant_roles']) == sorted(['adhoc_role', 'use_role', 'update_role', 'read_role', 'admin_role'])
@pytest.mark.django_db
def test_workflow_access_list(workflow_job_template, alice, bob, setup_managed_roles, get, admin_user):
"""Basic verification that WFJT access_list is functional"""
workflow_job_template.admin_role.members.add(alice)
workflow_job_template.organization.workflow_admin_role.members.add(bob)
url = reverse('api:workflow_job_template_access_list', kwargs={'pk': workflow_job_template.pk})
for u in (alice, bob, admin_user):
response = get(url, user=u, expect=200)
user_ids = [item['id'] for item in response.data['results']]
assert alice.pk in user_ids
assert bob.pk in user_ids

View File

@@ -0,0 +1,41 @@
import pytest
from awx.main.access import InstanceGroupAccess, NotificationTemplateAccess
from ansible_base.rbac.models import RoleDefinition
@pytest.mark.django_db
def test_instance_group_object_role_delete(rando, instance_group, setup_managed_roles):
"""Basic functionality of IG object-level admin role function AAP-25506"""
rd = RoleDefinition.objects.get(name='InstanceGroup Admin')
rd.give_permission(rando, instance_group)
access = InstanceGroupAccess(rando)
assert access.can_delete(instance_group)
@pytest.mark.django_db
def test_notification_template_object_role_change(rando, notification_template, setup_managed_roles):
"""Basic functionality of NT object-level admin role function AAP-25493"""
rd = RoleDefinition.objects.get(name='NotificationTemplate Admin')
rd.give_permission(rando, notification_template)
access = NotificationTemplateAccess(rando)
assert access.can_change(notification_template, {'name': 'new name'})
@pytest.mark.django_db
def test_organization_auditor_role(rando, setup_managed_roles, organization, inventory, project, jt_linked):
obj_list = (inventory, project, jt_linked)
for obj in obj_list:
assert obj.organization == organization, obj # sanity
assert [rando.has_obj_perm(obj, 'view') for obj in obj_list] == [False for i in range(3)], obj_list
rd = RoleDefinition.objects.get(name='Organization Audit')
rd.give_permission(rando, organization)
codename_set = set(rd.permissions.values_list('codename', flat=True))
assert not ({'view_inventory', 'view_jobtemplate', 'audit_organization'} - codename_set) # sanity
assert [obj in type(obj).access_qs(rando) for obj in obj_list] == [True for i in range(3)], obj_list
assert [rando.has_obj_perm(obj, 'view') for obj in obj_list] == [True for i in range(3)], obj_list

View File

@@ -2,9 +2,11 @@ import pytest
from django.contrib.contenttypes.models import ContentType
from django.urls import reverse as django_reverse
from django.test.utils import override_settings
from awx.api.versioning import reverse
from awx.main.models import JobTemplate, Inventory, Organization
from awx.main.access import JobTemplateAccess, WorkflowJobTemplateAccess
from ansible_base.rbac.models import RoleDefinition
@@ -66,13 +68,17 @@ def test_assign_managed_role(admin_user, alice, rando, inventory, post, setup_ma
@pytest.mark.django_db
def test_assign_custom_delete_role(admin_user, rando, inventory, delete, patch):
# TODO: just a delete_inventory, without change_inventory
rd, _ = RoleDefinition.objects.get_or_create(
name='inventory-delete', permissions=['delete_inventory', 'view_inventory'], content_type=ContentType.objects.get_for_model(Inventory)
name='inventory-delete',
permissions=['delete_inventory', 'view_inventory', 'change_inventory'],
content_type=ContentType.objects.get_for_model(Inventory),
)
rd.give_permission(rando, inventory)
inv_id = inventory.pk
inv_url = reverse('api:inventory_detail', kwargs={'pk': inv_id})
patch(url=inv_url, data={"description": "new"}, user=rando, expect=403)
# TODO: eventually this will be valid test, for now ignore
# patch(url=inv_url, data={"description": "new"}, user=rando, expect=403)
delete(url=inv_url, user=rando, expect=202)
assert Inventory.objects.get(id=inv_id).pending_deletion
@@ -88,3 +94,129 @@ def test_assign_custom_add_role(admin_user, rando, organization, post, setup_man
inv_id = r.data['id']
inventory = Inventory.objects.get(id=inv_id)
assert rando.has_obj_perm(inventory, 'change')
@pytest.mark.django_db
def test_jt_creation_permissions(setup_managed_roles, inventory, project, rando):
"""This tests that if you assign someone required permissions in the new API
using the managed roles, then that works to give permissions to create a job template"""
inv_rd = RoleDefinition.objects.get(name='Inventory Admin')
proj_rd = RoleDefinition.objects.get(name='Project Admin')
# establish prior state
access = JobTemplateAccess(rando)
assert not access.can_add({'inventory': inventory.pk, 'project': project.pk, 'name': 'foo-jt'})
inv_rd.give_permission(rando, inventory)
proj_rd.give_permission(rando, project)
assert access.can_add({'inventory': inventory.pk, 'project': project.pk, 'name': 'foo-jt'})
@pytest.mark.django_db
def test_workflow_creation_permissions(setup_managed_roles, organization, workflow_job_template, rando):
"""Similar to JT, assigning new roles gives creator permissions"""
org_wf_rd = RoleDefinition.objects.get(name='Organization WorkflowJobTemplate Admin')
assert workflow_job_template.organization == organization # sanity
# establish prior state
access = WorkflowJobTemplateAccess(rando)
assert not access.can_add({'name': 'foo-flow', 'organization': organization.pk})
org_wf_rd.give_permission(rando, organization)
assert access.can_add({'name': 'foo-flow', 'organization': organization.pk})
@pytest.mark.django_db
def test_assign_credential_to_user_of_another_org(setup_managed_roles, credential, admin_user, rando, org_admin, organization, post):
'''Test that a credential can only be assigned to a user in the same organization'''
# cannot assign credential to rando, as rando is not in the same org as the credential
rd = RoleDefinition.objects.get(name="Credential Admin")
credential.organization = organization
credential.save(update_fields=['organization'])
assert credential.organization not in Organization.access_qs(rando, 'member')
url = django_reverse('roleuserassignment-list')
resp = post(url=url, data={"user": rando.id, "role_definition": rd.id, "object_id": credential.id}, user=admin_user, expect=400)
assert "You cannot grant credential access to a User not in the credentials' organization" in str(resp.data)
# can assign credential to superuser
rando.is_superuser = True
rando.save()
post(url=url, data={"user": rando.id, "role_definition": rd.id, "object_id": credential.id}, user=admin_user, expect=201)
# can assign credential to org_admin
assert credential.organization in Organization.access_qs(org_admin, 'member')
post(url=url, data={"user": org_admin.id, "role_definition": rd.id, "object_id": credential.id}, user=admin_user, expect=201)
@pytest.mark.django_db
@override_settings(ALLOW_LOCAL_ASSIGNING_JWT_ROLES=False)
def test_team_member_role_not_assignable(team, rando, post, admin_user, setup_managed_roles):
member_rd = RoleDefinition.objects.get(name='Organization Member')
url = django_reverse('roleuserassignment-list')
r = post(url, data={'object_id': team.id, 'role_definition': member_rd.id, 'user': rando.id}, user=admin_user, expect=400)
assert 'Not managed locally' in str(r.data)
@pytest.mark.django_db
def test_adding_user_to_org_member_role(setup_managed_roles, organization, admin, bob, post, get):
'''
Adding user to organization member role via the legacy RBAC endpoints
should give them access to the organization detail
'''
url_detail = reverse('api:organization_detail', kwargs={'pk': organization.id})
get(url_detail, user=bob, expect=403)
role = organization.member_role
url = reverse('api:role_users_list', kwargs={'pk': role.id})
post(url, data={'id': bob.id}, user=admin, expect=204)
get(url_detail, user=bob, expect=200)
@pytest.mark.django_db
@pytest.mark.parametrize('actor', ['user', 'team'])
@pytest.mark.parametrize('role_name', ['Organization Admin', 'Organization Member', 'Team Admin', 'Team Member'])
def test_prevent_adding_actor_to_platform_roles(setup_managed_roles, role_name, actor, organization, team, admin, bob, post):
'''
Prevent user or team from being added to platform-level roles
'''
rd = RoleDefinition.objects.get(name=role_name)
endpoint = 'roleuserassignment-list' if actor == 'user' else 'roleteamassignment-list'
url = django_reverse(endpoint)
object_id = team.id if 'Team' in role_name else organization.id
data = {'object_id': object_id, 'role_definition': rd.id}
actor_id = bob.id if actor == 'user' else team.id
data[actor] = actor_id
r = post(url, data=data, user=admin, expect=400)
assert 'Not managed locally' in str(r.data)
@pytest.mark.django_db
@pytest.mark.parametrize('role_name', ['Controller Team Admin', 'Controller Team Member'])
def test_adding_user_to_controller_team_roles(setup_managed_roles, role_name, team, admin, bob, post, get):
'''
Allow user to be added to Controller Team Admin or Controller Team Member
'''
url_detail = reverse('api:team_detail', kwargs={'pk': team.id})
get(url_detail, user=bob, expect=403)
rd = RoleDefinition.objects.get(name=role_name)
url = django_reverse('roleuserassignment-list')
post(url, data={'object_id': team.id, 'role_definition': rd.id, 'user': bob.id}, user=admin, expect=201)
get(url_detail, user=bob, expect=200)
@pytest.mark.django_db
@pytest.mark.parametrize('role_name', ['Controller Organization Admin', 'Controller Organization Member'])
def test_adding_user_to_controller_organization_roles(setup_managed_roles, role_name, organization, admin, bob, post, get):
'''
Allow user to be added to Controller Organization Admin or Controller Organization Member
'''
url_detail = reverse('api:organization_detail', kwargs={'pk': organization.id})
get(url_detail, user=bob, expect=403)
rd = RoleDefinition.objects.get(name=role_name)
url = django_reverse('roleuserassignment-list')
post(url, data={'object_id': organization.id, 'role_definition': rd.id, 'user': bob.id}, user=admin, expect=201)
get(url, user=bob, expect=200)

View File

@@ -0,0 +1,120 @@
import pytest
from django.apps import apps
from ansible_base.rbac.managed import SystemAuditor
from ansible_base.rbac import permission_registry
from awx.main.access import check_user_access, get_user_queryset
from awx.main.models import User, AdHocCommandEvent
from awx.api.versioning import reverse
@pytest.fixture
def ext_auditor_rd():
info = SystemAuditor(overrides={'name': 'Alien Auditor', 'shortname': 'ext_auditor'})
rd, _ = info.get_or_create(apps)
return rd
@pytest.fixture
def ext_auditor(ext_auditor_rd):
u = User.objects.create(username='external-auditor-user')
ext_auditor_rd.give_global_permission(u)
return u
@pytest.fixture
def obj_factory(request):
def _rf(fixture_name):
obj = request.getfixturevalue(fixture_name)
# special case to make obj organization-scoped
if obj._meta.model_name == 'executionenvironment':
obj.organization = request.getfixturevalue('organization')
obj.save(update_fields=['organization'])
return obj
return _rf
@pytest.mark.django_db
def test_access_qs_external_auditor(ext_auditor_rd, rando, job_template):
ext_auditor_rd.give_global_permission(rando)
jt_cls = apps.get_model('main', 'JobTemplate')
ujt_cls = apps.get_model('main', 'UnifiedJobTemplate')
assert job_template in jt_cls.access_qs(rando)
assert job_template.id in jt_cls.access_ids_qs(rando)
assert job_template.id in ujt_cls.accessible_pk_qs(rando, 'read_role')
@pytest.mark.django_db
@pytest.mark.parametrize('model', sorted(permission_registry.all_registered_models, key=lambda cls: cls._meta.model_name))
class TestExternalAuditorRoleAllModels:
def test_access_can_read_method(self, obj_factory, model, ext_auditor, rando):
fixture_name = model._meta.verbose_name.replace(' ', '_')
obj = obj_factory(fixture_name)
assert check_user_access(rando, model, 'read', obj) is False
assert check_user_access(ext_auditor, model, 'read', obj) is True
def test_access_get_queryset(self, obj_factory, model, ext_auditor, rando):
fixture_name = model._meta.verbose_name.replace(' ', '_')
obj = obj_factory(fixture_name)
assert obj not in get_user_queryset(rando, model)
assert obj in get_user_queryset(ext_auditor, model)
def test_global_list(self, obj_factory, model, ext_auditor, rando, get):
fixture_name = model._meta.verbose_name.replace(' ', '_')
obj_factory(fixture_name)
url = reverse(f'api:{fixture_name}_list')
r = get(url, user=rando, expect=200)
initial_ct = r.data['count']
r = get(url, user=ext_auditor, expect=200)
assert r.data['count'] == initial_ct + 1
if fixture_name in ('job_template', 'workflow_job_template'):
url = reverse('api:unified_job_template_list')
r = get(url, user=rando, expect=200)
initial_ct = r.data['count']
r = get(url, user=ext_auditor, expect=200)
assert r.data['count'] == initial_ct + 1
def test_detail_view(self, obj_factory, model, ext_auditor, rando, get):
fixture_name = model._meta.verbose_name.replace(' ', '_')
obj = obj_factory(fixture_name)
url = reverse(f'api:{fixture_name}_detail', kwargs={'pk': obj.pk})
get(url, user=rando, expect=403) # NOTE: should be 401
get(url, user=ext_auditor, expect=200)
@pytest.mark.django_db
class TestExternalAuditorNonRoleModels:
def test_ad_hoc_command_view(self, ad_hoc_command_factory, rando, ext_auditor, get):
"""The AdHocCommandAccess class references is_system_auditor
this is to prove it works with other system-level view roles"""
ad_hoc_command = ad_hoc_command_factory()
url = reverse('api:ad_hoc_command_list')
r = get(url, user=rando, expect=200)
assert r.data['count'] == 0
r = get(url, user=ext_auditor, expect=200)
assert r.data['count'] == 1
assert r.data['results'][0]['id'] == ad_hoc_command.id
event = AdHocCommandEvent.objects.create(ad_hoc_command=ad_hoc_command)
url = reverse('api:ad_hoc_command_ad_hoc_command_events_list', kwargs={'pk': ad_hoc_command.id})
r = get(url, user=rando, expect=403)
r = get(url, user=ext_auditor, expect=200)
assert r.data['count'] == 1
url = reverse('api:ad_hoc_command_event_detail', kwargs={'pk': event.id})
r = get(url, user=rando, expect=403)
r = get(url, user=ext_auditor, expect=200)
assert r.data['id'] == event.id

View File

@@ -0,0 +1,62 @@
import pytest
from ansible_base.rbac.models import RoleDefinition, DABPermission, RoleUserAssignment
@pytest.mark.django_db
def test_roles_to_not_create(setup_managed_roles):
assert RoleDefinition.objects.filter(name='Organization Admin').count() == 1
SHOULD_NOT_EXIST = ('Organization Organization Admin', 'Organization Team Admin', 'Organization InstanceGroup Admin')
bad_rds = RoleDefinition.objects.filter(name__in=SHOULD_NOT_EXIST)
if bad_rds.exists():
bad_names = list(bad_rds.values_list('name', flat=True))
raise Exception(f'Found RoleDefinitions that should not exist: {bad_names}')
@pytest.mark.django_db
def test_project_update_role(setup_managed_roles):
"""Role to allow updating a project on the object-level should exist"""
assert RoleDefinition.objects.filter(name='Project Update').count() == 1
@pytest.mark.django_db
def test_org_child_add_permission(setup_managed_roles):
for model_name in ('Project', 'NotificationTemplate', 'WorkflowJobTemplate', 'Inventory'):
rd = RoleDefinition.objects.get(name=f'Organization {model_name} Admin')
assert 'add_' in str(rd.permissions.values_list('codename', flat=True)), f'The {rd.name} role definition expected to contain add_ permissions'
# special case for JobTemplate, anyone can create one with use permission to project/inventory
assert not DABPermission.objects.filter(codename='add_jobtemplate').exists()
@pytest.mark.django_db
def test_controller_specific_roles_have_correct_permissions(setup_managed_roles):
'''
Controller specific roles should have the same permissions as the platform roles
e.g. Controller Team Admin should have same permission set as Team Admin
'''
for rd_name in ['Controller Team Admin', 'Controller Team Member', 'Controller Organization Member', 'Controller Organization Admin']:
rd = RoleDefinition.objects.get(name=rd_name)
rd_platform = RoleDefinition.objects.get(name=rd_name.split('Controller ')[1])
assert set(rd.permissions.all()) == set(rd_platform.permissions.all())
@pytest.mark.django_db
@pytest.mark.parametrize('resource_name', ['Team', 'Organization'])
@pytest.mark.parametrize('action', ['Member', 'Admin'])
def test_legacy_RBAC_uses_controller_specific_roles(setup_managed_roles, resource_name, action, team, bob, organization):
'''
Assignment to legacy RBAC roles should use controller specific role definitions
e.g. Controller Team Admin, Controller Team Member, Controller Organization Member, Controller Organization Admin
'''
resource = team if resource_name == 'Team' else organization
if action == 'Member':
resource.member_role.members.add(bob)
else:
resource.admin_role.members.add(bob)
rd = RoleDefinition.objects.get(name=f'Controller {resource_name} {action}')
rd_platform = RoleDefinition.objects.get(name=f'{resource_name} {action}')
assert RoleUserAssignment.objects.filter(role_definition=rd, user=bob, object_id=resource.id).exists()
assert not RoleUserAssignment.objects.filter(role_definition=rd_platform, user=bob, object_id=resource.id).exists()

View File

@@ -1,4 +1,5 @@
from unittest import mock
import json
import pytest
@@ -6,17 +7,29 @@ from django.contrib.contenttypes.models import ContentType
from crum import impersonate
from awx.main.models.rbac import get_role_from_object_role, give_creator_permissions
from awx.main.fields import ImplicitRoleField
from awx.main.models.rbac import get_role_from_object_role, give_creator_permissions, get_role_codenames, get_role_definition
from awx.main.models import User, Organization, WorkflowJobTemplate, WorkflowJobTemplateNode, Team
from awx.api.versioning import reverse
from ansible_base.rbac.models import RoleUserAssignment, RoleDefinition
from ansible_base.rbac import permission_registry
@pytest.mark.django_db
@pytest.mark.parametrize(
'role_name',
['execution_environment_admin_role', 'project_admin_role', 'admin_role', 'auditor_role', 'read_role', 'execute_role', 'notification_admin_role'],
[
'execution_environment_admin_role',
'workflow_admin_role',
'project_admin_role',
'admin_role',
'auditor_role',
'read_role',
'execute_role',
'approval_role',
'notification_admin_role',
],
)
def test_round_trip_roles(organization, rando, role_name, setup_managed_roles):
"""
@@ -26,11 +39,41 @@ def test_round_trip_roles(organization, rando, role_name, setup_managed_roles):
"""
getattr(organization, role_name).members.add(rando)
assignment = RoleUserAssignment.objects.get(user=rando)
print(assignment.role_definition.name)
old_role = get_role_from_object_role(assignment.object_role)
assert old_role.id == getattr(organization, role_name).id
@pytest.mark.django_db
@pytest.mark.parametrize('model', sorted(permission_registry.all_registered_models, key=lambda cls: cls._meta.model_name))
def test_role_migration_matches(request, model, setup_managed_roles):
fixture_name = model._meta.verbose_name.replace(' ', '_')
obj = request.getfixturevalue(fixture_name)
role_ct = 0
for field in obj._meta.get_fields():
if isinstance(field, ImplicitRoleField):
if field.name == 'read_role':
continue # intentionally left as "Compat" roles
role_ct += 1
old_role = getattr(obj, field.name)
old_codenames = set(get_role_codenames(old_role))
rd = get_role_definition(old_role)
new_codenames = set(rd.permissions.values_list('codename', flat=True))
# all the old roles should map to a non-Compat role definition
if 'Compat' not in rd.name:
model_rds = RoleDefinition.objects.filter(content_type=ContentType.objects.get_for_model(obj))
rd_data = {}
for rd in model_rds:
rd_data[rd.name] = list(rd.permissions.values_list('codename', flat=True))
assert (
'Compat' not in rd.name
), f'Permissions for old vs new roles did not match.\nold {field.name}: {old_codenames}\nnew:\n{json.dumps(rd_data, indent=2)}'
assert new_codenames == set(old_codenames)
# In the old system these models did not have object-level roles, all others expect some model roles
if model._meta.model_name not in ('notificationtemplate', 'executionenvironment'):
assert role_ct > 0
@pytest.mark.django_db
def test_role_naming(setup_managed_roles):
qs = RoleDefinition.objects.filter(content_type=ContentType.objects.get(model='jobtemplate'), name__endswith='dmin')
@@ -141,3 +184,32 @@ def test_implicit_parents_no_assignments(organization):
with mock.patch('awx.main.models.rbac.give_or_remove_permission') as mck:
Team.objects.create(name='random team', organization=organization)
mck.assert_not_called()
@pytest.mark.django_db
def test_user_auditor_rel(organization, rando, setup_managed_roles):
assert rando not in organization.auditor_role
audit_rd = RoleDefinition.objects.get(name='Organization Audit')
audit_rd.give_permission(rando, organization)
assert list(rando.auditor_of_organizations) == [organization]
@pytest.mark.django_db
@pytest.mark.parametrize('resource_name', ['Organization', 'Team'])
@pytest.mark.parametrize('role_name', ['Member', 'Admin'])
def test_mapping_from_controller_role_definitions_to_roles(organization, team, rando, role_name, resource_name, setup_managed_roles):
"""
ensure mappings for controller roles are correct
e.g.
Controller Organization Member > organization.member_role
Controller Organization Admin > organization.admin_role
Controller Team Member > team.member_role
Controller Team Admin > team.admin_role
"""
resource = organization if resource_name == 'Organization' else team
old_role_name = f"{role_name.lower()}_role"
getattr(resource, old_role_name).members.add(rando)
assignment = RoleUserAssignment.objects.get(user=rando)
assert assignment.role_definition.name == f'Controller {resource_name} {role_name}'
old_role = get_role_from_object_role(assignment.object_role)
assert old_role.id == getattr(resource, old_role_name).id

View File

@@ -4,25 +4,19 @@ import pytest
# CRUM
from crum import impersonate
# Django
from django.contrib.contenttypes.models import ContentType
# AWX
from awx.main.models import UnifiedJobTemplate, Job, JobTemplate, WorkflowJobTemplate, WorkflowApprovalTemplate, Project, WorkflowJob, Schedule, Credential
from awx.main.models import UnifiedJobTemplate, Job, JobTemplate, WorkflowJobTemplate, Project, WorkflowJob, Schedule, Credential
from awx.api.versioning import reverse
from awx.main.constants import JOB_VARIABLE_PREFIXES
@pytest.mark.django_db
def test_subclass_types():
assert set(UnifiedJobTemplate._submodels_with_roles()) == set(
[
ContentType.objects.get_for_model(JobTemplate).id,
ContentType.objects.get_for_model(Project).id,
ContentType.objects.get_for_model(WorkflowJobTemplate).id,
ContentType.objects.get_for_model(WorkflowApprovalTemplate).id,
]
)
assert set(UnifiedJobTemplate._submodels_with_roles()) == {
JobTemplate,
Project,
WorkflowJobTemplate,
}
@pytest.mark.django_db

View File

@@ -46,6 +46,8 @@ def generate_fake_var(element):
def credential_kind(source):
"""Given the inventory source kind, return expected credential kind"""
if source == 'openshift_virtualization':
return 'kubernetes_bearer_token'
return source.replace('ec2', 'aws')

View File

@@ -1,5 +1,4 @@
import glob
import json
import os
from django.conf import settings
@@ -12,122 +11,98 @@ except ImportError:
from pip._internal.req.constructors import parse_req_from_line
def test_python_and_js_licenses():
def index_licenses(path):
# Check for GPL (forbidden) and LGPL (need to ship source)
# This is not meant to be an exhaustive check.
def check_license(license_file):
with open(license_file) as f:
data = f.read()
is_lgpl = 'GNU LESSER GENERAL PUBLIC LICENSE' in data.upper()
# The LGPL refers to the GPL in-text
# Case-sensitive for GPL to match license text and not PSF license reference
is_gpl = 'GNU GENERAL PUBLIC LICENSE' in data and not is_lgpl
return (is_gpl, is_lgpl)
def check_license(license_file):
with open(license_file) as f:
data = f.read()
is_lgpl = 'GNU LESSER GENERAL PUBLIC LICENSE' in data.upper()
is_gpl = 'GNU GENERAL PUBLIC LICENSE' in data and not is_lgpl
return is_gpl, is_lgpl
def find_embedded_source_version(path, name):
files = os.listdir(path)
tgz_files = [f for f in files if f.endswith('.tar.gz')]
for tgz in tgz_files:
pkg_name = tgz.split('-')[0].split('_')[0]
if pkg_name == name:
return tgz.split('-')[1].split('.tar.gz')[0]
return None
list = {}
for txt_file in glob.glob('%s/*.txt' % path):
filename = txt_file.split('/')[-1]
name = filename[:-4].lower()
(is_gpl, is_lgpl) = check_license(txt_file)
list[name] = {
'name': name,
'filename': filename,
'gpl': is_gpl,
'source_required': (is_gpl or is_lgpl),
'source_version': find_embedded_source_version(path, name),
}
return list
def find_embedded_source_version(path, name):
files = os.listdir(path)
tgz_files = [f for f in files if f.endswith('.tar.gz')]
for tgz in tgz_files:
pkg_name = tgz.split('-')[0].split('_')[0]
if pkg_name == name:
return tgz.split('-')[1].split('.tar.gz')[0]
return None
def read_api_requirements(path):
ret = {}
skip_pbr_license_check = False
for req_file in ['requirements.txt', 'requirements_git.txt']:
fname = '%s/%s' % (path, req_file)
for reqt in parse_requirements(fname, session=''):
parsed_requirement = parse_req_from_line(reqt.requirement, None)
name = parsed_requirement.requirement.name
version = str(parsed_requirement.requirement.specifier)
if version.startswith('=='):
version = version[2:]
if parsed_requirement.link:
if str(parsed_requirement.link).startswith(('http://', 'https://')):
(name, version) = str(parsed_requirement.requirement).split('==', 1)
else:
(name, version) = parsed_requirement.link.filename.split('@', 1)
if name.endswith('.git'):
name = name[:-4]
if name == 'receptor':
name = 'receptorctl'
if name == 'ansible-runner':
skip_pbr_license_check = True
ret[name] = {'name': name, 'version': version}
if 'pbr' in ret and skip_pbr_license_check:
del ret['pbr']
return ret
def index_licenses(path):
licenses = {}
for txt_file in glob.glob(f'{path}/*.txt'):
filename = os.path.basename(txt_file)
name = filename[:-4].lower()
is_gpl, is_lgpl = check_license(txt_file)
licenses[name] = {
'name': name,
'filename': filename,
'gpl': is_gpl,
'source_required': is_gpl or is_lgpl,
'source_version': find_embedded_source_version(path, name),
}
return licenses
def read_ui_requirements(path):
def json_deps(jsondata):
ret = {}
deps = jsondata.get('dependencies', {})
for key in deps.keys():
key = key.lower()
devonly = deps[key].get('dev', False)
if not devonly:
if key not in ret.keys():
depname = key.replace('/', '-')
if depname[0] == '@':
depname = depname[1:]
ret[depname] = {'name': depname, 'version': deps[key]['version']}
ret.update(json_deps(deps[key]))
return ret
with open('%s/package-lock.json' % path) as f:
jsondata = json.load(f)
return json_deps(jsondata)
def parse_requirement(reqt):
parsed_requirement = parse_req_from_line(reqt.requirement, None)
name = parsed_requirement.requirement.name
version = str(parsed_requirement.requirement.specifier)
if version.startswith('=='):
version = version[2:]
if parsed_requirement.link:
if str(parsed_requirement.link).startswith(('http://', 'https://')):
name, version = str(parsed_requirement.requirement).split('==', 1)
else:
name, version = parsed_requirement.link.filename.split('@', 1)
if name.endswith('.git'):
name = name[:-4]
if name == 'receptor':
name = 'receptorctl'
return name, version
def remediate_licenses_and_requirements(licenses, requirements):
errors = []
items = list(licenses.keys())
items.sort()
for item in items:
if item not in [r.lower() for r in requirements.keys()] and item != 'awx':
errors.append(" license file %s does not correspond to an existing requirement; it should be removed." % (licenses[item]['filename'],))
continue
# uWSGI has a linking exception
if licenses[item]['gpl'] and item != 'uwsgi':
errors.append(" license for %s is GPL. This software cannot be used." % (item,))
if licenses[item]['source_required']:
version = requirements[item]['version']
if version != licenses[item]['source_version']:
errors.append(" embedded source for %s is %s instead of the required version %s" % (item, licenses[item]['source_version'], version))
elif licenses[item]['source_version']:
errors.append(" embedded source version %s for %s is included despite not being needed" % (licenses[item]['source_version'], item))
items = list(requirements.keys())
items.sort()
for item in items:
if item.lower() not in licenses.keys():
errors.append(" license for requirement %s is missing" % (item,))
return errors
base_dir = settings.BASE_DIR
api_licenses = index_licenses('%s/../licenses' % base_dir)
ui_licenses = index_licenses('%s/../licenses/ui' % base_dir)
api_requirements = read_api_requirements('%s/../requirements' % base_dir)
ui_requirements = read_ui_requirements('%s/ui' % base_dir)
def read_api_requirements(path):
requirements = {}
skip_pbr_license_check = False
for req_file in ['requirements.txt', 'requirements_git.txt']:
fname = f'{path}/{req_file}'
for reqt in parse_requirements(fname, session=''):
name, version = parse_requirement(reqt)
if name == 'ansible-runner':
skip_pbr_license_check = True
requirements[name] = {'name': name, 'version': version}
if 'pbr' in requirements and skip_pbr_license_check:
del requirements['pbr']
return requirements
def remediate_licenses_and_requirements(licenses, requirements):
errors = []
errors += remediate_licenses_and_requirements(ui_licenses, ui_requirements)
errors += remediate_licenses_and_requirements(api_licenses, api_requirements)
for item in sorted(licenses.keys()):
if item not in [r.lower() for r in requirements.keys()] and item != 'awx':
errors.append(f" license file {licenses[item]['filename']} does not correspond to an existing requirement; it should be removed.")
continue
if licenses[item]['gpl'] and item != 'uwsgi':
errors.append(f" license for {item} is GPL. This software cannot be used.")
if licenses[item]['source_required']:
version = requirements[item]['version']
if version != licenses[item]['source_version']:
errors.append(f" embedded source for {item} is {licenses[item]['source_version']} instead of the required version {version}")
elif licenses[item]['source_version']:
errors.append(f" embedded source version {licenses[item]['source_version']} for {item} is included despite not being needed")
for item in sorted(requirements.keys()):
if item.lower() not in licenses.keys():
errors.append(f" license for requirement {item} is missing")
return errors
def test_python_licenses():
base_dir = settings.BASE_DIR
api_licenses = index_licenses(f'{base_dir}/../licenses')
api_requirements = read_api_requirements(f'{base_dir}/../requirements')
errors = remediate_licenses_and_requirements(api_licenses, api_requirements)
if errors:
raise Exception('Included licenses not consistent with requirements:\n%s' % '\n'.join(errors))

View File

@@ -73,11 +73,16 @@ class TestMigrationSmoke:
def test_migrate_DAB_RBAC(self, migrator):
old_state = migrator.apply_initial_migration(('main', '0190_alter_inventorysource_source_and_more'))
Organization = old_state.apps.get_model('main', 'Organization')
Team = old_state.apps.get_model('main', 'Team')
User = old_state.apps.get_model('auth', 'User')
org = Organization.objects.create(name='arbitrary-org', created=now(), modified=now())
user = User.objects.create(username='random-user')
org.read_role.members.add(user)
org.member_role.members.add(user)
team = Team.objects.create(name='arbitrary-team', organization=org, created=now(), modified=now())
team.member_role.members.add(user)
new_state = migrator.apply_tested_migration(
('main', '0192_custom_roles'),
@@ -85,3 +90,19 @@ class TestMigrationSmoke:
RoleUserAssignment = new_state.apps.get_model('dab_rbac', 'RoleUserAssignment')
assert RoleUserAssignment.objects.filter(user=user.id, object_id=org.id).exists()
assert RoleUserAssignment.objects.filter(user=user.id, role_definition__name='Controller Organization Member', object_id=org.id).exists()
assert RoleUserAssignment.objects.filter(user=user.id, role_definition__name='Controller Team Member', object_id=team.id).exists()
# Regression testing for bug that comes from current vs past models mismatch
RoleDefinition = new_state.apps.get_model('dab_rbac', 'RoleDefinition')
assert not RoleDefinition.objects.filter(name='Organization Organization Admin').exists()
# Test special cases in managed role creation
assert not RoleDefinition.objects.filter(name='Organization Team Admin').exists()
assert not RoleDefinition.objects.filter(name='Organization InstanceGroup Admin').exists()
# Test that a removed EE model permission has been deleted
new_state = migrator.apply_tested_migration(
('main', '0195_EE_permissions'),
)
DABPermission = new_state.apps.get_model('dab_rbac', 'DABPermission')
assert not DABPermission.objects.filter(codename='view_executionenvironment').exists()

View File

@@ -0,0 +1,148 @@
import pytest
from django.contrib.contenttypes.models import ContentType
from awx.main.access import ExecutionEnvironmentAccess
from awx.main.models import ExecutionEnvironment, Organization, Team
from awx.main.models.rbac import get_role_codenames
from awx.api.versioning import reverse
from django.urls import reverse as django_reverse
from ansible_base.rbac.models import RoleDefinition
@pytest.fixture
def ee_rd():
return RoleDefinition.objects.create_from_permissions(
name='EE object admin',
permissions=['change_executionenvironment', 'delete_executionenvironment'],
content_type=ContentType.objects.get_for_model(ExecutionEnvironment),
)
@pytest.fixture
def org_ee_rd():
return RoleDefinition.objects.create_from_permissions(
name='EE org admin',
permissions=['add_executionenvironment', 'change_executionenvironment', 'delete_executionenvironment', 'view_organization'],
content_type=ContentType.objects.get_for_model(Organization),
)
@pytest.mark.django_db
def test_old_ee_role_maps_to_correct_permissions(organization):
assert set(get_role_codenames(organization.execution_environment_admin_role)) == {
'view_organization',
'add_executionenvironment',
'change_executionenvironment',
'delete_executionenvironment',
}
@pytest.fixture
def org_ee(organization):
return ExecutionEnvironment.objects.create(name='some user ee', organization=organization)
@pytest.fixture
def check_user_capabilities(get, setup_managed_roles):
def _rf(user, obj, expected):
url = reverse('api:execution_environment_list')
r = get(url, user=user, expect=200)
for item in r.data['results']:
if item['id'] == obj.pk:
assert expected == item['summary_fields']['user_capabilities']
break
else:
raise RuntimeError(f'Could not find expected object ({obj}) in EE list result: {r.data}')
return _rf
# ___ begin tests ___
@pytest.mark.django_db
def test_any_user_can_view_global_ee(control_plane_execution_environment, rando):
assert ExecutionEnvironmentAccess(rando).can_read(control_plane_execution_environment)
@pytest.mark.django_db
def test_managed_ee_not_assignable(control_plane_execution_environment, ee_rd, rando, admin_user, post):
url = django_reverse('roleuserassignment-list')
r = post(url, {'role_definition': ee_rd.pk, 'user': rando.id, 'object_id': control_plane_execution_environment.pk}, user=admin_user, expect=400)
assert 'Can not assign object roles to managed Execution Environment' in str(r.data)
@pytest.mark.django_db
def test_org_member_required_for_assignment(org_ee, ee_rd, rando, admin_user, post):
url = django_reverse('roleuserassignment-list')
r = post(url, {'role_definition': ee_rd.pk, 'user': rando.id, 'object_id': org_ee.pk}, user=admin_user, expect=400)
assert 'User must have view permission to Execution Environment organization' in str(r.data)
@pytest.mark.django_db
def test_team_can_have_permission(org_ee, ee_rd, rando, admin_user, post):
org2 = Organization.objects.create(name='a different team')
team = Team.objects.create(name='a team', organization=org2)
team.member_role.members.add(rando)
assert org_ee not in ExecutionEnvironmentAccess(rando).get_queryset() # user can not view the EE
url = django_reverse('roleteamassignment-list')
# can give object roles to the team now
post(url, {'role_definition': ee_rd.pk, 'team': team.id, 'object_id': org_ee.pk}, user=admin_user, expect=201)
assert rando.has_obj_perm(org_ee, 'change')
assert org_ee in ExecutionEnvironmentAccess(rando).get_queryset() # user can view the EE now
@pytest.mark.django_db
def test_give_object_permission_to_ee(setup_managed_roles, org_ee, ee_rd, org_member, check_user_capabilities):
access = ExecutionEnvironmentAccess(org_member)
assert access.can_read(org_ee) # by virtue of being an org member
assert not access.can_change(org_ee, {'name': 'new'})
check_user_capabilities(org_member, org_ee, {'edit': False, 'delete': False, 'copy': False})
ee_rd.give_permission(org_member, org_ee)
assert access.can_change(org_ee, {'name': 'new', 'organization': org_ee.organization.id})
check_user_capabilities(org_member, org_ee, {'edit': True, 'delete': True, 'copy': False})
@pytest.mark.django_db
def test_need_related_organization_access(org_ee, ee_rd, org_member):
org2 = Organization.objects.create(name='another organization')
ee_rd.give_permission(org_member, org_ee)
org2.member_role.members.add(org_member)
access = ExecutionEnvironmentAccess(org_member)
assert access.can_change(org_ee, {'name': 'new', 'organization': org_ee.organization})
assert access.can_change(org_ee, {'name': 'new', 'organization': org_ee.organization.id})
assert not access.can_change(org_ee, {'name': 'new', 'organization': org2.id})
assert not access.can_change(org_ee, {'name': 'new', 'organization': org2})
# User can make the change if they have relevant permission to the new organization
org_ee.organization.execution_environment_admin_role.members.add(org_member)
org2.execution_environment_admin_role.members.add(org_member)
assert access.can_change(org_ee, {'name': 'new', 'organization': org2.id})
assert access.can_change(org_ee, {'name': 'new', 'organization': org2})
@pytest.mark.django_db
@pytest.mark.parametrize('style', ['new', 'old'])
def test_give_org_permission_to_ee(setup_managed_roles, org_ee, organization, org_member, check_user_capabilities, style, org_ee_rd):
access = ExecutionEnvironmentAccess(org_member)
assert not access.can_change(org_ee, {'name': 'new'})
check_user_capabilities(org_member, org_ee, {'edit': False, 'delete': False, 'copy': False})
if style == 'new':
org_ee_rd.give_permission(org_member, organization)
assert org_member.has_obj_perm(org_ee.organization, 'add_executionenvironment') # sanity
else:
organization.execution_environment_admin_role.members.add(org_member)
assert access.can_change(org_ee, {'name': 'new', 'organization': organization.id})
check_user_capabilities(org_member, org_ee, {'edit': True, 'delete': True, 'copy': True})
# Extra check, user can not remove the EE from the organization
assert not access.can_change(org_ee, {'name': 'new', 'organization': None})

View File

@@ -2,7 +2,7 @@ import pytest
from rest_framework.exceptions import PermissionDenied
from awx.main.access import JobAccess, JobLaunchConfigAccess, AdHocCommandAccess, InventoryUpdateAccess, ProjectUpdateAccess
from awx.main.access import JobAccess, JobLaunchConfigAccess, AdHocCommandAccess, InventoryUpdateAccess, ProjectUpdateAccess, SystemJobTemplateAccess
from awx.main.models import (
Job,
JobLaunchConfig,
@@ -350,3 +350,21 @@ class TestLaunchConfigAccess:
assert access.can_use(config)
assert rando.can_access(JobLaunchConfig, 'use', config)
@pytest.mark.django_db
class TestSystemJobTemplateAccess:
def test_system_job_template_auditor(self, system_auditor, system_job_template):
access = SystemJobTemplateAccess(system_auditor)
assert access.can_read(system_job_template)
assert not access.can_start(system_job_template)
def test_system_job_template_rando(self, rando, system_job_template):
access = SystemJobTemplateAccess(rando)
assert not access.can_read(system_job_template)
assert not access.can_start(system_job_template)
def test_system_job_template_superuser(self, admin_user, system_job_template):
access = SystemJobTemplateAccess(admin_user)
assert access.can_read(system_job_template)
assert access.can_start(system_job_template)

View File

@@ -182,8 +182,14 @@ def test_job_template_creator_access(project, organization, rando, post, setup_m
@pytest.mark.django_db
@pytest.mark.job_permissions
@pytest.mark.parametrize('lacking', ['project', 'inventory'])
def test_job_template_insufficient_creator_permissions(lacking, project, inventory, organization, rando, post):
@pytest.mark.parametrize(
'lacking,reason',
[
('project', 'You do not have use permission on Project'),
('inventory', 'You do not have use permission on Inventory'),
],
)
def test_job_template_insufficient_creator_permissions(lacking, reason, project, inventory, organization, rando, post):
if lacking != 'project':
project.use_role.members.add(rando)
else:
@@ -192,12 +198,13 @@ def test_job_template_insufficient_creator_permissions(lacking, project, invento
inventory.use_role.members.add(rando)
else:
inventory.read_role.members.add(rando)
post(
response = post(
url=reverse('api:job_template_list'),
data=dict(name='newly-created-jt', inventory=inventory.id, project=project.pk, playbook='helloworld.yml'),
user=rando,
expect=403,
)
assert reason in response.data[lacking]
@pytest.mark.django_db

View File

@@ -99,7 +99,9 @@ def test_notification_template_access_org_user(notification_template, user):
@pytest.mark.django_db
def test_notificaiton_template_orphan_access_org_admin(notification_template, organization, org_admin):
notification_template.organization = None
notification_template.save(update_fields=['organization'])
access = NotificationTemplateAccess(org_admin)
assert not org_admin.has_obj_perm(notification_template, 'change')
assert not access.can_change(notification_template, {'organization': organization.id})

View File

@@ -48,3 +48,17 @@ def test_org_resource_role(ext_auth, organization, rando, org_admin):
assert access.can_attach(organization, rando, 'member_role.members') == ext_auth
organization.member_role.members.add(rando)
assert access.can_unattach(organization, rando, 'member_role.members') == ext_auth
@pytest.mark.django_db
def test_delete_org_while_workflow_active(workflow_job_template):
'''
Delete org while workflow job is active (i.e. changing status)
'''
assert workflow_job_template.organization # sanity check
wj = workflow_job_template.create_unified_job() # status should be new
workflow_job_template.organization.delete()
wj.refresh_from_db()
assert wj.status != 'pending' # sanity check
wj.status = 'pending' # status needs to change in order to trigger workflow_job_template.save()
wj.save(update_fields=['status'])

View File

@@ -35,6 +35,13 @@ class TestWorkflowJobTemplateAccess:
assert org_member in wfjt.execute_role
assert org_member in wfjt.read_role
def test_non_super_admin_no_add_without_org(self, wfjt, organization, rando):
organization.member_role.members.add(rando)
wfjt.admin_role.members.add(rando)
access = WorkflowJobTemplateAccess(rando, save_messages=True)
assert not access.can_add({'name': 'without org'})
assert 'An organization is required to create a workflow job template for normal user' in access.messages['organization']
@pytest.mark.django_db
class TestWorkflowJobTemplateNodeAccess:

View File

@@ -5,7 +5,7 @@ from django.contrib.auth.models import User
from django.forms.models import model_to_dict
from rest_framework.exceptions import ParseError
from awx.main.access import BaseAccess, check_superuser, JobTemplateAccess, WorkflowJobTemplateAccess, SystemJobTemplateAccess, vars_are_encrypted
from awx.main.access import BaseAccess, check_superuser, JobTemplateAccess, WorkflowJobTemplateAccess, vars_are_encrypted
from awx.main.models import (
Credential,
@@ -239,14 +239,3 @@ def test_user_capabilities_method():
foo = object()
foo_capabilities = foo_access.get_user_capabilities(foo, ['edit', 'copy'])
assert foo_capabilities == {'edit': 'bar', 'copy': 'foo'}
def test_system_job_template_can_start(mocker):
user = mocker.MagicMock(spec=User, id=1, is_system_auditor=True, is_superuser=False)
assert user.is_system_auditor
access = SystemJobTemplateAccess(user)
assert not access.can_start(None)
user.is_superuser = True
access = SystemJobTemplateAccess(user)
assert access.can_start(None)

View File

@@ -1259,7 +1259,6 @@ class TestJobCredentials(TestJobExecution):
extra_vars = parse_extra_vars(args, private_data_dir)
assert extra_vars["turbo_button"] == "True"
return ['successful', 0]
def test_custom_environment_injectors_with_nested_extra_vars(self, private_data_dir, job, mock_me):
task = jobs.RunJob()

View File

@@ -1,5 +1,7 @@
import re
from functools import reduce
from django.core.exceptions import FieldDoesNotExist
from pyparsing import (
infixNotation,
opAssoc,
@@ -353,7 +355,7 @@ class SmartFilter(object):
try:
res = boolExpr.parseString('(' + filter_string + ')')
except ParseException:
except (ParseException, FieldDoesNotExist):
raise RuntimeError(u"Invalid query %s" % filter_string_raw)
if len(res) > 0:

View File

@@ -450,7 +450,12 @@ class Licenser(object):
if first_host:
automated_since = int(first_host.first_automation.timestamp())
else:
automated_since = int(Instance.objects.order_by('id').first().created.timestamp())
try:
automated_since = int(Instance.objects.order_by('id').first().created.timestamp())
except AttributeError:
# In the odd scenario that create_preload_data was not run, there are no hosts
# Then we CAN end up here before any instance has registered
automated_since = int(time.time())
instance_count = int(attrs.get('instance_count', 0))
attrs['current_instances'] = current_instances
attrs['automated_instances'] = automated_instances

View File

@@ -8,9 +8,22 @@ from django.db import connection
@contextmanager
def advisory_lock(*args, **kwargs):
def advisory_lock(*args, lock_session_timeout_milliseconds=0, **kwargs):
if connection.vendor == 'postgresql':
cur = None
idle_in_transaction_session_timeout = None
idle_session_timeout = None
if lock_session_timeout_milliseconds > 0:
with connection.cursor() as cur:
idle_in_transaction_session_timeout = cur.execute('SHOW idle_in_transaction_session_timeout').fetchone()[0]
idle_session_timeout = cur.execute('SHOW idle_session_timeout').fetchone()[0]
cur.execute(f"SET idle_in_transaction_session_timeout = '{lock_session_timeout_milliseconds}'")
cur.execute(f"SET idle_session_timeout = '{lock_session_timeout_milliseconds}'")
with django_pglocks_advisory_lock(*args, **kwargs) as internal_lock:
yield internal_lock
if lock_session_timeout_milliseconds > 0:
with connection.cursor() as cur:
cur.execute(f"SET idle_in_transaction_session_timeout = '{idle_in_transaction_session_timeout}'")
cur.execute(f"SET idle_session_timeout = '{idle_session_timeout}'")
else:
yield True

48
awx/main/utils/proxy.py Normal file
View File

@@ -0,0 +1,48 @@
# Copyright (c) 2024 Ansible, Inc.
# All Rights Reserved.
# DRF
from rest_framework.request import Request
"""
Note that these methods operate on request.environ. This data is from uwsgi.
It is the source data from which request.headers (read-only) is constructed.
"""
def is_proxy_in_headers(request: Request, proxy_list: list[str], headers: list[str]) -> bool:
"""
Determine if the request went through at least one proxy in the list.
Example:
request.environ = {
"HTTP_X_FOO": "8.8.8.8, 192.168.2.1",
"REMOTE_ADDR": "192.168.2.1",
"REMOTE_HOST": "foobar"
}
proxy_list = ["192.168.2.1"]
headers = ["HTTP_X_FOO", "REMOTE_ADDR", "REMOTE_HOST"]
The above would return True since 192.168.2.1 is a value for the header HTTP_X_FOO
request: The DRF/Django request. request.environ dict will be used for searching for proxies
proxy_list: A list of known and trusted proxies may be ip or hostnames
headers: A list of keys for which to consider values that may contain a proxy
"""
remote_hosts = set()
for header in headers:
for value in request.environ.get(header, '').split(','):
value = value.strip()
if value:
remote_hosts.add(value)
return bool(remote_hosts.intersection(set(proxy_list)))
def delete_headers_starting_with_http(request: Request, headers: list[str]):
for header in headers:
if header.startswith('HTTP_'):
request.environ.pop(header, None)

View File

@@ -8,7 +8,7 @@ import ipaddress
import aiohttp
from aiohttp import client_exceptions
import aioredis
import redis
from channels.layers import get_channel_layer
@@ -47,7 +47,6 @@ class WebsocketRelayConnection:
verify_ssl: bool = settings.BROADCAST_WEBSOCKET_VERIFY_CERT,
):
self.name = name
self.event_loop = asyncio.get_event_loop()
self.stats = stats
self.remote_host = remote_host
self.remote_port = remote_port
@@ -110,7 +109,10 @@ class WebsocketRelayConnection:
self.stats.record_connection_lost()
def start(self):
self.async_task = self.event_loop.create_task(self.connect())
self.async_task = asyncio.get_running_loop().create_task(
self.connect(),
name=f"WebsocketRelayConnection.connect.{self.name}",
)
return self.async_task
def cancel(self):
@@ -121,7 +123,10 @@ class WebsocketRelayConnection:
# metrics messages
# the "metrics" group is not subscribed to in the typical fashion, so we
# just explicitly create it
producer = self.event_loop.create_task(self.run_producer("metrics", websocket, "metrics"))
producer = asyncio.get_running_loop().create_task(
self.run_producer("metrics", websocket, "metrics"),
name="WebsocketRelayConnection.run_producer.metrics",
)
self.producers["metrics"] = {"task": producer, "subscriptions": {"metrics"}}
async for msg in websocket:
self.stats.record_message_received()
@@ -143,7 +148,10 @@ class WebsocketRelayConnection:
name = f"{self.remote_host}-{group}"
origin_channel = payload['origin_channel']
if not self.producers.get(name):
producer = self.event_loop.create_task(self.run_producer(name, websocket, group))
producer = asyncio.get_running_loop().create_task(
self.run_producer(name, websocket, group),
name=f"WebsocketRelayConnection.run_producer.{name}",
)
self.producers[name] = {"task": producer, "subscriptions": {origin_channel}}
logger.debug(f"Producer {name} started.")
else:
@@ -191,7 +199,7 @@ class WebsocketRelayConnection:
return
continue
except aioredis.errors.ConnectionClosedError:
except redis.exceptions.ConnectionError:
logger.info(f"Producer {name} lost connection to Redis, shutting down.")
return
@@ -297,16 +305,15 @@ class WebSocketRelayManager(object):
pass
async def run(self):
event_loop = asyncio.get_running_loop()
self.stats_mgr = RelayWebsocketStatsManager(event_loop, self.local_hostname)
self.stats_mgr = RelayWebsocketStatsManager(self.local_hostname)
self.stats_mgr.start()
database_conf = deepcopy(settings.DATABASES['default'])
database_conf['OPTIONS'] = deepcopy(database_conf.get('OPTIONS', {}))
for k, v in settings.LISTENER_DATABASES.get('default', {}).items():
database_conf[k] = v
if k != 'OPTIONS':
database_conf[k] = v
for k, v in settings.LISTENER_DATABASES.get('default', {}).get('OPTIONS', {}).items():
database_conf['OPTIONS'][k] = v
@@ -322,7 +329,10 @@ class WebSocketRelayManager(object):
)
await async_conn.set_autocommit(True)
on_ws_heartbeat_task = event_loop.create_task(self.on_ws_heartbeat(async_conn))
on_ws_heartbeat_task = asyncio.get_running_loop().create_task(
self.on_ws_heartbeat(async_conn),
name="WebSocketRelayManager.on_ws_heartbeat",
)
# Establishes a websocket connection to /websocket/relay on all API servers
while True:

View File

@@ -91,8 +91,7 @@ USE_L10N = True
USE_TZ = True
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'ui', 'build', 'static'),
os.path.join(BASE_DIR, 'ui_next', 'build'),
os.path.join(BASE_DIR, 'ui', 'build'),
os.path.join(BASE_DIR, 'static'),
]
@@ -262,6 +261,7 @@ START_TASK_LIMIT = 100
# We have the grace period so the task manager can bail out before the timeout.
TASK_MANAGER_TIMEOUT = 300
TASK_MANAGER_TIMEOUT_GRACE_PERIOD = 60
TASK_MANAGER_LOCK_TIMEOUT = TASK_MANAGER_TIMEOUT + TASK_MANAGER_TIMEOUT_GRACE_PERIOD
# Number of seconds _in addition to_ the task manager timeout a job can stay
# in waiting without being reaped
@@ -322,9 +322,8 @@ TEMPLATES = [
},
'DIRS': [
os.path.join(BASE_DIR, 'templates'),
os.path.join(BASE_DIR, 'ui', 'build'),
os.path.join(BASE_DIR, 'ui', 'public'),
os.path.join(BASE_DIR, 'ui_next', 'build', 'awx'),
os.path.join(BASE_DIR, 'ui', 'build', 'awx'),
],
},
]
@@ -492,6 +491,7 @@ CELERYBEAT_SCHEDULE = {
'cleanup_images': {'task': 'awx.main.tasks.system.cleanup_images_and_files', 'schedule': timedelta(hours=3)},
'cleanup_host_metrics': {'task': 'awx.main.tasks.host_metrics.cleanup_host_metrics', 'schedule': timedelta(hours=3, minutes=30)},
'host_metric_summary_monthly': {'task': 'awx.main.tasks.host_metrics.host_metric_summary_monthly', 'schedule': timedelta(hours=4)},
'periodic_resource_sync': {'task': 'awx.main.tasks.system.periodic_resource_sync', 'schedule': timedelta(minutes=15)},
}
# Django Caching Configuration
@@ -656,6 +656,13 @@ AWX_ANSIBLE_CALLBACK_PLUGINS = ""
# Automatically remove nodes that have missed their heartbeats after some time
AWX_AUTO_DEPROVISION_INSTANCES = False
# If False, do not allow creation of resources that are shared with the platform ingress
# e.g. organizations, teams, and users
ALLOW_LOCAL_RESOURCE_MANAGEMENT = True
# If True, allow users to be assigned to roles that were created via JWT
ALLOW_LOCAL_ASSIGNING_JWT_ROLES = False
# Enable Pendo on the UI, possible values are 'off', 'anonymous', and 'detailed'
# Note: This setting may be overridden by database settings.
PENDO_TRACKING_STATE = "off"
@@ -778,6 +785,11 @@ INSIGHTS_EXCLUDE_EMPTY_GROUPS = False
TERRAFORM_INSTANCE_ID_VAR = 'id'
TERRAFORM_EXCLUDE_EMPTY_GROUPS = True
# ------------------------
# OpenShift Virtualization
# ------------------------
OPENSHIFT_VIRTUALIZATION_EXCLUDE_EMPTY_GROUPS = True
# ---------------------
# ----- Custom -----
# ---------------------
@@ -817,7 +829,7 @@ MANAGE_ORGANIZATION_AUTH = True
DISABLE_LOCAL_AUTH = False
# Note: This setting may be overridden by database settings.
TOWER_URL_BASE = "https://towerhost"
TOWER_URL_BASE = "https://platformhost"
INSIGHTS_URL_BASE = "https://example.org"
INSIGHTS_AGENT_MIME = 'application/example'
@@ -998,6 +1010,7 @@ AWX_RUNNER_KEEPALIVE_SECONDS = 0
# Delete completed work units in receptor
RECEPTOR_RELEASE_WORK = True
RECPETOR_KEEP_WORK_ON_ERROR = False
# K8S only. Use receptor_log_level on AWX spec to set this properly
RECEPTOR_LOG_LEVEL = 'info'
@@ -1078,8 +1091,6 @@ AWX_MOUNT_ISOLATED_PATHS_ON_K8S = False
# This is overridden downstream via /etc/tower/conf.d/cluster_host_id.py
CLUSTER_HOST_ID = socket.gethostname()
UI_NEXT = True
# License compliance for total host count. Possible values:
# - '': No model - Subscription not counted from Host Metrics
# - 'unique_managed_hosts': Compliant = automated - deleted hosts (using /api/v2/host_metrics/)

File diff suppressed because it is too large Load Diff

View File

@@ -64,7 +64,7 @@
<div class="col-sm-6">
</div>
<div class="col-sm-6 footer-copyright">
Copyright &copy; 2021 <a href="http://www.redhat.com" target="_blank">Red Hat</a>, Inc. All Rights Reserved.
Copyright &copy; 2024 <a href="http://www.redhat.com" target="_blank">Red Hat</a>, Inc. All Rights Reserved.
</div>
</div>
</div>

View File

View File

@@ -1,11 +0,0 @@
jest.*.js
webpack.*.js
etc
coverage
build
node_modules
dist
images
instrumented
*test*.js

View File

@@ -1,168 +0,0 @@
{
"parser": "@babel/eslint-parser",
"ignorePatterns": ["./node_modules/"],
"parserOptions": {
"requireConfigFile": false,
"ecmaVersion": 6,
"sourceType": "module",
"ecmaFeatures": {
"jsx": true,
"modules": true
},
"babelOptions": {
"presets": ["@babel/preset-react"]
}
},
"plugins": ["react-hooks", "jsx-a11y", "i18next", "@babel"],
"extends": [
"airbnb",
"prettier",
"plugin:jsx-a11y/strict",
"plugin:i18next/recommended"
],
"settings": {
"react": {
"version": "detect"
},
"import/resolver": {
"node": {
"paths": ["src"]
}
}
},
"env": {
"browser": true,
"node": true,
"jest": true
},
"globals": {
"window": true
},
"rules": {
"i18next/no-literal-string": [
2,
{
"markupOnly": true,
"ignoreAttribute": [
"data-testid",
"dateFieldName",
"timeFieldName",
"to",
"streamType",
"path",
"component",
"variant",
"key",
"position",
"promptName",
"color",
"promptId",
"headingLevel",
"size",
"target",
"autoComplete",
"trigger",
"from",
"name",
"fieldId",
"css",
"gutter",
"dataCy",
"tooltipMaxWidth",
"mode",
"aria-labelledby",
"aria-hidden",
"aria-controls",
"aria-pressed",
"sortKey",
"ouiaId",
"credentialTypeNamespace",
"link",
"value",
"credentialTypeKind",
"linkTo",
"scrollToAlignment",
"displayKey",
"sortedColumnKey",
"maxHeight",
"maxWidth",
"role",
"aria-haspopup",
"dropDirection",
"resizeOrientation",
"src",
"theme",
"gridColumns",
"rows",
"href",
"modifier",
"data-cy",
"fieldName",
"splitButtonVariant",
"pageKey",
"textId",
"rel"
],
"ignore": [
"Ansible",
"Tower",
"JSON",
"YAML",
"lg",
"hh:mm AM/PM",
"Twilio"
],
"ignoreComponent": [
"AboutModal",
"code",
"Omit",
"PotentialLink",
"TypeRedirect",
"Radio",
"RunOnRadio",
"NodeTypeLetter",
"SelectableItem",
"Dash",
"Plural"
],
"ignoreCallee": ["describe"]
}
],
"camelcase": "off",
"arrow-parens": "off",
"comma-dangle": "off",
// https://github.com/benmosher/eslint-plugin-import/issues/479#issuecomment-252500896
"import/no-extraneous-dependencies": "off",
"max-len": [
"error",
{
"code": 100,
"ignoreStrings": true,
"ignoreTemplateLiterals": true
}
],
"no-continue": "off",
"no-debugger": "off",
"no-mixed-operators": "off",
"no-param-reassign": "off",
"no-plusplus": "off",
"no-underscore-dangle": "off",
"no-use-before-define": "off",
"no-multiple-empty-lines": ["error", { "max": 1 }],
"object-curly-newline": "off",
"no-trailing-spaces": ["error"],
"no-unused-expressions": ["error", { "allowShortCircuit": true }],
"react/jsx-props-no-spreading": ["off"],
"react/prefer-stateless-function": "off",
"react/prop-types": "off",
"react/sort-comp": ["error", {}],
"jsx-a11y/label-has-for": "off",
"jsx-a11y/label-has-associated-control": "off",
"react-hooks/rules-of-hooks": "error",
"react-hooks/exhaustive-deps": "warn",
"react/jsx-filename-extension": "off",
"no-restricted-exports": "off",
"react/function-component-definition": "off",
"prefer-regex-literals": "off"
}
}

View File

@@ -1,17 +0,0 @@
{"catalogs":[{
"path": "<rootDir>/locales/{locale}/messages",
"include": ["<rootDir>"],
"exclude": ["**/node_modules/**"]
}],
"compileNamespace": "cjs",
"extractBabelOptions": {},
"compilerBabelOptions": {},
"fallbackLocales": { "default": "en"},
"format": "po",
"locales": ["en","es","fr","ko","nl","zh","ja","zu"],
"orderBy": "messageId",
"pseudoLocale": "zu",
"rootDir": "./src",
"runtimeConfigModule": ["@lingui/core", "i18n"],
"sourceLocale": "en"
}

View File

@@ -1 +0,0 @@
engine-strict = true

View File

@@ -1,2 +0,0 @@
build
src/locales

View File

@@ -1,8 +0,0 @@
{
"printWidth": 80,
"tabWidth": 2,
"semi": true,
"singleQuote": true,
"trailingComma": "es5",
"bracketSpacing": true
}

View File

@@ -1,363 +0,0 @@
# Ansible AWX UI With PatternFly
Hi there! We're excited to have you as a contributor.
Have questions about this document or anything not covered here? Feel free to reach out to any of the contributors of this repository.
## Table of contents
- [Ansible AWX UI With PatternFly](#ansible-awx-ui-with-patternfly)
- [Table of contents](#table-of-contents)
- [Things to know prior to submitting code](#things-to-know-prior-to-submitting-code)
- [Setting up your development environment](#setting-up-your-development-environment)
- [Prerequisites](#prerequisites)
- [Node and npm](#node-and-npm)
- [Build the User Interface](#build-the-user-interface)
- [Accessing the AWX web interface](#accessing-the-awx-web-interface)
- [AWX REST API Interaction](#awx-rest-api-interaction)
- [Handling API Errors](#handling-api-errors)
- [Forms](#forms)
- [Working with React](#working-with-react)
- [App structure](#app-structure)
- [Patterns](#patterns)
- [Bootstrapping the application (root src/ files)](#bootstrapping-the-application-root-src-files)
- [Naming files](#naming-files)
- [Naming components that use the context api](#naming-components-that-use-the-context-api)
- [Class constructors vs Class properties](#class-constructors-vs-class-properties)
- [Binding](#binding)
- [Typechecking with PropTypes](#typechecking-with-proptypes)
- [Custom Hooks](#custom-hooks)
- [Naming Functions](#naming-functions)
- [Default State Initialization](#default-state-initialization)
- [Testing components that use contexts](#testing-components-that-use-contexts)
- [Internationalization](#internationalization)
- [Marking strings for translation and replacement in the UI](#marking-strings-for-translation-and-replacement-in-the-ui)
- [Setting up .po files to give to translation team](#setting-up-po-files-to-give-to-translation-team)
- [Marking an issue to be translated](#marking-an-issue-to-be-translated)
## Things to know prior to submitting code
- All code submissions are done through pull requests against the `devel` branch.
- If collaborating with someone else on the same branch, please use `--force-with-lease` instead of `--force` when pushing up code. This will prevent you from accidentally overwriting commits pushed by someone else. For more information, see https://git-scm.com/docs/git-push#git-push---force-with-leaseltrefnamegt
- We use a [code formatter](https://prettier.io/). Before adding a new commit or opening a PR, please apply the formatter using `npm run prettier`
- We adopt the following code style guide:
- functions should adopt camelCase
- constructors/classes should adopt PascalCase
- constants to be exported should adopt UPPERCASE
- For strings, we adopt the `sentence capitalization` since it is a [Patternfly style guide](https://www.patternfly.org/v4/ux-writing/capitalization).
## Setting up your development environment
The UI is built using [ReactJS](https://reactjs.org/docs/getting-started.html) and [Patternfly](https://www.patternfly.org/).
### Prerequisites
#### Node and npm
The AWX UI requires the following:
- Node >= 16.13.1 LTS
- NPM 8.x
Run the following to install all the dependencies:
```bash
(host) $ npm install
```
#### Build the User Interface
Run the following to build the AWX UI:
```bash
(host) $ npm run start
```
## Accessing the AWX web interface
You can now log into the AWX web interface at [https://127.0.0.1:3001](https://127.0.0.1:3001).
## AWX REST API Interaction
This interface is built on top of the AWX REST API. If a component needs to interact with the API then the model that corresponds to that base endpoint will need to be imported from the api module.
Example:
`import { OrganizationsAPI, UsersAPI } from '../../../api';`
All models extend a `Base` class which provides an interface to the standard HTTP methods (GET, POST, PUT etc). Methods that are specific to that endpoint should be added directly to model's class.
**Mixins** - For related endpoints that apply to several different models a mixin should be used. Mixins are classes with a number of methods and can be used to avoid adding the same methods to a number of different models. A good example of this is the Notifications mixin. This mixin provides generic methods for reading notification templates and toggling them on and off.
Note that mixins can be chained. See the example below.
Example of a model using multiple mixins:
```javascript
import NotificationsMixin from '../mixins/Notifications.mixin';
import InstanceGroupsMixin from '../mixins/InstanceGroups.mixin';
class Organizations extends InstanceGroupsMixin(NotificationsMixin(Base)) {
...
}
export default Organizations;
```
**Testing** - The easiest way to mock the api module in tests is to use jest's [automatic mock](https://jestjs.io/docs/en/es6-class-mocks#automatic-mock). This syntax will replace the class with a mock constructor and mock out all methods to return undefined by default. If necessary, you can still override these mocks for specific tests. See the example below.
Example of mocking a specific method for every test in a suite:
```javascript
import { OrganizationsAPI } from '../../../../src/api';
// Mocks out all available methods. Comparable to:
// OrganizationsAPI.readAccessList = jest.fn();
// but for every available method
jest.mock('../../../../src/api');
// Return a specific mock value for the readAccessList method
beforeEach(() => {
OrganizationsAPI.readAccessList.mockReturnValue({ foo: 'bar' });
});
// Reset mocks
afterEach(() => {
jest.clearAllMocks();
});
...
```
**Test Attributes** -
It should be noted that the `dataCy` prop, as well as its equivalent attribute `data-cy`, are used as flags for any UI test that wants to avoid relying on brittle CSS selectors such as `nth-of-type()`.
## Handling API Errors
API requests can and will fail occasionally so they should include explicit error handling. The three _main_ categories of errors from our perspective are: content loading errors, form submission errors, and other errors. The patterns currently in place for these are described below:
- **content loading errors** - These are any errors that occur when fetching data to initialize a page or populate a list. For these, we conditionally render a _content error component_ in place of the unresolved content.
- **form submission errors** - If an error is encountered when submitting a form, we display the error message on the form. For field-specific validation errors, we display the error message beneath the specific field(s). For general errors, we display the error message at the bottom of the form near the action buttons. An error that happens when requesting data to populate a form is not a form submission error, it is still a content error and is handled as such (see above).
- **other errors** - Most errors will fall into the first two categories, but for miscellaneous actions like toggling notifications, deleting a list item, etc. we display an alert modal to notify the user that their requested action couldn't be performed.
## Forms
Our forms should have a known, consistent, and fully-resolved starting state before it is possible for a user, keyboard-mouse, screen reader, or automated test to interact with them. If multiple network calls are needed to populate a form, resolve them all before displaying the form or showing a content error. When multiple requests are needed to create or update the resources represented by a form, resolve them all before transitioning the ui to a success or failure state.
## Working with React
### App structure
All source code lives in the `/src` directory and all tests are colocated with the components that they test.
Inside these folders, the internal structure is:
- **/api** - All classes used to interact with API's are found here. See [AWX REST API Interaction](#awx-rest-api-interaction) for more information.
- **/components** - All generic components that are meant to be used in multiple contexts throughout awx. Things like buttons, tabs go here.
- **/contexts** - Components which utilize react's context api.
- **/hooks** - Custom react [hooks](https://reactjs.org/docs/hooks-custom.html)
- **/locales** - [Internationalization](#internationalization) config and source files.
- **/screens** - Based on the various routes of awx.
- **/shared** - Components that are meant to be used specifically by a particular route, but might be shareable across pages of that route. For example, a form component which is used on both add and edit screens.
- **/util** - Stateless helper functions that aren't tied to react.
### Patterns
- A **screen** shouldn't import from another screen. If a component _needs_ to be shared between two or more screens, it is a generic and should be moved to `src/components`.
#### Bootstrapping the application (root src/ files)
In the root of `/src`, there are a few files which are used to initialize the react app. These are
- **index.js**
- Connects react app to root dom node.
- Sets up root route structure, navigation grouping and login modal
- Calls base context providers
- Imports .scss styles.
- **app.js**
- Sets standard page layout, about modal, and root dialog modal.
- **RootProvider.js**
- Sets up all context providers.
- Initializes i18n and router
### Naming files
Ideally, files should be named the same as the component they export, and tests with `.test` appended. In other words, `<FooBar>` would be defined in `FooBar.js`, and its tests would be defined in `FooBar.test.js`.
#### Naming components that use the context api
**File naming** - Since contexts export both consumer and provider (and potentially in withContext function form), the file can be simplified to be named after the consumer export. In other words, the file containing the `Network` context components would be named `Network.js`.
**Component naming and conventions** - In order to provide a consistent interface with react-router and [lingui](https://lingui.js.org/), as well as make their usage easier and less verbose, context components follow these conventions:
- Providers are wrapped in a component in the `FooProvider` format.
- The value prop of the provider should be pulled from state. This is recommended by the react docs, [here](https://reactjs.org/docs/context.html#caveats).
- The provider should also be able to accept its value by prop for testing.
- Any sort of code related to grabbing data to put on the context should be done in this component.
- Consumers are wrapped in a component in the `Foo` format.
- If it makes sense, consumers can be exported as a function in the `withFoo()` format. If a component is wrapped in this function, its context values are available on the component as props.
### Class constructors vs Class properties
It is good practice to use constructor-bound instance methods rather than methods as class properties. Methods as arrow functions provide lexical scope and are bound to the Component class instance instead of the class itself. This makes it so we cannot easily test a Component's methods without invoking an instance of the Component and calling the method directly within our tests.
BAD:
```javascript
class MyComponent extends React.Component {
constructor(props) {
super(props);
}
myEventHandler = () => {
// do a thing
};
}
```
GOOD:
```javascript
class MyComponent extends React.Component {
constructor(props) {
super(props);
this.myEventHandler = this.myEventHandler.bind(this);
}
myEventHandler() {
// do a thing
}
}
```
### Binding
It is good practice to bind our class methods within our class constructor method for the following reasons:
1. Avoid defining the method every time `render()` is called.
2. [Performance advantages](https://stackoverflow.com/a/44844916).
3. Ease of [testing](https://github.com/airbnb/enzyme/issues/365).
### Typechecking with PropTypes
Shared components should have their prop values typechecked. This will help catch bugs when components get refactored/renamed.
```javascript
About.propTypes = {
ansible_version: PropTypes.string,
isOpen: PropTypes.bool,
onClose: PropTypes.func.isRequired,
version: PropTypes.string,
};
About.defaultProps = {
ansible_version: null,
isOpen: false,
version: null,
};
```
### Custom Hooks
There are currently a few custom hooks:
1. [useRequest](https://github.com/ansible/awx/blob/devel/awx/ui/src/util/useRequest.js#L21) encapsulates main actions related to requests.
2. [useDismissableError](https://github.com/ansible/awx/blob/devel/awx/ui/src/util/useRequest.js#L71) provides controls for "dismissing" an error message.
3. [useDeleteItems](https://github.com/ansible/awx/blob/devel/awx/ui/src/util/useRequest.js#L98) handles deletion of items from a paginated item list.
4. [useSelected](https://github.com/ansible/awx/blob/devel/awx/ui/src/util/useSelected.js#L14) provides a way to read and update a selected list.
### Naming Functions
Here are the guidelines for how to name functions.
| Naming Convention | Description |
| ----------------- | --------------------------------------------------------------------------------- |
| `handle<x>` | Use for methods that process events |
| `on<x>` | Use for component prop names |
| `toggle<x>` | Use for methods that flip one value to the opposite value |
| `show<x>` | Use for methods that always set a value to show or add an element |
| `hide<x>` | Use for methods that always set a value to hide or remove an element |
| `create<x>` | Use for methods that make API `POST` requests |
| `read<x>` | Use for methods that make API `GET` requests |
| `update<x>` | Use for methods that make API `PATCH` requests |
| `destroy<x>` | Use for methods that make API `DESTROY` requests |
| `replace<x>` | Use for methods that make API `PUT` requests |
| `disassociate<x>` | Use for methods that pass `{ disassociate: true }` as a data param to an endpoint |
| `associate<x>` | Use for methods that pass a resource id as a data param to an endpoint |
| `can<x>` | Use for props dealing with RBAC to denote whether a user has access to something |
### Default State Initialization
When declaring empty initial states, prefer the following instead of leaving them undefined:
```javascript
this.state = {
somethingA: null,
somethingB: [],
somethingC: 0,
somethingD: {},
somethingE: '',
};
```
### Testing components that use contexts
We have several React contexts that wrap much of the app, including those from react-router, lingui, and some of our own. When testing a component that depends on one or more of these, you can use the `mountWithContexts()` helper function found in `testUtils/enzymeHelpers.js`. This can be used just like Enzyme's `mount()` function, except it will wrap the component tree with the necessary context providers and basic stub data.
If you want to stub the value of a context, or assert actions taken on it, you can customize a contexts value by passing a second parameter to `mountWithContexts`. For example, this provides a custom value for the `Config` context:
```javascript
const config = {
custom_virtualenvs: ['foo', 'bar'],
};
mountWithContexts(<OrganizationForm />, {
context: { config },
});
```
Now that these custom virtual environments are available in this `OrganizationForm` test we can assert that the component that displays
them is rendering properly.
The object containing context values looks for five known contexts, identified by the keys `linguiPublisher`, `router`, `config`, `network`, and `dialog` — the latter three each referring to the contexts defined in `src/contexts`. You can pass `false` for any of these values, and the corresponding context will be omitted from your test. For example, this will mount your component without the dialog context:
```javascript
mountWithContexts(<Organization />< {
context: {
dialog: false,
}
});
```
## Internationalization
Internationalization leans on the [lingui](https://github.com/lingui/js-lingui) project. [Official documentation here](https://lingui.js.org/). We use this library to mark our strings for translation. If you want to see this in action you'll need to take the following steps:
### Marking strings for translation and replacement in the UI
The lingui library provides various React helpers for dealing with both marking strings for translation, and replacing strings that have been translated. For consistency and ease of use, we have consolidated on one pattern for the codebase. To set strings to be translated in the UI:
- import the t template tag function from the @lingui/macro package.
- wrap your string using the following format: `` t`String to be translated` ``
**Note:** If you have a variable string with text that needs translating, you must wrap it in `` t`${variable} string` `` where it is defined. Then you must run `npm run extract-strings` to generate new `.po` files and submit those files along with your pull request.
**Note:** We try to avoid the `I18n` consumer, or `i18nMark` function lingui gives us access to in this repo. i18nMark does not actually replace the string in the UI (leading to the potential for untranslated bugs), and the other helpers are redundant. Settling on a consistent, single pattern helps us ease the mental overhead of the need to understand the ins and outs of the lingui API.
**Note:** Pluralization can be complicated so it is best to allow lingui handle cases where we have a string that may need to be pluralized based on number of items, or count. In that case lingui provides a `<Plural>` component, and a `plural()` function. When adding or updating strings in a `<Plural/>` tag you must run `npm run extra-strings` and submit the new `.po` files with your pull request. See documentation [here](https://lingui.js.org/guides/plurals.html?highlight=pluralization).
You can learn more about the ways lingui and its React helpers at [this link](https://lingui.js.org/tutorials/react-patterns.html).
### Setting up .po files to give to translation team
1. Make sure that the languages you intend to translate are set correctly in the `.linguirc` configuration file.
2. `npm run extract-strings` to create .po files for each language specified. The .po files will be placed in src/locales. When updating strings that are used by `<Plural>` or `plural()` you will need to run this command to get the strings to render properly. This command will create `.po` files for each of the supported languages that will need to be committed with your PR.
3. Open up the .po file for the language you want to test and add some translations. In production we would pass this .po file off to the translation team.
4. Once you've edited your .po file (or we've gotten a .po file back from the translation team) run `npm run compile-strings`. This command takes the .po files and turns them into a minified JSON object and can be seen in the `messages.js` file in each locale directory. These files get loaded at the App root level (see: App.js).
5. Change the language in your browser and reload the page. You should see your specified translations in place of English strings.
### Marking an issue to be translated
1. Issues marked with `component:I10n` should not be closed after the issue was fixed.
2. Remove the label `state:needs_devel`.
3. Add the label `state:pending_translations`. At this point, the translations will be batch translated by a maintainer, creating relevant entries in the PO files. Then after those translations have been merged, the issue can be closed.

View File

@@ -1,19 +0,0 @@
FROM node:16.13.1
ARG NPMRC_FILE=.npmrc
ENV NPMRC_FILE=${NPMRC_FILE}
ARG TARGET='https://awx:8043'
ENV TARGET=${TARGET}
ENV CI=true
WORKDIR /ui
ADD .eslintignore .eslintignore
ADD .eslintrc.json .eslintrc.json
ADD .linguirc .linguirc
ADD jsconfig.json jsconfig.json
ADD public public
ADD package.json package.json
ADD package-lock.json package-lock.json
COPY ${NPMRC_FILE} .npmrc
RUN npm install
ADD src src
EXPOSE 3001
CMD [ "npm", "start" ]

116
awx/ui/Makefile Normal file
View File

@@ -0,0 +1,116 @@
## UI_DIR: Relative path to the directory containing this Makefile
UI_DIR := $(patsubst %/,%,$(dir $(lastword $(MAKEFILE_LIST))))
## Path to your local clone of the UI repo
# NOTE: you will not be able to build within the docker-compose development environment if you use this option
UI_LOCAL ?=
## Git repo and branch to the UI repo
UI_GIT_REPO ?= https://github.com/ansible/ansible-ui.git
UI_GIT_BRANCH ?= main
## Product name to display on the UI used in UI build process
PRODUCT ?= AWX
.PHONY: ui
## Default build target of ui Makefile, builds ui/build
ui: ui/build
.PHONY: ui/build
## Build ui/build
ui/build: $(UI_DIR)/build
## True build target for ui.
$(UI_DIR)/build:
@$(MAKE) $(UI_DIR)/src/build/awx
@echo "=== Copying $(UI_DIR)/src/build to $(UI_DIR)/build ==="
@rm -rf $(UI_DIR)/build
@cp -r $(UI_DIR)/src/build $(UI_DIR)
@echo "=== Done building $(UI_DIR)/build ==="
.PHONY: ui/src/build
## Build ui/src/build
ui/src/build: $(UI_DIR)/src/build/awx
## True target for ui/src/build. Build ui from source.
$(UI_DIR)/src/build/awx: $(UI_DIR)/src $(UI_DIR)/src/node_modules/webpack
@echo "=== Building ui ==="
@cd $(UI_DIR)/src && PRODUCT="$(PRODUCT)" PUBLIC_PATH=/static/awx/ ROUTE_PREFIX=/ npm run build:awx
@mv $(UI_DIR)/src/build/awx/index.html $(UI_DIR)/src/build/awx/index_awx.html
.PHONY: ui/src
## Clone or link src of UI to ui/src, will re-clone/link/update if necessary.
ui/src: $(UI_DIR)/src
# TODO: Rewrite this big bash script in a more readable way.
## True target for ui/src.
$(UI_DIR)/src:
@echo "=== Setting up $(UI_DIR)/src ==="
@if [ ! -z "$(UI_LOCAL)" ]; then \
if [ -d $(UI_DIR)/src ]; then \
if [ "$$(readlink $(UI_DIR)/src)" = "$(UI_LOCAL)" ]; then \
echo "SKIP: ui/src. $(UI_DIR)/src already linked to $(UI_LOCAL)."; \
else \
echo "=== Linking $(UI_DIR)/src to $(UI_LOCAL) ==="; \
rm -rf $(UI_DIR)/src; \
ln -s $(UI_LOCAL) $(UI_DIR)/src; \
fi; \
else \
echo "=== Linking $(UI_DIR)/src to $(UI_LOCAL) ==="; \
ln -s $(UI_LOCAL) $(UI_DIR)/src; \
fi; \
elif [ ! -z "$(UI_GIT_REPO)" ]; then \
if [ -d $(UI_DIR)/src ]; then \
GIT_REMOTE_ORIGIN=$$(cd $(UI_DIR)/src && git remote get-url origin); \
GIT_REMOTE_BRANCH=$$(cd $(UI_DIR)/src && git rev-parse --abbrev-ref HEAD); \
if [ "$$GIT_REMOTE_ORIGIN" = "$(UI_GIT_REPO)" ] && [ "$$GIT_REMOTE_BRANCH" = "$(UI_GIT_BRANCH)" ]; then \
echo "=== Updating $(UI_DIR)/src from $(UI_GIT_BRANCH) of $(UI_GIT_REPO) ==="; \
git fetch && git pull; \
else \
echo "=== Cloning $(UI_DIR)/src from $(UI_GIT_BRANCH) of $(UI_GIT_REPO) ==="; \
rm -rf $(UI_DIR)/src; \
git clone --depth 1 --branch $(UI_GIT_BRANCH) $(UI_GIT_REPO) $(UI_DIR)/src || true; \
fi; \
else \
echo "=== Cloning $(UI_DIR)/src from $(UI_GIT_BRANCH) of $(UI_GIT_REPO) ==="; \
git clone --depth 1 --branch $(UI_GIT_BRANCH) $(UI_GIT_REPO) $(UI_DIR)/src || true; \
fi; \
else \
echo "FAILED: ui/src. UI_LOCAL and UI_GIT_REPO are not set."; \
exit 1; \
fi
.PHONY: ui/src/webpack
## Install webpack.
ui/src/webpack: $(UI_DIR)/src/node_modules/webpack
## True target for ui/src/webpack.
$(UI_DIR)/src/node_modules/webpack:
@echo "=== Installing webpack ==="
@cd $(UI_DIR)/src && npm install webpack
.PHONY: clean/ui
## Clean ui
clean/ui: clean/ui/build clean/ui/src
.PHONY: clean/ui/src
## Clean ui src
clean/ui/src:
rm -rf $(UI_DIR)/src
.PHONY: clean/ui/build
## Clean ui build
clean/ui/build:
rm -rf $(UI_DIR)/build
.PHONY: $(UI_DIR)/clean
## Alias for clean/ui, so we can run `make clean` directly in ui
$(UI_DIR)/clean: clean/ui
.PHONY: $(UI_DIR)/clean/src
## Alias for clean/ui/src, so we can run `make clean/src` directly in ui
$(UI_DIR)/clean/src: clean/ui/src
.PHONY: $(UI_DIR)/clean/build
## Alias for clean/ui/build, so we can run `make clean/build` directly in ui
$(UI_DIR)/clean/build: clean/ui/build

View File

@@ -1,115 +1,47 @@
# AWX-UI
# Instruction to build ui directly from this directory
## Requirements
- node >= 16.13.1, npm >= 8.x make, git
## Set src of the ui repo
## Development
The API development server will need to be running. See [CONTRIBUTING.md](../../CONTRIBUTING.md).
### Via GIT
```shell
# install
npm --prefix=awx/ui install
# Start the ui development server. While running, the ui will be reachable
# at https://127.0.0.1:3001 and updated automatically when code changes.
npm --prefix=awx/ui start
```bash
export UI_GIT_REPO=https://<git repo>
```
### Build for the Development Containers
If you just want to build a ui for the container-based awx development
environment and do not need to work on the ui code, use these make targets:
or
```shell
# The ui will be reachable at https://localhost:8043 or
# http://localhost:8013
make ui-devel
# clean up
make clean-ui
```bash
export UI_GIT_REPO=git@<git repo>
```
### Using an External Server
If you normally run awx on an external host/server (in this example, `awx.local`),
you'll need use the `TARGET` environment variable when starting the ui development
server:
optionally set branch (default is main)
```shell
TARGET='https://awx.local:8043' npm --prefix awx/ui start
```bash
export UI_GIT_BRANCH=main
```
## Testing
```shell
# run code formatting check
npm --prefix awx/ui run prettier-check
### Via symlink to existing clone
# run lint checks
npm --prefix awx/ui run lint
NOTE: UI_LOCAL have higher precedence than UI_GIT_REPO, if UI_LOCAL is set, UI_GIT_REPO will be ignored.
# run all unit tests
npm --prefix awx/ui run test
# run a single test (in this case the login page test):
npm --prefix awx/ui test -- src/screens/Login/Login.test.jsx
# start the test watcher and run tests on files that you've changed
npm --prefix awx/ui run test-watch
# start the tests and get the coverage report after the tests have completed
npm --prefix awx/ui run test -- --coverage
```
#### Note:
- Once the test watcher is up and running you can hit `a` to run all the tests.
- All commands are run on your host machine and not in the api development containers.
## Updating Dependencies
It is not uncommon to run the ui development tooling outside of the awx development
container. That said, dependencies should always be modified from within the
container to ensure consistency.
```shell
# make sure the awx development container is running and open a shell
docker exec -it tools_awx_1 bash
# start with a fresh install of the current dependencies
(tools_awx_1)$ make clean-ui && npm --prefix=awx/ui ci
# add an exact development dependency
(tools_awx_1)$ npm --prefix awx/ui install --save-dev --save-exact dev-package@1.2.3
# add an exact production dependency
(tools_awx_1)$ npm --prefix awx/ui install --save --save-exact prod-package@1.23
# remove a development dependency
(tools_awx_1)$ npm --prefix awx/ui uninstall --save-dev dev-package
# remove a production dependency
(tools_awx_1)$ npm --prefix awx/ui uninstall --save prod-package
# exit the container
(tools_awx_1)$ exit
# add the updated package.json and package-lock.json files to scm
git add awx/ui/package.json awx/ui/package-lock.json
```
#### Note:
- Building the ui can use up a lot of resources. If you're running docker for mac or similar
virtualization, the default memory limit may not be enough and you should increase it.
## Building for Production
```shell
# built files are placed in awx/ui/build
npm --prefix awx/ui run build
```bash
export UI_LOCAL = /path/to/your/ui
```
## CI Container
## Build
To run:
```shell
cd awx/awx/ui
docker build -t awx-ui .
docker run --name tools_ui_1 --network _sources_default --link 'tools_awx_1:awx' -e TARGET="https://awx:8043" -p '3001:3001' --rm -v $(pwd)/src:/ui/src awx-ui
```bash
make ui
```
**Note:** This is for CI, test systems, zuul, etc. For local development, see [usage](https://github.com/ansible/awx/blob/devel/awx/ui/README.md#Development)
## Rebuild
```bash
make -B ui
```
## Clean
```bash
make clean/ui
```

Some files were not shown because too many files have changed in this diff Show More