* Add new enablement settings from DAB RBAC
* Initial implementation of system auditor as role without testing
* Fix system auditor role, remove duplicate assignments
* Make the system auditor role managed
* Flake8 fix
* Remove another thing from old solution
* Fix a few test failures
* Add extra setting to disable custom system roles via API
* Add test for custom role prohibition
Develop ability to list permissions for existing roles
Create a model registry for RBAC-tracked models
Write the data migration logic for creating
the preloaded role definitions
Write migration to migrate old Role into ObjectRole model
This loops over the old Role model, knowing it is unique
on object and role_field
Most of the logic is concerned with identifying the
needed permissions, and then corresponding role definition
As needed, object roles are created and users then teams
are assigned
Write re-computation of cache logic for teams
and then for object role permissions
Migrate new RBAC internals to ansible_base
Migrate tests to ansible_base
Implement solution for visible_roles
Expose URLs for DAB RBAC
* Middleware classes can be instantiated multiple times in testing. To
make this a non-issue, move the init code for named urls out of the
middleware init and into the app init.
* This makes it easier to use other testing facilities, like
LiveServerTestCase, without having to mock the named url middleware
init.
* `drf_reverse()` was introduced here 1a75b1836e
* There is a comment about monkey patching. I can't find the monkey patch it is referencing.
* AWX `drf_reverse()` is a copy paste of this https://github.com/encode/django-rest-framework/blob/master/rest_framework/reverse.py#L32
* The only difference is DRF's version calls `preserve_builtin_query_params()`
* `preserve_builtin_query_params()` only does something if `api_settings.URL_FORMAT_OVERRIDE` is defined.
* We don't use `REST_FRAMEWORK.URL_FORMAT_OVERRIDE`
* add resources api to controller
* update setting
models are not the source of truth in AWX
* Force creation of ServiceID object in tests
* fix typo
* settings fix for CI
---------
Co-authored-by: Alan Rominger <arominge@redhat.com>
* authorized/not authorized tests for wsrelay endpoint
* not authorized test for web browser websockets
* skeleton of a test for authorized web browser websockets
test_listener_port
test_peers_from_control_nodes
test_peers_from_control_nodes_without_listener_port
are covered in the following tests:
test_no_op
test_creates_canonical_address
test_deletes_canonical_address
test_updates_canonical_address
test_canonical_address_validation_error
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Add functional test case for inspecting
established receptor connections.
InstanceLink starts in ADDING state, and should
move to ESTABLISHED state if the connection
is detected in the receptor status output.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Adds validation and a unit test to ensure:
- peers_from_control_nodes=True should fail if
listener_port is not set
- peers_from_control_nodes=False should be NOOP if
listener_port is not set
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Add validation to prevent any managed node
from modifying "peers" through the API
Peering from these nodes should be handled
by setting peers_from_control_nodes only.
Managed nodes are control nodes and
ingress hop nodes.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
InstanceLink target should not be null.
Should be safe to set to null=False, because we have
a custom RunPython method to explicitly set
target to a proper key.
Also, add new test to test_migrations
which ensures data integrity after migrating
the receptor address model changes.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
If a Instance endpoint is patched with
{"peers_from_control_nodes" True}
but a listener_port is not defined on the instance,
or is part of the patch payload, do not create a
receptor address.
Only update or create a receptor address if listener_port
is set, either in the payload or already on the instance.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Removes ability to directly create and delete
receptor addresses for a given node.
Instead, receptor addresses are created automatically
if listener_port is set on the Instance.
For example patching "hop" instance
with {"listener_port": 6667}
will create a canonical receptor address with port
6667.
Likewise, peers_from_control_nodes on the instance
sets the peers_from_control_nodes on the canonical
address (if listener port is also set).
protocol is a read-only field that simply reflects
the canonical address protocol.
Other Changes:
- rename k8s_routable to is_internal
- add protocol to ReceptorAddress
- remove peers_from_control_nodes and listener_port
from Instance model
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Creates a non-deletable address that acts as
the "main" address for this instance.
All other addresses for that instance must
be non-canonical.
When listener_port on an instance is set, automatically
create a canonical receptor address where:
- address is hostname of instance
- port is listener_port
- canonical is True
Additionally, protocol field is added to instance to
denote the receptor listener protocol to use (ws, tcp).
The receptor config listener information is derived from
the listener_port and protocol information. Having a
canonical address that mirrors the listener_port ensures that
an address exists that matches the receptor config information.
Other changes:
- Add managed field to receptor address.
If managed is True, no fields on on this address can be edited
via the API.
If canonical is True, only the address cannot be edited.
- Add managed field to instance. If managed is True, users
cannot set node_state to deprovisioning (i.e. cannot delete node)
This change to our mechanism to prevent users from deleting
the mesh ingress hop node.
- Field is_internal is now renamed to k8s_routable
- Add reverse_peers on instance which is a list of instance IDs
that peer to this instance (via an address)
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Updated existing tests to support the
ReceptorAddress model
- cannot peer to self
- cannot peer to node that is already peered to me
- cannot peer to node more than once (via 2+ addresses)
- cannot set is_internal True
Other changes:
Change post save signal to only call
schedule_write_receptor_config() when an actual change is detected.
Make functional tests more robust by
checking for specific validation error in the
response.
I.e. instead of just checking for 400, just for 400
and that the error message corresponds to the
validation we are testing for.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
* Add username and password to handle_auth and update exception message
Revise naming of ldap username and password
* Add url for LDAP and userpass to method_auth
* Add information regarding LDAP and username and password to credential plugins documentation
Revise ldap_auth to userpass_auth and revised exception to better reflect functionality
* Revise method_auth to ensure certs can be used with username and ensure namespace functionality is not hindered
Add support for receiving webhooks from Bitbucket Data Center, and add support for posting build statuses back
Note that this is very explicitly only for Bitbucket Data Center.
The entire webhook format and API is entirely different for Bitbucket Cloud.
* Narrow the scope of RBAC evaluations
* Update tests for RBAC method changes
* Simplify querset for credentials in org
* Fix call pattern to pass in team role obj
* Adding hosts bulk deletion feature
Signed-off-by: Avi Layani <alayani@redhat.com>
* fix the type of the argument
Signed-off-by: Avi Layani <alayani@redhat.com>
* fixing activity_entry tracking
Signed-off-by: Avi Layani <alayani@redhat.com>
* Revert "fixing activity_entry tracking"
This reverts commit c8eab52c2ccc5abe215d56d1704ba1157e5fbbd0.
Since the bulk_delete is not related to an inventory, only hosts which
can be from different inventories.
* get only needed vars to reduce memory consumption
Signed-off-by: Avi Layani <alayani@redhat.com>
* filtering the data to reduce memory increase the number of queries
Signed-off-by: Avi Layani <alayani@redhat.com>
* update the activity stream for inventories
Signed-off-by: Avi Layani <alayani@redhat.com>
* fix the changes dict initialiazation
Signed-off-by: Avi Layani <alayani@redhat.com>
---------
Signed-off-by: Avi Layani <alayani@redhat.com>
* Add TLS certificate auth for HashiCorp Vault
Add support for AWX to authenticate with HashiCorp Vault using
TLS client certificates.
Also updates the documentation for the HashiCorp Vault secret management
plugins to include both the new TLS options and the missing Kubernetes
auth method options.
Signed-off-by: Andrew Austin <aaustin@redhat.com>
* Refactor docker-compose vault for TLS cert auth
Add TLS configuration to the docker-compose Vault configuration and
use that method by default in vault plumbing.
This ensures that the result of bringing up the docker-compose stack
with vault enabled and running the plumb-vault playbook is a fully
working credential retrieval setup using TLS client cert authentication.
Signed-off-by: Andrew Austin <aaustin@redhat.com>
* Remove incorrect trailing space
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
* Make vault init idempotent
- improve error handling for vault_initialization
- ignore error if vault cert auth is already configured
- removed unused register
* Add VAULT_TLS option
Make TLS for HashiCorp Vault optional and configurable via VAULT_TLS env var
* Add retries for vault init
Sometime it took longer for vault to fully come up and init will fail
---------
Signed-off-by: Andrew Austin <aaustin@redhat.com>
Co-authored-by: Hao Liu <44379968+TheRealHaoLiu@users.noreply.github.com>
Co-authored-by: Hao Liu <haoli@redhat.com>
* allow pytest --migrations to succeed
* We actually subvert migrations from running in test via pytest.ini
--no-migrations option. This has led to bit rot for the sqlite
migrations happy path. This changeset pays off that tech debt and
allows for an sqlite migration happy path.
* This paves the way for programatic invocation of individual migrations
and weaving of the creation of resources (i.e. Instance, Job Template,
etc). With this, a developer can instantiate various database states,
trigger a migration, assert the state of the db, and then have pytest
rollback all of that.
* I will note that in practice, running these migrations is dog shit
slow BUT this work also opens up the possibility of saving and
re-using sqlite3 database files. Normally, caching is not THE answer
and causes more harm than good. But in this case, our migrations are
mostly write-once (I say mostly because this change set violates
that :) so cache invalidation isn't a major issue.
* functional test for migrations on sqlite
* We commonly subvert running migrations in test land. Test land uses
sqlite. By not constantly exercising this code path it atrophies. The
smoke test here is to continuously exercise that code path.
* Add ci test to run migration tests separately, they take =~ 2-3
minutes each on my laptop.
* The smoke tests also serves as an example of how to write migration
tests.
* run migration tests in ci
Get rid of PeersSerializer and just use SlugRelatedField,
which should be more a straightforward approach.
Other changes:
- cleanup code related to the already-removed api/v2/peers
endpoint
- add "hybrid" node type into more instance_peers test cases
API changes
- cannot change peers or enable
peers_from_control_nodes on VM deployments
- allow setting ip_address
- use ip_address over hostname in the generated
group_vars/all.yml
- Drop api/v2/peers endpoint
DB changes
- add ip_address unique constraint, but ignore "" entries
Other changes
- provision_instance should take listener_port option
Tests
- test that new controls doesn't disturb other peers
relationships
- test ip_address over hostname
inspect_established_receptor_connections should
not change link state is current state is Removing.
Other changes:
- rename inspect_execution_nodes to inspect_execution_and_hop_nodes
- Default link state is Adding
- Set min listener_port value to 1024
- inspect_established_receptor_connections now
runs as part of cluster_node_heartbeat task
Add Disconnected link state
introspect_receptor_connections is a periodic
task that examines active receptor connections
and cross-checks it with the InstanceLink info.
Any links that should be active but are not
will be put into a Disconnected state. If
active, it will be in an Established state.
UI - Add hop creation and peers mgmt (#13922)
* add UI for mgmt peers, instance edit and add
* add peer info on detail and bug fix on detail
* remove unused chip and change peer label
* rename lookup, put Instance type disable on edit
---------
Co-authored-by: tanganellilore <lorenzo.tanagnelli@hotmail.it>
Dispatcher refactoring to get pg_notify publish payload
as separate method
Refactor periodic module under dispatcher entirely
Use real numbers for schedule reference time
Run based on due_to_run method
Review comments about naming and code comments
