Adds fields client_id and client_secret which
will result in authentication via service account
on console.redhat.com
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Adds new modules for CRUD operations on the
following endpoints:
- api/v2/role_definitions
- api/v2/role_user_assignments
- api/v2/role_team_assignments
Note: assignment is Create or Delete only
Additional changes:
- Currently DAB endpoints do not have "type"
field on the resource list items. So this modifies
the create_or_update_if_needed to allow manually
specifying item type.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
As we do for control nodes, disable the
install_bundle endpoint for ingress nodes.
This can be done by checking if instance managed
is True.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Listener Addresses is a better name to
emphasize these are routable addresses to
reach a listener service on the node.
Also removed expand toggle on the listener
addresses list items, as the expanded mode
had no additional information.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Make protocol be blank on instance if there
is no canonical address for this instance.
It was defaulting to "tcp" before.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
In receptor address post-save method:
- Fixed detecting if address was missing
a link from control nodes
- Use InstanceLink create_or_update to prevent
adding duplicate InstanceLink source and target
peers
In instance serializer create_or_update,
delete receptor addresses first before doing
instance create or update. This ensures that we don't
trigger unnecessary post-save methods that might
attempt to manipulate receptor addresses that
will just be removed later.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
test_listener_port
test_peers_from_control_nodes
test_peers_from_control_nodes_without_listener_port
are covered in the following tests:
test_no_op
test_creates_canonical_address
test_deletes_canonical_address
test_updates_canonical_address
test_canonical_address_validation_error
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Add functional test case for inspecting
established receptor connections.
InstanceLink starts in ADDING state, and should
move to ESTABLISHED state if the connection
is detected in the receptor status output.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Adds validation and a unit test to ensure:
- peers_from_control_nodes=True should fail if
listener_port is not set
- peers_from_control_nodes=False should be NOOP if
listener_port is not set
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Add validation to prevent any managed node
from modifying "peers" through the API
Peering from these nodes should be handled
by setting peers_from_control_nodes only.
Managed nodes are control nodes and
ingress hop nodes.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
InstanceLink target should not be null.
Should be safe to set to null=False, because we have
a custom RunPython method to explicitly set
target to a proper key.
Also, add new test to test_migrations
which ensures data integrity after migrating
the receptor address model changes.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Adds remove_receptor_address to delete a
receptor address from the database
Also, enforce that only 1 canonical address
can be added to an instance via
the add_receptor_address command.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
- Add forwards method to create a receptor address
for any existing Instance that has listener_port defined
- Add forwards method to modify each InstanceLink object
that changes target to the newly created receptor addresses
This migration was implemented as follows:
1. Add a target_new to InstanceLink which is a foreign key
to ReceptorAddress
2. create receptor addresses
3. link to these receptor addresses using the target_new field
4. rename target_new to target
5. drop listener_port and peers_from_control_nodes from Instance
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
If a Instance endpoint is patched with
{"peers_from_control_nodes" True}
but a listener_port is not defined on the instance,
or is part of the patch payload, do not create a
receptor address.
Only update or create a receptor address if listener_port
is set, either in the payload or already on the instance.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
command has 3 "targets", all of which should be a
ReceptorAddress object
- peers, disconnect, exact
Add logic to make sure each entry in those lists
are receptor addresses.
When creating InstanceLink objects, make sure
target is ReceptorAddress, not an Instance.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
After removing CRUD from receptor addresses, we need
to remove the module.
- remove receptor_address module
- Add listener_port to instance module
- Add peers_from_control_nodes to instance module
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Removes ability to directly create and delete
receptor addresses for a given node.
Instead, receptor addresses are created automatically
if listener_port is set on the Instance.
For example patching "hop" instance
with {"listener_port": 6667}
will create a canonical receptor address with port
6667.
Likewise, peers_from_control_nodes on the instance
sets the peers_from_control_nodes on the canonical
address (if listener port is also set).
protocol is a read-only field that simply reflects
the canonical address protocol.
Other Changes:
- rename k8s_routable to is_internal
- add protocol to ReceptorAddress
- remove peers_from_control_nodes and listener_port
from Instance model
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Creates a non-deletable address that acts as
the "main" address for this instance.
All other addresses for that instance must
be non-canonical.
When listener_port on an instance is set, automatically
create a canonical receptor address where:
- address is hostname of instance
- port is listener_port
- canonical is True
Additionally, protocol field is added to instance to
denote the receptor listener protocol to use (ws, tcp).
The receptor config listener information is derived from
the listener_port and protocol information. Having a
canonical address that mirrors the listener_port ensures that
an address exists that matches the receptor config information.
Other changes:
- Add managed field to receptor address.
If managed is True, no fields on on this address can be edited
via the API.
If canonical is True, only the address cannot be edited.
- Add managed field to instance. If managed is True, users
cannot set node_state to deprovisioning (i.e. cannot delete node)
This change to our mechanism to prevent users from deleting
the mesh ingress hop node.
- Field is_internal is now renamed to k8s_routable
- Add reverse_peers on instance which is a list of instance IDs
that peer to this instance (via an address)
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Make receptoraddress list views
searchable by "address"
Other changes:
- Add help text to source and target of the
InstanceLink model
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
- Add receptor_address module which allows
users to create addresses for instances
- Update awx_collection functional and integration
tests to support new peering design
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Updated existing tests to support the
ReceptorAddress model
- cannot peer to self
- cannot peer to node that is already peered to me
- cannot peer to node more than once (via 2+ addresses)
- cannot set is_internal True
Other changes:
Change post save signal to only call
schedule_write_receptor_config() when an actual change is detected.
Make functional tests more robust by
checking for specific validation error in the
response.
I.e. instead of just checking for 400, just for 400
and that the error message corresponds to the
validation we are testing for.
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
- cannot peer to self
- cannot peer to instance that is already peered to self
Other changes:
- ReceptorAddress protocol field restricted to choices: tcp, ws, wss
- fix awx-manage list_instances when instance.last_seen is None
- InstanceLink make source and target unique together
- Add help text to the ReceptorAddress fields
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
- websocket_path can only be set if protocol is ws
- is_internal must be False
- only 1 address per instance can have
peers_from_control_nodes set to True
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
register_peers has inputs:
source: source instance
peers: list of instances the source should peer to
InstanceLink "target" is now expected to be a ReceptorAddress
For each peer, we can just use the first receptor address. If
multiple receptor addresses exist, throw a command error.
Currently this command is only used on VM-deployments, where
there is only a single receptor address per instance, so this
should work fine.
Other changes:
drop listener_port field from Instance. Listener port is now just
"port" on ReceptorAddress
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
group_vars all.yaml changes:
- peer entry has two fields, address and port
- receptor_port is inferred from the first
receptor_address entry that uses protocol tcp
other changes:
ActivityStream now records when receptor_addresses
are peered to
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
- write_receptor_config peers to ReceptorAddress entries
that have peers_from_control_nodes enabled
- peers_from_control_nodes and listener_port removed from Instance model
- peers_from_control_nodes added to ReceptorAddress model
- ReceptorAddress is now unique by address and protocol combination
- Write receptor config task is dispatched upon ReceptorAddress creation
or deletion, and when control node is first created
- InstanceLinkSerializer adds a target_address field and has logic
to grab the instance hostname associated with the peered ReceptorAddress
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
Add post save and post delete hooks to
call write_receptor_config when
a receptor address is added / removed.
Add peers_from_control_nodes to
provision_instance
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
- Add database contraints to make sure addresses
are unique
If port is defined:
address, port, protocol, websocket_path are unique together
if port is not defined:
address, protocol, websocket_path are unique together
- Allow deleting address via API
- Add ReceptorAddressAccess to determine permissions
- awx-manage add_receptor_address returns changed: True
if successful
Might help to install receptor last,
that way when nodes are first connected to the mesh
they already have podman installed and can potentially
run jobs. Otherwise it might be possible for controller
to launch jobs against nodes that aren't fully set up.
Add a check_peers_changed() utility method
to determine if peers in attrs matches
the current instance peers.
Other changes:
- Set ip_address default to "", and do not
allow null.
Get rid of PeersSerializer and just use SlugRelatedField,
which should be more a straightforward approach.
Other changes:
- cleanup code related to the already-removed api/v2/peers
endpoint
- add "hybrid" node type into more instance_peers test cases
API changes
- cannot change peers or enable
peers_from_control_nodes on VM deployments
- allow setting ip_address
- use ip_address over hostname in the generated
group_vars/all.yml
- Drop api/v2/peers endpoint
DB changes
- add ip_address unique constraint, but ignore "" entries
Other changes
- provision_instance should take listener_port option
Tests
- test that new controls doesn't disturb other peers
relationships
- test ip_address over hostname
Dynamically flipping from Established
to Disconnected is not the intended
usage of InstanceLink State.
- Link state starts in Adding and becomes
Established once any control node first sees the link
is in the status KnownConnectionCosts
inspect_established_receptor_connections should
not change link state is current state is Removing.
Other changes:
- rename inspect_execution_nodes to inspect_execution_and_hop_nodes
- Default link state is Adding
- Set min listener_port value to 1024
- inspect_established_receptor_connections now
runs as part of cluster_node_heartbeat task
- periodically ping postres on port 5432 and only start
migrations if successful.
- prevents crash loop when attempting migrations before
postgres is ready.
return more context for bulk host create
now return list of minimal info about host objects
[
{
"name": "lakjdsafoiaweirnladlk",
"enabled": true,
"instance_id": "",
"description": "",
"variables": "",
"id": 4593,
"url": "/api/v2/hosts/4593/",
"inventory": "/api/v2/inventories/1/"
}
]
Updated tests, but needed to work around some weird behavior with
sqlite. Apparently it behaves differently around assigning ID's to the
result of bulk_create and that is messed up my use of `reverse` to look
up the url of the hosts
If a job fails, we do receptor work results and put that output
into result_traceback.
We should only do this if
1. Receptor unit has failed
2. Runner callback processed 0 events
Otherwise we risk putting too much data into this field.
- Prevents changing hostname, listener_port, or node_type for instances
that already exist
- API default node_type is execution
- API default node_state is installed
awx-web container does not have access to receptor socket, and the
execution node health check requires receptorctl.
This change runs the health check asynchronously in the task container.
Seth Foster
