33748 Commits

Author SHA1 Message Date
Seth Foster
1b44bebed3 Peers_from_control_nodes requires listener port
Adds validation and a unit test to ensure:

- peers_from_control_nodes=True should fail if
listener_port is not set
- peers_from_control_nodes=False should be NOOP if
listener_port is not set

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
a4cf55bdba Require receptor collection 2.0.3
This release supports ingress hop node, and
is backwards compatible with previous AWX
version.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
c333d0e82f Prevent modifying peers on managed node
Add validation to prevent any managed node
from modifying "peers" through the API

Peering from these nodes should be handled
by setting peers_from_control_nodes only.

Managed nodes are control nodes and
ingress hop nodes.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
David O Neill
b093c89a84 Form hardening and node type exclusion
Disabled add/edit/remove for managed nodes
Tightened validation on Peers from control nodes
2024-02-02 10:37:41 -05:00
Jeff Bradberry
f98493aa61 Support wss as ws-listener in the Receptor config 2024-02-02 10:37:41 -05:00
Hao Liu
c36d2b0485 Update requirements.yml 2024-02-02 10:37:41 -05:00
Jeff Bradberry
8ddb604bf1 Template the listener protocol into the receptor install bundle (#14792)
Previously we were hard-coding tcp, now we need to also support ws/wss.
2024-02-02 10:37:41 -05:00
Seth Foster
cd9dd43be7 Make InstanceLink target non-nullable
InstanceLink target should not be null.

Should be safe to set to null=False, because we have
a custom RunPython method to explicitly set
target to a proper key.

Also, add new test to test_migrations
which ensures data integrity after migrating
the receptor address model changes.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Jeff Bradberry
82323390a7 Reconstitute migration file
Do the things as a proper 3-step process -- add new stuff, copy the data, drop the old.
2024-02-02 10:37:41 -05:00
Seth Foster
4c5ac1d3da Add management command to remove address
Adds remove_receptor_address to delete a
receptor address from the database

Also, enforce that only 1 canonical address
can be added to an instance via
the add_receptor_address command.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
David O Neill
9c06370e33 InstanceAdd sends null for port_listener 2024-02-02 10:37:41 -05:00
David O Neill
449b95d1eb Fix remaning tests, removed unused code 2024-02-02 10:37:41 -05:00
David O Neill
1712540c8e Updates for receptor reaslese to ui for protocol and is_managed 2024-02-02 10:37:41 -05:00
Seth Foster
7cf639d8eb Add migration to support InstanceLink changes
- Add forwards method to create a receptor address
for any existing Instance that has listener_port defined

- Add forwards method to modify each InstanceLink object
that changes target to the newly created receptor addresses

This migration was implemented as follows:

1. Add a target_new to InstanceLink which is a foreign key
to ReceptorAddress
2. create receptor addresses
3. link to these receptor addresses using the target_new field
4. rename target_new to target
5. drop listener_port and peers_from_control_nodes from Instance

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
dbfcc40d7c Only create receptor address if port is defined
If a Instance endpoint is patched with
{"peers_from_control_nodes" True}

but a listener_port is not defined on the instance,
or is part of the patch payload, do not create a
receptor address.

Only update or create a receptor address if listener_port
is set, either in the payload or already on the instance.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Hao Liu
73d2c92ae3 Fix condition for creating receptor_address
If listener_port is not explicitly defined don't create a receptor_address
2024-02-02 10:37:41 -05:00
Seth Foster
24a4242147 Add protocol to receptor address serializer
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
92ce85b688 Remove unused warnings import
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
9531f8377a Ensure register_peers target is ReceptorAddress
command has 3 "targets", all of which should be a
ReceptorAddress object

- peers, disconnect, exact

Add logic to make sure each entry in those lists
are receptor addresses.

When creating InstanceLink objects, make sure
target is ReceptorAddress, not an Instance.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Hao Liu
15a16b3dd1 Update bootstrap_development.sh 2024-02-02 10:37:41 -05:00
Seth Foster
a37e7bf147 Add canonical=True when creating ReceptorAddress in tests
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Hao Liu
a2fcd2f97a Fix ui-lint error 2024-02-02 10:37:41 -05:00
Hao Liu
c394ffdd19 Fix lint error, remove unused import 2024-02-02 10:37:41 -05:00
Seth Foster
69102cf265 Remove receptor_address module from collection
After removing CRUD from receptor addresses, we need
to remove the module.

- remove receptor_address module
- Add listener_port to instance module
- Add peers_from_control_nodes to instance module

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
a188798543 Make canonical field default to False
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
60108ebd10 Rename migration dependency
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Jeff Bradberry
8c7c00451a Join across the InstanceLink.target to the underlying Instance
to avoid having to do a lookup for every instance in the list for the
mesh visualizer endpoint.
2024-02-02 10:37:41 -05:00
Seth Foster
7a1ed406da Remove CRUD for Receptor Addresses
Removes ability to directly create and delete
receptor addresses for a given node.

Instead, receptor addresses are created automatically
if listener_port is set on the Instance.

For example patching "hop" instance

with {"listener_port": 6667}

will create a canonical receptor address with port
6667.

Likewise, peers_from_control_nodes on the instance
sets the peers_from_control_nodes on the canonical
address (if listener port is also set).

protocol is a read-only field that simply reflects
the canonical address protocol.

Other Changes:

- rename k8s_routable to is_internal
- add protocol to ReceptorAddress
- remove peers_from_control_nodes and listener_port
from Instance model

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
César Francisco San Nicolás Martínez
f916ffe1e9 Adjust migration names and dependencies 2024-02-02 10:37:41 -05:00
César Francisco San Nicolás Martínez
901dbd697e Comment unused dependency 2024-02-02 10:37:41 -05:00
David O Neill
d8b4a9825e Cleanup
- Remove peer selection on add and edit instance
- Added conconcial name and order comlums names same on endpoints and
  peers
- Other cleanup items
- rename name to instance name
2024-02-02 10:37:41 -05:00
Hao Liu
6db66c5f81 Fix provision instance not respecting protocol 2024-02-02 10:37:41 -05:00
David O Neill
82ad7dcf40 Mesh UI support
- add endpoint
- delete endpoint (wip)
- associate
- disassociate
2024-02-02 10:37:41 -05:00
David O Neill
93500f9fea Fix lint trailing whitespace 2024-02-02 10:37:41 -05:00
Seth Foster
9ba70c151d Add canonical receptor address
Creates a non-deletable address that acts as
the "main" address for this instance.

All other addresses for that instance must
be non-canonical.

When listener_port on an instance is set, automatically
create a canonical receptor address where:
  - address is hostname of instance
  - port is listener_port
  - canonical is True

Additionally, protocol field is added to instance to
denote the receptor listener protocol to use (ws, tcp).

The receptor config listener information is derived from
the listener_port and protocol information. Having a
canonical address that mirrors the listener_port ensures that
an address exists that matches the receptor config information.

Other changes:
- Add managed field to receptor address.
If managed is True, no fields on on this address can be edited
via the API.
If canonical is True, only the address cannot be edited.

- Add managed field to instance. If managed is True, users
cannot set node_state to deprovisioning (i.e. cannot delete node)

This change to our mechanism to prevent users from deleting
the mesh ingress hop node.

- Field is_internal is now renamed to k8s_routable

- Add reverse_peers on instance which is a list of instance IDs
that peer to this instance (via an address)

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
David O Neill
46dc61253f UI Updates for receptor peering 2024-02-02 10:37:41 -05:00
Seth Foster
6cb2cd18b0 Fix proper indent to instance module
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
5d1dd8ec41 Fix inconsistent tab width
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
9f69daf787 Add choices to module protocol field
Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
16ece5de7e Remove unused variables and imports
Addresses flake8 and linting failures

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
ab0e9265c5 Add search fields to views
Make receptoraddress list views
searchable by "address"

Other changes:
- Add help text to source and target of the
InstanceLink model

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
04cbbbccfa Update awx_collection to support ReceptorAddress
- Add receptor_address module which allows
users to create addresses for instances

- Update awx_collection functional and integration
tests to support new peering design

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
d1cacf64de Add functional and unit tests
Updated existing tests to support the
ReceptorAddress model

- cannot peer to self
- cannot peer to node that is already peered to me
- cannot peer to node more than once (via 2+ addresses)
- cannot set is_internal True

Other changes:
Change post save signal to only call
schedule_write_receptor_config() when an actual change is detected.

Make functional tests more robust by
checking for specific validation error in the
response.

I.e. instead of just checking for 400, just for 400
and that the error message corresponds to the
validation we are testing for.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
5385eb0fb3 Add validation when setting peers
- cannot peer to self
- cannot peer to instance that is already peered to self

Other changes:
- ReceptorAddress protocol field restricted to choices: tcp, ws, wss
- fix awx-manage list_instances when instance.last_seen is None
- InstanceLink make source and target unique together
- Add help text to the ReceptorAddress fields

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
7d7503279d Add API validation when creating ReceptorAddress
- websocket_path can only be set if protocol is ws
- is_internal must be False
- only 1 address per instance can have
peers_from_control_nodes set to True

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Hao Liu
d860d1d91b Temp change to aid dev
Temporarily switch to feature branch for receptor-collection in instance_install_bundle to make dev easier
2024-02-02 10:37:41 -05:00
Seth Foster
3a17c45b64 Register_peers support for receptor_addresses
register_peers has inputs:

source: source instance
peers: list of instances the source should peer to

InstanceLink "target" is now expected to be a ReceptorAddress

For each peer, we can just use the first receptor address. If
multiple receptor addresses exist, throw a command error.

Currently this command is only used on VM-deployments, where
there is only a single receptor address per instance, so this
should work fine.

Other changes:
drop listener_port field from Instance. Listener port is now just
"port" on ReceptorAddress

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
bca68bcdf1 Add install bundle support
group_vars all.yaml changes:
- peer entry has two fields, address and port
- receptor_port is inferred from the first
receptor_address entry that uses protocol tcp

other changes:
ActivityStream now records when receptor_addresses
are peered to

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
c32f234ebb Add peers_from_control_nodes to ReceptorAddress
- write_receptor_config peers to ReceptorAddress entries
that have peers_from_control_nodes enabled

- peers_from_control_nodes and listener_port removed from Instance model

- peers_from_control_nodes added to ReceptorAddress model

- ReceptorAddress is now unique by address and protocol combination

- Write receptor config task is dispatched upon ReceptorAddress creation
or deletion, and when control node is first created

- InstanceLinkSerializer adds a target_address field and has logic
to grab the instance hostname associated with the peered ReceptorAddress

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00
Seth Foster
5cb3d3b078 Update receptor conf when address changes
Add post save and post delete hooks to
call write_receptor_config when
a receptor address is added / removed.

Add peers_from_control_nodes to
provision_instance

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-02-02 10:37:41 -05:00