Commit Graph

289 Commits

Author SHA1 Message Date
Seth Foster
965127637b Make ip_address read only
Setting a different value for ip_address
and hostname does not work with the current
way we create receptor certs.
2023-08-29 13:06:54 -04:00
Seth Foster
c47acc5988 Change PeersSerializer to SlugRelatedField
Get rid of PeersSerializer and just use SlugRelatedField,
which should be more a straightforward approach.

Other changes:
- cleanup code related to the already-removed api/v2/peers
endpoint
- add "hybrid" node type into more instance_peers test cases
2023-08-29 13:06:54 -04:00
Seth Foster
81e06dace2 Add listener_port to provision_instance
API changes
- cannot change peers or enable
peers_from_control_nodes on VM deployments
- allow setting ip_address
- use ip_address over hostname in the generated
group_vars/all.yml
- Drop api/v2/peers endpoint

DB changes
- add ip_address unique constraint, but ignore "" entries

Other changes
- provision_instance should take listener_port option

Tests
- test that new controls doesn't disturb other peers
relationships
- test ip_address over hostname
2023-08-29 13:06:54 -04:00
Seth Foster
3bd25c682e Allow setting ip_address for execution nodes 2023-08-29 13:06:54 -04:00
Lorenzo Tanganelli
f7fdb7fe8d Add peers readonly api and instancelink constraint (#13916)
Add Disconnected link state

introspect_receptor_connections is a periodic
task that examines active receptor connections
and cross-checks it with the InstanceLink info.

Any links that should be active but are not
will be put into a Disconnected state. If
active, it will be in an Established state.

UI - Add hop creation and peers mgmt (#13922)

* add UI for mgmt peers, instance edit and add

* add peer info on detail and bug fix on detail

* remove unused chip and change peer label

* rename lookup, put Instance type disable on edit

---------

Co-authored-by: tanganellilore <lorenzo.tanagnelli@hotmail.it>
2023-08-29 13:06:54 -04:00
Seth Foster
d8abd4912b Add support in hop nodes in API 2023-08-29 13:06:54 -04:00
Martin Slemr
6c5590e0e6 HostMetricSummaryMonthly command + views + scheduled task (#13999)
Co-authored-by: Alan Rominger <arominge@redhat.com>
2023-07-12 16:40:09 -04:00
Alan Rominger
06430741ab Fix 400 error from job labels sublist (#13972)
This was caused by an incorrect parent_key ref from label to job
  also applies to workflow_job labels

This fixes a regression introduced by a recent merge (#13957)
2023-05-10 11:37:59 -04:00
Shane McDonald
f1196fc019 Clean up string formatting issues from black migration 2023-05-10 08:19:23 -04:00
Alan Rominger
77175d2862 Consolidate get_queryset methods (#13906)
In a prior merge, we added the ability to slap filter_read_permission = False on a view to get a certain functionality where it didn't filter a sublist the view is showing.

This logic already existed in a highly duplicated form among a number of views, so this deletes those methods in favor of the flag.
2023-04-28 09:10:18 -04:00
Gabriel Muniz
d8af19d169 Fix organization not showing all galaxy credentials for org admin (#13676)
* Fix organization not showing all galaxy credentials for org admin

* Add basic test to ensure counts

* refactored approach to allow removal of redundant code

* Allow configurable prefetch_related

* implicitly get related fields

* Removed extra queryset code
2023-04-25 15:33:42 -04:00
Gabriel Muniz
cd5cc64d6a Fix 500 on missing inventory for provisioning callbacks (#13862)
* Fix 500 on missing inventory for provisioning callbacks

* Added test to cover bug fix

* Reworded msg to clear what is missing to start the callback
2023-04-19 09:27:41 -04:00
Alan Rominger
16ad27099e [constructed-inventory] Save facts on model for original host (#13700)
* Save facts on model for original host

Redirect to original host for ansible facts

Use current inventory hosts for facts instance_id filter
Thanks for Gabe for identifying this bug

* Fix spelling of queryset

Co-authored-by: Rick Elrod <rick@elrod.me>

* Fix sign error with facts expiry - from review

---------

Co-authored-by: Rick Elrod <rick@elrod.me>
2023-03-28 11:20:24 -05:00
Gabe Muniz
7112da9cdc Various validations for const. inv. serialization
- prevent constructed inventory host,group,inventory_source creation
- disable deleting constructed inventory hosts
- remove the ability to add constructed inventory sources
- remove ability to add constructed inventories to constructed inventories
- block updates to constructed source type
- added tests for group/host/source creation
2023-03-28 11:20:24 -05:00
Martin Slemr
8ec6e556a1 HostMetricSummaryMonthly API commented out 2023-03-23 14:13:16 -04:00
Martin Slemr
7285d82f00 HostMetric migration 2023-03-23 14:06:55 -04:00
Alan Rominger
e38f87eb1d Remove custom API filters and suggest solution via templates 2023-03-23 14:06:55 -04:00
Martin Slemr
e6050804f9 HostMetric review,migration,permissions 2023-03-23 14:06:55 -04:00
Martin Slemr
f919178734 HostMetricSummaryMonthly API and Migrations 2023-03-23 14:06:55 -04:00
Martin Slemr
b18ad77035 Host Metrics update/soft delete 2023-03-23 14:06:55 -04:00
Martin Slemr
d80759cd7a HostMetrics migration 2023-03-23 14:06:55 -04:00
Martin Slemr
ef4e77d78f Host Metrics List API 2023-03-23 14:06:55 -04:00
Gabriel Muniz
a63067da38 Add instance groups roles (#13584)
* adding roles to instance groups
added ResourceMixin to Instancegroup and changed the filtered_queryset

* added necessary changes to rebuild relationship between IG and roles

* added description to InstanceGroupAccess

* preliminary ui plug for demo purposes

* preliminary ui plug for demo purposes
added inventory special logic for use_role to allow attaching instance groups
added more tests to handle those cases

* Add access_list to InstanceGroup

* scratch branch to test migration work

* refactored to shorten logic

* Added migration and am removing logic that enabled Org admin permissions

* Add Obj admin role to JT, Inv, Org

* Changed tests to reflect new permissions

* refactored some of the tests

* cleaned up more tests and reworded help on InstanceGroupAccess

* Removed unnecessary delete of Route for instance group perms change

* Fix UI tests and migration

* fixed permissions on prompt for InstanceGroups

* added related object roles endpoint

* added ui/api function for options instance_groups

* separate the migrations in order to avoid issues with migrations not being finished

* changed migrations parent class to disable the activity stream error in migrations

* Added logging to migration as activitystream is disabled

* added clarifying comment to jobtemlateaccess and linted UI addition

* renamed migrations to avoid collisions

* Rename migrations to avoid collisions
2023-03-14 21:37:22 -04:00
Elijah DeLee
3efc7d5bc4 fix access problems (#15)
* fix access problems and add  Add bulk job max settings to api

filter workflow job nodes better

This will both improve performance by limiting the queryset for the node
sublists as well as fix our access problem.

override can_read instead of modify queryset in access.py

We do this because we are not going to expose bulk jobs to the list
views, which is complicatd and has poor performance implications.

Instead, we just care about individual Workflows that clients get linked
to not being broken.

fix comment

remove the get functions from the conf.py for bulk api max value

comment the api expose of the bulk job variables

reformt conf.py with make black

trailing space

add more assertion to the bulk host create test
2023-03-08 12:58:12 -05:00
Elijah DeLee
02e5ba5f94 Move view around and inherit from right view to get OPTIONS
we needed to inherit from GenericAPIView to get the options to render
correctly

q!

add execution env support

add organization validation to the workflowjob

Update awx/api/serializers.py

Co-authored-by: Elijah DeLee <kdelee@redhat.com>

Update awx/api/serializers.py

Co-authored-by: Elijah DeLee <kdelee@redhat.com>
2023-03-08 12:58:12 -05:00
Elijah DeLee
5c47c24e28 Introduce bulk jobs
Provide a view that allows users to launch many jobs with one POST
request. Under the hood, this creates a workflow with a number of jobs
all in a "flat" structure --  much like a sliced job, but with arbitrary
"joblets".

For ~ 100 nodes looking at ~ 200 some queries, which is more than the
proof of concept, but still an order of magnitude better than individual
job launches.

Still more work to implement other m2m objects, and need to address what
Organization should be assigned to a WorkflowJob launched by a BulkJob.

They need this so they can step into the workflow_job_nodes and get the
status of all the containing jobs.

Also want to test when there are MANY job templates etc in the system
because the querires like
UnifiedJobTemplate.accessible_pk_qs(request.user, 'execute_role').all()
queries scare me, seems like it could be a lot of things.

use "many=True" instead of ListField

Still seeing identical number of queries when creatin 100 jobs, going to
investigate more

only validate type in nested serializer

then, we actually get the database object after we do the RBAC checks
This drops us down from hundreds of queries to launch 100 jobs,
to less than 100 queries to launch 100 jobs (I got around 24 queries to
launch 100 jobs with credentials)

pave way for more promptable things

add "limit" as possible prompt on launch to bulk jobs
re-organize how we add credentials to pave way for the other m2m items
not having to repeat too much code

add labels to the bulk job

add the other fields to the workflowjobnode

move urls around
2023-03-08 12:58:12 -05:00
Lila Yasin
39ee4285ce Working on running spellcheck on everything ahead of merging the shellcheck/code check CI addition. (#13453) 2023-03-01 10:19:00 -03:00
Gabe Muniz
fb2647ff7b changing the signature of workflowapprovallist
included workflow approval as a read only endpoint to pass collection tests
2023-02-17 14:57:54 -05:00
Alan Rominger
f5785976be Update to comply with new black rules 2023-02-01 14:59:38 -05:00
Alan Rominger
4a7f4d0ed4 Remove uneditable Instance fields from API browser 2022-12-06 15:20:04 -05:00
Jeff Bradberry
477a63d1b4 Replace the querysets provided by django-qsstats-magic
with ones that are constructed to do the same thing, without any
extraneous functionality.
2022-11-10 11:00:39 -05:00
Rick Elrod
1c65339a24 No InventoryUpdates when source Project is failed (#13063)
Previously, in some cases, an InventoryUpdate sourced by an SCM project
would still run and be successful even after the project it is sourced
from failed to update. This would happen because the InventoryUpdate
would revert the project back to its last working revision. This
behavior is confusing and inconsistent with how we handle jobs (which
just refuse to launch when the project is failed).

This change pulls out the logic that the job launch serializer and
RunJob#pre_run_hook had implemented (independently) to check if the
project is in a failed state, and puts it into a method on the Project
model. This is then checked in the project launch serializer as well as
the inventory update serializer, along with
SourceControlMixin#sync_and_copy as a fallback for things that don't run
the serializer validation (such as scheduled jobs and WFJT jobs).

Signed-off-by: Rick Elrod <rick@elrod.me>
2022-11-03 13:23:34 -05:00
Alex Corey
41fd6ea37f Prevents health checks on all node types except for Execution nodes 2022-10-25 10:11:45 -04:00
Alan Rominger
192f45bbd0 Make canceling view non-atomic to fix 500 errors with job bursts (#13072)
* Make canceling view non-atomic to fix 500 errors with job bursts

* Update test calls for cancel method changes
2022-10-20 15:02:54 -04:00
Alan Rominger
4d9b8400da Merge pull request #12887 from AlanCoding/more_rules
[tech debt] Add new flake8 rules to do some meaningful corrections
2022-10-18 20:27:00 -04:00
Alan Rominger
58772d79c7 Remove unnecessary imports by deleting NOQA markers 2022-10-05 09:09:03 -04:00
Rick Elrod
208254ab81 A few super minor nits in api views/serializers (#12996)
Signed-off-by: Rick Elrod <rick@elrod.me>
2022-10-03 19:24:57 -05:00
Alan Rominger
d3eb2c1975 Add new flak8 rules to do some meaningful corrections 2022-09-27 20:36:42 -04:00
Jeff Bradberry
65179d9cd0 Add a new Instance.health_check_started field
This will enable us to provide more useful information for the user,
now that all user-triggered health checks are async.

Also, de-bounce the health check endpoint to not allow additional
health check tasks to be triggered when one is already in progress.
2022-09-27 17:09:41 -04:00
Hao Liu
795569227a Fix import ordering partially
Signed-off-by: Hao Liu <haoli@redhat.com>
2022-09-23 11:50:09 -04:00
Jeff Bradberry
08c18d71bf Move InstanceLink creation and updating to the async tasks
So that they get applied in situations that do not go through the API.
2022-09-23 09:46:14 -04:00
Seth Foster
eaa4f2483f Run instance health check in task container
awx-web container does not have access to receptor socket, and the
execution node health check requires receptorctl.

This change runs the health check asynchronously in the task container.
2022-09-23 09:46:14 -04:00
Jeff Bradberry
b6946c7e35 Update API to support setting instances to Deprovisioning
- allow the node_state to be set to deprovisioning
- set the links that touch the instance to removing
- only allow on K8S
- only allow to be done to execution nodes
2022-09-23 09:46:13 -04:00
Jeff Bradberry
3bc86ca8cb Follow up on new execution node creation
- hop nodes are descoped
- links need to be created on execution node creation
- expose the 'edit' capabilities on the instance serializer
2022-09-23 09:46:13 -04:00
Alex Corey
7e627e1d1e Adds Instance Peers Tab and update Instance Details view with more data (#12655)
* Adds InstancePeers tab and updates details view

* attempt to fix failing api tests
2022-09-23 09:46:12 -04:00
TheRealHaoLiu
7956fc3c31 add instance install bundle endpoint
add scaffolding for instance install_bundle endpoint

- add instance_install_bundle view (does not do anything yet)
- add `instance_install_bundle` related field to serializer
- add `/install_bundle` to instance URL
- `/install_bundle` only available for execution and hop node
- `/install_bundle` endpoint response contain a downloadable tgz with moc data

TODO: add actual data to the install bundle response

Signed-off-by: Hao Liu <haoli@redhat.com>
2022-09-23 09:46:12 -04:00
Jeff Bradberry
e4518f7b13 Changes in posting constraints due to rescoping to OCP/K8S-only
- node_state is now read only
- node_state gets set automatically to Installed in the create view
- raise a validation error when creating on non-K8S
- allow SystemAdministrator the 'add' permission for Instances
- expose the new listener_port field
2022-09-23 09:46:12 -04:00
Sarabraj Singh
350efc12f5 machinery to allow POSTing payloads to instances/ endpoint 2022-09-23 09:46:12 -04:00
Jeff Bradberry
3bcd539b3d Make sure that the health checks handle the state transitions properly
- nodes with states Provisioning, Provisioning Fail, Deprovisioning,
  and Deprovisioning Fail should bypass health checks and should never
  transition due to the existing machinery
- nodes with states Unavailable and Installed can transition to Ready
  if they check out as healthy
- nodes in the Ready state should transition to Unavailable if they
  fail a check
2022-09-23 09:46:11 -04:00
Alan Rominger
e231e08869 Fix bug with missing parent field and diff with parent
Remove corresponding views for job instance_groups

Validate job_slice_count in API

Remove defaults from some job launch view prompts
  the null default is preferable
2022-09-22 16:08:23 -04:00