32908 Commits

Author SHA1 Message Date
Elijah DeLee
02e5ba5f94 Move view around and inherit from right view to get OPTIONS
we needed to inherit from GenericAPIView to get the options to render
correctly

q!

add execution env support

add organization validation to the workflowjob

Update awx/api/serializers.py

Co-authored-by: Elijah DeLee <kdelee@redhat.com>

Update awx/api/serializers.py

Co-authored-by: Elijah DeLee <kdelee@redhat.com>
2023-03-08 12:58:12 -05:00
Elijah DeLee
81ba6c0234 add migration for is bulk job 2023-03-08 12:58:12 -05:00
Elijah DeLee
5c47c24e28 Introduce bulk jobs
Provide a view that allows users to launch many jobs with one POST
request. Under the hood, this creates a workflow with a number of jobs
all in a "flat" structure --  much like a sliced job, but with arbitrary
"joblets".

For ~ 100 nodes looking at ~ 200 some queries, which is more than the
proof of concept, but still an order of magnitude better than individual
job launches.

Still more work to implement other m2m objects, and need to address what
Organization should be assigned to a WorkflowJob launched by a BulkJob.

They need this so they can step into the workflow_job_nodes and get the
status of all the containing jobs.

Also want to test when there are MANY job templates etc in the system
because the querires like
UnifiedJobTemplate.accessible_pk_qs(request.user, 'execute_role').all()
queries scare me, seems like it could be a lot of things.

use "many=True" instead of ListField

Still seeing identical number of queries when creatin 100 jobs, going to
investigate more

only validate type in nested serializer

then, we actually get the database object after we do the RBAC checks
This drops us down from hundreds of queries to launch 100 jobs,
to less than 100 queries to launch 100 jobs (I got around 24 queries to
launch 100 jobs with credentials)

pave way for more promptable things

add "limit" as possible prompt on launch to bulk jobs
re-organize how we add credentials to pave way for the other m2m items
not having to repeat too much code

add labels to the bulk job

add the other fields to the workflowjobnode

move urls around
2023-03-08 12:58:12 -05:00
Elijah DeLee
752289e175 create new bulk host create endpoint
allow system admins, org admins, and inventory admins to bulk create
hosts.

Testing on an "open" licensed awx as system admin, I created 1000 hosts with 6 queries in ~ 0.15 seconds
Testing on an "open" licensed awx as organization admin, I created 1000 hosts with 11 queries in ~ 0.15 seconds

fix org max host check

also only do permission denied if license is a trial

add /api/v2/bulk to list bulk apis available

add api description templates

One motiviation to not take a list of hosts with mixed inventories is to
keep things simple re: RBAC and keeping a constant number of queries.

If there is great clamor for accepting list of hosts to insert into
arbitrary different inventories, we could probably make it happen - we'd
need to pop the inventory off of each of the hosts, run the
HostSerializer validate, then in top level BulkHostCreateSerializer
fetch all the inventories/check permissions/org host limits for those
inventories/etc. But that makes this that much more complicated.

add test for rbac access

test also helped me find a bug in a query, fixed that

add test to assert num queries scales as expected

also move other test to dedicated file

also test with super user like I meant to

record activity stream for the inventory

this records that a certain number of hosts were added by a certain user
we could consider if there is any other additional information we want
to include
2023-03-08 12:58:12 -05:00
Hao Liu
a24aaba6bc
Merge pull request #13663 from slemrmartin/fix-analytics-collectors
Fix analytics collector
2023-03-08 12:03:16 -05:00
Martin Slemr
349785550c Fix analytics collector 2023-03-08 17:33:23 +01:00
Hao Liu
7cca6c4cd9
Merge pull request #13656 from TheRealHaoLiu/feature-branch-build
Automatically build image for feature branch
2023-03-07 16:53:55 -05:00
Hao Liu
3945db60eb Automatically build image for feature branch
- also will now publish awx image for devel
2023-03-07 16:24:53 -05:00
Hao Liu
252b0dda9f
Merge pull request #13633 from TheRealHaoLiu/phony-dockerfile
[fix] Dockerfile collision between awx-kube-build and docker-compose-build
2023-03-07 15:42:58 -05:00
Martin Slemr
0a2f1622f6
Analytics: instance_info.json v1.3 (#13408) 2023-03-07 14:24:04 -03:00
Jesse Wattenbarger
00817d6b89
Merge pull request #13604 from jjwatt/jjwatt-make-foreach
Change docker-clean build rule in Makefile
2023-03-07 11:54:25 -05:00
Alan Rominger
06808ef4c4
Merge pull request #13608 from AlanCoding/keepalive
Use ansible-runner change to get periodic keep-alive messages in K8S
2023-03-06 14:34:37 -05:00
Hao Liu
f0c967c1b2
Merge pull request #13645 from TheRealHaoLiu/fix-websocket
Revert "Remove trailing $ from websocket_urlpatterns to work with cus…
2023-03-02 21:36:21 -05:00
Hao Liu
2ca0b7bc01 Revert "Remove trailing $ from websocket_urlpatterns to work with custom path to fix #12241"
This reverts commit 5e28f5dca162ec01d6fd32207404f66fa2276604.
2023-03-02 21:14:53 -05:00
Jesse Wattenbarger
217dc57c24 Change docker-clean build rule in Makefile
- Use a make foreach macro and rmi instead of grep and xargs.
2023-03-02 14:56:12 -05:00
Alex Corey
1411d11a0e
Merge pull request #13506 from AlexSCorey/13422-JTTabOnCreds
Conditionally applies the job templates tab to credentials that can be on a JT
2023-03-02 13:15:48 -05:00
Alex Corey
2fe1ea94bd Conditionally applies the job templates tab to credentials that can be on a JT 2023-03-02 12:57:20 -05:00
Hao Liu
a47cfc55ab
Merge pull request #13574 from tomsiewert/use-compose-plugin
Make docker-compose command configurable in Makefile
2023-03-01 15:41:33 -05:00
Hao Liu
0eb9de02f3
Merge pull request #13627 from infamousjoeg/fix-13597-webservice_id-default
Fixes #13597 webservice_id default value added
2023-03-01 15:29:53 -05:00
Lila Yasin
39ee4285ce
Working on running spellcheck on everything ahead of merging the shellcheck/code check CI addition. (#13453) 2023-03-01 10:19:00 -03:00
Christian Adams
2dcda04a9e
Merge pull request #13445 from stanislav-zaprudskiy/disable_instance_command
Add `disable_instance` management command
2023-02-28 15:37:38 -05:00
Christian Adams
52d46c88e4
External users should not be able to change their password (#13491)
* Azure AD users should not be able to change their password

* Multiple auth changes

Moving get_external_user function into awx.sso.common
Altering get_external_user to not look at current config, just user object values
Altering how api/conf.py detects external auth config (and making reusable function in awx.sso.common)
Altering logic in api.serializers in _update_pasword to use awx.sso.common

* Adding unit tests

---------

Co-authored-by: John Westcott IV <john.westcott.iv@redhat.com>
2023-02-28 15:44:34 -03:00
Hao Liu
c2df22e0f0
Merge pull request #13632 from TheRealHaoLiu/reshaving-the-yak
[chore] update project_update playbook to be compliant with ansible-lint
2023-02-28 13:17:45 -05:00
Alan Rominger
90f54b98cd
Update keepalive setting help_text to be more direct
Co-authored-by: Shane McDonald <me@shanemcd.com>
2023-02-28 09:04:07 -05:00
Michael Abashian
b143df3183
Fix broken UI test 2023-02-28 09:04:07 -05:00
Alan Rominger
6fa22f5be2
Add UI for the new setting 2023-02-28 09:04:07 -05:00
Alan Rominger
d5de1f9d11
Make use of new keepalive messages from ansible-runner
Make setting API configurable and process keepalive events
  when seen in the event callback

Use env var in pod spec and make it specific to K8S
2023-02-28 09:04:07 -05:00
Hao Liu
7cca39d069 change make Dockerfile to phony
awx-kube-build and docker-compose-build share the same Dockerfile

if u run awx-kube-build than docker-compose-build in succession the second command wont run the Dockerfile target and cause the image to be built with the incorrect Dockerfile
2023-02-27 20:53:07 -05:00
Hao Liu
cf21eab7f4 [chore] update project_update playbook to be compliant with ansible-lint
reshaving the yak

Co-Authored-By: Gabriel Muniz <gmuniz@redhat.com>
2023-02-27 18:32:10 -05:00
Joe Garcia
98b2f51c18
fix kwargs[] to kwargs.get() 2023-02-27 11:52:44 -05:00
Joe Garcia
327352feaf
Add default value to webservice_id kwarg 2023-02-27 11:26:52 -05:00
Alan Rominger
ccaace8b30
Merge pull request #13541 from npithonDR/devel
Fix error for byweekday in schedule_rruleset
2023-02-27 10:24:48 -05:00
Hao Liu
2902b40084
Merge pull request #13623 from TheRealHaoLiu/revert-project-update-playbook
Revert project_update.yml
2023-02-27 08:47:24 -05:00
Hao Liu
9669b9dd2f Revert project_update.yml
Due to problem found in testing reverting

019e6a52fe
2023-02-27 08:23:27 -05:00
Shane McDonald
d27aada817
Merge pull request #13619 from shanemcd/non-root-path-dev-env
Allow serving app from non-root path in dev env
2023-02-24 09:52:34 -05:00
Shane McDonald
2fca07ee4c Allow serving app from non-root path in dev env
Usage:

$ EXTRA_SOURCES_ANSIBLE_OPTS='-e ingress_path=/awx' make docker-compose
$ curl http://localhost:8013/awx/api/v2/ping/
2023-02-24 09:29:17 -05:00
npithonDR
335ac636b5
Merge pull request #1 from AlanCoding/npithon
Follow comments, split non-list objects
2023-02-24 08:42:00 +01:00
Shane McDonald
f4bcc03ac7
Merge pull request #12242 from adpavlov/12241-websocket-custom-path
Fix websockets when application is served from a non-root path
2023-02-23 12:25:22 -05:00
Alan Rominger
3051384f95
Follow suggestion from comment, split if NOT list 2023-02-23 12:05:32 -05:00
Alan Rominger
811ecb8673
Follow suggestion from comment, split if NOT list 2023-02-23 12:05:21 -05:00
Alexander Pavlov
5e28f5dca1 Remove trailing $ from websocket_urlpatterns to work with custom path to fix #12241
Signed-off-by: Alexander Pavlov <alexander.pavlov@amdocs.com>
2023-02-23 12:02:47 -05:00
Hao Liu
d088d36448
Merge pull request #13618 from TheRealHaoLiu/head-to-tail
[fix] switch from head to tail in project update playbook when clearing project dir
2023-02-23 11:13:03 -05:00
Hao Liu
89e41597a6 switch from head to tail
from @relrod

`head` will close the input fd when it no longer needs it (or exits). find will try to write to the closed fd and somewhere along the way, it will receive SIGPIPE as a result. This is why `yes | head -5 ` doesn't run forever.
2023-02-23 10:46:48 -05:00
Hao Liu
283adc30a8
Merge pull request #13526 from TheRealHaoLiu/project_update_playbook_lint
[chore] Update project_update playbook to be compliant with ansible-lint
2023-02-22 21:39:42 -05:00
Hao Liu
019e6a52fe Update project_update playbook to be compliant with ansible-lint 2023-02-22 19:30:24 -05:00
Hao Liu
35e5610642
Merge pull request #13615 from TheRealHaoLiu/update-kind-devel-doc
update kind development environment instruction
2023-02-22 19:25:03 -05:00
Hao Liu
3a303875bb update kind development environment instruction 2023-02-22 16:18:53 -05:00
Alan Rominger
4499a50019
Merge pull request #13595 from sean-m-sullivan/devel
fix inventory prompt on launch for workflow nodes
2023-02-22 10:23:02 -05:00
Alan Rominger
3fe46e2e27
Merge pull request #13606 from AlanCoding/copy_login
Give proper 401 code to user not logged in
2023-02-21 16:31:23 -05:00
Alan Rominger
6d3f39fe92
Give proper 401 code to user not logged in 2023-02-21 13:34:29 -05:00