866 Commits

Author SHA1 Message Date
Hao Liu
8828ea706e
add make target for building custom awx kube image (#12789) 2022-08-30 20:19:36 +00:00
Shane McDonald
1dfdff4a9e Speed up image build when UI is pre-built on host 2022-08-30 12:36:25 -04:00
Shane McDonald
5ca898541f Fix auto-generated dev env admin password
Fallout from https://github.com/ansible/awx/pull/12753
2022-08-26 13:07:46 -04:00
Shane McDonald
d752e6ce6d Allow for setting custom admin password in dev environment 2022-08-26 11:29:11 -04:00
Christian M. Adams
687b4ac71d
Fix make target for compiling api strings 2022-08-24 16:36:25 -04:00
Hao Liu
7c885dcadb
add help command to make (#12669)
add `make help`
that prints all available make targets
help text generated from comments above the make target starting with `##`

Signed-off-by: Hao Liu <haoli@redhat.com>
2022-08-16 20:36:47 -04:00
Shane McDonald
a0d5f1fb03
Merge pull request #12428 from djyasin/updating_setuppy
Updated setup.py --version to python3 -m setuptools_scm.
2022-06-30 12:17:54 -04:00
Shane McDonald
6a3f9690b0 Remove setup.py entirely 2022-06-27 14:15:32 -04:00
Shane McDonald
cbea36745e Transition from setup.py to setup.cfg 2022-06-27 13:30:01 -04:00
Lila
63ea6bb5b3 Updated setup.py --version to python3 -m setuptools_scm. 2022-06-24 10:22:56 -04:00
Alan Rominger
db105c21e4
Set default false values 2022-06-16 15:46:42 -04:00
Alan Rominger
1dd535a859
Remove old way of doing grafana graphs 2022-06-16 15:31:45 -04:00
Alan Rominger
f4ef7d6927
Add volumes to the clean command 2022-06-16 14:03:22 -04:00
Alan Rominger
fb292d9706
Move visualization containers into docker-compose 2022-06-16 10:25:02 -04:00
Jeff Bradberry
e8b2998578 Narrow down the inotifywait criteria for reloading the dev environment
- listen specifically within awx/awx, so that changes in awxkit or
  awx_collection don't trigger spurious reloads
- expand the exclude pattern to ignore the test directories
2022-06-13 16:08:20 -04:00
John Westcott IV
1d35bba8c3
Variablizing the awx_template_version for building to allow release process to update the version in the module_util (#12248) 2022-06-02 12:28:57 -04:00
Seth Foster
6f68f3cba6
Add make prometheus and make grafana commands to dev environment 2022-05-31 17:07:15 -04:00
John Westcott IV
a86740c3c9
Adding ability to start and plumb splunk instance (#12183) 2022-05-09 09:50:28 -04:00
Shane McDonald
89c2a4c6ed Alternative code reloader for dev env
I verified what Seth found in https://github.com/ansible/awx/pull/12052, but would really hate to lose this functionality. Curious if folks on the API team can try this and see if it works for them.
2022-04-14 09:42:17 -04:00
Satoe Imaishi
229ed53c0e Add setuptools_scm[toml] and pkgconfig, xmlsec's build dep 2022-03-29 01:57:44 -04:00
Shane McDonald
1d181757e8 Migrate all development image builds to GHA
- This also moves the development image from quay.io to ghcr.io.
- These changes will need to be backported to all of our supported branches.
2022-03-23 10:03:55 -04:00
John Westcott IV
4de27117e8
Adding option to enable and configure an OpenLDAP server next to AWX (#11843) 2022-03-10 10:29:04 -05:00
nixocio
f1efc578cb Split UI test run
Split UI test run

See: https://github.com/ansible/awx/issues/10678
2022-03-03 16:22:32 -05:00
Shane McDonald
f6e18bbf06 Publish to galaxy and pypi in promote workflow 2022-03-01 15:42:13 -05:00
nixocio
87b1f0d0de Bump node to LTS version
Bump node to LTS version
2022-02-14 12:41:11 -05:00
Elijah DeLee
604cbc1737
Consume control capacity (#11665)
* Select control node before start task

Consume capacity on control nodes for controlling tasks and consider
remainging capacity on control nodes before selecting them.

This depends on the requirement that control and hybrid nodes should all
be in the instance group named 'controlplane'. Many tests do not satisfy that
requirement. I'll update the tests in another commit.

* update tests to use controlplane

We don't start any tasks if we don't have a controlplane instance group

Due to updates to fixtures, update tests to set node type and capacity
explicitly so they get expected result.

* Fixes for accounting of control capacity consumed

Update method is used to account for currently consumed capacity for
instance groups in the in-memory capacity tracking data structure we initialize in
after_lock_init and then update via calculate_capacity_consumed (both in
task_manager.py)

Also update fit_task_to_instance to consider control impact on instances

Trust that these functions do the right thing looking for a
node with capacity, and cut out redundant check for the whole group's
capacity per Alan's reccomendation.

* Refactor now redundant code

Deal with control type tasks before we loop over the preferred instance
groups, which cuts out the need for some redundant logic.

Also, fix a bug where I was missing assigning the execution node in one case!

* set job explanation on tasks that need capacity

move the job explanation for jobs that need capacity to a function
so we can re-use it in the three places we need it.

* project updates always run on the controlplane

Instance group ordering makes no sense on project updates because they
always need to run on the control plane.

Also, since hybrid nodes should always run the control processes for the
jobs running on them as execution nodes, account for this when looking for a
execution node.

* fix misleading message

the variables and wording were both misleading, fix to be more accurate
description in the two different cases where this log may be emitted.

* use settings correctly

use settings.DEFAULT_CONTROL_PLANE_QUEUE_NAME instead of a hardcoded
name
cache the controlplane_ig object during the after lock init to avoid
an uneccesary query
eliminate mistakenly duplicated AWX_CONTROL_PLANE_TASK_IMPACT and use
only AWX_CONTROL_NODE_TASK_IMPACT

* add test for control capacity consumption

add test to verify that when there are 2 jobs and only capacity for one
that one will move into waiting and the other stays in pending

* add test for hybrid node capacity consumption

assert that the hybrid node is used for both control and execution and
capacity is deducted correctly

* add test for task.capacity_type = control

Test that control type tasks have the right capacity consumed and
get assigned to the right instance group

Also fix lint in the tests

* jobs_running not accurate for control nodes

We can either NOT use "idle instances" for control nodes, or we need
to update the jobs_running property on the Instance model to count
jobs where the node is the controller_node.

I didn't do that because it may be an expensive query, and it would be
hard to make it match with jobs_running on the InstanceGroup which
filters on tasks assigned to the instance group.

This change chooses to stop considering "idle" control nodes an option,
since we can't acurrately identify them.

The way things are without any change, is we are continuing to over consume capacity on control nodes
because this method sees all control nodes as "idle" at the beginning
of the task manager run, and then only counts jobs started in that run
in the in-memory tracking. So jobs which last over a number of task
manager runs build up consuming capacity, which is accurately reported
via Instance.consumed_capacity

* Reduce default task impact for control nodes

This is something we can experiment with as far as what users
want at install time, but start with just 1 for now.

* update capacity docs

Describe usage of the new setting and the concept of control impact.

Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Rebeccah <rhunter@redhat.com>
2022-02-14 10:13:22 -05:00
Shane McDonald
6ed429ada2 Scope api schema.json to target branch 2022-02-07 17:54:01 -05:00
John Westcott IV
c8b906ffb7
Workflow changes (#11692)
Modifying workflows to install python for make commands
Squashing CI tasks to remove repeated steps
Modifying pre-commit.sh to not fail if there are no python file changes
2022-02-07 15:42:35 -05:00
Shane McDonald
44968cc01e
Upgrade to Python 3.9 2022-01-24 12:21:20 -05:00
Jeff Bradberry
db999b82ed
Merge pull request #11431 from jbradberry/receptor-mesh-models
Modify Instance and introduce InstanceLink
2022-01-11 10:55:54 -05:00
John Westcott IV
c92468062d
SAML user attribute flags issue #5303 (PR #11430)
* Adding SAML option in SAML configuration to specify system auditor and system superusers by role or attribute
* Adding keycloak container and documentation on how to start keycloak alongside AWX (including configuration of both)
2022-01-10 16:52:44 -05:00
Jeff Bradberry
386aa898ec Remove the make init target
we want to fold that in to bootstrap_environment.sh.
2022-01-10 11:37:19 -05:00
nixocio
510b40a776 Update how ui tests are invoked on CI
Update how ui tests are invoked on CI as an attempt to speed up test
run.
2021-12-07 09:18:32 -05:00
Alan Rominger
099efb883d
Allow customizing the receptor image in the development environment (#11374)
* Allow for customizing the receptor image

* Hook in receptor image to docker-compose template

* Fix missing -e to pass into Dockerfile playbook

* Add some docs
2021-11-19 14:00:23 -05:00
Shane McDonald
f3669f3be6
Fix make install_collection
The version obtained from setuptools_scm is not compatible with ansible-galaxy collection install.
2021-10-19 10:26:23 -04:00
Shane McDonald
4ab2539c8a
Merge pull request #11242 from shanemcd/awx-operator-ci-check
Add awx-operator CI check
2021-10-13 10:28:23 -04:00
Shane McDonald
f1520e1a70
Allow for building headless mode
This will only be used in CI and maybe other places where we dont need a UI
2021-10-12 18:59:24 -04:00
Alan Rominger
74af187568
Fix Makefile conditional used for docker-refresh (#11238) 2021-10-12 13:52:52 -04:00
Shane McDonald
fcdda8d7a7
Remove old test comparing VERSION files 2021-10-05 19:11:03 -04:00
Shane McDonald
1f0b936e82
Remove VERSION files, obtain version from git tags. 2021-10-05 19:11:00 -04:00
Shane McDonald
d38c109d49
Update pip and setuptools 2021-10-04 13:07:16 -04:00
Shane McDonald
57c9b14198
Fix docker-compose targets 2021-10-03 13:40:26 -04:00
Shane McDonald
43a77e8667
Clean up unused parts of Makefile 2021-10-02 14:22:14 -04:00
Alan Rominger
0ac3a377fd
Make some needed updates to docker-refresh target (#11089) 2021-09-17 09:11:52 -04:00
Alan Rominger
6a17e5b65b
Allow manually running a health check, and make other adjustments to the health check trigger (#11002)
* Full finalize the planned work for health checks of execution nodes

* Implementation of instance health_check endpoint

* Also do version conditional to node_type

* Do not use receptor mesh to check main cluster nodes health

* Fix bugs from testing health check of cluster nodes, add doc

* Add a few fields to health check serializer missed before

* Light refactoring of error field processing

* Fix errors clearing error, write more unit tests

* Update health check info in docs

* Bump migration of health check after rebase

* Mark string for translation

* Add related health_check link for system auditors too

* Handle health_check cluster node timeout, add errors for peer judgement
2021-09-03 16:37:37 -04:00
Alan Rominger
711e5e09ba
Delete images by id instead of tag in docker-refresh (#10957) 2021-08-27 11:51:58 -04:00
Jim Ladd
77933e97c0 create default IG when bringing up dev env 2021-08-26 11:15:14 -07:00
Alan Rominger
ee4792dbf8
Add an option to create a cluster with control-only nodes (#10946) 2021-08-26 13:37:13 -04:00
Alan Rominger
c3ad479fc6
Minor tweaks for the mesh_code branch from review (#10902) 2021-08-24 08:41:35 -04:00
Shane McDonald
460c7c3379
Allow for dynamically scaling automation mesh in dev env 2021-08-24 08:41:32 -04:00