Commit Graph

543 Commits

Author SHA1 Message Date
Alan Rominger
329caad681 In admin reaper skip work units w/o params 2021-11-10 08:50:14 +08:00
Alan Rominger
e54db3ce50 Gracefully handle receptorctl RuntimeError in health check 2021-11-10 08:50:11 +08:00
Alan Rominger
f34c96ecf5 Error handling when node is missing from mesh for jobs and checks 2021-11-10 08:50:11 +08:00
Alan Rominger
7b35902d33 Respect settings to keep files and work units
Add new logic to cleanup orphaned work units
  from administrative tasks

Remove noisy log which is often irrelevant
  about running-cleanup-on-execution-nodes
  we already have other logs for this
2021-11-10 08:50:10 +08:00
Amol Gautam
f79a57c3e2 Changed Work Submission parameter for K8s work 2021-10-11 08:10:26 -07:00
Alan Rominger
b70793db5c Consolidate cleanup actions under new ansible-runner worker cleanup command (#11160)
* Primary development of integrating runner cleanup command

* Fixup image cleanup signals and their tests

* Use alphabetical sort to solve the cluster coordination problem

* Update test to new pattern

* Clarity edits to interface with ansible-runner cleanup method

* Another change corresponding to ansible-runner CLI updates

* Fix incomplete implementation of receptor remote cleanup

* Share receptor utils code between worker_info and cleanup

* Complete task logging from calling runner cleanup command

* Wrap up unit tests and some contract changes that fall out of those

* Fix bug in CLI construction

* Fix queryset filter bug
2021-10-05 16:32:03 -04:00
Amol Gautam
24a6edef9e AWX dev environment changes for receptor work signing feature
-- Updated devel build to take most recent receptor binary
-- Added signWork parameter when sedning job to receptor
-- Modified docker-compose tasks to generate RSA key pair to use for work-signing
-- Modified docker-compose templates and jinja templates for implementing work-sign
-- Modified Firewall rules on the receptor jinja config

Add firewall rules to dev env
2021-10-05 11:41:34 -07:00
Marcelo Moreira de Mello
2204e03123 Fixed logic to avoid tracebacks when node_name is invalid 2021-09-27 11:28:28 -04:00
Marcelo Moreira de Mello
045785c36f Refactored get_conn_type() method to use Enum 2021-09-23 10:51:50 -04:00
Marcelo Moreira de Mello
45600d034d Initial StreamTLS support for receptor nodes 2021-09-23 10:50:17 -04:00
Alan Rominger
6a17e5b65b Allow manually running a health check, and make other adjustments to the health check trigger (#11002)
* Full finalize the planned work for health checks of execution nodes

* Implementation of instance health_check endpoint

* Also do version conditional to node_type

* Do not use receptor mesh to check main cluster nodes health

* Fix bugs from testing health check of cluster nodes, add doc

* Add a few fields to health check serializer missed before

* Light refactoring of error field processing

* Fix errors clearing error, write more unit tests

* Update health check info in docs

* Bump migration of health check after rebase

* Mark string for translation

* Add related health_check link for system auditors too

* Handle health_check cluster node timeout, add errors for peer judgement
2021-09-03 16:37:37 -04:00
Alan Rominger
424dbe8208 Use ansible-runner imports for cpu and memory calculation (#10954)
* Use ansible-runner imports for cpu and memory calculation

* Fix bug with capacity and memory adjustment
2021-08-27 21:46:53 -04:00
Alan Rominger
daf4310176 Clean up work_type processing and fix execution vs control capacity (#10930)
* Clean up added work_type processing for mesh_code branch

* track both execution and control capacity

* Remove unused execution_capacity property

* Count all forms of capacity to make test pass

* Force jobs to be on execution nodes, updates on control nodes

* Introduce capacity_type property to abstract some details out

* Update test to cover all job types at same time

* Register OpenShift nodes as control types

* Remove unqualified consumed_capacity from task manager and make unit tests work

* Remove unqualified consumed_capacity from task manager and make unit tests work

* Update unit test to execution vs control TM logic changes

* Fix bug, else handling for work_type method
2021-08-26 07:24:14 -04:00
Alan Rominger
42484cf98d Obtain receptor sockfile from the receptor.conf file (#10932) 2021-08-24 11:20:21 -04:00
Alan Rominger
940c189c12 Corresponding AWX changes for runner --worker-info schema update (#10926) 2021-08-24 08:41:36 -04:00
Alan Rominger
928c35ede5 Model changes for instance last_seen field to replace modified (#10870)
* Model changes for instance last_seen field to replace modified

* Break up refresh_capacity into smaller units

* Rename execution node methods, fix last_seen clustering

* Use update_fields to make it clear save only affects capacity

* Restructing to pass unit tests

* Fix bug where a PATCH did not update capacity value
2021-08-24 08:41:35 -04:00
Alan Rominger
3b1e40d227 Use the ansible-runner worker --worker-info to perform execution node capacity checks (#10825)
* Introduce utilities for --worker-info health check integration

* Handle case where ansible-runner is not installed

* Add ttl parameter for health check

* Reformulate return data structure and add lots of error cases

* Move up the cleanup tasks, close sockets

* Integrate new --worker-info into the execution node capacity check

* Undo the raw value override from the PoC

* Additional refinement to execution node check frequency

* Put in more complete network diagram

* Followup on comment to remove modified from from health check responsibilities
2021-08-24 08:41:35 -04:00
Alan Rominger
13300bdbd4 Update rebase to keep old control plane capacity check
Also do some basic work to separate control versus execution capacity
  this is to assure that we don't send jobs to the control node
2021-08-24 08:40:19 -04:00
Ryan Petrello
05cb876df5 implement an initial development environment for receptor-based clusters 2021-08-24 08:40:18 -04:00
Jeff Bradberry
b0c511a7a2 Make non-polymorphic refs that slip through still work with our SET_NULL 2021-07-13 13:41:44 -04:00
Chris Meyers
84116349ab reload settings on fork or new thread
* Reloading settings will re-import cache. Django cache is thread aware,
but keeping an old reference to the cache avoids the awareness.
2021-06-28 13:53:11 -04:00
Christian M. Adams
06b04007a0 Rename managed_by_tower to managed 2021-06-22 10:49:36 -04:00
Alan Rominger
21aa1fc11f Handle inventory types where Automation Hub collection names differ
Move imports added by Bill to be in-line, because utils should not import models at top

Remove more get_licenser inline imports
2021-06-16 13:39:52 -04:00
Alan Rominger
92c6ddf13c Prefer installer defaults over user-defined global EEs
Add a basic level of testing around get_default_execution_environment
2021-06-14 14:13:39 -04:00
Bill Nottingham
f460f70513 Remove host insights view 2021-06-09 16:34:32 -04:00
Jeff Bradberry
8ede74a7f6 Deal with the possibility of get_default_pod_spec not finding an EE 2021-06-09 13:48:23 -04:00
Jeff Bradberry
9aa56b1247 Update the EE resolver logic
so that the control plane managed EE is kept separate.
2021-06-09 10:03:35 -04:00
Bill Nottingham
ca07946c24 Adjust subscription calculation & warning behavior
- Add a field for hosts automated across

  This is populated by the new table we've added.
- Update the subs check to check against this, not imported hosts.
- Reword messages on inventory import
2021-06-07 15:47:55 -04:00
softwarefactory-project-zuul[bot]
b64f966db1 Merge pull request #10090 from rebeccahhh/custom_venv_command
add a new awx-manage command `custom_venvs`

add an awx-manage command that gets pip freeze data from custom_venv and outputs to command line stdout
SUMMARY

part of #7062  - this command is a glorified pip freeze + some extra stuff, people could navigate to each of their custom virtual environments themselves and run a pip freeze, but this allows them to not, and everyone likes their life to be easier. The extra stuff allows users to see the connections that their existing virtual envs have in awx to things like organizations, jobs, inventory updates, and projects.

ISSUE TYPE


Feature Pull Request

COMPONENT NAME


API

AWX VERSION

awx: 19.1.0

ADDITIONAL INFORMATION

This is built off of existing code and there is a line that gets custom venv paths from the settings module, that line does not seem to be working. I have written around that but want to make a note of it.

Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
2021-06-04 18:44:52 +00:00
Chris Meyers
1a1d66d2a2 bump db partition migration 2021-06-04 09:17:09 -07:00
Chris Meyers
30871bd6cf close db and cache connection in new threads 2021-06-04 09:17:09 -07:00
Jim Ladd
2a23b4c719 bump migration 2021-06-04 09:17:09 -07:00
Jim Ladd
ef9f9129ba bump migration 2021-06-04 09:17:09 -07:00
Chris Meyers
6ce227a6b6 bump migrations 2021-06-04 09:17:09 -07:00
Chris Meyers
137111351c bump migrations after devel rebase 2021-06-04 09:17:09 -07:00
Jim Ladd
0f9f3f58e2 bump migration 2021-06-04 09:17:08 -07:00
Jim Ladd
fb97687d14 lint 2021-06-04 09:17:08 -07:00
Jim Ladd
0f53d9b911 bump db partition migration 2021-06-04 09:17:08 -07:00
Jim Ladd
6b4effc85a bump partition migration to 135 2021-06-04 09:17:08 -07:00
Jim Ladd
a7cabec3d0 bump partition migration to 0132 2021-06-04 09:17:07 -07:00
Jim Ladd
8501a45531 lint fixes 2021-06-04 09:17:07 -07:00
Jim Ladd
14b610dabf bump partition migration 2021-06-04 09:17:07 -07:00
Jim Ladd
373edbf8c0 Update reference to partition migration 2021-06-04 09:17:07 -07:00
Ryan Petrello
08b96a0bd7 correct filter events on the migration event horizon
events that existed *prior* to the partition migration will have
`job_created=1970-01-01` auto-applied at migration time; as such,
queries for these events e.g., /api/v2/job/N/job_events/
use 1970-01-01 in related event searche

events created *after* the partition migration (net-new playbook runs
will have `job_created` values that *exactly match* the related
`UnifiedJob.created` field.
2021-06-04 09:17:07 -07:00
Jim Ladd
ec484f81cf Partition *all* job event tables 2021-06-04 09:17:06 -07:00
Jim Ladd
82df3ebddb add option to create partitions that span a single minute
* turned on by default currently for testing
2021-06-04 09:17:06 -07:00
Jim Ladd
612e91263c auto-create partition 2021-06-04 09:17:06 -07:00
Jim Ladd
445042c0f4 Create partition only if it doesn't exist 2021-06-04 09:17:06 -07:00
Jim Ladd
0c289205de Give new primary key constraint unique name, create first live partition 2021-06-04 09:17:06 -07:00
Jim Ladd
ba45592d93 create helper method to create partitions
* create_partition()
2021-06-04 09:17:06 -07:00