Commit Graph
100 Commits
Author SHA1 Message Date
Alan Rominger e7dbe90cb5 Merge pull request #10727 from ansible/mesh_code
Code changes to support execution-only nodes in receptor mesh
2021-08-24 13:39:19 -04:00
Alan Rominger 42484cf98d Obtain receptor sockfile from the receptor.conf file (#10932) 2021-08-24 11:20:21 -04:00
Alan Rominger 940c189c12 Corresponding AWX changes for runner --worker-info schema update (#10926) 2021-08-24 08:41:36 -04:00
Alan Rominger c3ad479fc6 Minor tweaks for the mesh_code branch from review (#10902) 2021-08-24 08:41:35 -04:00
Alan Rominger 928c35ede5 Model changes for instance last_seen field to replace modified (#10870)
* Model changes for instance last_seen field to replace modified

* Break up refresh_capacity into smaller units

* Rename execution node methods, fix last_seen clustering

* Use update_fields to make it clear save only affects capacity

* Restructing to pass unit tests

* Fix bug where a PATCH did not update capacity value
2021-08-24 08:41:35 -04:00
Alan Rominger 3b1e40d227 Use the ansible-runner worker --worker-info to perform execution node capacity checks (#10825)
* Introduce utilities for --worker-info health check integration

* Handle case where ansible-runner is not installed

* Add ttl parameter for health check

* Reformulate return data structure and add lots of error cases

* Move up the cleanup tasks, close sockets

* Integrate new --worker-info into the execution node capacity check

* Undo the raw value override from the PoC

* Additional refinement to execution node check frequency

* Put in more complete network diagram

* Followup on comment to remove modified from from health check responsibilities
2021-08-24 08:41:35 -04:00
Alan Rominger 4e84c7c4c4 Use the existing get_receptor_ctl method (#10813) 2021-08-24 08:41:35 -04:00
Alan Rominger f47eb126e2 Adopt the node_type field in receptor logic (#10802)
* Adopt the node_type field in receptor logic

* Refactor Instance.objects.register so we do not reset capacity to 0
2021-08-24 08:41:34 -04:00
Alan Rominger 5d4ab13386 Add topology of docker-compose to docs, remove old mount (#10773) 2021-08-24 08:41:34 -04:00
Alan Rominger b53d3bc81d Undo some things not compatible with hybrid node hack (#10763) 2021-08-24 08:41:34 -04:00
Alan Rominger 46ccc58749 Make the AWX nodes fully connected in the development environment (#10758) 2021-08-24 08:41:34 -04:00
Alan Rominger 289beb85d2 Add developer docs for incoming receptor mesh features (#10747)
* Add developer docs for incoming receptor mesh features

* Additional wording about the receptor mesh process

* Wrap up docs feedback changes and polishing

* Add in way more terminology introductions, delete statement about past

* Fix typo around OCP-incluster type
2021-08-24 08:41:34 -04:00
Alan Rominger 9881bb72b8 Treat the awx_1 node as a hybrid node for now, use local work type (#10726) 2021-08-24 08:40:21 -04:00
Alan Rominger f597205fa7 Run capacity checks with container isolation (#10688)
This requires swapping out the container images
  for the execution nodes from awx-ee to the awx image

For completeness, the hop node image is switched to the raw
  receptor image

A few outright bugs are fixed here
  memory calculation just was not right at all
  the execution_capacity calculation was reverse of intention

Drop in a few TODOs about error handling from debugging
2021-08-24 08:40:19 -04:00
Alan Rominger e7be86867d Fix rebase bug specific to ad hoc commands 2021-08-24 08:40:19 -04:00
Alan Rominger 13300bdbd4 Update rebase to keep old control plane capacity check
Also do some basic work to separate control versus execution capacity
  this is to assure that we don't send jobs to the control node
2021-08-24 08:40:19 -04:00
Alan Rominger b09da48835 Remove some diff that we dont want from PoC 2021-08-24 08:40:19 -04:00
Alan Rominger 39e23db523 Make minor changes to add needed imports 2021-08-24 08:40:19 -04:00
Alan Rominger b10a8b0fa9 Initial functionality tweaks 2021-08-24 08:40:18 -04:00
Alan Rominger 397fab793d Avoid waiting for deletion in error cases 2021-07-13 13:49:09 -04:00
Alan Rominger 8772ca2e3a Development of patch for inventory source migration error 2021-07-13 11:47:01 -04:00
Alan Rominger 3e6cbd5114 Wait until inventory is fully deleted in silent_delete 2021-07-13 07:50:58 -04:00
Alan Rominger 122f282e5d Add back in fixture 2021-07-09 13:32:31 -04:00
Alan Rominger 9b319cf2bf Fix pep8 failure 2021-07-09 13:08:57 -04:00
Alan Rominger cb8441ed3d Remove added assertion that was wrong 2021-07-09 12:30:06 -04:00
Alan Rominger 08cb497689 Run flake8 on the AWX collection in CI 2021-07-09 12:01:28 -04:00
Alan Rominger 17f9b57028 Remove unused exception about custom venvs 2021-07-07 11:38:37 -04:00
Alan Rominger e96080a512 No result_traceback is blank, not null 2021-07-07 11:37:30 -04:00
Alan Rominger b329d9cbf4 Make quay.io the registry URL default 2021-06-29 13:26:51 -04:00
Alan Rominger f0cd6b2457 Run flake8 on awxkit 2021-06-29 11:32:59 -04:00
Alan Rominger b8f5dda6da Fix ansible-test pep8 bug and add notable release 2021-06-28 22:37:10 -04:00
Alan Rominger f126a6343b Fix bug setting execution_node to null (not blank) (#5169) 2021-06-28 10:51:06 -04:00
Alan Rominger f1273d5810 Show tracebacks from dependency failures (#5154) 2021-06-28 10:51:05 -04:00
Alan Rominger 390e1f9a0a Fix obvious logical bug with project folder pre-creation (#5155) 2021-06-28 10:51:04 -04:00
Alan Rominger d0b7d970c4 Create partition for 2 job types that bypass TM (#5138)
* Create partition for 2 job types that bypass TM

* Mock create_partition in unit tests
2021-06-28 10:51:03 -04:00
Alan Rominger 02cccbe608 Close file before returning 2021-06-25 14:47:44 -04:00
Alan Rominger 1831b2591a Null iso IG jobs before deleting (#5122)
This is a workaround for polymorphic SET_NULL bug
2021-06-22 10:57:16 -04:00
Alan Rominger 1422bb2043 Fix tower upgrade (#5114)
* Hack to avoid creating both tower and controller CredentialType

* Remove line used for testing
2021-06-22 10:49:41 -04:00
Alan Rominger 4a85983eb7 Symlink modules for 2.9 compat (#5103)
* Symlink modules for 2.9 compat

* Avoid symlinks in completeness tests
2021-06-22 10:49:40 -04:00
Alan Rominger 0508a9267c Minor patch to right wording for 400 error (#5109) 2021-06-22 10:49:39 -04:00
Alan Rominger 53fe08af61 Update collection inventory_source options 2021-06-22 10:49:38 -04:00
Alan Rominger a8083296e6 Make safe variable duplications for rename (#5093)
* Make safe variable duplications to controller

* Back out some changes to be smarter

* Update test to new vars
2021-06-22 10:49:38 -04:00
Alan Rominger cc616206b3 Pass current apps to CredentialType setup method 2021-06-22 10:49:37 -04:00
Alan Rominger 6db4732bf3 Rename the TOWER_ settings 2021-06-22 10:49:36 -04:00
Alan Rominger ead7907173 Update another test for tower to controller rename 2021-06-22 10:49:36 -04:00
Alan Rominger 8c1bc97c2f Fix up unit tests after tower to controller rename 2021-06-22 10:49:36 -04:00
Alan Rominger 5a7a1b8f20 Remove inventory_as_dict which would mess with downstream templating (#5085) 2021-06-22 10:49:34 -04:00
Alan Rominger 15444bef70 Show inventory source POST in OPTIONS for inventory admins (#5076) 2021-06-22 10:49:31 -04:00
Alan Rominger 0b6208047c Correct the function of the -q option for migration helper commands 2021-06-16 16:26:38 -04:00
Alan Rominger f14129de9b Further copy editing of EE docs 2021-06-16 16:18:21 -04:00
Alan Rominger 2309feb6bd Make ExecutionEnvironment awxkit class able to handle null values in payload 2021-06-16 15:41:06 -04:00
Alan Rominger 52d6f36b7c Incorporate review comments about EE docs 2021-06-16 14:24:48 -04:00
Alan Rominger 21aa1fc11f Handle inventory types where Automation Hub collection names differ
Move imports added by Bill to be in-line, because utils should not import models at top

Remove more get_licenser inline imports
2021-06-16 13:39:52 -04:00
Alan Rominger 4478052b71 Update and expand docs/ folder for EEs 2021-06-16 11:23:59 -04:00
Alan Rominger 92c6ddf13c Prefer installer defaults over user-defined global EEs
Add a basic level of testing around get_default_execution_environment
2021-06-14 14:13:39 -04:00
Alan Rominger b39b80b036 Make ExecutionEnvironment awxkit class able to handle null values in payload 2021-06-11 13:32:46 -04:00
Alan Rominger 579d49033a Remove debugging log message 2021-06-08 13:33:54 -04:00
Alan Rominger 210d5084f0 Move skip flag up from event_data and pop it off 2021-06-08 13:33:54 -04:00
Alan Rominger 53e8a9e709 Fix bug 2021-06-08 13:33:53 -04:00
Alan Rominger 15effd7ade Add some conditions for always-send and never-send event types
Always send websocket messages for
  high priority events like playbook_on_stats

Never send websocket messages for
  events with no output
  unless they are a high priority event type
2021-06-08 13:33:53 -04:00
Alan Rominger b919befc90 Add option to record websocket received time 2021-06-08 13:33:53 -04:00
Alan Rominger 4052603238 make sure log format does not error 2021-06-08 13:33:53 -04:00
Alan Rominger b306c6f258 Put new setting in defaults so unit tests will run 2021-06-08 13:33:52 -04:00
Alan Rominger 4b6b8f2bdd Finish up the immediate or average rate method 2021-06-08 13:33:52 -04:00
Alan Rominger 70420dc3e4 THIS DOES NOT WORK pass events if they fit either timing criteria 2021-06-08 13:33:52 -04:00
Alan Rominger 50ca2d47ce Further log adjustments 2021-06-08 13:33:52 -04:00
Alan Rominger faa0a6cf9a fix up log wording 2021-06-08 13:33:52 -04:00
Alan Rominger 01228cea02 Implement max event websocket rate as setting 2021-06-08 13:33:50 -04:00
Alan Rominger cbb461ab71 Fix bug 2021-06-08 13:33:23 -04:00
Alan Rominger b551608f16 Move websocket skip logic into event_handler 2021-06-08 13:33:22 -04:00
Alan Rominger b43d8e2c7f Enforce a 30-per-second max event websocket rate 2021-06-08 13:32:05 -04:00
Alan Rominger 5ab449c90f Add event-specific template to all event types 2021-06-08 11:01:14 -04:00
Alan Rominger d79f73ab7a Remove references to Ansible venv path because it does not exist 2021-06-07 21:14:35 -04:00
Alan Rominger b26eaa3bd2 Remove uses of ansible_virtualenv_path 2021-06-07 21:14:35 -04:00
Alan Rominger dad5533816 Enter flake8 rules in tox config as actual config entries 2021-06-07 16:21:50 -04:00
Alan Rominger 5beb68f527 Remove templates from views that were removed 2021-06-07 11:15:19 -04:00
Alan Rominger 2eb1e4bbe3 Add OPTIONS documentation for new job limit feature 2021-06-07 11:11:22 -04:00
Alan Rominger efcbea1fc5 Skip sliced jobs from the workflow EE logic 2021-06-03 16:20:47 -04:00
Alan Rominger cb1df4a334 Remove code and settings no longer used 2021-05-24 09:44:58 -04:00
Alan Rominger 5dac46cd7d Update with force option if branch_override 2021-05-17 11:57:46 -04:00
Alan Rominger e5f059806b Add instance_group to inventory update serializer 2021-05-13 09:00:43 -04:00
Alan Rominger 6649b435ce Fix flake8 error 2021-05-13 08:57:19 -04:00
Alan Rominger ae320ab228 Do not set openstack env var to blank string 2021-05-13 08:57:19 -04:00
Alan Rominger 29c961e52a Remove comment 2021-05-13 08:57:19 -04:00
Alan Rominger fd466c5cff Finish converting the runner strings 2021-05-13 08:57:19 -04:00
Alan Rominger 623cf0b4cd Start migrating the /runner path references 2021-05-13 08:57:18 -04:00
Alan Rominger d33a748eea Clean up and test patch changing methods 2021-05-13 08:57:18 -04:00
Alan Rominger 1f1cdf8859 start on path helper methods 2021-05-13 08:57:18 -04:00
Alan Rominger 334be9eb25 Use durable switch from container to host path 2021-05-13 08:57:18 -04:00
Alan Rominger 8f9373085a Fix credential env folder, test_tasks.py 2021-05-13 08:57:18 -04:00
Alan Rominger 11c5d577d6 Fix rel path for other inventories 2021-05-13 08:57:17 -04:00
Alan Rominger 0e17023ba3 Inventory directory already pre-created 2021-05-13 08:57:17 -04:00
Alan Rominger 3c785fbff3 update unit tests to new behavior 2021-05-13 08:57:16 -04:00
Alan Rominger 0061c57577 update inventory injector tests 2021-05-13 08:56:38 -04:00
Alan Rominger f59da78328 Use inventory and env private_data_dir subfolders
This avoids writing files to the top level
  of the ansible-runner private_data_dir

Inventory is moved to be in the standard "inventory" folder

Credential related files are moved inside of the "env" folder

Also pre-create these folders when preparing for a job run

With this, args is the only top-level file still remaining
2021-05-13 08:56:38 -04:00
Alan Rominger 164255e516 Remove fields not needed from .only 2021-05-11 16:26:12 -04:00
Alan Rominger 0f5629b514 Add more fields to .only since they get referenced 2021-05-11 16:14:55 -04:00
Alan Rominger 7ec0464072 minikube instructions adjustments from debugging 2021-05-10 11:30:14 -04:00
Alan Rominger ee68dd00aa Remove inventory-related docs which are outdated or wrong 2021-04-28 14:00:05 -04:00
Alan Rominger 67f7998ab9 Modify formatting in response to black update 2021-04-26 10:51:27 -04:00