* colorize uwsgi and celery logs; DEBUG lines are green, WARN lines
are yellow, ERROR lines (and tracebacks) are red
* pretty-print fact callback receiver JSON
* simplify the uwsgi log format so it's more legible
* Tower now injects facts into jobs via memcached for use by Ansible
playbooks. On the Ansible side, this is accomplished by the existing
mechanism, an Ansible Fact Cache Plugin + memcached. On the Tower side,
memcached is leveraged heavily.
* use tower-expect command to determine job status when running
the isolated heartbeat playbook
* grok JSON output of playbook to obtain result information
* run playbook against multiple isolated hosts at the same time
(addresses scalability concerns)
modern Tower deployments (both production and dev container-based)
always build ansible and Tower in separate, distinct virtualenvs, not
the global site-packages
Previously we were running the playbook on a host-by-host
basis, but this changes it to pass in the list of all
isolated isntances the machine is responsible for.
Using the `json` Ansible stdout module, we are able to
parse the output for information on each host.
* During insights project updates, if the etag from insights matches the
last etag (scm_revision) then don't get all the maintenance playbooks.
Otherwise, get all the maintenance playbooks and update scm_revision
with the etag.
* Task Manager logic wasn't assigning default instance group on system
jobs
* Task credential changes assumed the model would have a credential
* Fix up an innocuous error symlinking rdb.py if it already exists
* rampart_groups_setup_playbook:
Updating changelog for Instance Groups
Fix an incorrect reference on instance group jobs list
Purge remaining references to rampart groups
Simplify can_access for instance groups on job templates
Adding Instance Group permissions and tests
Increase test coverage for task scheduler inventory updates
Exit logic fixes for instance group tools
View Fixes for instance groups
new view to allow associations but no creations
Updating acceptance documentation and system docs
Updating unit tests for task manager refactoring
Update views and serializers to support instance group (ramparts)
Implementing models for instance groups, updating task manager
Updating the setup playbook to support instance group installation
Add nginx to server start and switch back to first tmux win
Fix an issue where the local queue wouldn't use the rabbitmq name
* fixed a problem where the SCM last revision was not getting reset
* change the 400 editing error to just reset the last revision
* handle `source_vars` in the same way as custom scripts
* block unallowed `source_vars` in the validator
* hide POST in OPTIONS for inventory source sublist if not allowed
* includes top level views for instances and instance groups and
extending those views to be able to view running jobs
* Associative endpoints on Organizations, Inventories, and Job
Templates
* Related and summary field entries where appropriate
* Adding job model references to executing instance group
* Fix up default queue properties for clustering from the settings file
* Update production and default settings for instance queues in settings
* New InstanceGroup model and associative relationship with Instances
* Associative instances between Organizations, Inventory, and Job
Templates and InstanceGroups
* Migrations for adding fields and tables for Instance Groups
* Adding activity stream reference for instance groups
* Task Manager Refactoring:
** Simplifying task manager relationships and move away from the
interstitial hash tables
** Simplify dependency determination logic
** Reduce task manager runtime complexity by removing the partial
references and moving the logic into the task manager directly or
relying on Job model logic for determinism
For manually initiated inventory updates, also cancel the source
project of "sync" type, like jobs do
For automatic inventory updates spawned from source project update,
of launch type "scm", handle contigency cases
* Dynamic Inventory Source
Template against ansible 2.3 dynamic inventory sources.
The major change is removal of `rax.py`. Most upstream scripts except
`foreman.py` has quite trivial coding style changes, or minor functional
extensions that does not affect Tower inventory update runs.
`foreman.py`, on the other hand, went through quite a major refactoring,
but functionalities stay the same.
Major python dependency updates include apache-libcloud (1.3.0 -->
2.0.0), boto (2.45.0 --> 2.46.1) and shade (1.19.0 --> 1.20.0). Minor
python dependency updates include indirect updates via `pip-compile`,
which are determined by base dependencies.
Some minor `task.py` extensions:
- `.ini` file for ec2 has one more field `stack_filter=False`, which
reveals changes in `ec2.py`.
- `.ini` file for cloudforms will catch these four options from
`source_vars_dict` of inventory update: `'version', 'purge_actions',
'clean_group_keys', 'nest_tags'`. These four options have always been
available in `cloudforms.py` but `cloudforms.ini.example` has not
mentioned them until the latest version. For consistency with upstream
docs, we should make these fields available for tower user to customize.
- YAML file of openstack will catch ansible options `use_hostnames`,
`expand_hostvars` and `fail_on_errors` from `source_vars_dict` of
inventory update as a response to issue #6075.
* Remove Rackspace support
Supports of Rackspace as both a dynamic inventory source and a cloud
credential are fully removed. Data migrations have been added to support
arbitrary credential types feature and delete rackspace inventory
sources.
Note also requirement `jsonschema` has been moved from
`requirements.txt` to `requirements.in` as a primary dependency to
reflect it's usage in `/main/fields.py`.
Connected issue: #6080.
* `pexpect` major update
`pexpect` stands at the very core of our task system and underwent a
major update from 3.1 to 4.2.1. Although verified during devel, please
still be mindful of any suspicious issues on celery side even after this
PR gets merged.
* Miscellaneous
- requests now explicitly declared in `requirements.in` at version 2.11.1
in response to upstream issue
- celery: 3.1.17 -> 3.1.25
- django-extensions: 1.7.4 -> 1.7.8
- django-polymorphic: 0.7.2 -> 1.2
- django-split-settings: 0.2.2 -> 0.2.5
- django-taggit: 0.21.3 -> 0.22.1
- irc: 15.0.4 -> 15.1.1
- pygerduty: 0.35.1 -> 0.35.2
- pyOpenSSL: 16.2.0 -> 17.0.0
- python-saml: 2.2.0 -> 2.2.1
- redbaron: 0.6.2 -> 0.6.3
- slackclient: 1.0.2 -> 1.0.5
- tacacs_plus: 0.1 -> 0.2
- xmltodict: 0.10.2 -> 0.11.0
- pip: 8.1.2 -> 9.0.1
- setuptools: 23.0.0 -> 35.0.2
- (requirements_ansible.in only)kombu: 3.0.35 -> 3.0.37
* Ensure "fresh bits" in dependent jobs
- run IUs in same context and dependent PU
- review docs for update-on-launch expectations
* Avoid customer variable precedence upgrade issues
- only allow SCM types to have overwrite_vars=true
- do not run other cloud sources through the backported script
* Make extra sure to echo stdout and stderr in event of error
on InventorySource model
* scm_project -> source_project
on InventorySourceSerializer
* scm_inventories -> scm_inventory_sources
on InventoryUpdate model
* scm_project_update -> source_project_update
