Commit Graph

413 Commits

Author SHA1 Message Date
Chris Meyers
5d205f1e1b remove deprecated mongo fact code and deps
* We have the requirements that users with Tower < 3.0.x must upgrade to
3.0.x before > 3.0.x. Thus, we are save to delete all mongo migration
code and deps.
2016-11-09 16:25:27 -05:00
Chris Meyers
b66256eb98 fixes test_settings unit tests 2016-11-08 11:10:52 -05:00
Matthew Jones
88a8810815 Merge pull request #3729 from sundeep-co-in/STAGE
enable django i18n
2016-11-04 15:52:33 -04:00
Matthew Jones
06dd5bb5d2 Fix an invalid database backend in unit test settings 2016-11-03 16:09:17 -04:00
AlanCoding
83e31b8618 remove timeout settings example, attach minimum timeout values 2016-11-02 14:59:51 -04:00
AlanCoding
de66318381 move timeout settings into 3 different settings vars 2016-11-02 14:59:51 -04:00
Chris Meyers
5c71f1d6f7 allow all hosts during development 2016-11-02 10:45:42 -05:00
Chris Meyers
250a71907d update for commit hook 2016-11-02 08:25:51 -05:00
sundeep-co-in
688b5fef68 resolve conflict: settings/defaults.py 2016-11-02 15:19:57 +05:30
Chris Meyers
13c89ab78c HAify job schedules and more task_manager renaming 2016-11-01 13:50:42 -05:00
Chris Meyers
555f0bb90f project and jobs running correctly 2016-11-01 13:50:42 -05:00
Matthew Jones
c6cf02a602 Merge branch 'release_3.0.3' into devel
* release_3.0.3: (55 commits)
  Revert "Revert "Add needed types for selinux change""
  Revert "Add needed types for selinux change"
  interpret backslash escapes when displaying url in welcome message
  Bump the SELinux policy version
  Add needed types for selinux change
  Update SELinux policy to allow httpd_t to execute files in lib_t and var_lib_t
  Bumping changelog for 3.0.3
  Update rax.py inventory
  Revert "filter internal User.admin_roles from the /roles API list view"
  fix spelling of disassociated
  Resolves 404 when assigning resources/users to organizations in card view. Sidesteps a bug in the Refresh() utility, where pagination calculations are not made against filtered results.
  Sync azure changes to Tower virtual environment
  Add regions here as well.
  Also bump boto for new regions, per ryansb.
  More regions!
  Revert "bump shade version"
  bump shade version
  Hack copying of job_template.related.survey_spec into ui job copy flow, resolves #3737
  Revert "bump shade version"
  bump shade version
  ...
2016-11-01 11:49:28 -04:00
sundeep-co-in
4237b2e207 resolve conflict: index.html 2016-10-31 23:40:56 +05:30
Bill Nottingham
83cd055f10 Merge pull request #3569 from wenottingham/forever_blowing_bubbles
Use bubblewrap (https://github.com/projectatomic/bubblewrap) instead of proot.
2016-10-28 11:55:14 -04:00
sundeep-co-in
d65e121b19 flake8 compliance 2016-10-28 14:07:08 +05:30
AlanCoding
1fec29dcb3 remove awx.compat, since it looks like it is not necessary 2016-10-26 17:22:03 -04:00
sundeep-co-in
576db5120f resolve conflict 2016-10-25 16:18:14 +05:30
Matthew Jones
73b5b47e94 flake8 cleanup 2016-10-20 15:27:59 -04:00
Matthew Jones
6e22460f1e Add scm revision to the job model
This also cleans up flake8 issues
2016-10-20 15:18:05 -04:00
Matthew Jones
e6bcc039f2 Rearchitect project update strategy
* Instead of using a fanout for project updates initial project
  updates will sync the latest commit hash
* Prior to a node running a job it will ensure that the latest project
  is synced
2016-10-20 13:30:44 -04:00
Bill Nottingham
5c69ed2e9f More regions! 2016-10-19 16:15:09 -04:00
sundeep-co-in
78a8ce9479 django i18n 2016-10-18 18:10:39 +05:30
Aaron Tan
21feae7c11 Rename default timeout and remove runtime_flag. 2016-10-17 16:56:39 -04:00
Aaron Tan
96e3228c9d Change global settings structure and fix potential celery issue. 2016-10-17 16:56:39 -04:00
AlanCoding
8d89d68d5e flake8 fixes for Jenkins 2016-10-14 09:46:41 -04:00
Chris Church
8883738a7c Fix issue when string list settings field is null. Resolves #3683. 2016-10-12 14:47:22 -04:00
James Laska
77a56b3dfd Properly recognize foreman.enabled
The ENABLED_VALUE must always be a string.  Thanks to @cchurch for the
suggestion.

Resolves #3632
2016-10-07 09:50:18 -04:00
Wayne Witzel III
5d8e7c17c2 Merge branch 'devel' into jtabor-sockets 2016-10-07 09:50:03 -04:00
James Laska
5881736769 Properly detect enabled foreman inventory
Related #3632
2016-10-07 09:49:31 -04:00
Matthew Jones
babe29ebfa Implement cluster health checks
* Add a local node queue to execute targeted jobs
* Add a setting for active cluster node id (per-node)
* Base the heartbeat time on the `modified` time on the Instance table
* Add periodic task that calls save() on the instance to update the
  heartbeat time if services are up
* Purge/update any ha/instance management commands
* Fix up CELERY_ROUTES settings data structure
2016-10-06 16:05:39 -04:00
Wayne Witzel III
1a9843d5d1 adding initial channels integration 2016-10-04 14:04:52 -07:00
Wayne Witzel III
530a5c0c88 merging devel 2016-10-03 10:35:50 -04:00
Chris Meyers
203df91a5d more robust test mode checking 2016-10-03 09:28:44 -04:00
Bill Nottingham
8265934c2d Use bubblewrap (https://github.com/projectatomic/bubblewrap) instead of proot. 2016-09-29 16:23:30 -04:00
Chris Meyers
dd3c3c9f29 Merge branch 'feature-ha_task_manager' into devel 2016-09-29 10:27:17 -04:00
Wayne Witzel III
258ddba333 add channels to reqs and defaults 2016-09-27 18:39:19 -04:00
Chris Meyers
d65120538d scheduler messages need not be durable 2016-09-27 16:05:30 -04:00
Chris Meyers
04f69727f2 fully message driven job execution
TODO:
* Need a distributed lock (leverage postgres)
* Less memory-intensive graph representation
* Maybe serializer/deserializer graph to database
  * Iterative graph building instead of full rebuild.
2016-09-27 14:16:18 -04:00
Chris Meyers
cdb65ccac9 replace task manager with event driven scheduler 2016-09-27 14:16:18 -04:00
Chris Meyers
cc90204b0f task manager using messages
* First pass, adapt singleton task manager to process messages and run
jobs based on events instead of a busy loop.
* Still need to make message handing run in celery, not in a consumption
loop
2016-09-27 14:05:29 -04:00
Chris Church
6ebe45b1bd Configure Tower in Tower:
* Add separate Django app for configuration: awx.conf.
* Migrate from existing main.TowerSettings model to conf.Setting.
* Add settings wrapper to allow get/set/del via django.conf.settings.
* Update existing references to tower_settings to use django.conf.settings.
* Add a settings registry to allow for each Django app to register configurable settings.
* Support setting validation and conversion using Django REST Framework fields.
* Add /api/v1/settings/ to display a list of setting categories.
* Add /api/v1/settings/<slug>/ to display all settings in a category as a single object.
* Allow PUT/PATCH to update setting singleton, DELETE to reset to defaults.
* Add "all" category to display all settings across categories.
* Add "changed" category to display only settings configured in the database.
* Support per-user settings via "user" category (/api/v1/settings/user/).
* Support defaults for user settings via "user-defaults" category (/api/v1/settings/user-defaults/).
* Update serializer metadata to support category, category_slug and placeholder on OPTIONS responses.
* Update serializer metadata to handle child fields of a list/dict.
* Hide raw data form in browsable API for OPTIONS and DELETE.
* Combine existing licensing code into single "TaskEnhancer" class.
* Move license helper functions from awx.api.license into awx.conf.license.
* Update /api/v1/config/ to read/verify/update license using TaskEnhancer and settings wrapper.
* Add support for caching settings accessed via settings wrapper.
* Invalidate cached settings when Setting model changes or is deleted.
* Preload all database settings into cache on first access via settings wrapper.
* Add support for read-only settings than can update their value depending on other settings.
* Use setting_changed signal whenever a setting changes.
* Register configurable authentication, jobs, system and ui settings.
* Register configurable LDAP, RADIUS and social auth settings.
* Add custom fields and validators for URL, LDAP, RADIUS and social auth settings.
* Rewrite existing validator for Credential ssh_private_key to support validating private keys, certs or combinations of both.
* Get all unit/functional tests working with above changes.
* Add "migrate_to_database_settings" command to determine settings to be migrated into the database and comment them out when set in Python settings files.
* Add support for migrating license key from file to database.
* Remove database-configuable settings from local_settings.py example files.
* Update setup role to no longer install files for database-configurable settings.

f 94ff6ee More settings work.
f af4c4e0 Even more db settings stuff.
f 96ea9c0 More settings, attempt at singleton serializer for settings.
f 937c760 More work on singleton/category views in API, add code to comment out settings in Python files, work on command to migrate settings to database.
f 425b0d3 Minor fixes for sprint demo.
f ea402a4 Add support for read-only settings, cleanup license engine, get license support working with DB settings.
f ec289e4 Rename migration, minor fixmes, update setup role.
f 603640b Rewrite key/cert validator, finish adding social auth fields, hook up signals for setting_changed, use None to imply a setting is not set.
f 67d1b5a Get functional/unit tests passing.
f 2919b62 Flake8 fixes.
f e62f421 Add redbaron to requirements, get file to database migration working (except for license).
f c564508 Add support for migrating license file.
f 982f767 Add support for regex in social map fields.
2016-09-26 22:14:47 -04:00
Wayne Witzel III
1268c8e205 update authentication class in postprocess and add unit test 2016-09-23 08:22:26 -04:00
Matthew Jones
f3a8eb9daf Merge pull request #3509 from ansible/ha_installer
Improvements to the setup/installer to support new HA workflows
2016-09-16 15:53:38 -04:00
Matthew Jones
d8ea0920c5 Update configuration and playbook for rabbitmq
* Purge remaining zeromq settings
* Make sure we have CALLBACK_QUEUE defined where needed
* Update setup playbook to not be tied to old ha primary/secondary concept
2016-09-16 11:35:46 -04:00
Matthew Jones
3de4aae548 Fixing up HA induced flake8 issues 2016-09-15 13:51:17 -04:00
Chris Meyers
bf3c68d8b4 Merge pull request #3497 from chrismeyersfsu/fix-flake8_deps
Fix flake8 deps
2016-09-15 13:33:41 -04:00
Chris Meyers
f06b212266 flake8 fixes 2016-09-15 10:25:36 -04:00
Matthew Jones
f5d2c5c18a Merge branch 'ramparts_and_ha' into devel
* ramparts_and_ha: (21 commits)
  Rename database migrations for devel integration
  Integrate callback receiver refactoring
  Fix an issue running jobs in the cluster
  Implement a more dynamic celery queue system
  Purge old munin monitors and tools
  Refactor Tower HA Instance logic and models
  Docker compose improvements
  Initial Docker Compose workflow for Tower cluster
  Add memcached role for setup playbook
  Removing qpid from deb packaging
  Refactor rabbitmq role
  Integrate memcached into setup playbook
  Remove mongodb shutdown task
  Remove dependency on erlang_sd_notify
  Add initial rabbitmq role
  Initial rabbitmq setup playbook integration
  Update development environment for rabbit
  Replace qpid with rabbitmq
  Remove redis role from setup playbook
  Update qpid packaging, remove migrations
  ...
2016-09-15 10:19:57 -04:00
Matthew Jones
56d4210047 Implement Performance improvements shown in #3492
* Make callback workers tunable
* Disable callback worker recycling - doesn't appear to be needed
  anymore
* Tweak pexpect behavior by limiting its read buffer size and search
  window
* Use copy instead of deepcopy for job event callback emitter censor
2016-09-14 16:08:47 -04:00
Matthew Jones
ab395b0009 Integrate callback receiver refactoring
* Drop ZMQ as the communication mechanism between job_event_callback and
  callback_receiver
* Setup queue and exchange for callback broker communication
* Refactor event plugin and callback receiver to efficiently handle
  message submission and processing
* Integrate django caching for parent processing
2016-09-14 11:42:13 -04:00