Commit Graph
100 Commits
Author SHA1 Message Date
Matthew Jones 91cbaa1096 Install supervisor into the dev environment 2019-03-21 15:58:24 -04:00
Matthew Jones 5cf7cc21c8 Revert "Fix chrome can not be started with unit-tests due to missing shared libraries"
This reverts commit d558ffd699.
2019-03-07 10:07:08 -05:00
Matthew Jones 499fd7b2f1 Revert "Fix chrome can not be started with unit-tests due to missing shared libraries"
This reverts commit 3e5f328b52.
2019-03-07 10:06:30 -05:00
Matthew Jones 058e2c0d81 Make schema generator handle alternative iso 8601 datetimes 2019-01-22 11:44:56 -05:00
Matthew Jones 7330102961 Remove a warning message for dispatcher pool for tests 2018-11-19 11:19:57 -05:00
Matthew Jones 61916b86b5 Fix a bug that did not allow project_admin's to create a project.
This was a regression from previous functionality
2018-11-19 11:05:48 -05:00
Matthew Jones f737fc066f Generate schema suitable for comparing for schema changes 2018-11-15 15:29:35 -05:00
Matthew Jones 80ac44565a Make sure we reference the actual hostname 2018-11-02 10:37:58 -04:00
Matthew Jones 46d6dce738 Mass rename of shard -> split 2018-10-31 11:04:12 -04:00
Matthew Jones 61a706274b Adding architecture doc for job sharding 2018-10-31 11:04:10 -04:00
Matthew Jones 44ffcf86de Properly take prompted inventory into account
This also will rename shard jobs to add an index to the job name
2018-10-31 11:04:10 -04:00
Matthew Jones 673068464a Add support for runner_on_start
This will be available in ansible 2.8
2018-10-30 13:23:55 -04:00
Matthew Jones 0477581dea Fix up flake8 2018-10-16 11:30:07 -04:00
Matthew Jones d5c557c639 Proper parameterization for scm tests 2018-10-16 11:30:06 -04:00
Matthew Jones 8e60cb1270 Purge an unneeded ansible 2.4 version check 2018-10-16 11:30:05 -04:00
Matthew Jones 119b9475ea Force openshift user behavior for uids over 2500 2018-10-16 11:30:04 -04:00
Matthew Jones e0c7a7bece Mention zuul in contributing 2018-10-05 14:32:47 -04:00
Matthew Jones 640f9474fc Remove shippable configuration 2018-10-05 14:26:03 -04:00
Matthew Jones 3a8bacb8ef Add an initial check and gate job configuration for zuul
Updates for running ui tests and linters
2018-10-05 13:39:59 -04:00
Matthew Jones 22dd6ddfea Remove the logstash container from the base dev docker compose
Now with less java running while you code!
2018-06-05 09:42:38 -04:00
Matthew Jones 72f2994cc5 Merge branch 'release_3.3.0' into devel 2018-05-17 16:07:47 -04:00
Matthew Jones 2f3d7b17f6 Fix an issue where missing instance group would cause an error
We'll now default to queue submission to the basic management queue
2018-04-26 09:11:08 -04:00
Matthew Jones 4af8a53232 Remove Instance Group concept/usage from WorkflowJobs
This also relaxes some of the task manager rules on Instance Groups
down the full stack such that workflow jobs tend to shortcut the
processing or omit it altogether.

This lets the workflow job spawning logic exist outside of the
instance group queues, which it doesn't need to participate in in the
first place.
2018-04-25 08:29:49 -04:00
Matthew Jones 05419d010b Update group cluster policies on save, not just created
Currently updating policy settings doesn't trigger a re-evaluation of
instance group policies, this makes sure we re-evaluate in the event
that anything changes.
2018-04-24 21:40:11 -04:00
Matthew Jones 192dc82458 Update documentation for default pod resource requests
Including information on how to override the default resources
2018-03-15 12:01:02 -04:00
Matthew Jones 3ba7095ba4 Fixing some issues defining resource requests in openshift and k8s
* Allow overriding all container resource requests by setting defaults/
* Fix an issue where template vars were reversed in the deployment config
* Remove `limit` usage to allow for resource ballooning if it's available
* Fix type error when using templated values in the config map for resources
2018-03-15 12:00:53 -04:00
Matthew Jones b0cf4de072 Implement container-cluster aware capacity determination
* Added two settings values for declaring absolute cpu and memory
  capacity that will be picked up by the capacity utility methods
* installer inventory variables for controlling the amount of cpu and
  memory container requests/limits for the awx task containers
* Added fixed values for cpu and memory container requests for other
  containers
* configmap uses the declared inventory variables to define the
  capacity inputs that will be used by AWX to correspond to the same
  inputs for requests/limits on the deployment.
2018-03-14 14:35:45 -04:00
Matthew Jones acde2520d0 Sort cloud regions in a stable way
* All comes first
* Then US regions
* Then all other regions alphabetically
2018-03-13 15:31:28 -04:00
Matthew Jones dcf0b49840 Adding information on Kubernetes RBAC considerations for Helm 2018-03-13 13:48:10 -04:00
Matthew Jones 45ad94f057 Adding more helpful job extra vars
* Adds email, first name, last name as extra vars to job launches
* Remove old ad-hoc command extra vars population... use our
  base-class method instead
2018-03-13 13:33:54 -04:00
Matthew Jones b11b1acc68 Update middleware warning for latest minor version 2018-03-08 12:54:26 -05:00
Matthew Jones d19ef60d97 Remove old south migrations from before a previous django upgrade 2018-03-06 14:47:09 -05:00
Matthew Jones e99184656e Apply rabbitmq and setting kubernetes changes post-celery rollback 2018-03-05 16:22:27 -05:00
Matthew Jones 105b82c436 Apply celery rollback changes to kubernetes configmap 2018-03-05 15:32:24 -05:00
Matthew Jones 8505783350 Merge remote-tracking branch 'tower/release_3.2.3' into devel
* tower/release_3.2.3:
  fix unicode bugs with log statements
  use --export option for ansible-inventory
  add support for new "BECOME" prompt in Ansible 2.5+ for adhoc commands
  enforce strings for secret password inputs on Credentials
  fix a bug for "users should be able to change type of unused credential"
  fix xss vulnerabilities - on host recent jobs popover - on schedule name tooltip
  fix a bug when testing UDP-based logging configuration
  bump templates form credential_types page limit
  Wait for Slack RTM API websocket connection to be established
  don't process artifacts from custom `set_stat` calls asynchronously
  don't overwrite env['ANSIBLE_LIBRARY'] when fact caching is enabled
  only allow facts to cache in the proper file system location
  replace our memcached-based fact cache implementation with local files
  add support for new "BECOME" prompt in Ansible 2.5+
  fix a bug in inventory generation for isolated nodes
  properly handle unicode for isolated job buffers
2018-02-20 12:22:25 -05:00
Matthew Jones ffe5a92eb9 Update isolated instance capacity calculaltion 2018-02-13 21:51:50 -05:00
Matthew Jones 925d9efecf Fixing up isolated node execution after cluster changes
* Rework queue detection to include control groups and isolated instances
* Fix up development tooling around isolated nodes
* Update unit tests
2018-02-13 21:51:38 -05:00
Matthew Jones 4002f2071d Adding instance group policy unit tests
also remove async call for applying topology change
2018-02-07 11:14:53 -05:00
Matthew Jones c61efc0af8 Add information on enabled flag 2018-02-05 15:44:26 -05:00
Matthew Jones 8e94a9e599 Adding capacity docs
Updating capacity for callback jobs to include parent process impact
2018-02-05 09:49:01 -05:00
Matthew Jones 70bf78e29f Apply capacity algorithm changes
* This also adds fields to the instance view for tracking cpu and
  memory usage as well as information on what the capacity ranges are
* Also adds a flag for enabling/disabling instances which removes them
  from all queues and has them stop processing new work
* The capacity is now based almost exclusively on some value relative
  to forks
* capacity_adjustment allows you to commit an instance to a certain
  amount of forks, cpu focused or memory focused
* Each job run adds a single fork overhead (that's the reasoning
  behind the +1)
2018-02-01 16:57:09 -05:00
Matthew Jones 6a85fc38dd Add scalable cluster kubernetes support 2018-02-01 16:57:09 -05:00
Matthew Jones 6e9930a45f Use on_commit hook for triggering ig policy
* also Apply console handlers to loggers for dev environment
2018-02-01 16:56:43 -05:00
Matthew Jones d9e774c4b6 Updates for automatic triggering of policies
* Switch policy router queue to not be "tower" so that we don't
  fall into a chicken/egg scenario
* Show fixed policy list in serializer so a user can determine if
  an instance is manually managed
* Change IG membership mixin to not directly handle applying topology
  changes. Instead it just makes sure the policy instance list is
  accurate
* Add create/delete hooks for instances and groups to trigger policy
  re-evaluation
* Update policy algorithm for fairer distribution
* Fix an issue where CELERY_ROUTES wasn't renamed after celery/django
  upgrade
* Update unit tests to be more explicit
* Update count calculations used by algorithm to only consider
  non-manual instances
* Adding unit tests and fixture
* Don't propagate logging messages from awx.main.tasks and
  awx.main.scheduler
* Use advisory lock to prevent policy eval conflicts
* Allow updating instance groups from view
2018-02-01 16:56:16 -05:00
Matthew Jones 56abfa732e Adding initial instance group policies
and policy evaluation planner
2018-02-01 16:56:15 -05:00
Matthew Jones c819560d39 Add automatic deprovisioning support, only enabled for openshift
* Implement a config watcher for service restarts
* If the configmap bind point changes then restart all services
2018-02-01 16:51:40 -05:00
Matthew Jones 624289bed7 Add support for directly managing instance groups
* Associating/Disassociating an instance with a group
* Triggering a topology rebuild on that change
* Force rabbitmq cleanup of offline nodes
* Automatically check for dependent service startup
* Fetch and set hostname for celery so it doesn't clobber other
  celeries
* Rely on celery init signals to dyanmically set listen queues
* Removing old total_capacity instance manager property
2018-02-01 16:46:44 -05:00
Matthew Jones 6ede1dfbea Update openshift installer to support rabbitmq autoscale
* Switch rabbitmq container out for one that supports autoscale
* Add etcd pod to support autoscale negotiation
2018-02-01 16:38:10 -05:00
Matthew Jones 7dc0fce1aa Use x-forwarded-for by default in openshift and kubernetes 2018-01-11 12:00:01 -05:00
Matthew Jones 7ea6d7bf4d Clean up documentation for kubernetes installer 2018-01-10 09:39:07 -05:00
Matthew Jones fad4a549d0 Remove oc command usage from docker registry k8s reference 2018-01-10 09:38:00 -05:00
Matthew Jones d0b3cac72a Remove nodeport definition 2018-01-10 09:29:12 -05:00
Matthew Jones 44f0b003fc Kubernetes install documentation 2018-01-10 09:25:59 -05:00
Matthew Jones 56aed597b2 Add initial support for kubernetes to the installer 2018-01-10 09:25:59 -05:00
Matthew Jones f33ee03b98 Remove nodeport customization
This isn't strictly necessary for the Openshift routes and can
sometimes cause problems when the resource is already defined in openshift
2018-01-10 09:23:46 -05:00
Matthew Jones 478bcc0b07 Removing old unused tests 2017-12-14 11:34:43 -05:00
Matthew Jones 2fb0144914 Add libcurl-devel to official image build 2017-12-13 16:14:55 -05:00
Matthew Jones 9dbcc5934e Merge remote-tracking branch 'tower/release_3.2.2' into devel 2017-12-13 12:25:47 -05:00
Matthew Jones 282290e151 Fix an issue referencing postgres port from openshift deployment 2017-12-12 10:52:02 -05:00
Matthew Jones 34c206fab0 Bump psql-container pg version to 9.6 2017-12-11 12:01:28 -05:00
Matthew Jones da0b686369 Adding jupyter notebook support to the AWX development environment
* Jupyter starts alongside the other awx services and is available on
  0.0.0.0:8888
* make target: make jupyter
* default settings in settings/development.py
* Added jupyter, matplotlib, numpy to dev dependencies
2017-12-05 23:46:18 -05:00
Matthew Jones a3071c2a1f Make sure we define postgres port customization during install 2017-12-04 11:08:40 -05:00
Matthew Jones cf0cc2e2f2 Add system requirements to install docs 2017-12-04 07:56:34 -05:00
Matthew Jones 8a65c6e1c8 Present the tower inventory sync failure in a better way
This allows it to be handled better by ansible 2.4+
2017-11-16 12:59:12 -05:00
Matthew Jones 03eca250d9 Fix an openshift issue writing the inventory file
Openshift was throwing an error here, though I'm not sure why it makes
a whole lot of difference to call fdopen() vs open(). This was
introduced when this method was unified under the new
ansible-inventory system. This fixes it for all cases. mkstemp(),
while not necessary, is a useful addition to keep from leaking
inventory details unnecessarily.
2017-11-15 13:12:54 -05:00
Matthew Jones 65d01d508b Fix an issue with handler tasks after celery upgrade
There's a bug in celery 4.X when using bound tasks as error
handlers. We don't actually need it to be bound especially since the
request object is now available in the function signature
2017-11-15 13:12:06 -05:00
Matthew Jones 9ee77a95c6 Delay instantiation of the celery app for the inspector
This keeps the instance from re-using a pool that might have already
expired and is unusable for the inspector that needs to run as part of
the task manager
2017-11-14 10:33:47 -05:00
Matthew Jones 89a05e9bbc Handle json decoder errors from tower inventory source 2017-11-06 14:15:53 -05:00
Matthew Jones 74c9b9cf6a Adding pycrypto distro package
Without this a lot of things break and it's no longer marked as a
dependency for the ansible core project
2017-11-06 11:18:45 -05:00
Matthew Jones 5e37d6ea7e Remove unused TOWER_HOST and AWX_HOST env vars
These were conflicting with the new Tower credential
2017-11-02 10:44:06 -04:00
Matthew Jones bf281f6ea9 Specify a PGDATA directory to prevent container re-create issues 2017-10-31 10:20:08 -04:00
Matthew Jones 85be3c7692 Align inventory variables with Ansible modules 2017-10-27 08:12:14 -04:00
Matthew Jones 5f3ebc26e0 Adding license checks for Tower inventory source
* For Tower the license must match between the source and destination
* For AWX the check is disabled
* Hosts imported from another Tower don't count against your license
  in the local Tower
* Fix up some issues with enablement
* Prevent slashes from being used in the instance filter
* Add &all=1 filter to make sure we pick up all hosts
2017-10-27 08:12:14 -04:00
Matthew Jones d282966aa1 Use towervars to enable turning on remote tracking vars on Tower src
* This allows the local Tower to track enabled state and unique
  instance id for each host imported from the remote Tower
2017-10-27 08:12:14 -04:00
Matthew Jones 71e132ce0f Show instance filter ui element with tower inventory source 2017-10-27 08:12:14 -04:00
Matthew Jones fdc7f58bb4 Support passing instance filters to tower inventory src
* Switch ignore ssl errors to default on
* Application inventory source defaults for Tower src
2017-10-26 13:51:05 -04:00
Matthew Jones 6c597ad165 Adding initial credential and invsrc for Tower
* New credential type for Tower
* Inventory source definitions and migrations for Tower
* Initial Tower inventory source script
2017-10-26 13:51:05 -04:00
Matthew Jones 5635f5fb49 Merge branch 'release_3.2.1' into devel
* release_3.2.1:
  fallback to empty dict when processing extra_data
  fix migration problem from 3.1.1
  move 0005a migration to 0005b
  feedback on ad hoc prohibited vars error msg
  Fix the way we include i18n files in sdist
  Fix migrations to support 3.1.2 -> 3.2.1+ upgrade path
  fix missing parameter to update_capacity method
  fix WARNING log when launching ad hoc command
  Validate against ansible variables on ad hoc launch
  do not allow ansible connection type of local for ad_hoc
  work around an ansible 2.4 inventory caching bug
  fix scan job migration unicode issue
  Assert isolated nodes have capacity set to 0 and restored based on version
  Set capacity to zero if the isolated node has an old version
2017-10-19 13:30:26 -04:00
Matthew Jones da5e6883d4 Lower the default uid by which we'll rewrite passwd
This fixes some issues with openshift under certain security policies
2017-10-13 14:27:30 -04:00
Matthew Jones 6fa283fc98 Merge branch 'release_3.2.0' into devel
* release_3.2.0: (66 commits)
  fix workflow maker lookup issues
  adding extra logic check for ansible_facts in smart search
  adding "admin_role" as a default query param for insights cred lookup
  changing insights cred lookup to not use hard coded cred type
  fix rounding of capacity percentage
  Catch potential unicode errors when looking up addrinfo
  fixing typo with adding query params for instance groups modal
  move percentage capacitty to variable
  Add unit test for inventory_sources_already_updated
  Check for inventory sources already updated from start args
  Fixed inventory completed jobs pagination bug by setting default page size
  Remove the logic blocking dependent inventory updates on callbacks
  fix instance group percentage
  Remove host-filter-modal import
  Fix partial hover highlight of host filter modal row
  Removed leading slash on basePath
  Fixed host nested groups pagination
  Added trailing slash to basePath
  Fixed nested groups pagination
  Fixed host_filter searching related fields
  ...
2017-10-10 16:09:43 -04:00
Matthew Jones 128eaaf60f Fix an issue where mercurial wasn't getting installed in the image
Also fixes an issue assigning proxy environment variables
2017-09-28 11:37:10 -04:00
Matthew Jones 9219f0f682 Catch potential unicode errors when looking up addrinfo 2017-09-27 20:14:47 -04:00
Matthew Jones 3e38a0c17d Add unit test for inventory_sources_already_updated 2017-09-27 16:52:05 -04:00
Matthew Jones f4c9617f95 Check for inventory sources already updated from start args
In the case where the host didn't exist in the inventory source and
was found in the first inventory sync
2017-09-27 16:31:22 -04:00
Matthew Jones 7861cda6fe Remove the logic blocking dependent inventory updates on callbacks 2017-09-27 16:00:05 -04:00
Matthew Jones 4d24318f70 Update install documentation with official image information 2017-09-26 14:56:46 -04:00
Matthew Jones b89d1781a2 Modify openshift roles to support official AWX images 2017-09-26 09:58:07 -04:00
Matthew Jones 2e7b4529f8 Add support for installing AWX from official images
* New options in the inventory file
* Add defaults to some variables
2017-09-25 21:52:23 -04:00
Matthew Jones 3201436edb Add image builder playbook for installer 2017-09-25 12:23:15 -04:00
Matthew Jones 73a37a281d Allow pre-populating the default AWX superuser account 2017-09-22 10:10:51 -04:00
Matthew Jones 55259b4445 Update minishift documentation relative to postgresql
Also bump the default claim value to 5G
2017-09-21 22:19:08 -04:00
Matthew Jones 523734a6af omit old unit tests from "test" make target 2017-09-21 21:33:34 -04:00
Matthew Jones 6b7126ab6b Updating contributing for What should I work on? 2017-09-19 13:46:45 -04:00
Matthew Jones 476dae5418 Add krb package dependencies to support windows connections 2017-09-19 09:23:27 -04:00
Matthew Jones f5ccb51ef2 Fix an issue where we ignored the secret key during install 2017-09-18 15:11:04 -04:00
Matthew Jones 87b58e6bc2 Apply ephemeral cache for some memoize tasks recently merged 2017-09-18 12:31:33 -04:00
Matthew Jones 84f9c49b6f Propogate cert validation to inventory updates for vmware 2017-09-18 12:21:27 -04:00
Matthew Jones 64415872a0 Merge branch 'release_3.2.0' into devel 2017-09-18 10:55:45 -04:00
Matthew Jones 0ee3b0e59b Fix an issue not populating the version metadata for isolated sdist 2017-09-18 10:51:56 -04:00
Matthew Jones ebbd42322d Fix an issue with hasattr call 2017-09-18 09:08:44 -04:00