Commit Graph
100 Commits
Author SHA1 Message Date
Matthew Jones a98db65d4a Documenting process isolation for acceptance 2017-02-01 13:44:05 -05:00
Matthew Jones e389084298 Add a periodic celery task to clean up local stdout files
This works in tandem with the on-demand stdout file generator and
decouples stdout locally from the persistent database representation
ob the job itself
2017-01-31 15:52:45 -05:00
Matthew Jones 4c61449f89 Generate stdout from database if not present on disk
Useful for download stdout when not available on all cluster nodes
2017-01-31 14:29:08 -05:00
Matthew Jones 4a015da982 Purge old mongo Makefile targets, update psql target for new docker 2017-01-31 14:28:48 -05:00
Matthew Jones 53256f4c2d Integrate proper cleanup for tmp paths for inventory updates 2017-01-31 11:31:44 -05:00
Matthew Jones 587a843acd Shorten the default control persist path
Customer needed a few more characters, this should give a bit more breathingroom
2017-01-30 13:49:20 -05:00
Matthew Jones d9a7f2fadb Silently ignore psutil errors when the process is already gone
During ssh CP cleanuip
2017-01-30 13:39:05 -05:00
Matthew Jones 66426aa7dd Store project update and inventory update stdout in the database
So things work across the cluster
2017-01-27 16:51:03 -05:00
Matthew Jones bfda6ff8b5 Fix up some unit tests 2017-01-26 16:01:54 -05:00
Matthew Jones 6388e9b055 Putting back the inadvertantly removed job_events 2017-01-26 12:00:13 -05:00
Matthew Jones 37d8e57c4f These old endpoints are hacky, ugly, and no longer used
They were strictly shortcuts for UI querying which can be performed
more efficiently on `/job_events` without any special processing
2017-01-26 11:55:44 -05:00
Matthew Jones e15273c993 Fixes some issues where Tower would not emit the password on prompts
Carat seems to only want to match newlines or the beginning of the
string, since we seem to get CR+LF this seems to keep the carat from
matching so here we're expanding the regex matches.

This also extends support for DZDO where it was only originally added
on ad-hoc commands inadvertantly.

Remove PMRUN support since that was never merged into core
2017-01-26 11:19:23 -05:00
Matthew Jones 4771b2efc0 Merge branch 'remove_jobevent_parent_usage' into release_3.1.0
* remove_jobevent_parent_usage:
  Don't iterate over parents, perform a final update at the end
  Drop assignment of job event parent in favor of parent_uuid
2017-01-24 21:07:58 -05:00
Matthew Jones 22a3e87720 Don't iterate over parents, perform a final update at the end 2017-01-24 16:21:50 -05:00
Matthew Jones 2af98e5f58 Drop assignment of job event parent in favor of parent_uuid
* This eliminates the save lookup of the parent job event and
  associated memcached call(s)
* Removes some other legacy parent determination mechanisms
* Alternate way to query for the parent/child relationship
2017-01-24 15:40:33 -05:00
Matthew Jones b3f4f3bea5 Fix issue reassembling ad-hoc event partials 2017-01-23 16:30:44 -05:00
Matthew Jones abdd91bd1f Fixes for data corruption/exception in cache usage
Specifically as it relates to serializers and job event writing at
high speeds
2017-01-23 14:39:03 -05:00
Matthew Jones 8a9b2fcaea Implement alternate ports for nginx
* This also allows disabling https mode in the nginx configuration
* Reconfigure the development container to not specifically require
  https, so the haproxy cluster configuration can work
2017-01-23 14:34:15 -05:00
Matthew Jones df249eda2e Raise default job event worker queue size 2017-01-23 10:47:47 -05:00
Matthew Jones 47efe8c2e8 Add deprecation warning to job start 2017-01-19 11:01:31 -05:00
Matthew Jones a06b44ae38 Add sync launch type 2017-01-19 09:47:10 -05:00
Matthew Jones 94ffce07a4 Fix an issue processing job events when uuid isn't present 2017-01-19 09:47:10 -05:00
Matthew Jones 6ed86e1934 Revert "bump pkg info tower version to 3.1.0"
This reverts commit dea4c873e8d89f3e2ed5e74f8d3eb480e57f927d.
2017-01-18 14:30:06 -05:00
Matthew Jones 73f92bba5d Integrate multiprocessing writer worker in callback receiver 2017-01-18 12:05:29 -05:00
Matthew Jones 1cdeb4d2af Refactor event emitter to not use event partials
event emitter now caches to memcache where it is eventually picked up
by the stdout event emitter. This obviates event reassembly in the
callback receiver.
2017-01-17 15:59:43 -05:00
Matthew Jones 0d4d872a76 Import translation files 2017-01-12 15:15:13 -05:00
Matthew Jones 23f4898eea Re-wirte ad-hoc notification templates to organization to match docs 2017-01-11 11:25:08 -05:00
Matthew Jones 8796af7e28 Make session and csrf cookies secure by default 2017-01-11 10:39:46 -05:00
Matthew Jones 974fe1d244 Fix an issue with set_stats for unsupported Ansible versions 2017-01-11 10:24:17 -05:00
Matthew Jones e594a3ee3e Implement a worker-based callback receiver system 2017-01-10 14:25:41 -05:00
Matthew Jones 3006e357f6 Add a hint header for max job events for the UI 2017-01-05 15:16:36 -05:00
Matthew Jones 89304b2ee0 Don't ignore ANSIBLE_NET env items when hiding sensitive information 2017-01-05 12:35:50 -05:00
Matthew Jones 60fa34e8cc Fix a redis reference issue
We are no longer using redis for Celery
2017-01-04 14:40:40 -05:00
Matthew Jones 415c7166b6 Adding source for LGPL deps 2017-01-04 14:26:53 -05:00
Matthew Jones f9d9196270 Adding deprecation warnings
Also updating New in/Added in text
2017-01-04 14:23:43 -05:00
Matthew Jones 6188f2d9b5 Update third party licenses 2017-01-04 12:39:39 -05:00
Matthew Jones e45201aa87 Added scm_type to project summary fields 2017-01-03 15:37:38 -05:00
Matthew Jones 9d289e4349 Fix some issues related to purging the old paging decorator 2017-01-03 15:31:30 -05:00
Matthew Jones 9f777f6153 Add some provisions for https into haproxy and the cluster 2017-01-03 13:59:29 -05:00
Matthew Jones 888ec25c3c Adding new privilege escalation methods from core 2016-12-15 16:22:21 -05:00
Matthew Jones a25853a3a5 Updating translation artifacts
* also installing translation tools into dev environment
* Removing fedora install instructions, no one here uses fedora
2016-12-14 15:58:47 -05:00
Matthew Jones 1cd2a762be Reset max bytes display 2016-12-14 15:20:04 -05:00
Matthew Jones a61e729ebb Purge event res dict if it is over a certain size
Also purge/update some old settings values
2016-12-14 15:05:28 -05:00
Matthew Jones f89bd15bf5 Show started and finished on job summary fields 2016-12-13 15:27:48 -05:00
Matthew Jones e1942083e3 Remove sneeringer's old unused decorator code 2016-12-12 16:14:22 -05:00
Matthew Jones 93ce2a5dfd Set proot/bubblewrap enabled by default 2016-12-12 15:26:31 -05:00
Matthew Jones 8b8c29b9dc Purge has_schedules
Also... migration for big field docs updates
2016-12-12 12:33:09 -05:00
Matthew Jones bb7a201c2f Add missing field to ad-hoc command partial 2016-12-12 12:25:19 -05:00
Matthew Jones 349d497bb4 Make max page size tunable 2016-12-12 12:10:33 -05:00
Matthew Jones 08edbb1b72 Set an upper limit of 200 on the max page size 2016-12-12 12:01:48 -05:00
Matthew Jones 2d6dff1a3b Better error handling on deprovision_node management command 2016-12-12 10:55:03 -05:00
Matthew Jones 4e10d55501 Updating clustering acceptance docs 2016-12-07 11:08:39 -05:00
Matthew Jones 24b858c6fb Switch away from deepcopy to improve performance 2016-12-06 13:33:14 -05:00
Matthew Jones cd11a9c1c2 peg compose tag to active branch 2016-12-06 13:32:33 -05:00
Matthew Jones 71c600cad0 more missing punctuation 2016-12-06 13:18:34 -05:00
Matthew Jones 6efeeeb083 Adding missing period for timeout 2016-12-06 12:29:56 -05:00
Matthew Jones 51461a27da Adding help text to various job fields 2016-12-06 12:18:17 -05:00
Matthew Jones ebb0f06b17 Updating host field help text 2016-12-06 12:12:08 -05:00
Matthew Jones d7ffdf7020 Updating project help text 2016-12-06 12:08:23 -05:00
Matthew Jones eebdd13aa1 Updating schedules help text 2016-12-06 11:39:08 -05:00
Matthew Jones e5195032f6 Add back in some older dependencies that we had un-tracked
We'll need to figure out why dm.xmlsec.binding is segfaulting without these
2016-12-03 15:51:56 -05:00
Matthew Jones b17ca58fdd Trigger local cache flush before broadcasting on settings change 2016-12-02 16:46:48 -05:00
Matthew Jones faf4c84eca Purging old workload generator 2016-12-02 14:19:28 -05:00
Matthew Jones 765b7d9cc2 Add TOWER_HOST in the form of TOWER_URL_BASE to job env 2016-12-01 12:01:17 -05:00
Matthew Jones 9cecb7d870 Cleaning up some parts of the logging branch
* Fixing some flake8 warnings
* Removing some superflous print statements
2016-11-29 15:02:20 -05:00
Matthew Jones 1921f8cf04 Re-sync tower and ansible dependencies 2016-11-29 15:02:20 -05:00
Matthew Jones 7df11c59c0 Adding requests-futures dependency 2016-11-29 13:40:35 -05:00
Matthew Jones cad112e8da Add python-logstash dependency 2016-11-29 13:29:13 -05:00
Matthew Jones 34add726b7 Hide active tasks if anonymous user 2016-11-28 13:35:43 -05:00
Matthew Jones ddaf4ff2d0 Merge branch 'fix_fact_caching' into devel
* fix_fact_caching:
  Convert the fact data structure to the right type
  Fixing up fact cachine related unit tests
  Port fact caching system to rabbitmq
2016-11-21 23:15:13 -05:00
Matthew Jones ccd284526d Fix serializer related unit tests 2016-11-21 23:11:12 -05:00
Matthew Jones fd56a7acbb Remove unneeded summary fields element 2016-11-21 14:15:18 -05:00
Matthew Jones 9d044c1f41 Store the job's project refresh if it happened
* Create a field on the job to track this
* Add a summarizeable model to the summarizable fk fields so we can
  pull that in where needed
2016-11-21 14:11:20 -05:00
Matthew Jones c6ebdb9b26 Switch to using callback queue by default
* Fix an issue where we grab the wrong setting for the callback
  connection
* Remove old ansible.cfg that we shipped with the setup playbook for
  centos 6
2016-11-21 13:26:32 -05:00
Matthew Jones f77caf8b8b Convert the fact data structure to the right type 2016-11-18 12:13:09 -05:00
Matthew Jones 35b9786e06 Fixing up fact cachine related unit tests 2016-11-18 10:23:07 -05:00
Matthew Jones 3b07d9745d Port fact caching system to rabbitmq
* Purge all references to zmq also
* New setting to control the queue it works on
2016-11-18 09:44:17 -05:00
Matthew Jones 30984a3a79 Fix up flake8 errors 2016-11-10 13:07:48 -05:00
Matthew Jones 1a3d452a2d Fix an auth issue in the browsable api
Trying to get the object could raise an unhandleable permission error if
the user wasn't logged in for certain views
2016-11-10 11:46:25 -05:00
Matthew Jones 5186e7d8b4 Remove some unneeded func tests for instances 2016-11-10 11:46:25 -05:00
Matthew Jones 78b8876ed9 Support expiring of capacity if a node is down
For a certain amount of time
2016-11-10 09:52:04 -05:00
Matthew Jones 343966f744 Implement gathering overall task capacity
For use when running/planning jobs
2016-11-07 13:45:01 -05:00
Matthew Jones fa0780b9d0 Optimize uwsgi launching in dev container and prod
* Set --master flag so we have a preforking master process to manage
  subprocs
* Set max-requests to 1000 so uwsgi will recycle processes after 1000
  requests
* Set --no-orphans so uwsgi will track child processes and clean them up
  in order to respawn children
* Turn stats on in the container and install uwsgitop in dev
  requirements so we can track uwsgi stats
2016-11-04 13:38:23 -04:00
Matthew Jones 06dd5bb5d2 Fix an invalid database backend in unit test settings 2016-11-03 16:09:17 -04:00
Matthew Jones b0e40b6c29 Remove proot from dev base image in favor of bubblewrap
Remove python utils from base system, they are included with the venvs
2016-11-03 16:06:03 -04:00
Matthew Jones 8e77deea27 Push celery queue stats to memcached periodically
We'll piggyback off the task that checks for inconsistent jobs between
celery and Tower itself. These are read off via /api/v1/ping
2016-11-03 12:10:38 -04:00
Matthew Jones e01c86e88f Fix missing import for failure signal handler 2016-11-03 09:43:18 -04:00
Matthew Jones 1908b56ef7 Add event handler to manifest 2016-11-02 14:16:27 -04:00
Matthew Jones 4cb2e5aa02 Add failure event handler for supervisor process 2016-11-02 13:44:01 -04:00
Matthew Jones 23bf9cd2be Cleanup tower services and exit handling
* Replace apache with nginx in various service scripts
* Add a failure handler that will shut down all services if any of them
  fail
* Use INT signal for uwsgi shutdown procedure
2016-11-02 13:00:09 -04:00
Matthew Jones 96355854a8 Bump django version for CVE
https://www.djangoproject.com/weblog/2016/nov/01/security-releases/
2016-11-01 14:40:32 -04:00
Matthew Jones a1de353543 Shifting migrations after 3.0.3 merge 2016-11-01 14:40:32 -04:00
Matthew Jones c6cf02a602 Merge branch 'release_3.0.3' into devel
* release_3.0.3: (55 commits)
  Revert "Revert "Add needed types for selinux change""
  Revert "Add needed types for selinux change"
  interpret backslash escapes when displaying url in welcome message
  Bump the SELinux policy version
  Add needed types for selinux change
  Update SELinux policy to allow httpd_t to execute files in lib_t and var_lib_t
  Bumping changelog for 3.0.3
  Update rax.py inventory
  Revert "filter internal User.admin_roles from the /roles API list view"
  fix spelling of disassociated
  Resolves 404 when assigning resources/users to organizations in card view. Sidesteps a bug in the Refresh() utility, where pagination calculations are not made against filtered results.
  Sync azure changes to Tower virtual environment
  Add regions here as well.
  Also bump boto for new regions, per ryansb.
  More regions!
  Revert "bump shade version"
  bump shade version
  Hack copying of job_template.related.survey_spec into ui job copy flow, resolves #3737
  Revert "bump shade version"
  bump shade version
  ...
2016-11-01 11:49:28 -04:00
Matthew Jones defd271c90 Make sure we bootstrap the static dir prior to starting 2016-10-28 14:21:19 -04:00
Matthew Jones 7c7d2e37ed Fix some issues syncing playbooks
* Build a list of playbooks and store it in the database at sync time
* Fix an issue running playbook sync on jobs for scan jobs
* Remove a TODO that was unneeded
2016-10-24 11:32:21 -04:00
Matthew Jones 3d8eb48986 Test for HA license before allowing cluster job start 2016-10-20 16:50:49 -04:00
Matthew Jones 73b5b47e94 flake8 cleanup 2016-10-20 15:27:59 -04:00
Matthew Jones 6e22460f1e Add scm revision to the job model
This also cleans up flake8 issues
2016-10-20 15:18:05 -04:00
Matthew Jones ca32c5fd21 scm_revision should be used for job_type run 2016-10-20 14:56:15 -04:00
Matthew Jones 4562faa858 Pass scm revision in env and extra var for job run 2016-10-20 14:42:28 -04:00