Commit Graph
100 Commits
Author SHA1 Message Date
Matthew Jones a79c53bb17 Switch to disabling sftp batch instead of scp
For ad-hoc jobs where we are experiencing issues related to disabling
control persist to support proot.  See commit
f69846e7871cae41b19ecda33563d6365dc9c3e0 for earlier analysis
2016-05-12 15:33:58 -04:00
Matthew Jones 1faf3cceb8 Fix up notification related unicode errors
I'm not sure why I used smart_text instead of smart_str here originally
2016-05-12 10:54:00 -04:00
Matthew Jones 76728fcba7 Set a user agent for the webhook if not provided 2016-05-12 10:54:00 -04:00
Matthew Jones 8a0b40e478 Force scp usage instead of sftp for ad-hoc jobs 2016-05-11 16:03:57 -04:00
Matthew Jones c6eedce3d8 Further purging of cleanup_deleted 2016-05-11 15:07:56 -04:00
Matthew Jones fda5563aa8 Add azure module dependency to tower venv 2016-05-10 16:21:55 -04:00
Matthew Jones d5afbc20ac Require cloud on inventory source for notification
Inventory Sources must now be a cloud type in order to associate a
notification template with it.
2016-05-10 14:50:34 -04:00
Matthew Jones 01ea3b602d Add ipython to dev dependencies
So tower-manage gets ipython instead of the vanilla interpreter
2016-05-10 10:34:26 -04:00
Matthew Jones b0f4dc06c4 Fix fail/skip scenario for scan service jobs
In the case where no services could be found we'll set the status as
skipped instead of failed
2016-05-09 15:30:56 -04:00
Matthew Jones b6f8d12119 Map ProjectUpdate for notifications on its view 2016-05-09 12:48:46 -04:00
Matthew Jones 7a611cdb8a Fix an issue with the related through tables on notifications 2016-05-09 12:44:54 -04:00
Matthew Jones dad72e297c Update the default path for celerybeat-schedule
If the celerybeat database becomes corrupt then it can prevent starting
some services for Tower.  This happens in an obscure way and currently
that database is stored at the top of the source tree.   This moves it
into the container managed space so that failure scenario won't persist
if the container is re-created
2016-05-09 11:36:53 -04:00
Matthew Jones 61525d8da4 Include more details on schedules endpoint 2016-05-06 14:45:00 -04:00
Matthew Jones ea25742ea5 Bump cryptography vendored library version 2016-05-06 11:52:02 -04:00
Matthew Jones 32720b2cc4 Add Schedule to the system job migration 2016-05-05 16:06:26 -04:00
Matthew Jones 6cece17024 Purge cleanup_deleted tasks as they aren't needed
This also adds a periodic static task to cleanup auth tokens
2016-05-05 16:04:36 -04:00
Matthew Jones 3c90ecba0b Fix up notifications active field removal 2016-05-05 15:27:57 -04:00
Matthew Jones 0a89581868 Fix up flake8 errors 2016-05-05 14:23:41 -04:00
Matthew Jones 5895b3a343 Refactor Notification's naming
Notifier -> NotificationTemplate
notifier -> notification_template
2016-05-05 13:53:46 -04:00
Matthew Jones 675b596fb0 Perform json and yaml validation on Job Templates 2016-05-05 12:00:12 -04:00
Matthew Jones e5bebf7b51 Change docker project default directory 2016-05-05 11:04:53 -04:00
Matthew Jones 200ffaf7ca Fixes a small flake8 issue 2016-05-04 15:26:37 -04:00
Matthew Jones 6dad37d556 Add periods to various serializer validation errs 2016-05-04 14:52:24 -04:00
Matthew Jones b8ed6c079e Strip newlines from eula at UI team's request 2016-05-04 14:42:10 -04:00
Matthew Jones 0e7767c6a6 Fix fact gathering module being absent
Somewhere recently we broke the ability to gather multiple modules in
one scan job pass.   This fixes that, it also further improves the scan
job performance.
2016-05-04 14:00:15 -04:00
Matthew Jones e3f9b631e7 Clean up some notification task bugs 2016-05-04 13:52:41 -04:00
Matthew Jones 76ce77aaf7 Remove stray inventory source debug statement 2016-05-04 13:18:50 -04:00
Matthew Jones 39f922d2fe Fix up azure source identification logic
We now need to deal with two azure inventory scripts.  The logic here
was incorrect after the PR importing the new module.   This brings it
back in line with the old logic for selecting the class azure inventory script.
2016-05-04 13:17:19 -04:00
Matthew Jones c453adbc30 Bump libcloud version for ansible tasks 2016-05-03 14:47:46 -04:00
Matthew Jones db541b4e5e Add ad-hoc extra vars migration 2016-05-03 14:04:59 -04:00
Matthew Jones a17bb5677b Implement extra vars on ad-hoc commands 2016-05-03 12:02:27 -04:00
Matthew Jones 93b0054190 Remove the dumb-init deb that gets left behind in the container image 2016-05-03 12:01:58 -04:00
Matthew Jones 062ace975c Add the super-handy pprofile to dev reqs 2016-05-03 12:01:58 -04:00
Matthew Jones 9cc9ef6b33 Use shallow copy in fact cache plugin
This deep copy would cause ansible-playbook to get hung up and fail to
spawn new subprocesses thus forcing extremely large scan jobs to
serialize their connections through one process.    This had the effect
of forcing `forks=1` behavior.

This change was for citco, a customer who is running scan jobs against
600 hosts with a forks value of 200.
2016-05-02 13:34:47 -04:00
Matthew Jones 256487a963 Strip hash marks from slack channels at send time 2016-04-25 09:49:36 -04:00
Matthew Jones 9ac8ee3ef0 Updating api side vendored licenses 2016-04-22 15:23:20 -04:00
Matthew Jones cf7b71ae2c Merge branch 'release_2.4.5' into devel
* release_2.4.5:
  Remove distribute from the setup virtualenv if installed (#1631)
  Update changelogs for 2.4.5 release
  Only export changed targets in reprepro
  Improve the efficiency of the stdout dump database migration
  Added logic to not show the loop summary events in the UI by looking at event_data.event_loop.
  Handle runner items from ansible v2
  bump boto
  fix case of Ansible v2 _result.cmd is list for release 2.4.5
  Handle both string and list hosts in our hosts->name conversion
  Default play names to the hosts the play was run against
  Use better isinstance(x) type checking
  Backporting test fixes from PR #1020: Fix error with ad hoc command events when running in check mode.
  Fix for tasks breaking when using 'yum' with ansible 1.9.4
  Bump 2.4.5 version, changelogs, and reprepo
2016-04-22 10:54:23 -04:00
Matthew Jones fe360ca8e3 Updates to network credentials
Separating out the network credential used fields
2016-04-21 16:43:55 -04:00
Matthew Jones 42247fa3c1 Add job events top level api object 2016-04-21 15:30:35 -04:00
Matthew Jones fcc9c1e6e4 Switch org projects list back to allowing create 2016-04-21 15:09:47 -04:00
Matthew Jones e1808a6ed7 Update indentation for flake8 2016-04-21 14:13:03 -04:00
Matthew Jones 0cd4b52582 Remove azure dependency from tower venv 2016-04-21 14:03:27 -04:00
Matthew Jones 296f70ce17 Implement Azure RM creds and inventory
* Vendor ansible's azure_rm inventory script
* Add new inventory type
* Add new credential type
* Expand host instance_id column from varchar 100 to 1024 to accept the
  long instance ids returned by Azure
* Make the inventory_import azure match rename more explicit.
2016-04-21 13:51:30 -04:00
Matthew Jones 16fbb7d14c Add default playbook to job template preload 2016-04-20 11:29:21 -04:00
Matthew Jones d87213705b Adjust job launch blocking logic
Previously a job template would always block another job template launch
regardless of the details of the job template.   We now restrict that
blocking logic to only block in the case that the job template was
launched with the same inventory.   We keep the exclusion where if the
launch type is 'callback' and the limits differ then they won't be blocked
2016-04-18 15:57:31 -04:00
Matthew Jones 226cb9acdf Protect job template and project template notifiers in the case where
the project or org may not be available.
2016-04-18 14:24:53 -04:00
Matthew Jones ba7f48abf1 Updating tests around data preload 2016-04-18 11:27:39 -04:00
Matthew Jones d1801bdf9f Demo data flake8 cleanup 2016-04-15 15:14:10 -04:00
Matthew Jones 8c4e4f4c0d Creating pre-loaded demo data
* Bootstrap some initial data if there is no default org
* Renamed default org script appropriately
2016-04-15 14:42:51 -04:00
Matthew Jones 6f79ec8830 Revert "Merge pull request #1534 from chrismeyersfsu/fix-config_performance"
This reverts commit 7cc144fa59f157a5e1789a913e0d9a31720141ec, reversing
changes made to 86af9950e754b06f40cb293a36ab365f589273db.
2016-04-14 11:40:07 -04:00
Matthew Jones 6150addc2e Updates to makefile to coerce pip install and usage for dependencies 2016-04-13 11:18:52 -04:00
Matthew Jones 28ffa57258 Implement rbac and related unit tests for notifications 2016-04-12 17:00:38 -04:00
Matthew Jones b4ef0304e3 Add funcsigs as a direct dependency for the venvs 2016-04-12 10:48:49 -04:00
Matthew Jones d6c2b931f9 Remove stray comma from network credential in JobOptions 2016-04-11 14:49:23 -04:00
Matthew Jones f437d39d06 Add psutil to ansible venv dependencies 2016-04-11 12:57:56 -04:00
Matthew Jones 673c5929f9 Improve the efficiency of the stdout dump database migration 2016-04-11 11:40:56 -04:00
Matthew Jones 62218cfd7d Fix virtualenv path association in job runs 2016-04-08 11:56:51 -04:00
Matthew Jones a0fd546bdf Force jenkins pip dependencies to override the base 2016-04-07 10:33:20 -04:00
Matthew Jones 86cca77b43 Make sure requirements_jenkins invokes the other requirements sections 2016-04-06 20:33:44 -04:00
Matthew Jones 547506a3bf Remove contextual statsd reference 2016-04-06 16:47:36 -04:00
Matthew Jones c0afa3ca0f More commenting out of statsd 2016-04-06 16:42:30 -04:00
Matthew Jones 3ba591cea2 Remove statsd from callback
This doesn't work with our current virtualenv setup and ansible
dependencies.  Will revisit later
2016-04-06 16:29:50 -04:00
Matthew Jones 0a6df40066 Improvements to docker compose workflow with new packaging changes 2016-04-06 12:24:19 -04:00
Matthew Jones ae9222f716 Fix flake8 after packaging changes merge 2016-04-06 11:03:34 -04:00
Matthew Jones b89d183433 Import pytest for skipping settings tests 2016-04-04 14:19:08 -04:00
Matthew Jones 5460bac498 Add historical notifications to notifier summary 2016-04-04 13:38:34 -04:00
Matthew Jones 469bff1d0b Fix required fields in notifier serializer
Handle the situation where the password field was left as-is when
posting the existing encrypted field
2016-04-04 12:04:31 -04:00
Matthew Jones b01fe663af Fix up tests for vacated settings 2016-04-04 11:57:40 -04:00
Matthew Jones b6706662a4 Disable database settings until after 3.0 2016-04-04 09:59:24 -04:00
Matthew Jones f2284a0098 Handle runner items from ansible v2
Also denote whether the trailing runner_on_ was a loop event
2016-03-23 11:38:11 -04:00
Matthew Jones 7f8fae566e Further decouple survey spec from enablement
We now show the survey summary field only if the contents of the survey
spec or valid (not the default {})
2016-03-22 14:59:56 -04:00
Matthew Jones 0e2184902e Handle runner items from ansible v2
Also denote whether the trailing runner_on_ was a loop event
2016-03-22 12:31:05 -04:00
Matthew Jones f286dc7486 Fix an issue with the email notifier
Incorrect body format assumptions in the email notifier
2016-03-21 15:40:41 -04:00
Matthew Jones 46bbc83d0d Changes for Tower virtualenv support
* Break requirements down into ansible and tower reqs
* Generate separate Ansible and Tower virtual environments
* Install appropriate requirements files into each one
* Modify development tools to use these venvs instead of our old
  site-packages
* Modify settings to indicate venv enablement and location of venvs
* Modify tasks to use the proper virtual environment for its purpose
2016-03-21 12:11:19 -04:00
Matthew Jones c7e33d0471 Changes for Tower virtualenv support
* Break requirements down into ansible and tower reqs
* Generate separate Ansible and Tower virtual environments
* Install appropriate requirements files into each one
* Modify development tools to use these venvs instead of our old
  site-packages
* Modify settings to indicate venv enablement and location of venvs
* Modify tasks to use the proper virtual environment for its purpose
2016-03-21 11:47:19 -04:00
Matthew Jones 526a6ec7dd Remove unneeded fetch for r/o fields 2016-03-16 13:12:13 -04:00
Matthew Jones 45f95bf2b2 Disallow related elements to be treated as choices
DRF will try to resolve potential candidates into the OPTIONS endpoint.
This is mainly to support their POST field in the browseable API.   We
don't need this and it can yield some expensive queries so we bypass
generating choices for any RelatedField fields
2016-03-16 13:11:05 -04:00
Matthew Jones 63b01bb04b Fix up the docker-refresh Makefile target
This allows you to cleanup the images without requiring a rebuild necessarily
2016-03-16 13:11:05 -04:00
Matthew Jones 7e818319b5 Pass, don't return pass when checking mongo connectivity 2016-03-14 13:53:34 -04:00
Matthew Jones 001eb1f69f Handle a mongo OperationFailure
This seems to happenw hen the database is up, allows us to connect but
we don't have permission to access the objects (or they don't exist
yet).
2016-03-14 11:57:34 -04:00
Matthew Jones eab223d229 Make sure we are covering system jobs and template on notifications 2016-03-11 15:11:08 -05:00
Matthew Jones cdc342a6ee Fix fact database migration
For new installs the database may not be online and thus the migration
can't happen.  This will allow it to continue but we may want more
messaging to the user running potential upgrades
2016-03-11 12:59:21 -05:00
Matthew Jones c2ea9c3c00 Make survey always visible if license enables
Previously this would only be visible if survey is enabled.  It should
be possible to define and view a survey even if this is disabled.   The
enablement flag controls whether the survey prompt is shown and required.
2016-03-10 16:09:34 -05:00
Matthew Jones 4f8763ff90 Changes for Tower virtualenv support
* Break requirements down into ansible and tower reqs
* Generate separate Ansible and Tower virtual environments
* Install appropriate requirements files into each one
* Modify development tools to use these venvs instead of our old
  site-packages
* Modify settings to indicate venv enablement and location of venvs
* Modify tasks to use the proper virtual environment for its purpose
2016-03-09 14:33:42 -05:00
Matthew Jones 823e54a0b9 Fix an error sending success notifications 2016-03-08 14:14:57 -05:00
Matthew Jones 714d13eefd Update docker development workflow to not require make develop 2016-03-02 12:09:42 -05:00
Matthew Jones 1f19c13ac9 Fix up a Notification unit test issue after the merge 2016-03-01 12:16:10 -05:00
Matthew Jones 533c629040 Add irc requirements dependency 2016-02-29 23:50:16 -05:00
Matthew Jones 99ec07b8a5 Merge branch 'notifications_work' into devel
* notifications_work: (23 commits)
  Updates to notification unit tests after @wwitzel3's feedback
  Fix some notifications issues and write some tests
  Add notification system documentation
  Clean up flake8 related issues
  Fixing up some unicode issues
  Implement tower ui view url on models
  Sanity check and force proper types in admin check
  Proper type for in check
  Adding migration and base notification type
  Add a periodic administrative notification
  Refactor message generator
  Support notification password field encryption
  Notification configuration type checking
  Refactor NotificationTemplate to Notifier
  Implement irc notification backend
  Add webhook notification backend
  Pagerduty and Hipchat backends plus some cleanup
  Notification serializers, views, and tasks
  Implement notification serializer and validations
  Notification endpoints and url expositions
  ...
2016-02-29 16:50:33 -05:00
Matthew Jones b892ee5f96 Updates to notification unit tests after @wwitzel3's feedback 2016-02-29 13:25:49 -05:00
Matthew Jones 0ee12901fe Fix some notifications issues and write some tests
* Fixes some notifier merging issues
* Fixes some more unicode problems
* Implements unit tests
2016-02-29 12:30:00 -05:00
Matthew Jones a9bf29d2a7 Add the id field to the default summary fields
This is to support the UI.

ref: #1063
2016-02-26 11:30:34 -05:00
Matthew Jones b35d7a3c6b Add notification system documentation 2016-02-24 13:26:43 -05:00
Matthew Jones 7eb74cd011 Bump 2.4.5 version, changelogs, and reprepo 2016-02-24 09:51:12 -05:00
Matthew Jones 41d5393af0 Clean up flake8 related issues 2016-02-23 15:30:07 -05:00
Matthew Jones 4b1493f456 Fixing up some unicode issues 2016-02-23 15:26:29 -05:00
Matthew Jones 75ef0dd395 Implement tower ui view url on models 2016-02-23 14:19:42 -05:00
Matthew Jones b88892be49 Sanity check and force proper types in admin check 2016-02-23 12:33:30 -05:00
Matthew Jones 050ed8a200 Proper type for in check
Still an __in for when we need to add another trigger obj
2016-02-23 12:18:31 -05:00
Matthew Jones 39c942e98b Adding migration and base notification type 2016-02-23 11:20:08 -05:00