Commit Graph
100 Commits
Author SHA1 Message Date
Matthew Jones 531f00b1ac rm the old docker run container after stopping 2015-10-09 12:04:34 -04:00
Matthew Jones e543a806ee Bump version number for devel to 3.0 2015-10-09 11:58:48 -04:00
Matthew Jones aa9357fb14 Include new social auth settings in compose 2015-10-09 11:58:48 -04:00
Matthew Jones ad95548d48 Fix up docker compose local settings 2015-10-09 11:58:48 -04:00
Matthew Jones bfc9512dc7 Also remove .deps_built with make clean 2015-10-09 11:58:48 -04:00
Matthew Jones 950a299c87 Add swig to dependencies for compose 2015-10-09 11:58:47 -04:00
Matthew Jones f85fb0551d Add libxmlsec1-dev to the docker compose workflow 2015-10-09 11:58:47 -04:00
Matthew Jones b06f7fb883 Remove logging from fact init 2015-10-02 16:56:13 -04:00
Matthew Jones dbd0a01410 Merge branch 'release_2.3.1' into devel
* release_2.3.1:
  Add 2.3.1 to the reprepro distribution manifest
  Update version to 2.3.1 and update changelog
  Remove unused time import
  Fix proot controlpersist cleanup issue
  Propagate necessary build changes from devel
  Update reprepro components
  Reenable the reprepro i386 repository
  Fix supervisor migration
  Fix reprepro/conf path in Makefile
  prefer the first version when packages are ambiguous
  Correct error message
  Fix omission
  Correct conditional stmt
  Improve logging when sourcerpm fails
  Fix typo
  Fix OFFICIAL deb Makefile
2015-10-02 16:37:45 -04:00
Matthew Jones ac37667649 Clean up fact module imports 2015-10-02 16:08:02 -04:00
Matthew Jones c65d068d55 Add note about wanting to use register_connection
I think this would be the cleaner way to go about things but there are
some unanswered questions about that route.
2015-10-02 16:04:47 -04:00
Matthew Jones 1425718d9e Improve the efficiency of mongo connections
- Remove the connect action in the fact __init__ module
- Monkeypatch mongoengine.Document's _get_db to force a new connection
  or raise an error if we don't have the proper settings.
2015-10-02 15:57:50 -04:00
Matthew Jones 71081a5b63 Update version to 2.3.1 and update changelog 2015-10-01 14:40:21 -04:00
Matthew Jones 0146b8acb0 Remove unused time import 2015-10-01 14:40:20 -04:00
Matthew Jones b4ed547b2b Fix proot controlpersist cleanup issue
* Also removed psutil version and feature check from tasks.  We now
  include the most recent version of psutil all around
* Old versions of psutil must have had these Process() items as
  properties, they are functions now so we weren't properly evaluating
  them.  With this change we are and things are back to working normally
2015-10-01 14:40:20 -04:00
Matthew Jones 6a3a36513a Add failure test for sts tokens 2015-10-01 10:49:58 -04:00
Matthew Jones 5de5b0209b Remove unused time import 2015-09-30 16:20:41 -04:00
Matthew Jones e034c0b326 Fix proot controlpersist cleanup issue
* Also removed psutil version and feature check from tasks.  We now
  include the most recent version of psutil all around
* Old versions of psutil must have had these Process() items as
  properties, they are functions now so we weren't properly evaluating
  them.  With this change we are and things are back to working normally
2015-09-30 15:56:31 -04:00
Matthew Jones acc974289f Unit test for AWS STS token support 2015-09-30 13:42:16 -04:00
Matthew Jones 5ee3c78e7f Update ec2 module to latest ansible version 2015-09-30 13:40:06 -04:00
Matthew Jones 34e028b8c7 Adding security token field to support ec2 sts
And any future credential that might want to use "security_token"
2015-09-30 13:39:36 -04:00
Matthew Jones 29b4865af8 Ability to pause pexpect output with env var
Since inventory updates don't support extra vars
2015-09-29 11:36:31 -04:00
Matthew Jones 5be53d4a79 Add a sleep step to job runs if configured
If the extra var PEXPECT_SLEEP is given to a job then it will
artificially delay the job run by that number of seconds allowing QA to
hook in and cancel the job
2015-09-28 15:43:46 -04:00
Matthew Jones a3410920a9 Remove management job schedule count restriction 2015-09-25 16:00:19 -04:00
Matthew Jones 3d7f25218a Add api side support for pendo
Also adds setup playbook support for pendo.  This change anticipates
that the default for new installations will be 'detailed'
2015-09-25 15:52:42 -04:00
Matthew Jones a7f1e69352 Fix an issue with default schedules
These weren't being created correctly, the logic meant an extra schedule
would be created on cleanup facts erroneously selected to be cleanup
activity stream
2015-09-25 14:17:31 -04:00
Matthew Jones 2135a6651b Hash the license key to show a deployment key
This can be used by pendo and other things that need to identify an
individual deployment without exposing the actual license key
2015-09-25 13:21:08 -04:00
Matthew Jones 3a6807c5b4 Add environment variable that sets proot's tmp dir
Only really useful with patched proot containing tmpdir fix
2015-09-25 11:08:03 -04:00
Matthew Jones 256af567de Merge branch 'release_2.3' into devel
* release_2.3: (24 commits)
  Updated package changelogs
  Rephrase task name
  Make purge supervisor.conf conditional on 2.2 el6
  Fix another name typo.
  Update python-ldap license text.
  Add LGPL source directly so it gets shipped.
  Update font-awesome licenses per legal feedback
  Remove bundled_licenses - we are shipping the source for bundled items, and therefore do not need to provide this separately.
  Add more license texts.
  Add full Apache licenses to files, per Pam.
  Attempt to harden mongodb tgz download
  Correct DJRF/DJRF-mongoengine licenses, rename qsstats license to match pkg name.
  Fix some typos in file names, and add licenses for some things we have modified in our own repos.
  Update Ubuntu depends and ppa's
  Include source for bundled packages in bundle.
  Include license information in packaging
  Add READMEs for the license directories.
  vendored python licenses
  Add bundled licenses documentation
  Revert to stock libzmq3 for deb depends
  ...
2015-09-22 15:57:32 -04:00
Matthew Jones 66168fb6dc Merge branch 'release_2.3' into stable
* release_2.3: (169 commits)
  Updated package changelogs
  Rephrase task name
  Make purge supervisor.conf conditional on 2.2 el6
  Fix another name typo.
  Update python-ldap license text.
  Add LGPL source directly so it gets shipped.
  Update font-awesome licenses per legal feedback
  Remove bundled_licenses - we are shipping the source for bundled items, and therefore do not need to provide this separately.
  Add more license texts.
  Add full Apache licenses to files, per Pam.
  Attempt to harden mongodb tgz download
  Correct DJRF/DJRF-mongoengine licenses, rename qsstats license to match pkg name.
  Fix some typos in file names, and add licenses for some things we have modified in our own repos.
  Update Ubuntu depends and ppa's
  Include source for bundled packages in bundle.
  Include license information in packaging
  Add READMEs for the license directories.
  vendored python licenses
  Add bundled licenses documentation
  Revert to stock libzmq3 for deb depends
  ...
2015-09-22 15:37:31 -04:00
Matthew Jones 4cf2f32451 Remove the contents of tmp/ on clean 2015-09-22 11:04:40 -04:00
Matthew Jones 077819c4c5 Add some default schedules for cleanup jobs 2015-09-21 14:53:53 -04:00
Matthew Jones 5fa0ce9d58 Include apt-get update with install
This forces the Docker apt-get update to not attempt to re-use the cache
2015-09-21 14:52:43 -04:00
Matthew Jones 519b0e8eef Merge branch 'release_2.3' into devel
* release_2.3: (54 commits)
  Upgrade changelog for 2.3
  Purge superlance
  Purge an old supervisor config
  Require chris-lea libzmq3
  Use the htpasswd command, not the module
  Fix a playbook syntax error for postgres
  Ensure postgres is started earlier in setup role
  Allow customizing npm path
  Include RPM-GPG-KEY-ansible-release for all releases
  Simplify GPG keys
  Correct broken RPM-GPG-KEY
  Use gpg --passphrase for signing CHECKSUM
  Handle rpm --addsign pasphrase prompts
  Adding docker-compose development workflow
  Fix license in spec file.
  Updates to selinux policy for some munin plugins
  ansible 1.8.4 requires sudo:true when using sudo_user
  Fix expect script timeout
  Disable timeout and correct sign.exp error
  Fix expect script hang
  ...
2015-09-11 16:03:52 -04:00
Matthew Jones 0ac4359bec Purge superlance
We no longer get memmon with our package vendoring changes.  Don't think
we need it anyway as we've got the socket service fixed.
2015-09-11 14:43:28 -04:00
Matthew Jones 8785af996d Purge an old supervisor config
Apparently awx_supervisor.conf was still hanging around in the debian
install.   We'll make sure we won't lay that file down and bump the
supervisor migration to make sure the file is removed permanently
2015-09-11 14:43:28 -04:00
Matthew Jones a2b6ffb237 Put down a file to track make develop in compose 2015-09-10 11:03:45 -04:00
Matthew Jones 97f85d2344 Rev django auth ldap to version 1.2.6
This fixes (probably among other things) an issue where we wouldn't get
any logging messages out of django for misconfigured ldap.
2015-09-10 10:59:24 -04:00
Matthew Jones f61a01b504 Switch to using ansible wait_for instead of sleep
In the startup script for docker-compose
2015-09-10 09:32:01 -04:00
Matthew Jones 940910b9a3 Adding docker-compose development workflow 2015-09-09 17:17:48 -04:00
Matthew Jones 7263367c6d Adding docker-compose development workflow 2015-09-09 17:16:35 -04:00
Matthew Jones 5727d722b6 Merge remote-tracking branch 'upstream/release_2.3' into devel
* upstream/release_2.3: (91 commits)
  Include python-{paramiko,ecdsa} dependencies
  Remove extra epel testing stanzas
  Unit test for ec2 credentialless inventory
  Fix issue with ec2 iam sync with no credential.
  Use the htpasswd command instead the ansible module
  Pip is no longer needed
  check local user root or not in ./configure
  Remove unneeded when check for super user addition
  Improve distro detection in setup.sh
  Fix superuser check on upgrade
  Minor improvements to setup.sh
  Remove ansible prerequisite check from configure
  Attempt to install ansible within setup.sh
  Allow munin processes to access postgres
  Move up base package dependency install
  fixes jenkins failures
  Proper flake8 fix
  fixes executing processes with correct PYTHONPATH will pickup .pth files
  Show the repo for bundled package file dump
  Proper flake8 fix
  ...
2015-09-03 15:46:10 -04:00
Matthew Jones 7774b17960 Unit test for ec2 credentialless inventory 2015-09-03 14:05:17 -04:00
Matthew Jones 15005d3e6b Fix issue with ec2 iam sync with no credential.
When checking if an inventory sync can start allow an ec2 sync to run
even if credentials are not supplied
2015-09-03 13:37:38 -04:00
Matthew Jones 5f6f16b5ff Rev version to 2.4.0 2015-09-03 09:44:20 -04:00
Matthew Jones c37716a2e7 Proper flake8 fix 2015-09-01 18:35:12 -04:00
Matthew Jones f0a23b7ed3 Proper flake8 fix 2015-09-01 16:15:39 -04:00
Matthew Jones 4a5111308f Fix flake8 warning from tests 2015-09-01 15:23:35 -04:00
Matthew Jones 09f4c9fbbe Fixed job redaction for projects and unit tests 2015-08-31 16:24:38 -04:00
Matthew Jones bd3faea671 Fix path check in start_development docker script 2015-08-31 15:15:49 -04:00
Matthew Jones 2f39393cf3 Implement fact receiver thread worker
Optionally allow processing of fact receiver messages in a worker
thread.  This works around an issue where data could take a while to
page into Mongo and cross the zmq socket timeout.
2015-08-28 10:59:52 -04:00
Matthew Jones 549809dfa4 Fix chunked stdout display issue
* We removed stdout redaction from jobs and job templates but only for
  the full stdout dump, this extends it to the limited display also
2015-08-25 16:05:03 -04:00
Matthew Jones 896b0e18f3 Merge remote-tracking branch 'chrismeyers/feature-configure_offline' into release_2.3
* chrismeyers/feature-configure_offline:
  configure support inventory file
  retain existing behavior with added --inventory-file param
2015-08-24 14:38:30 -04:00
Matthew Jones 1ead738f96 Updates to offline packager docker environment 2015-08-21 16:40:35 -04:00
Matthew Jones 1471a3bb3e Allow proot's base tmp path to be configurable 2015-08-21 15:28:39 -04:00
Matthew Jones 7080996670 Cleanup stdout dump during 2.2 migration
postgres is adding extra newline/carriage return escapes to the stdout
output.  Here we'll use sed to fix those so they can be read correctly
2015-08-21 13:28:34 -04:00
Matthew Jones 4ce2300d27 Bump version number for 2.3.0 release 2015-08-21 11:55:19 -04:00
Matthew Jones c5ce9cdc12 Fix stdout plaintext render
* Was rendering as json (though text/plain)
* Switched to rendering it as normal text with stripped ansi
* Fixes: https://trello.com/c/TMyOak0H/142-api-v1-jobs-n-stdout-format-txt-emits-json-output
2015-08-21 10:20:57 -04:00
Matthew Jones 230a1f3be5 Merge remote-tracking branch 'upstream/release_2.2.3' into upstream_devel
* upstream/release_2.2.3: (33 commits)
  Bump version to 2.2.3
  Optimize inventory read check for Job Templates
  Updating changelog for 2.2 release
  Bump version to 2.2.2
  Use smart_str instead of .encode on passwords
  Change default sensitive redaction behavior
  Update changelogs for 2.2.1
  Add an error message when running a job and trying to use an OpenSSH formatted key on an older version of OpenSSH.
  Revert "Add sudo: true to the backup playbook"
  Handle Ubuntu 12.04 psutil recursive cancel
  fixed log viewer modal partial
  fixed alignment of download standard out button in modals
  About Tower version number fix
  Pin package version for older release
  add standard out download button to various places in the UI
  Bump 2.2 branch to version 2.2.1
  Don't create a group that is its own parent when an EC2 tag has an empty value.  Fixes https://trello.com/c/2zc0odvX
  Limit max depth when building mapping of group depths to avoid hitting recursion limit.  Fixes https://trello.com/c/2zc0odvX
  Fix .ini web links for v2
  Add support for detecting encrypted openssh format private keys. Fixes https://trello.com/c/ZeVOXN5U
  ...
2015-08-20 11:47:07 -04:00
Matthew Jones a76b1a0432 Merge branch 'release_2.2.3' into stable
* release_2.2.3:
  Bump version to 2.2.3
  Optimize inventory read check for Job Templates
2015-08-20 11:15:48 -04:00
Matthew Jones f66ed43788 Bump version to 2.2.3 2015-08-19 13:26:55 -04:00
Matthew Jones 2bdc79f3ba Optimize inventory read check for Job Templates
Our old check used an extremely inefficient boolean logic AND and
generated a bad query.   This does a simpler check by querying the
inventories that the user has access to and using that as a simple
condition when performing the job template permission query
2015-08-19 13:26:10 -04:00
Matthew Jones 26f942a593 Merge remote-tracking branch 'upstream/release_2.2' into stable
* upstream/release_2.2: (31 commits)
  Updating changelog for 2.2 release
  Bump version to 2.2.2
  Use smart_str instead of .encode on passwords
  Change default sensitive redaction behavior
  Update changelogs for 2.2.1
  Add an error message when running a job and trying to use an OpenSSH formatted key on an older version of OpenSSH.
  Revert "Add sudo: true to the backup playbook"
  Handle Ubuntu 12.04 psutil recursive cancel
  fixed log viewer modal partial
  fixed alignment of download standard out button in modals
  About Tower version number fix
  Pin package version for older release
  add standard out download button to various places in the UI
  Bump 2.2 branch to version 2.2.1
  Don't create a group that is its own parent when an EC2 tag has an empty value.  Fixes https://trello.com/c/2zc0odvX
  Limit max depth when building mapping of group depths to avoid hitting recursion limit.  Fixes https://trello.com/c/2zc0odvX
  Fix .ini web links for v2
  Add support for detecting encrypted openssh format private keys. Fixes https://trello.com/c/ZeVOXN5U
  Fix psutil usage on el6 for job cancel
  Set recursive on child process canceling
  ...
2015-08-19 10:39:54 -04:00
Matthew Jones 35ffb37492 Add cleaning site-packages in the Makefile 2015-08-17 11:33:45 -04:00
Matthew Jones ee8a76f32b Use smart_str instead of .encode on passwords
We originally tried to prune these .encode() calls out of the tree in
favor of smart_str but we must have missed this one.   This allows
migration 0066 to work if utf-8 characters are used in the password
2015-08-13 17:00:46 -04:00
Matthew Jones e221097a37 Use smart_str instead of .encode on passwords
We originally tried to prune these .encode() calls out of the tree in
favor of smart_str but we must have missed this one.   This allows
migration 0066 to work if utf-8 characters are used in the password
2015-08-13 16:55:39 -04:00
Matthew Jones d4dccff20c Change default sensitive redaction behavior
* Only project updates will contain sensitive data redaction logic when
  displaying stdout
2015-08-13 15:48:56 -04:00
Matthew Jones 6c2d6a63e7 Change default sensitive redaction behavior
* Only project updates will contain sensitive data redaction logic when
  displaying stdout
2015-08-13 15:48:16 -04:00
Matthew Jones 4968848958 Merge pull request #371 from cchurch/openssh_key_check
Add error message when OpenSSH private keys aren't supported
2015-08-11 16:53:08 -04:00
Matthew Jones 21acaa999d Handle Ubuntu 12.04 psutil recursive cancel
Ubuntu has a crazy old version of psutil, we'll fall back to sigkill for
that platform
2015-08-11 10:32:29 -04:00
Matthew Jones 9aa89e08d9 Handle Ubuntu 12.04 psutil recursive cancel
Ubuntu has a crazy old version of psutil, we'll fall back to sigkill for
that platform
2015-08-11 10:24:54 -04:00
Matthew Jones 43d873acab Fix path check in start_development docker script 2015-08-10 14:36:39 -04:00
Matthew Jones e5bd721b4b Add cleaning site-packages in the Makefile 2015-08-10 11:02:51 -04:00
Matthew Jones a992881b17 Install honcho in the docker development container 2015-08-10 11:00:27 -04:00
Matthew Jones 5969c6a380 Check for site-packages and if it doesn't exist symlink to the built directory 2015-08-10 10:24:29 -04:00
Matthew Jones 7484267186 Bump 2.2 branch to version 2.2.1 2015-08-07 11:19:09 -04:00
Matthew Jones 1e95ac1c54 Merge pull request #361 from cchurch/recursive_inventory
Fix inventory issues when a group is its own parent
2015-08-06 21:30:45 -04:00
Matthew Jones 3b96c3ca52 Fix psutil usage on el6 for job cancel
psutil will fail to import on el6 due to not being able to access a
pseudo terminal.  This issues a SIGKILL to the proot process in order to
force the stop
2015-08-05 22:01:44 -04:00
Matthew Jones 31b8491d53 Set recursive on child process canceling
Sometimes ansible spawns many subprocesses that can get orphaned and
stuck if we only kill the direct descendent child processes
2015-08-05 22:00:27 -04:00
Matthew Jones 57778eb429 mock stdout unit test patch failure fix
Adjust some mock.patch behavior to match the new stdout behavior where
we need to check a file's size as well as its existence
2015-08-05 21:55:02 -04:00
Matthew Jones b1c12ca120 Fix empty standard out behavior
Previously we could just check for the absence of the stdout file to
know we didn't yet/wouldn't ever have any stdout content.  Now that we
are creating the stdout file ahead of time and persisting it we need to
also check to see if the file contains any data.   This is also
necessary for some unit tests.
2015-08-05 21:54:23 -04:00
Matthew Jones dc0d732ad4 Fix merge issues on stdout branch
The act of committing a PR that including part of these changes and then
reverting it meant that some of the changes from the original branch
didn't land in this new PR.   This commit adds the missing bits.

Lesson learned:  Create a new branch when submitting a PR on a
previously reverted PR
2015-08-05 21:53:46 -04:00
Matthew Jones c0ea174e93 Rework stdout display for file storage only
* Cleanup old code related to stdout download using temporary files
* Remove copy-into-database code
* Modify stdout download code to reference stdout file
2015-08-05 21:52:47 -04:00
Matthew Jones f20d93a274 Merge pull request #358 from cchurch/openssh_private_key_support
Support private keys in OpenSSH format
2015-08-05 19:44:56 -04:00
Matthew Jones a7ae1d8348 Merge pull request #359 from chrismeyersfsu/requirements.txt3
all vendor dependencies expressed in requirements.txt
2015-08-05 19:44:16 -04:00
Matthew Jones 64c3b1ec69 Merge pull request #345 from chrismeyersfsu/fix-centos_6.5
unittest works on centos 6.5
2015-08-05 19:43:57 -04:00
Matthew Jones 91bd96f458 Merge pull request #360 from matburt/fix_el6_job_cancel
Fix overall job cancel behavior
2015-08-05 19:39:11 -04:00
Matthew Jones ab042c269b Set recursive on child process canceling
Sometimes ansible spawns many subprocesses that can get orphaned and
stuck if we only kill the direct descendent child processes
2015-08-05 11:44:50 -04:00
Matthew Jones efcff9b5f7 Fix psutil usage on el6 for job cancel
psutil will fail to import on el6 due to not being able to access a
pseudo terminal.  This issues a SIGKILL to the proot process in order to
force the stop
2015-08-05 11:20:48 -04:00
Matthew Jones 14df41b47b mock stdout unit test patch failure fix
Adjust some mock.patch behavior to match the new stdout behavior where
we need to check a file's size as well as its existence
2015-08-05 10:02:07 -04:00
Matthew Jones 7e45ee8b55 Fix empty standard out behavior
Previously we could just check for the absence of the stdout file to
know we didn't yet/wouldn't ever have any stdout content.  Now that we
are creating the stdout file ahead of time and persisting it we need to
also check to see if the file contains any data.   This is also
necessary for some unit tests.
2015-08-04 14:25:05 -04:00
Matthew Jones c231601b76 Fix merge issues on stdout branch
The act of committing a PR that including part of these changes and then
reverting it meant that some of the changes from the original branch
didn't land in this new PR.   This commit adds the missing bits.

Lesson learned:  Create a new branch when submitting a PR on a
previously reverted PR
2015-08-04 09:57:39 -04:00
Matthew Jones bddcb06fab Merge branch 'stdout_file_download' into upstream_master
* stdout_file_download:
  Rework stdout display for file storage only
2015-08-03 15:30:17 -04:00
Matthew Jones 1c777c5441 Rework stdout display for file storage only
* Cleanup old code related to stdout download using temporary files
* Remove copy-into-database code
* Modify stdout download code to reference stdout file
2015-08-03 15:14:51 -04:00
Matthew Jones 2f05ac7276 Merge pull request #349 from matburt/master
Optimize inventory read check for Job Templates
2015-08-03 09:57:51 -04:00
Matthew Jones 47a3e69636 Optimize inventory read check for Job Templates
Our old check used an extremely inefficient boolean logic AND and
generated a bad query.   This does a simpler check by querying the
inventories that the user has access to and using that as a simple
condition when performing the job template permission query
2015-07-31 00:06:12 -04:00
Matthew Jones ef443ec8b2 Add psutil dependency to the docker dev container 2015-07-28 13:30:59 -04:00
Matthew Jones e9370d0199 Merge pull request #326 from matburt/fix_cancel_behavior_in_proot
Fix proot job cancel behavior
2015-07-27 20:49:59 -04:00
Matthew Jones 4f73ba9e3e Merge pull request #329 from ansible/revert-320-stdout_file_download
Revert "Improve stdout behavior by implementing a downloadable stdout."
2015-07-24 14:54:36 -04:00
Matthew Jones 5df23b95f6 Revert "Improve stdout behavior by implementing a downloadable stdout." 2015-07-24 14:54:13 -04:00