External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-05-19 14:57:39 -02:30
Code Issues Packages Projects Releases Wiki Activity

Configure Tower in Tower:

Browse Source 
* Add separate Django app for configuration: awx.conf.
* Migrate from existing main.TowerSettings model to conf.Setting.
* Add settings wrapper to allow get/set/del via django.conf.settings.
* Update existing references to tower_settings to use django.conf.settings.
* Add a settings registry to allow for each Django app to register configurable settings.
* Support setting validation and conversion using Django REST Framework fields.
* Add /api/v1/settings/ to display a list of setting categories.
* Add /api/v1/settings/<slug>/ to display all settings in a category as a single object.
* Allow PUT/PATCH to update setting singleton, DELETE to reset to defaults.
* Add "all" category to display all settings across categories.
* Add "changed" category to display only settings configured in the database.
* Support per-user settings via "user" category (/api/v1/settings/user/).
* Support defaults for user settings via "user-defaults" category (/api/v1/settings/user-defaults/).
* Update serializer metadata to support category, category_slug and placeholder on OPTIONS responses.
* Update serializer metadata to handle child fields of a list/dict.
* Hide raw data form in browsable API for OPTIONS and DELETE.
* Combine existing licensing code into single "TaskEnhancer" class.
* Move license helper functions from awx.api.license into awx.conf.license.
* Update /api/v1/config/ to read/verify/update license using TaskEnhancer and settings wrapper.
* Add support for caching settings accessed via settings wrapper.
* Invalidate cached settings when Setting model changes or is deleted.
* Preload all database settings into cache on first access via settings wrapper.
* Add support for read-only settings than can update their value depending on other settings.
* Use setting_changed signal whenever a setting changes.
* Register configurable authentication, jobs, system and ui settings.
* Register configurable LDAP, RADIUS and social auth settings.
* Add custom fields and validators for URL, LDAP, RADIUS and social auth settings.
* Rewrite existing validator for Credential ssh_private_key to support validating private keys, certs or combinations of both.
* Get all unit/functional tests working with above changes.
* Add "migrate_to_database_settings" command to determine settings to be migrated into the database and comment them out when set in Python settings files.
* Add support for migrating license key from file to database.
* Remove database-configuable settings from local_settings.py example files.
* Update setup role to no longer install files for database-configurable settings.

f 94ff6ee More settings work.
f af4c4e0 Even more db settings stuff.
f 96ea9c0 More settings, attempt at singleton serializer for settings.
f 937c760 More work on singleton/category views in API, add code to comment out settings in Python files, work on command to migrate settings to database.
f 425b0d3 Minor fixes for sprint demo.
f ea402a4 Add support for read-only settings, cleanup license engine, get license support working with DB settings.
f ec289e4 Rename migration, minor fixmes, update setup role.
f 603640b Rewrite key/cert validator, finish adding social auth fields, hook up signals for setting_changed, use None to imply a setting is not set.
f 67d1b5a Get functional/unit tests passing.
f 2919b62 Flake8 fixes.
f e62f421 Add redbaron to requirements, get file to database migration working (except for license).
f c564508 Add support for migrating license file.
f 982f767 Add support for regex in social map fields.
This commit is contained in:
Chris Church
2016-09-26 22:14:47 -04:00
parent 609a3e6f2f
commit 6ebe45b1bd
92 changed files with 4401 additions and 1791 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
awx/main/__init__.py
View File

@@ -1,2 +1,4 @@
# Copyright (c) 2015 Ansible, Inc.
# All Rights Reserved.
default_app_config = 'awx.main.apps.MainConfig'

26
awx/main/access.py
View File

@@ -7,6 +7,7 @@ import sys
import logging
# Django
from django.conf import settings
from django.db.models import Q
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType
@@ -19,9 +20,8 @@ from awx.main.utils import * # noqa
from awx.main.models import * # noqa
from awx.main.models.unified_jobs import ACTIVE_STATES
from awx.main.models.mixins import ResourceMixin
from awx.api.license import LicenseForbids
from awx.main.task_engine import TaskSerializer
from awx.main.conf import tower_settings
from awx.main.task_engine import TaskEnhancer
from awx.conf.license import LicenseForbids
__all__ = ['get_user_queryset', 'check_user_access',
'user_accessible_objects',
@@ -192,8 +192,7 @@ class BaseAccess(object):
return self.can_change(obj, data)
def check_license(self, add_host=False, feature=None, check_expiration=True):
reader = TaskSerializer()
validation_info = reader.from_database()
validation_info = TaskEnhancer().validate_enhancements()
if ('test' in sys.argv or 'py.test' in sys.argv[0] or 'jenkins' in sys.argv) and not os.environ.get('SKIP_LICENSE_FIXUP_FOR_TEST', ''):
validation_info['free_instances'] = 99999999
validation_info['time_remaining'] = 99999999
@@ -311,7 +310,7 @@ class UserAccess(BaseAccess):
if self.user.is_superuser or self.user.is_system_auditor:
return User.objects.all()
if tower_settings.ORG_ADMINS_CAN_SEE_ALL_USERS and \
if settings.ORG_ADMINS_CAN_SEE_ALL_USERS and \
(self.user.admin_of_organizations.exists() or self.user.auditor_of_organizations.exists()):
return User.objects.all()
@@ -1919,20 +1918,6 @@ class CustomInventoryScriptAccess(BaseAccess):
def can_delete(self, obj):
return self.can_admin(obj)
class TowerSettingsAccess(BaseAccess):
'''
- I can see settings when
- I am a super user
- I can edit settings when
- I am a super user
- I can clear settings when
- I am a super user
'''
model = TowerSettings
class RoleAccess(BaseAccess):
'''
- I can see roles when
@@ -2009,7 +1994,6 @@ register_access(UnifiedJobTemplate, UnifiedJobTemplateAccess)
register_access(UnifiedJob, UnifiedJobAccess)
register_access(ActivityStream, ActivityStreamAccess)
register_access(CustomInventoryScript, CustomInventoryScriptAccess)
register_access(TowerSettings, TowerSettingsAccess)
register_access(Role, RoleAccess)
register_access(NotificationTemplate, NotificationTemplateAccess)
register_access(Notification, NotificationAccess)

9
awx/main/apps.py Normal file
View File

@@ -0,0 +1,9 @@
# Django
from django.apps import AppConfig
from django.utils.translation import ugettext_lazy as _
class MainConfig(AppConfig):
name = 'awx.main'
verbose_name = _('Main')

209
awx/main/conf.py
View File

@@ -1,50 +1,175 @@
# Copyright (c) 2015 Ansible, Inc..
# All Rights Reserved.
# Python
import json
import logging
import os
from django.conf import settings as django_settings
from django.db.utils import ProgrammingError
from django.db import OperationalError
from awx.main.models.configuration import TowerSettings
# Django
from django.utils.translation import ugettext_lazy as _
# Tower
from awx.conf import fields, register
logger = logging.getLogger('awx.main.conf')
class TowerConfiguration(object):
register(
'ACTIVITY_STREAM_ENABLED',
field_class=fields.BooleanField,
label=_('Enable Activity Stream'),
help_text=_('Enable capturing activity for the Tower activity stream.'),
category=_('System'),
category_slug='system',
)
# TODO: Caching so we don't have to hit the database every time for settings
def __getattr__(self, key):
settings_manifest = django_settings.TOWER_SETTINGS_MANIFEST
if key not in settings_manifest:
raise AttributeError("Tower Setting with key '{0}' is not defined in the manifest".format(key))
default_value = settings_manifest[key]['default']
ts = TowerSettings.objects.filter(key=key)
try:
if not ts.exists():
try:
val_actual = getattr(django_settings, key)
except AttributeError:
val_actual = default_value
return val_actual
return ts[0].value_converted
except (ProgrammingError, OperationalError), e:
# Database is not available yet, usually during migrations so lets use the default
logger.debug("Database settings not available yet, using defaults ({0})".format(e))
return default_value
register(
'ACTIVITY_STREAM_ENABLED_FOR_INVENTORY_SYNC',
field_class=fields.BooleanField,
label=_('Enable Activity Stream for Inventory Sync'),
help_text=_('Enable capturing activity for the Tower activity stream when running inventory sync.'),
category=_('System'),
category_slug='system',
)
def __setattr__(self, key, value):
settings_manifest = django_settings.TOWER_SETTINGS_MANIFEST
if key not in settings_manifest:
raise AttributeError("Tower Setting with key '{0}' does not exist".format(key))
settings_entry = settings_manifest[key]
try:
settings_actual = TowerSettings.objects.get(key=key)
except TowerSettings.DoesNotExist:
settings_actual = TowerSettings(key=key,
description=settings_entry['description'],
category=settings_entry['category'],
value_type=settings_entry['type'])
settings_actual.value_converted = value
settings_actual.save()
register(
'ORG_ADMINS_CAN_SEE_ALL_USERS',
field_class=fields.BooleanField,
label=_('All Users Visible to Organization Admins'),
help_text=_('Controls whether any Organization Admin can view all users, even those not associated with their Organization.'),
category=_('System'),
category_slug='system',
)
tower_settings = TowerConfiguration()
register(
'TOWER_ADMIN_ALERTS',
field_class=fields.BooleanField,
label=_('Enable Tower Administrator Alerts'),
help_text=_('Allow Tower to email Admin users for system events that may require attention.'),
category=_('System'),
category_slug='system',
)
register(
'TOWER_URL_BASE',
field_class=fields.URLField,
schemes=('http', 'https'),
allow_plain_hostname=True, # Allow hostname only without TLD.
label=_('Base URL of the Tower host'),
help_text=_('This setting is used by services like notifications to render '
'a valid url to the Tower host.'),
category=_('System'),
category_slug='system',
)
register(
'REMOTE_HOST_HEADERS',
field_class=fields.StringListField,
label=_('Remote Host Headers'),
help_text=_('HTTP headers and meta keys to search to determine remote host '
'name or IP. Add additional items to this list, such as '
'"HTTP_X_FORWARDED_FOR", if behind a reverse proxy.\n\n'
'Note: The headers will be searched in order and the first '
'found remote host name or IP will be used.\n\n'
'In the below example 8.8.8.7 would be the chosen IP address.\n'
'X-Forwarded-For: 8.8.8.7, 192.168.2.1, 127.0.0.1\n'
'Host: 127.0.0.1\n'
'REMOTE_HOST_HEADERS = [\'HTTP_X_FORWARDED_FOR\', '
'\'REMOTE_ADDR\', \'REMOTE_HOST\']'),
category=_('System'),
category_slug='system',
)
def _load_default_license_from_file():
try:
license_file = os.environ.get('AWX_LICENSE_FILE', '/etc/tower/license')
if os.path.exists(license_file):
license_data = json.load(open(license_file))
logger.debug('Read license data from "%s".', license_file)
return license_data
except:
logger.warning('Could not read license from "%s".', license_file, exc_info=True)
return {}
register(
'LICENSE',
field_class=fields.DictField,
default=_load_default_license_from_file,
label=_('Tower License'),
help_text=_('The license controls which features and functionality are '
'enabled in Tower. Use /api/v1/config/ to update or change '
'the license.'),
category=_('System'),
category_slug='system',
)
register(
'AD_HOC_COMMANDS',
field_class=fields.StringListField,
label=_('Ansible Modules Allowed for Ad Hoc Jobs'),
help_text=_('List of modules allowed to be used by ad-hoc jobs.'),
category=_('Jobs'),
category_slug='jobs',
)
register(
'AWX_PROOT_ENABLED',
field_class=fields.BooleanField,
label=_('Enable PRoot for Job Execution'),
help_text=_('Isolates an Ansible job from protected parts of the Tower system to prevent exposing sensitive information.'),
category=_('Jobs'),
category_slug='jobs',
)
register(
'AWX_PROOT_BASE_PATH',
field_class=fields.CharField,
label=_('Base PRoot execution path'),
help_text=_('The location that PRoot will create its temporary working directory.'),
category=_('Jobs'),
category_slug='jobs',
)
register(
'AWX_PROOT_HIDE_PATHS',
field_class=fields.StringListField,
label=_('Paths to hide from PRoot jobs'),
help_text=_('Extra paths to hide from PRoot isolated processes.'),
category=_('Jobs'),
category_slug='jobs',
)
register(
'AWX_PROOT_SHOW_PATHS',
field_class=fields.StringListField,
label=_('Paths to expose to PRoot jobs'),
help_text=_('Explicit whitelist of paths to expose to PRoot jobs.'),
category=_('Jobs'),
category_slug='jobs',
)
register(
'STDOUT_MAX_BYTES_DISPLAY',
field_class=fields.IntegerField,
min_value=0,
label=_('Standard Output Maximum Display Size'),
help_text=_('Maximum Size of Standard Output in bytes to display before requiring the output be downloaded.'),
category=_('Jobs'),
category_slug='jobs',
)
register(
'SCHEDULE_MAX_JOBS',
field_class=fields.IntegerField,
min_value=1,
label=_('Maximum Scheduled Jobs'),
help_text=_('Maximum number of the same job template that can be waiting to run when launching from a schedule before no more are created.'),
category=_('Jobs'),
category_slug='jobs',
)
register(
'AWX_ANSIBLE_CALLBACK_PLUGINS',
field_class=fields.StringListField,
label=_('Ansible Callback Plugins'),
help_text=_('List of paths for extra callback plugins to be used when running jobs.'),
category=_('Jobs'),
category_slug='jobs',
)

2
awx/main/management/commands/cleanup_facts.py
View File

@@ -13,7 +13,7 @@ from django.utils.timezone import now
# AWX
from awx.main.models.fact import Fact
from awx.api.license import feature_enabled
from awx.conf.license import feature_enabled
OLDER_THAN = 'older_than'
GRANULARITY = 'granularity'

8
awx/main/management/commands/inventory_import.py
View File

@@ -26,10 +26,9 @@ from django.utils.encoding import smart_text
# AWX
from awx.main.models import * # noqa
from awx.main.task_engine import TaskEnhancer
from awx.main.utils import ignore_inventory_computed_fields, check_proot_installed, wrap_args_with_proot
from awx.main.signals import disable_activity_stream
from awx.main.task_engine import TaskSerializer as LicenseReader
from awx.main.conf import tower_settings
logger = logging.getLogger('awx.main.commands.inventory_import')
@@ -358,7 +357,7 @@ class ExecutableJsonLoader(BaseLoader):
data = {}
stdout, stderr = '', ''
try:
if self.is_custom and getattr(tower_settings, 'AWX_PROOT_ENABLED', False):
if self.is_custom and getattr(settings, 'AWX_PROOT_ENABLED', False):
if not check_proot_installed():
raise RuntimeError("proot is not installed but is configured for use")
kwargs = {'proot_temp_dir': self.source_dir} # TODO: Remove proot dir
@@ -1191,8 +1190,7 @@ class Command(NoArgsCommand):
self._create_update_group_hosts()
def check_license(self):
reader = LicenseReader()
license_info = reader.from_database()
license_info = TaskEnhancer().validate_enhancements()
if not license_info or len(license_info) == 0:
self.logger.error(LICENSE_NON_EXISTANT_MESSAGE)
raise CommandError('No Tower license found!')

2
awx/main/management/commands/update_instance.py
View File

@@ -5,7 +5,7 @@ from django.core.management.base import CommandError
from django.db import transaction
from awx.main.management.commands._base_instance import BaseCommandInstance
from awx.api.license import feature_enabled
from awx.conf.license import feature_enabled
from awx.main.models import Instance
instance_str = BaseCommandInstance.instance_str

4
awx/main/middleware.py
View File

@@ -5,13 +5,13 @@ import logging
import threading
import uuid
from django.conf import settings
from django.contrib.auth.models import User
from django.db.models.signals import post_save
from django.db import IntegrityError
from django.utils.functional import curry
from awx.main.models import ActivityStream
from awx.main.conf import tower_settings
from awx.api.authentication import TokenAuthentication
@@ -79,6 +79,6 @@ class AuthTokenTimeoutMiddleware(object):
if not TokenAuthentication._get_x_auth_token_header(request):
return response
response['Auth-Token-Timeout'] = int(tower_settings.AUTH_TOKEN_EXPIRATION)
response['Auth-Token-Timeout'] = int(settings.AUTH_TOKEN_EXPIRATION)
return response

22
awx/main/migrations/0036_v310_remove_tower_settings.py Normal file
View File

@@ -0,0 +1,22 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('main', '0035_v310_jobevent_uuid'),
]
# These settings are now in the separate awx.conf app.
operations = [
migrations.RemoveField(
model_name='towersettings',
name='user',
),
migrations.DeleteModel(
name='TowerSettings',
),
]

35
awx/main/migrations/_old_access.py
View File

@@ -13,6 +13,7 @@ import sys
import logging
# Django
from django.conf import settings
from django.db.models import F, Q
from django.contrib.auth.models import User
@@ -22,9 +23,7 @@ from rest_framework.exceptions import ParseError, PermissionDenied
# AWX
from awx.main.utils import * # noqa
from awx.main.models import * # noqa
from awx.api.license import LicenseForbids
from awx.main.task_engine import TaskSerializer
from awx.main.conf import tower_settings
from awx.conf.license import LicenseForbids
__all__ = ['get_user_queryset', 'check_user_access']
@@ -153,8 +152,8 @@ class BaseAccess(object):
return self.can_change(obj, None)
def check_license(self, add_host=False, feature=None, check_expiration=True):
reader = TaskSerializer()
validation_info = reader.from_database()
from awx.main.task_engine import TaskEnhancer
validation_info = TaskEnhancer().validate_enhancements()
if ('test' in sys.argv or 'py.test' in sys.argv[0] or 'jenkins' in sys.argv) and not os.environ.get('SKIP_LICENSE_FIXUP_FOR_TEST', ''):
validation_info['free_instances'] = 99999999
validation_info['time_remaining'] = 99999999
@@ -202,7 +201,7 @@ class UserAccess(BaseAccess):
qs = self.model.objects.distinct()
if self.user.is_superuser:
return qs
if tower_settings.ORG_ADMINS_CAN_SEE_ALL_USERS and self.user.deprecated_admin_of_organizations.all().exists():
if settings.ORG_ADMINS_CAN_SEE_ALL_USERS and self.user.deprecated_admin_of_organizations.all().exists():
return qs
return qs.filter(
Q(pk=self.user.pk) |
@@ -1624,29 +1623,6 @@ class CustomInventoryScriptAccess(BaseAccess):
return False
class TowerSettingsAccess(BaseAccess):
'''
- I can see settings when
- I am a super user
- I can edit settings when
- I am a super user
- I can clear settings when
- I am a super user
'''
model = TowerSettings
def get_queryset(self):
if self.user.is_superuser:
return self.model.objects.all()
return self.model.objects.none()
def can_change(self, obj, data):
return self.user.is_superuser
def can_delete(self, obj):
return self.user.is_superuser
register_access(User, UserAccess)
register_access(Organization, OrganizationAccess)
register_access(Inventory, InventoryAccess)
@@ -1672,4 +1648,3 @@ register_access(UnifiedJobTemplate, UnifiedJobTemplateAccess)
register_access(UnifiedJob, UnifiedJobAccess)
register_access(ActivityStream, ActivityStreamAccess)
register_access(CustomInventoryScript, CustomInventoryScriptAccess)
register_access(TowerSettings, TowerSettingsAccess)

2
awx/main/models/__init__.py
View File

@@ -16,7 +16,6 @@ from awx.main.models.ad_hoc_commands import * # noqa
from awx.main.models.schedules import * # noqa
from awx.main.models.activity_stream import * # noqa
from awx.main.models.ha import * # noqa
from awx.main.models.configuration import * # noqa
from awx.main.models.rbac import * # noqa
from awx.main.models.mixins import * # noqa
from awx.main.models.notifications import * # noqa
@@ -99,7 +98,6 @@ activity_stream_registrar.connect(AdHocCommand)
# activity_stream_registrar.connect(Profile)
activity_stream_registrar.connect(Schedule)
activity_stream_registrar.connect(CustomInventoryScript)
activity_stream_registrar.connect(TowerSettings)
activity_stream_registrar.connect(NotificationTemplate)
activity_stream_registrar.connect(Notification)
activity_stream_registrar.connect(Label)

5
awx/main/models/ad_hoc_commands.py
View File

@@ -22,7 +22,6 @@ from jsonfield import JSONField
from awx.main.models.base import * # noqa
from awx.main.models.unified_jobs import * # noqa
from awx.main.utils import decrypt_field
from awx.main.conf import tower_settings
from awx.main.models.notifications import JobNotificationMixin
logger = logging.getLogger('awx.main.models.ad_hoc_commands')
@@ -115,7 +114,7 @@ class AdHocCommand(UnifiedJob, JobNotificationMixin):
if type(self.module_name) not in (str, unicode):
raise ValidationError("Invalid type for ad hoc command")
module_name = self.module_name.strip() or 'command'
if module_name not in tower_settings.AD_HOC_COMMANDS:
if module_name not in settings.AD_HOC_COMMANDS:
raise ValidationError('Unsupported module for ad hoc commands.')
return module_name
@@ -148,7 +147,7 @@ class AdHocCommand(UnifiedJob, JobNotificationMixin):
return reverse('api:ad_hoc_command_detail', args=(self.pk,))
def get_ui_url(self):
return urljoin(tower_settings.TOWER_URL_BASE, "/#/ad_hoc_commands/{}".format(self.pk))
return urljoin(settings.TOWER_URL_BASE, "/#/ad_hoc_commands/{}".format(self.pk))
@property
def task_auth_token(self):

84
awx/main/models/configuration.py
View File

@@ -1,84 +0,0 @@
# Copyright (c) 2015 Ansible, Inc.
# All Rights Reserved.
# Python
import json
# Django
from django.db import models
from django.utils.encoding import force_text
from django.utils.translation import ugettext_lazy as _
# Tower
from awx.main.models.base import CreatedModifiedModel
class TowerSettings(CreatedModifiedModel):
class Meta:
app_label = 'main'
SETTINGS_TYPE_CHOICES = [
('string', _("String")),
('int', _('Integer')),
('float', _('Decimal')),
('json', _('JSON')),
('bool', _('Boolean')),
('password', _('Password')),
('list', _('List'))
]
key = models.CharField(
max_length=255,
unique=True
)
description = models.TextField()
category = models.CharField(max_length=128)
value = models.TextField(
blank=True,
)
value_type = models.CharField(
max_length=12,
choices=SETTINGS_TYPE_CHOICES
)
user = models.ForeignKey(
'auth.User',
related_name='settings',
default=None,
null=True,
editable=False,
)
@property
def value_converted(self):
if self.value_type == 'json':
converted_type = json.loads(self.value)
elif self.value_type == 'password':
converted_type = self.value
elif self.value_type == 'list':
if self.value:
converted_type = [x.strip() for x in self.value.split(',')]
else:
converted_type = []
elif self.value_type == 'bool':
converted_type = force_text(self.value).lower() in ('true', 'yes', '1')
elif self.value_type == 'string':
converted_type = self.value
else:
t = __builtins__[self.value_type]
converted_type = t(self.value)
return converted_type
@value_converted.setter
def value_converted(self, value):
if self.value_type == 'json':
self.value = json.dumps(value)
elif self.value_type == 'list':
try:
self.value = ','.join(map(force_text, value))
except TypeError:
self.value = force_text(value)
elif self.value_type == 'bool':
self.value = force_text(bool(value))
else:
self.value = force_text(value)

137
awx/main/models/credential.py
View File

@@ -1,9 +1,6 @@
# Copyright (c) 2015 Ansible, Inc.
# All Rights Reserved.
import base64
import re
# Django
from django.db import models
from django.utils.translation import ugettext_lazy as _
@@ -14,6 +11,7 @@ from django.core.urlresolvers import reverse
from awx.main.fields import ImplicitRoleField
from awx.main.constants import CLOUD_PROVIDERS
from awx.main.utils import decrypt_field
from awx.main.validators import validate_ssh_private_key
from awx.main.models.base import * # noqa
from awx.main.models.mixins import ResourceMixin
from awx.main.models.rbac import (
@@ -241,11 +239,13 @@ class Credential(PasswordFieldsModel, CommonModelNameNotUnique, ResourceMixin):
else:
ssh_key_data = self.ssh_key_data
try:
key_data = validate_ssh_private_key(ssh_key_data)
pem_objects = validate_ssh_private_key(ssh_key_data)
for pem_object in pem_objects:
if pem_object.get('key_enc', False):
return True
except ValidationError:
return False
else:
return bool(key_data['key_enc'])
pass
return False
@property
def needs_ssh_key_unlock(self):
@@ -379,126 +379,3 @@ class Credential(PasswordFieldsModel, CommonModelNameNotUnique, ResourceMixin):
if 'cloud' not in update_fields:
update_fields.append('cloud')
super(Credential, self).save(*args, **kwargs)
def validate_ssh_private_key(data):
"""Validate that the given SSH private key or certificate is,
in fact, valid.
"""
# Map the X in BEGIN X PRIVATE KEY to the key type (ssh-keygen -t).
# Tower jobs using OPENSSH format private keys may still fail if the
# system SSH implementation lacks support for this format.
key_types = {
'RSA': 'rsa',
'DSA': 'dsa',
'EC': 'ecdsa',
'OPENSSH': 'ed25519',
'': 'rsa1',
}
# Key properties to return if valid.
key_data = {
'key_type': None, # Key type (from above mapping).
'key_seg': '', # Key segment (all text including begin/end).
'key_b64': '', # Key data as base64.
'key_bin': '', # Key data as binary.
'key_enc': None, # Boolean, whether key is encrypted.
'cert_seg': '', # Cert segment (all text including begin/end).
'cert_b64': '', # Cert data as base64.
'cert_bin': '', # Cert data as binary.
}
data = data.strip()
validation_error = ValidationError('Invalid private key.')
# Sanity check: We may potentially receive a full PEM certificate,
# and we want to accept these.
cert_begin_re = r'(-{4,})\s*BEGIN\s+CERTIFICATE\s*(-{4,})'
cert_end_re = r'(-{4,})\s*END\s+CERTIFICATE\s*(-{4,})'
cert_begin_match = re.search(cert_begin_re, data)
cert_end_match = re.search(cert_end_re, data)
if cert_begin_match and not cert_end_match:
raise validation_error
elif not cert_begin_match and cert_end_match:
raise validation_error
elif cert_begin_match and cert_end_match:
cert_dashes = set([cert_begin_match.groups()[0], cert_begin_match.groups()[1],
cert_end_match.groups()[0], cert_end_match.groups()[1]])
if len(cert_dashes) != 1:
raise validation_error
key_data['cert_seg'] = data[cert_begin_match.start():cert_end_match.end()]
# Find the private key, and also ensure that it internally matches
# itself.
# Set up the valid private key header and footer.
begin_re = r'(-{4,})\s*BEGIN\s+([A-Z0-9]+)?\s*PRIVATE\sKEY\s*(-{4,})'
end_re = r'(-{4,})\s*END\s+([A-Z0-9]+)?\s*PRIVATE\sKEY\s*(-{4,})'
begin_match = re.search(begin_re, data)
end_match = re.search(end_re, data)
if not begin_match or not end_match:
raise validation_error
# Ensure that everything, such as dash counts and key type, lines up,
# and raise an error if it does not.
dashes = set([begin_match.groups()[0], begin_match.groups()[2],
end_match.groups()[0], end_match.groups()[2]])
if len(dashes) != 1:
raise validation_error
if begin_match.groups()[1] != end_match.groups()[1]:
raise validation_error
key_type = begin_match.groups()[1] or ''
try:
key_data['key_type'] = key_types[key_type]
except KeyError:
raise ValidationError('Invalid private key: unsupported type %s' % key_type)
# The private key data begins and ends with the private key.
key_data['key_seg'] = data[begin_match.start():end_match.end()]
# Establish that we are able to base64 decode the private key;
# if we can't, then it's not a valid key.
#
# If we got a certificate, validate that also, in the same way.
header_re = re.compile(r'^(.+?):\s*?(.+?)(\\??)$')
for segment_name in ('cert', 'key'):
segment_to_validate = key_data['%s_seg' % segment_name]
# If we have nothing; skip this one.
# We've already validated that we have a private key above,
# so we don't need to do it again.
if not segment_to_validate:
continue
# Ensure that this segment is valid base64 data.
base64_data = ''
line_continues = False
lines = segment_to_validate.splitlines()
for line in lines[1:-1]:
line = line.strip()
if not line:
continue
if line_continues:
line_continues = line.endswith('\\')
continue
line_match = header_re.match(line)
if line_match:
line_continues = line.endswith('\\')
continue
base64_data += line
try:
decoded_data = base64.b64decode(base64_data)
if not decoded_data:
raise validation_error
key_data['%s_b64' % segment_name] = base64_data
key_data['%s_bin' % segment_name] = decoded_data
except TypeError:
raise validation_error
# Determine if key is encrypted.
if key_data['key_type'] == 'ed25519':
# See https://github.com/openssh/openssh-portable/blob/master/sshkey.c#L3218
# Decoded key data starts with magic string (null-terminated), four byte
# length field, followed by the ciphername -- if ciphername is anything
# other than 'none' the key is encrypted.
key_data['key_enc'] = not bool(key_data['key_bin'].startswith('openssh-key-v1\x00\x00\x00\x00\x04none'))
else:
key_data['key_enc'] = bool('ENCRYPTED' in key_data['key_seg'])
return key_data

3
awx/main/models/inventory.py
View File

@@ -30,7 +30,6 @@ from awx.main.models.notifications import (
JobNotificationMixin,
)
from awx.main.utils import _inventory_updates
from awx.main.conf import tower_settings
__all__ = ['Inventory', 'Host', 'Group', 'InventorySource', 'InventoryUpdate', 'CustomInventoryScript']
@@ -1244,7 +1243,7 @@ class InventoryUpdate(UnifiedJob, InventorySourceOptions, JobNotificationMixin):
return reverse('api:inventory_update_detail', args=(self.pk,))
def get_ui_url(self):
return urljoin(tower_settings.TOWER_URL_BASE, "/#/inventory_sync/{}".format(self.pk))
return urljoin(settings.TOWER_URL_BASE, "/#/inventory_sync/{}".format(self.pk))
def is_blocked_by(self, obj):
if type(obj) == InventoryUpdate:

9
awx/main/models/jobs.py
View File

@@ -31,7 +31,6 @@ from awx.main.models.notifications import (
from awx.main.utils import decrypt_field, ignore_inventory_computed_fields
from awx.main.utils import emit_websocket_notification
from awx.main.redact import PlainTextCleaner
from awx.main.conf import tower_settings
from awx.main.fields import ImplicitRoleField
from awx.main.models.mixins import ResourceMixin
@@ -483,9 +482,9 @@ class JobTemplate(UnifiedJobTemplate, JobOptions, ResourceMixin):
@property
def cache_timeout_blocked(self):
if Job.objects.filter(job_template=self, status__in=['pending', 'waiting', 'running']).count() > getattr(tower_settings, 'SCHEDULE_MAX_JOBS', 10):
if Job.objects.filter(job_template=self, status__in=['pending', 'waiting', 'running']).count() > getattr(settings, 'SCHEDULE_MAX_JOBS', 10):
logger.error("Job template %s could not be started because there are more than %s other jobs from that template waiting to run" %
(self.name, getattr(tower_settings, 'SCHEDULE_MAX_JOBS', 10)))
(self.name, getattr(settings, 'SCHEDULE_MAX_JOBS', 10)))
return True
return False
@@ -552,7 +551,7 @@ class Job(UnifiedJob, JobOptions, JobNotificationMixin):
return reverse('api:job_detail', args=(self.pk,))
def get_ui_url(self):
return urljoin(tower_settings.TOWER_URL_BASE, "/#/jobs/{}".format(self.pk))
return urljoin(settings.TOWER_URL_BASE, "/#/jobs/{}".format(self.pk))
@property
def task_auth_token(self):
@@ -1376,7 +1375,7 @@ class SystemJob(UnifiedJob, SystemJobOptions, JobNotificationMixin):
return reverse('api:system_job_detail', args=(self.pk,))
def get_ui_url(self):
return urljoin(tower_settings.TOWER_URL_BASE, "/#/management_jobs/{}".format(self.pk))
return urljoin(settings.TOWER_URL_BASE, "/#/management_jobs/{}".format(self.pk))
def is_blocked_by(self, obj):
return True

9
awx/main/models/organization.py
View File

@@ -23,7 +23,6 @@ from awx.main.models.rbac import (
ROLE_SINGLETON_SYSTEM_AUDITOR,
)
from awx.main.models.mixins import ResourceMixin
from awx.main.conf import tower_settings
__all__ = ['Organization', 'Team', 'Permission', 'Profile', 'AuthToken']
@@ -262,7 +261,7 @@ class AuthToken(BaseModel):
if not now:
now = tz_now()
if not self.pk or not self.is_expired(now=now):
self.expires = now + datetime.timedelta(seconds=tower_settings.AUTH_TOKEN_EXPIRATION)
self.expires = now + datetime.timedelta(seconds=settings.AUTH_TOKEN_EXPIRATION)
if save:
self.save()
@@ -279,12 +278,12 @@ class AuthToken(BaseModel):
if now is None:
now = tz_now()
invalid_tokens = AuthToken.objects.none()
if tower_settings.AUTH_TOKEN_PER_USER != -1:
if settings.AUTH_TOKEN_PER_USER != -1:
invalid_tokens = AuthToken.objects.filter(
user=user,
expires__gt=now,
reason='',
).order_by('-created')[tower_settings.AUTH_TOKEN_PER_USER:]
).order_by('-created')[settings.AUTH_TOKEN_PER_USER:]
return invalid_tokens
def generate_key(self):
@@ -313,7 +312,7 @@ class AuthToken(BaseModel):
valid_n_tokens_qs = self.user.auth_tokens.filter(
expires__gt=now,
reason='',
).order_by('-created')[0:tower_settings.AUTH_TOKEN_PER_USER]
).order_by('-created')[0:settings.AUTH_TOKEN_PER_USER]
valid_n_tokens = valid_n_tokens_qs.values_list('key', flat=True)
return bool(self.key in valid_n_tokens)

3
awx/main/models/projects.py
View File

@@ -28,7 +28,6 @@ from awx.main.models.unified_jobs import * # noqa
from awx.main.models.mixins import ResourceMixin
from awx.main.utils import update_scm_url
from awx.main.fields import ImplicitRoleField
from awx.main.conf import tower_settings
from awx.main.models.rbac import (
ROLE_SINGLETON_SYSTEM_ADMINISTRATOR,
ROLE_SINGLETON_SYSTEM_AUDITOR,
@@ -433,7 +432,7 @@ class ProjectUpdate(UnifiedJob, ProjectOptions, JobNotificationMixin):
return reverse('api:project_update_detail', args=(self.pk,))
def get_ui_url(self):
return urlparse.urljoin(tower_settings.TOWER_URL_BASE, "/#/scm_update/{}".format(self.pk))
return urlparse.urljoin(settings.TOWER_URL_BASE, "/#/scm_update/{}".format(self.pk))
def _update_parent_instance(self):
parent_instance = self._get_parent_instance()

7
awx/main/registrar.py
View File

@@ -1,11 +1,8 @@
# Copyright (c) 2015 Ansible, Inc.
# All Rights Reserved.
import logging
from django.db.models.signals import pre_save, post_save, pre_delete, m2m_changed
logger = logging.getLogger('awx.main.registrar')
class ActivityStreamRegistrar(object):
@@ -13,9 +10,7 @@ class ActivityStreamRegistrar(object):
self.models = []
def connect(self, model):
from awx.main.conf import tower_settings
if not getattr(tower_settings, 'ACTIVITY_STREAM_ENABLED', True):
return
# Always register model; the signal handlers will check if activity stream is enabled.
from awx.main.signals import activity_stream_create, activity_stream_update, activity_stream_delete, activity_stream_associate
if model not in self.models:

11
awx/main/signals.py
View File

@@ -8,6 +8,7 @@ import threading
import json
# Django
from django.conf import settings
from django.db.models.signals import post_save, pre_delete, post_delete, m2m_changed
from django.dispatch import receiver
@@ -21,7 +22,6 @@ from awx.api.serializers import * # noqa
from awx.main.utils import model_instance_diff, model_to_dict, camelcase_to_underscore, emit_websocket_notification
from awx.main.utils import ignore_inventory_computed_fields, ignore_inventory_group_removal, _inventory_updates
from awx.main.tasks import update_inventory_computed_fields
from awx.main.conf import tower_settings
__all__ = []
@@ -297,10 +297,10 @@ def update_host_last_job_after_job_deleted(sender, **kwargs):
class ActivityStreamEnabled(threading.local):
def __init__(self):
self.enabled = getattr(tower_settings, 'ACTIVITY_STREAM_ENABLED', True)
self.enabled = True
def __nonzero__(self):
return bool(self.enabled)
return bool(self.enabled and getattr(settings, 'ACTIVITY_STREAM_ENABLED', True))
activity_stream_enabled = ActivityStreamEnabled()
@@ -330,7 +330,6 @@ model_serializer_mapping = {
JobTemplate: JobTemplateSerializer,
Job: JobSerializer,
AdHocCommand: AdHocCommandSerializer,
TowerSettings: TowerSettingsSerializer,
NotificationTemplate: NotificationTemplateSerializer,
Notification: NotificationSerializer,
}
@@ -354,7 +353,7 @@ def activity_stream_create(sender, instance, created, **kwargs):
#TODO: Weird situation where cascade SETNULL doesn't work
# it might actually be a good idea to remove all of these FK references since
# we don't really use them anyway.
if type(instance) is not TowerSettings:
if instance._meta.model_name != 'setting': # Is not conf.Setting instance
getattr(activity_entry, object1).add(instance)
def activity_stream_update(sender, instance, **kwargs):
@@ -377,7 +376,7 @@ def activity_stream_update(sender, instance, **kwargs):
object1=object1,
changes=json.dumps(changes))
activity_entry.save()
if type(instance) is not TowerSettings:
if instance._meta.model_name != 'setting': # Is not conf.Setting instance
getattr(activity_entry, object1).add(instance)
def activity_stream_delete(sender, instance, **kwargs):

2
awx/main/south_migrations/0071_v240_changes.py
View File

@@ -4,7 +4,7 @@ from south.db import db
from south.v2 import DataMigration
from django.db import models
from django.utils.timezone import now
from awx.api.license import feature_enabled
from awx.conf.license import feature_enabled
class Migration(DataMigration):

24
awx/main/tasks.py
View File

@@ -48,8 +48,7 @@ from awx.main.constants import CLOUD_PROVIDERS
from awx.main.models import * # noqa
from awx.main.models import UnifiedJob
from awx.main.queue import FifoQueue
from awx.main.conf import tower_settings
from awx.main.task_engine import TaskSerializer, TASK_TIMEOUT_INTERVAL
from awx.main.task_engine import TaskEnhancer
from awx.main.utils import (get_ansible_version, get_ssh_version, decrypt_field, update_scm_url,
emit_websocket_notification,
check_proot_installed, build_proot_temp_dir, wrap_args_with_proot)
@@ -105,10 +104,9 @@ def send_notifications(notification_list, job_id=None):
@task(bind=True, queue='default')
def run_administrative_checks(self):
if not tower_settings.TOWER_ADMIN_ALERTS:
if not settings.TOWER_ADMIN_ALERTS:
return
reader = TaskSerializer()
validation_info = reader.from_database()
validation_info = TaskEnhancer().validate_enhancements()
if validation_info.get('instance_count', 0) < 1:
return
used_percentage = float(validation_info.get('current_instances', 0)) / float(validation_info.get('instance_count', 100))
@@ -118,7 +116,7 @@ def run_administrative_checks(self):
"Ansible Tower host usage over 90%",
tower_admin_emails,
fail_silently=True)
if validation_info.get('time_remaining', 0) < TASK_TIMEOUT_INTERVAL:
if validation_info.get('date_warning', False):
send_mail("Ansible Tower license will expire soon",
"Ansible Tower license will expire soon",
tower_admin_emails,
@@ -417,7 +415,7 @@ class BaseTask(Task):
# NOTE:
# Derived class should call add_ansible_venv() or add_tower_venv()
if self.should_use_proot(instance, **kwargs):
env['PROOT_TMP_DIR'] = tower_settings.AWX_PROOT_BASE_PATH
env['PROOT_TMP_DIR'] = settings.AWX_PROOT_BASE_PATH
return env
def build_safe_env(self, instance, **kwargs):
@@ -530,7 +528,7 @@ class BaseTask(Task):
instance = self.update_model(instance.pk)
if instance.cancel_flag:
try:
if tower_settings.AWX_PROOT_ENABLED and self.should_use_proot(instance):
if settings.AWX_PROOT_ENABLED and self.should_use_proot(instance):
# NOTE: Refactor this once we get a newer psutil across the board
if not psutil:
os.kill(child.pid, signal.SIGKILL)
@@ -727,9 +725,9 @@ class RunJob(BaseTask):
'''
plugin_dir = self.get_path_to('..', 'plugins', 'callback')
plugin_dirs = [plugin_dir]
if hasattr(tower_settings, 'AWX_ANSIBLE_CALLBACK_PLUGINS') and \
tower_settings.AWX_ANSIBLE_CALLBACK_PLUGINS:
plugin_dirs.append(tower_settings.AWX_ANSIBLE_CALLBACK_PLUGINS)
if hasattr(settings, 'AWX_ANSIBLE_CALLBACK_PLUGINS') and \
settings.AWX_ANSIBLE_CALLBACK_PLUGINS:
plugin_dirs.extend(settings.AWX_ANSIBLE_CALLBACK_PLUGINS)
plugin_path = ':'.join(plugin_dirs)
env = super(RunJob, self).build_env(job, **kwargs)
env = self.add_ansible_venv(env)
@@ -944,7 +942,7 @@ class RunJob(BaseTask):
'''
Return whether this task should use proot.
'''
return getattr(tower_settings, 'AWX_PROOT_ENABLED', False)
return getattr(settings, 'AWX_PROOT_ENABLED', False)
def post_run_hook(self, job, **kwargs):
'''
@@ -1624,7 +1622,7 @@ class RunAdHocCommand(BaseTask):
'''
Return whether this task should use proot.
'''
return getattr(tower_settings, 'AWX_PROOT_ENABLED', False)
return getattr(settings, 'AWX_PROOT_ENABLED', False)
def post_run_hook(self, ad_hoc_command, **kwargs):
'''

24
awx/main/tests/base.py
View File

@@ -31,8 +31,8 @@ from django.utils.encoding import force_text
# AWX
from awx.main.models import * # noqa
from awx.main.management.commands.run_task_system import run_taskmanager
from awx.main.task_engine import TaskEnhancer
from awx.main.utils import get_ansible_version
from awx.main.task_engine import TaskEngager as LicenseWriter
from awx.sso.backends import LDAPSettings
from awx.main.tests.URI import URI # noqa
@@ -143,35 +143,25 @@ class BaseTestMixin(MockCommonlySlowTestMixin):
return __name__ + '-generated-' + string + rnd_str
def create_test_license_file(self, instance_count=10000, license_date=int(time.time() + 3600), features=None):
writer = LicenseWriter(
settings.LICENSE = TaskEnhancer(
company_name='AWX',
contact_name='AWX Admin',
contact_email='awx@example.com',
license_date=license_date,
instance_count=instance_count,
license_type='enterprise',
features=features)
handle, license_path = tempfile.mkstemp(suffix='.json')
os.close(handle)
writer.write_file(license_path)
self._temp_paths.append(license_path)
os.environ['AWX_LICENSE_FILE'] = license_path
cache.clear()
features=features,
).enhance()
def create_basic_license_file(self, instance_count=100, license_date=int(time.time() + 3600)):
writer = LicenseWriter(
settings.LICENSE = TaskEnhancer(
company_name='AWX',
contact_name='AWX Admin',
contact_email='awx@example.com',
license_date=license_date,
instance_count=instance_count,
license_type='basic')
handle, license_path = tempfile.mkstemp(suffix='.json')
os.close(handle)
writer.write_file(license_path)
self._temp_paths.append(license_path)
os.environ['AWX_LICENSE_FILE'] = license_path
cache.clear()
license_type='basic',
).enhance()
def create_expired_license_file(self, instance_count=1000, grace_period=False):
license_date = time.time() - 1

28
awx/main/tests/data/ssh.py
View File

@@ -1,3 +1,31 @@
TEST_SSH_RSA1_KEY_DATA = '''-----BEGIN PRIVATE KEY-----
uFZFyag7VVqI+q/oGnQu+wj/pMi5ox+Qz5L3W0D745DzwgDXOeObAfNlr9NtIKbn
sZ5E0+rYB4Q/U0CYr5juNJQV1dbxq2Em1160axboe2QbvX6wE6Sm6wW9b9cr+PoF
MoYQebUnCY0ObrLbrRugSfZc17lyxK0ZGRgPXKhpMg6Ecv8XpvhjUYU9Esyqfuco
/p26Q140/HsHeHYNma0dQHCEjMr/qEzOY1qguHj+hRf3SARtM9Q+YNgpxchcDDVS
O+n+8Ljd/p82bpEJwxmpXealeWbI6gB9/R6wcCL+ZyCZpnHJd/NJ809Vtu47ZdDi
E6jvqS/3AQhuQKhJlLSDIzezB2VKKrHwOvHkg/+uLoCqHN34Gk6Qio7x69SvXy88
a7q9D1l/Zx60o08FyZyqlo7l0l/r8EY+36cuI/lvAvfxc5VHVEOvKseUjFRBiCv9
MkKNxaScoYsPwY7SIS6gD93tg3eM5pA0nfMfya9u1+uq/QCM1gNG3mm6Zd8YG4c/
Dx4bmsj8cp5ni/Ffl/sKzKYq1THunJEFGXOZRibdxk/Fal3SQrRAwy7CgLQL8SMh
IWqcFm25OtSOP1r1LE25t5pQsMdmp0IP2fEF0t/pXPm1ZfrTurPMqpo4FGm2hkki
U3sH/o6nrkSOjklOLWlwtTkkL4dWPlNwc8OYj8zFizXJkAfv1spzhv3lRouNkw4N
Mm22W7us2f3Ob0H5C07k26h6VuXX+0AybD4tIIcUXCLoNTqA0HvqhKpEuHu3Ck10
RaB8xHTxgwdhGVaNHMfy9B9l4tNs3Tb5k0LyeRRGVDhWCFo6axYULYebkj+hFLLY
+JE5RzPDFpTf1xbuT+e56H/lLFCUdDu0bn+D0W4ifXaVFegak4r6O4B53CbMqr+R
t6qDPKLUIuVJXK0J6Ay6XgmheXJGbgKh4OtDsc06gsTCE1nY4f/Z82AQahPBfTtF
J2z+NHdsLPn//HlxspGQtmLpuS7Wx0HYXZ+kPRSiE/vmITw85R2u8JSHQicVNN4C
2rlUo15TIU3tTx+WUIrHKHPidUNNotRb2p9n9FoSidU6upKnQHAT/JNv/zcvaia3
Bhl/wagheWTDnFKSmJ4HlKxplM/32h6MfHqsMVOl4F6eZWKaKgSgN8doXyFJo+sc
yAC6S0gJlD2gQI24iTI4Du1+UGh2MGb69eChvi5mbbdesaZrlR1dRqZpHG+6ob4H
nYLndRvobXS5l6pgGTDRYoUgSbQe21a7Uf3soGl5jHqLWc1zEPwrxV7Wr31mApr6
8VtGZcLSr0691Q1NLO3eIfuhbMN2mssX/Sl4t+4BibaucNIMfmhKQi8uHtwAXb47
+TMFlG2EQhZULFM4fLdF1vaizInU3cBk8lsz8i71tDc+5VQTEwoEB7Gksy/XZWEt
6SGHxXUDtNYa+G2O+sQhgqBjLIkVTV6KJOpvNZM+s8Vzv8qoFnD7isKBBrRvF1bP
GOXEG1jd7nSR0WSwcMCHGOrFEELDQPw3k5jqEdPFgVODoZPr+drZVnVz5SAGBk5Y
wsCNaDW+1dABYFlqRTepP5rrSu9wHnRAZ3ZGv+DHoGqenIC5IBR0sQ==
-----END PRIVATE KEY-----'''
TEST_SSH_KEY_DATA = '''-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAyQ8F5bbgjHvk4SZJsKI9OmJKMFxZqRhvx4LaqjLTKbBwRBsY
1/C00NPiZn70dKbeyV7RNVZxuzM6yd3D3lwTdbDu/eJ0x72t3ch+TdLt/aenyy10

33
awx/main/tests/functional/api/test_activity_streams.py
View File

@@ -6,28 +6,27 @@ from awx.main.models.activity_stream import ActivityStream
from awx.main.access import ActivityStreamAccess
from django.core.urlresolvers import reverse
from django.conf import settings
def mock_feature_enabled(feature, bypass_database=None):
def mock_feature_enabled(feature):
return True
@pytest.fixture
def activity_stream_entry(organization, org_admin):
return ActivityStream.objects.filter(organization__pk=organization.pk, user=org_admin, operation='associate').first()
@pytest.mark.skipif(not getattr(settings, 'ACTIVITY_STREAM_ENABLED', True), reason="Activity stream not enabled")
@mock.patch('awx.api.views.feature_enabled', new=mock_feature_enabled)
@pytest.mark.django_db
def test_get_activity_stream_list(monkeypatch, organization, get, user):
def test_get_activity_stream_list(monkeypatch, organization, get, user, settings):
settings.ACTIVITY_STREAM_ENABLED = True
url = reverse('api:activity_stream_list')
response = get(url, user('admin', True))
assert response.status_code == 200
@pytest.mark.skipif(not getattr(settings, 'ACTIVITY_STREAM_ENABLED', True), reason="Activity stream not enabled")
@mock.patch('awx.api.views.feature_enabled', new=mock_feature_enabled)
@pytest.mark.django_db
def test_basic_fields(monkeypatch, organization, get, user):
def test_basic_fields(monkeypatch, organization, get, user, settings):
settings.ACTIVITY_STREAM_ENABLED = True
u = user('admin', True)
activity_stream = ActivityStream.objects.filter(organization=organization).latest('pk')
activity_stream.actor = u
@@ -44,10 +43,10 @@ def test_basic_fields(monkeypatch, organization, get, user):
assert 'organization' in response.data['summary_fields']
assert response.data['summary_fields']['organization'][0]['name'] == 'test-org'
@pytest.mark.skipif(not getattr(settings, 'ACTIVITY_STREAM_ENABLED', True), reason="Activity stream not enabled")
@mock.patch('awx.api.views.feature_enabled', new=mock_feature_enabled)
@pytest.mark.django_db
def test_middleware_actor_added(monkeypatch, post, get, user):
def test_middleware_actor_added(monkeypatch, post, get, user, settings):
settings.ACTIVITY_STREAM_ENABLED = True
u = user('admin-poster', True)
url = reverse('api:organization_list')
@@ -66,21 +65,19 @@ def test_middleware_actor_added(monkeypatch, post, get, user):
assert response.status_code == 200
assert response.data['summary_fields']['actor']['username'] == 'admin-poster'
@pytest.mark.skipif(not getattr(settings, 'ACTIVITY_STREAM_ENABLED', True), reason="Activity stream not enabled")
@mock.patch('awx.api.views.feature_enabled', new=mock_feature_enabled)
@pytest.mark.django_db
def test_rbac_stream_resource_roles(activity_stream_entry, organization, org_admin):
def test_rbac_stream_resource_roles(activity_stream_entry, organization, org_admin, settings):
settings.ACTIVITY_STREAM_ENABLED = True
assert activity_stream_entry.user.first() == org_admin
assert activity_stream_entry.organization.first() == organization
assert activity_stream_entry.role.first() == organization.admin_role
assert activity_stream_entry.object_relationship_type == 'awx.main.models.organization.Organization.admin_role'
@pytest.mark.skipif(not getattr(settings, 'ACTIVITY_STREAM_ENABLED', True), reason="Activity stream not enabled")
@mock.patch('awx.api.views.feature_enabled', new=mock_feature_enabled)
@pytest.mark.django_db
def test_rbac_stream_user_roles(activity_stream_entry, organization, org_admin):
def test_rbac_stream_user_roles(activity_stream_entry, organization, org_admin, settings):
settings.ACTIVITY_STREAM_ENABLED = True
assert activity_stream_entry.user.first() == org_admin
assert activity_stream_entry.organization.first() == organization
assert activity_stream_entry.role.first() == organization.admin_role
@@ -88,9 +85,9 @@ def test_rbac_stream_user_roles(activity_stream_entry, organization, org_admin):
@pytest.mark.django_db
@pytest.mark.activity_stream_access
@pytest.mark.skipif(not getattr(settings, 'ACTIVITY_STREAM_ENABLED', True), reason="Activity stream not enabled")
@mock.patch('awx.api.views.feature_enabled', new=mock_feature_enabled)
def test_stream_access_cant_change(activity_stream_entry, organization, org_admin):
def test_stream_access_cant_change(activity_stream_entry, organization, org_admin, settings):
settings.ACTIVITY_STREAM_ENABLED = True
access = ActivityStreamAccess(org_admin)
# These should always return false because the activity stream can not be edited
assert not access.can_add(activity_stream_entry)
@@ -99,12 +96,12 @@ def test_stream_access_cant_change(activity_stream_entry, organization, org_admi
@pytest.mark.django_db
@pytest.mark.activity_stream_access
@pytest.mark.skipif(not getattr(settings, 'ACTIVITY_STREAM_ENABLED', True), reason="Activity stream not enabled")
@mock.patch('awx.api.views.feature_enabled', new=mock_feature_enabled)
def test_stream_queryset_hides_shows_items(
activity_stream_entry, organization, user, org_admin,
project, org_credential, inventory, label, deploy_jobtemplate,
notification_template, group, host, team):
notification_template, group, host, team, settings):
settings.ACTIVITY_STREAM_ENABLED = True
# this user is not in any organizations and should not see any resource activity
no_access_user = user('no-access-user', False)
queryset = ActivityStreamAccess(no_access_user).get_queryset()

4
awx/main/tests/functional/api/test_fact_versions.py
View File

@@ -13,10 +13,10 @@ from awx.main.utils import timestamp_apiformat
from django.core.urlresolvers import reverse
from django.utils import timezone
def mock_feature_enabled(feature, bypass_database=None):
def mock_feature_enabled(feature):
return True
def mock_feature_disabled(feature, bypass_database=None):
def mock_feature_disabled(feature):
return False
def setup_common(hosts, fact_scans, get, user, epoch=timezone.now(), get_params={}, host_count=1):

4
awx/main/tests/functional/api/test_fact_view.py
View File

@@ -6,10 +6,10 @@ from awx.main.utils import timestamp_apiformat
from django.core.urlresolvers import reverse
from django.utils import timezone
def mock_feature_enabled(feature, bypass_database=None):
def mock_feature_enabled(feature):
return True
def mock_feature_disabled(feature, bypass_database=None):
def mock_feature_disabled(feature):
return False
# TODO: Consider making the fact_scan() fixture a Class, instead of a function, and move this method into it

4
awx/main/tests/functional/api/test_organizations.py
View File

@@ -99,7 +99,7 @@ def test_organization_inventory_list(organization, inventory_factory, get, alice
@pytest.mark.django_db
@mock.patch('awx.api.views.feature_enabled', lambda feature,bypass_db=None: True)
@mock.patch('awx.api.views.feature_enabled', lambda feature: True)
def test_create_organization(post, admin, alice):
new_org = {
'name': 'new org',
@@ -111,7 +111,7 @@ def test_create_organization(post, admin, alice):
@pytest.mark.django_db
@mock.patch('awx.api.views.feature_enabled', lambda feature,bypass_db=None: True)
@mock.patch('awx.api.views.feature_enabled', lambda feature: True)
def test_create_organization_xfail(post, alice):
new_org = {
'name': 'new org',

2
awx/main/tests/functional/api/test_survey_spec.py
View File

@@ -6,7 +6,7 @@ from django.core.urlresolvers import reverse
from awx.main.models.jobs import JobTemplate, Job
from awx.main.models.activity_stream import ActivityStream
from awx.api.license import LicenseForbids
from awx.conf.license import LicenseForbids
from awx.main.access import JobTemplateAccess

4
awx/main/tests/functional/commands/test_cleanup_facts.py
View File

@@ -16,10 +16,10 @@ from awx.main.management.commands.cleanup_facts import CleanupFacts, Command
from awx.main.models.fact import Fact
from awx.main.models.inventory import Host
def mock_feature_enabled(feature, bypass_database=None):
def mock_feature_enabled(feature):
return True
def mock_feature_disabled(feature, bypass_database=None):
def mock_feature_disabled(feature):
return False
@pytest.mark.django_db

66
awx/main/tests/functional/core/test_licenses.py
View File

@@ -1,28 +1,23 @@
# Copyright (c) 2015 Ansible, Inc.
# All Rights Reserved.
import json
import mock
import os
import tempfile
import time
import pytest
from datetime import datetime
from awx.main.models import Host
from awx.main.task_engine import TaskSerializer, TaskEngager
from awx.main.task_engine import TaskEnhancer
@pytest.mark.django_db
def test_license_writer(inventory, admin):
writer = TaskEngager(
task_enhancer = TaskEnhancer(
company_name='acmecorp',
contact_name='Michael DeHaan',
contact_email='michael@ansibleworks.com',
license_date=25000, # seconds since epoch
instance_count=500)
data = writer.get_data()
data = task_enhancer.enhance()
Host.objects.bulk_create(
[
@@ -42,13 +37,7 @@ def test_license_writer(inventory, admin):
assert data['license_date'] == 25000
assert data['license_key'] == "11bae31f31c6a6cdcb483a278cdbe98bd8ac5761acd7163a50090b0f098b3a13"
strdata = writer.get_string()
strdata_loaded = json.loads(strdata)
assert strdata_loaded == data
reader = TaskSerializer()
vdata = reader.from_string(strdata)
vdata = task_enhancer.validate_enhancements()
assert vdata['available_instances'] == 500
assert vdata['current_instances'] == 12
@@ -63,70 +52,41 @@ def test_license_writer(inventory, admin):
@pytest.mark.django_db
def test_expired_licenses():
reader = TaskSerializer()
writer = TaskEngager(
task_enhancer = TaskEnhancer(
company_name='Tower',
contact_name='Tower Admin',
contact_email='tower@ansible.com',
license_date=int(time.time() - 3600),
instance_count=100,
trial=True)
strdata = writer.get_string()
vdata = reader.from_string(strdata)
task_enhancer.enhance()
vdata = task_enhancer.validate_enhancements()
assert vdata['compliant'] is False
assert vdata['grace_period_remaining'] < 0
writer = TaskEngager(
task_enhancer = TaskEnhancer(
company_name='Tower',
contact_name='Tower Admin',
contact_email='tower@ansible.com',
license_date=int(time.time() - 2592001),
instance_count=100,
trial=False)
strdata = writer.get_string()
vdata = reader.from_string(strdata)
task_enhancer.enhance()
vdata = task_enhancer.validate_enhancements()
assert vdata['compliant'] is False
assert vdata['grace_period_remaining'] < 0
writer = TaskEngager(
task_enhancer = TaskEnhancer(
company_name='Tower',
contact_name='Tower Admin',
contact_email='tower@ansible.com',
license_date=int(time.time() - 3600),
instance_count=100,
trial=False)
strdata = writer.get_string()
vdata = reader.from_string(strdata)
task_enhancer.enhance()
vdata = task_enhancer.validate_enhancements()
assert vdata['compliant'] is False
assert vdata['grace_period_remaining'] > 0
@pytest.mark.django_db
def test_aws_license():
os.environ['AWX_LICENSE_FILE'] = 'non-existent-license-file.json'
h, path = tempfile.mkstemp()
with os.fdopen(h, 'w') as f:
json.dump({'instance_count': 100}, f)
def fetch_ami(_self):
_self.attributes['ami-id'] = 'ami-00000000'
return True
def fetch_instance(_self):
_self.attributes['instance-id'] = 'i-00000000'
return True
with mock.patch('awx.main.task_engine.TEMPORARY_TASK_FILE', path):
with mock.patch('awx.main.task_engine.TemporaryTaskEngine.fetch_ami', fetch_ami):
with mock.patch('awx.main.task_engine.TemporaryTaskEngine.fetch_instance', fetch_instance):
reader = TaskSerializer()
license = reader.from_file()
assert license['is_aws']
assert license['time_remaining']
assert license['free_instances'] > 0
assert license['grace_period_remaining'] > 0
os.unlink(path)

2
awx/main/tests/functional/test_rbac_api.py
View File

@@ -5,7 +5,7 @@ from django.db import transaction
from django.core.urlresolvers import reverse
from awx.main.models.rbac import Role, ROLE_SINGLETON_SYSTEM_ADMINISTRATOR
def mock_feature_enabled(feature, bypass_database=None):
def mock_feature_enabled(feature):
return True
#@mock.patch('awx.api.views.feature_enabled', new=mock_feature_enabled)

7
awx/main/tests/old/ad_hoc.py
View File

@@ -20,7 +20,6 @@ from crum import impersonate
# AWX
from awx.main.utils import * # noqa
from awx.main.models import * # noqa
from awx.main.conf import tower_settings
from awx.main.tests.base import BaseJobExecutionTest
from awx.main.tests.data.ssh import (
TEST_SSH_KEY_DATA,
@@ -572,14 +571,14 @@ class AdHocCommandApiTest(BaseAdHocCommandTest):
# Try to relaunch ad hoc command when module has been removed from
# allowed list of modules.
try:
ad_hoc_commands = tower_settings.AD_HOC_COMMANDS
tower_settings.AD_HOC_COMMANDS = []
ad_hoc_commands = settings.AD_HOC_COMMANDS
settings.AD_HOC_COMMANDS = []
with self.current_user('admin'):
response = self.get(url, expect=200)
self.assertEqual(response['passwords_needed_to_start'], [])
response = self.post(url, {}, expect=400)
finally:
tower_settings.AD_HOC_COMMANDS = ad_hoc_commands
settings.AD_HOC_COMMANDS = ad_hoc_commands
# Try to relaunch after the inventory has been marked inactive.
self.inventory.delete()

3
awx/main/tests/old/users.py
View File

@@ -15,7 +15,6 @@ from django.test.utils import override_settings
# AWX
from awx.main.models import * # noqa
from awx.main.tests.base import BaseTest
from awx.main.conf import tower_settings
__all__ = ['AuthTokenTimeoutTest', 'AuthTokenLimitTest', 'AuthTokenProxyTest', 'UsersTest', 'LdapTest']
@@ -38,7 +37,7 @@ class AuthTokenTimeoutTest(BaseTest):
response = self._generic_rest(dashboard_url, expect=200, method='get', return_response_object=True, client_kwargs=kwargs)
self.assertIn('Auth-Token-Timeout', response)
self.assertEqual(response['Auth-Token-Timeout'], str(tower_settings.AUTH_TOKEN_EXPIRATION))
self.assertEqual(response['Auth-Token-Timeout'], str(settings.AUTH_TOKEN_EXPIRATION))
class AuthTokenLimitTest(BaseTest):
def setUp(self):

6
awx/main/tests/unit/conftest.py Normal file
View File

@@ -0,0 +1,6 @@
import pytest
@pytest.fixture(autouse=True)
def _disable_database_settings(mocker):
mocker.patch('awx.conf.settings.SettingsWrapper._get_supported_settings', return_value=[])

56
awx/main/tests/unit/test_credentials.py
View File

@@ -1,56 +0,0 @@
from django.core.exceptions import ValidationError
from awx.main.models.credential import validate_ssh_private_key
import pytest
def test_valid_rsa_key():
begin = """-----BEGIN RSA PRIVATE KEY-----"""
end = """-----END RSA PRIVATE KEY-----"""
unvalidated_key = build_key(begin, body, end)
key_data = validate_ssh_private_key(unvalidated_key)
assert key_data['key_type'] == 'rsa'
def test_invalid_key():
unvalidated_key = build_key(key_begin, body, "END KEY")
with pytest.raises(ValidationError):
validate_ssh_private_key(unvalidated_key)
def test_key_type_empty():
unvalidated_key = build_key(key_begin, body, key_end)
key_data = validate_ssh_private_key(unvalidated_key)
assert key_data['key_type'] == 'rsa1'
def build_key(begin, body, end):
return """%s%s%s""" % (begin, body, end)
key_begin = """-----BEGIN PRIVATE KEY-----"""
key_end = """-----END PRIVATE KEY-----"""
body = """
uFZFyag7VVqI+q/oGnQu+wj/pMi5ox+Qz5L3W0D745DzwgDXOeObAfNlr9NtIKbn
sZ5E0+rYB4Q/U0CYr5juNJQV1dbxq2Em1160axboe2QbvX6wE6Sm6wW9b9cr+PoF
MoYQebUnCY0ObrLbrRugSfZc17lyxK0ZGRgPXKhpMg6Ecv8XpvhjUYU9Esyqfuco
/p26Q140/HsHeHYNma0dQHCEjMr/qEzOY1qguHj+hRf3SARtM9Q+YNgpxchcDDVS
O+n+8Ljd/p82bpEJwxmpXealeWbI6gB9/R6wcCL+ZyCZpnHJd/NJ809Vtu47ZdDi
E6jvqS/3AQhuQKhJlLSDIzezB2VKKrHwOvHkg/+uLoCqHN34Gk6Qio7x69SvXy88
a7q9D1l/Zx60o08FyZyqlo7l0l/r8EY+36cuI/lvAvfxc5VHVEOvKseUjFRBiCv9
MkKNxaScoYsPwY7SIS6gD93tg3eM5pA0nfMfya9u1+uq/QCM1gNG3mm6Zd8YG4c/
Dx4bmsj8cp5ni/Ffl/sKzKYq1THunJEFGXOZRibdxk/Fal3SQrRAwy7CgLQL8SMh
IWqcFm25OtSOP1r1LE25t5pQsMdmp0IP2fEF0t/pXPm1ZfrTurPMqpo4FGm2hkki
U3sH/o6nrkSOjklOLWlwtTkkL4dWPlNwc8OYj8zFizXJkAfv1spzhv3lRouNkw4N
Mm22W7us2f3Ob0H5C07k26h6VuXX+0AybD4tIIcUXCLoNTqA0HvqhKpEuHu3Ck10
RaB8xHTxgwdhGVaNHMfy9B9l4tNs3Tb5k0LyeRRGVDhWCFo6axYULYebkj+hFLLY
+JE5RzPDFpTf1xbuT+e56H/lLFCUdDu0bn+D0W4ifXaVFegak4r6O4B53CbMqr+R
t6qDPKLUIuVJXK0J6Ay6XgmheXJGbgKh4OtDsc06gsTCE1nY4f/Z82AQahPBfTtF
J2z+NHdsLPn//HlxspGQtmLpuS7Wx0HYXZ+kPRSiE/vmITw85R2u8JSHQicVNN4C
2rlUo15TIU3tTx+WUIrHKHPidUNNotRb2p9n9FoSidU6upKnQHAT/JNv/zcvaia3
Bhl/wagheWTDnFKSmJ4HlKxplM/32h6MfHqsMVOl4F6eZWKaKgSgN8doXyFJo+sc
yAC6S0gJlD2gQI24iTI4Du1+UGh2MGb69eChvi5mbbdesaZrlR1dRqZpHG+6ob4H
nYLndRvobXS5l6pgGTDRYoUgSbQe21a7Uf3soGl5jHqLWc1zEPwrxV7Wr31mApr6
8VtGZcLSr0691Q1NLO3eIfuhbMN2mssX/Sl4t+4BibaucNIMfmhKQi8uHtwAXb47
+TMFlG2EQhZULFM4fLdF1vaizInU3cBk8lsz8i71tDc+5VQTEwoEB7Gksy/XZWEt
6SGHxXUDtNYa+G2O+sQhgqBjLIkVTV6KJOpvNZM+s8Vzv8qoFnD7isKBBrRvF1bP
GOXEG1jd7nSR0WSwcMCHGOrFEELDQPw3k5jqEdPFgVODoZPr+drZVnVz5SAGBk5Y
wsCNaDW+1dABYFlqRTepP5rrSu9wHnRAZ3ZGv+DHoGqenIC5IBR0sQ==
"""

11
awx/main/tests/unit/test_tasks.py
View File

@@ -10,9 +10,7 @@ from awx.main.tasks import (
send_notifications,
run_administrative_checks,
)
from awx.main.task_engine import TaskSerializer
from awx.main.task_engine import TaskEnhancer
@contextmanager
def apply_patches(_patches):
@@ -51,12 +49,11 @@ def test_send_notifications_list(mocker):
@pytest.mark.parametrize("current_instances,call_count", [(91, 2), (89,1)])
def test_run_admin_checks_usage(mocker, current_instances, call_count):
patches = list()
patches.append(mocker.patch('awx.main.tasks.tower_settings'))
patches.append(mocker.patch('awx.main.tasks.User'))
mock_ts = mocker.Mock(spec=TaskSerializer)
mock_ts.from_database.return_value = {'instance_count': 100, 'current_instances': current_instances}
patches.append(mocker.patch('awx.main.tasks.TaskSerializer', return_value=mock_ts))
mock_te = mocker.Mock(spec=TaskEnhancer)
mock_te.validate_enhancements.return_value = {'instance_count': 100, 'current_instances': current_instances, 'date_warning': True}
patches.append(mocker.patch('awx.main.tasks.TaskEnhancer', return_value=mock_te))
mock_sm = mocker.Mock()
patches.append(mocker.patch('awx.main.tasks.send_mail', wraps=mock_sm))

91
awx/main/tests/unit/test_validators.py Normal file
View File

@@ -0,0 +1,91 @@
from django.core.exceptions import ValidationError
from awx.main.validators import (
validate_private_key,
validate_certificate,
validate_ssh_private_key,
)
from awx.main.tests.data.ssh import (
TEST_SSH_RSA1_KEY_DATA,
TEST_SSH_KEY_DATA,
TEST_SSH_KEY_DATA_LOCKED,
TEST_OPENSSH_KEY_DATA,
TEST_OPENSSH_KEY_DATA_LOCKED,
TEST_SSH_CERT_KEY,
)
import pytest
def test_valid_rsa_key():
valid_key = TEST_SSH_KEY_DATA
pem_objects = validate_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'rsa'
assert not pem_objects[0]['key_enc']
with pytest.raises(ValidationError):
validate_certificate(valid_key)
pem_objects = validate_ssh_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'rsa'
assert not pem_objects[0]['key_enc']
def test_valid_locked_rsa_key():
valid_key = TEST_SSH_KEY_DATA_LOCKED
pem_objects = validate_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'rsa'
assert pem_objects[0]['key_enc']
with pytest.raises(ValidationError):
validate_certificate(valid_key)
pem_objects = validate_ssh_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'rsa'
assert pem_objects[0]['key_enc']
def test_invalid_rsa_key():
invalid_key = TEST_SSH_KEY_DATA.replace('-----END', '----END')
with pytest.raises(ValidationError):
validate_private_key(invalid_key)
with pytest.raises(ValidationError):
validate_certificate(invalid_key)
with pytest.raises(ValidationError):
validate_ssh_private_key(invalid_key)
def test_valid_openssh_key():
valid_key = TEST_OPENSSH_KEY_DATA
pem_objects = validate_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'ed25519'
assert not pem_objects[0]['key_enc']
with pytest.raises(ValidationError):
validate_certificate(valid_key)
pem_objects = validate_ssh_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'ed25519'
assert not pem_objects[0]['key_enc']
def test_valid_locked_openssh_key():
valid_key = TEST_OPENSSH_KEY_DATA_LOCKED
pem_objects = validate_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'ed25519'
assert pem_objects[0]['key_enc']
with pytest.raises(ValidationError):
validate_certificate(valid_key)
pem_objects = validate_ssh_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'ed25519'
assert pem_objects[0]['key_enc']
def test_valid_rsa1_key():
valid_key = TEST_SSH_RSA1_KEY_DATA
pem_objects = validate_ssh_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'rsa1'
assert not pem_objects[0]['key_enc']
with pytest.raises(ValidationError):
validate_certificate(valid_key)
pem_objects = validate_ssh_private_key(valid_key)
assert pem_objects[0]['key_type'] == 'rsa1'
assert not pem_objects[0]['key_enc']
def test_cert_with_key():
cert_with_key = TEST_SSH_CERT_KEY
with pytest.raises(ValidationError):
validate_private_key(cert_with_key)
with pytest.raises(ValidationError):
validate_certificate(cert_with_key)
pem_objects = validate_ssh_private_key(cert_with_key)
assert pem_objects[0]['type'] == 'CERTIFICATE'
assert pem_objects[1]['key_type'] == 'rsa'
assert not pem_objects[1]['key_enc']

14
awx/main/utils.py
View File

@@ -97,14 +97,14 @@ class RequireDebugTrueOrTest(logging.Filter):
return settings.DEBUG or 'test' in sys.argv
def memoize(ttl=60):
def memoize(ttl=60, cache_key=None):
'''
Decorator to wrap a function and cache its result.
'''
from django.core.cache import cache
def _memoizer(f, *args, **kwargs):
key = slugify('%s %r %r' % (f.__name__, args, kwargs))
key = cache_key or slugify('%s %r %r' % (f.__name__, args, kwargs))
value = cache.get(key)
if value is None:
value = f(*args, **kwargs)
@@ -475,6 +475,7 @@ def cache_list_capabilities(page, prefetch_list, model, user):
obj.capabilities_cache[display_method] = True
@memoize()
def get_system_task_capacity():
'''
Measure system memory and use it as a baseline for determining the system's capacity
@@ -550,8 +551,8 @@ def build_proot_temp_dir():
'''
Create a temporary directory for proot to use.
'''
from awx.main.conf import tower_settings
path = tempfile.mkdtemp(prefix='ansible_tower_proot_', dir=tower_settings.AWX_PROOT_BASE_PATH)
from django.conf import settings
path = tempfile.mkdtemp(prefix='ansible_tower_proot_', dir=settings.AWX_PROOT_BASE_PATH)
os.chmod(path, stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR)
return path
@@ -564,14 +565,13 @@ def wrap_args_with_proot(args, cwd, **kwargs):
- /var/log/supervisor
- /tmp (except for own tmp files)
'''
from awx.main.conf import tower_settings
from django.conf import settings
new_args = [getattr(settings, 'AWX_PROOT_CMD', 'proot'), '-v',
str(getattr(settings, 'AWX_PROOT_VERBOSITY', '0')), '-r', '/']
hide_paths = ['/etc/tower', '/var/lib/awx', '/var/log',
tempfile.gettempdir(), settings.PROJECTS_ROOT,
settings.JOBOUTPUT_ROOT]
hide_paths.extend(getattr(tower_settings, 'AWX_PROOT_HIDE_PATHS', None) or [])
hide_paths.extend(getattr(settings, 'AWX_PROOT_HIDE_PATHS', None) or [])
for path in sorted(set(hide_paths)):
if not os.path.exists(path):
continue
@@ -591,7 +591,7 @@ def wrap_args_with_proot(args, cwd, **kwargs):
show_paths.append(settings.ANSIBLE_VENV_PATH)
if settings.TOWER_USE_VENV:
show_paths.append(settings.TOWER_VENV_PATH)
show_paths.extend(getattr(tower_settings, 'AWX_PROOT_SHOW_PATHS', None) or [])
show_paths.extend(getattr(settings, 'AWX_PROOT_SHOW_PATHS', None) or [])
for path in sorted(set(show_paths)):
if not os.path.exists(path):
continue

168
awx/main/validators.py Normal file
View File

@@ -0,0 +1,168 @@
# Copyright (c) 2015 Ansible, Inc.
# All Rights Reserved.
# Python
import base64
import re
# Django
from django.utils.translation import ugettext_lazy as _
from django.core.exceptions import ValidationError
def validate_pem(data, min_keys=0, max_keys=None, min_certs=0, max_certs=None):
"""
Validate the given PEM data is valid and contains the required numbers of
keys and certificates.
Return a list of PEM objects, where each object is a dict with the following
keys:
- 'all': The entire string for the PEM object including BEGIN/END lines.
- 'type': The type of PEM object ('PRIVATE KEY' or 'CERTIFICATE').
- 'data': The string inside the BEGIN/END lines.
- 'b64': Key/certificate as a base64-encoded string.
- 'bin': Key/certificate as bytes.
- 'key_type': Only when type == 'PRIVATE KEY', one of 'rsa', 'dsa',
'ecdsa', 'ed25519' or 'rsa1'.
- 'key_enc': Only when type == 'PRIVATE KEY', boolean indicating if key is
encrypted.
"""
# Map the X in BEGIN X PRIVATE KEY to the key type (ssh-keygen -t).
# Tower jobs using OPENSSH format private keys may still fail if the
# system SSH implementation lacks support for this format.
private_key_types = {
'RSA': 'rsa',
'DSA': 'dsa',
'EC': 'ecdsa',
'OPENSSH': 'ed25519',
'': 'rsa1',
}
# Build regular expressions for matching each object in the PEM file.
pem_obj_re = re.compile(
r'^(-{4,}) *BEGIN ([A-Z ]+?) *\1[\r\n]+' +
r'(.+?)[\r\n]+\1 *END \2 *\1[\r\n]?(.*?)$', re.DOTALL,
)
pem_obj_header_re = re.compile(r'^(.+?):\s*?(.+?)(\\??)$')
pem_objects = []
key_count, cert_count = 0, 0
data = data.lstrip()
while data:
match = pem_obj_re.match(data)
if not match:
raise ValidationError(_('Invalid certificate or key: %r...') % data[:100])
data = match.group(4).lstrip()
# Check PEM object type, check key type if private key.
pem_obj_info = {}
pem_obj_info['all'] = match.group(0)
pem_obj_info['type'] = pem_obj_type = match.group(2)
if pem_obj_type.endswith('PRIVATE KEY'):
key_count += 1
pem_obj_info['type'] = 'PRIVATE KEY'
key_type = pem_obj_type.replace('PRIVATE KEY', '').strip()
try:
pem_obj_info['key_type'] = private_key_types[key_type]
except KeyError:
raise ValidationError(_('Invalid private key: unsupported type "%s"') % key_type)
elif pem_obj_type == 'CERTIFICATE':
cert_count += 1
else:
raise ValidationError(_('Unsupported PEM object type: "%s"') % pem_obj_type)
# Ensure that this PEM object is valid base64 data.
pem_obj_info['data'] = match.group(3)
base64_data = ''
line_continues = False
for line in pem_obj_info['data'].splitlines():
line = line.strip()
if not line:
continue
if line_continues:
line_continues = line.endswith('\\')
continue
line_match = pem_obj_header_re.match(line)
if line_match:
line_continues = line.endswith('\\')
continue
base64_data += line
try:
decoded_data = base64.b64decode(base64_data)
if not decoded_data:
raise TypeError
pem_obj_info['b64'] = base64_data
pem_obj_info['bin'] = decoded_data
except TypeError:
raise ValidationError(_('Invalid base64-encoded data'))
# If private key, check whether it is encrypted.
if pem_obj_info.get('key_type', '') == 'ed25519':
# See https://github.com/openssh/openssh-portable/blob/master/sshkey.c#L3218
# Decoded key data starts with magic string (null-terminated), four byte
# length field, followed by the ciphername -- if ciphername is anything
# other than 'none' the key is encrypted.
pem_obj_info['key_enc'] = not bool(pem_obj_info['bin'].startswith('openssh-key-v1\x00\x00\x00\x00\x04none'))
elif pem_obj_info.get('key_type', ''):
pem_obj_info['key_enc'] = bool('ENCRYPTED' in pem_obj_info['data'])
pem_objects.append(pem_obj_info)
# Validate that the number of keys and certs provided are within the limits.
key_count_dict = dict(min_keys=min_keys, max_keys=max_keys, key_count=key_count)
if key_count < min_keys:
if min_keys == 1:
if max_keys == min_keys:
raise ValidationError(_('Exactly one private key is required.'))
else:
raise ValidationError(_('At least one private key is required.'))
else:
raise ValidationError(_('At least %(min_keys)d private keys are required, only %(key_count)d provided.') % key_count_dict)
elif max_keys is not None and key_count > max_keys:
if max_keys == 1:
raise ValidationError(_('Only one private key is allowed, %(key_count)d provided.') % key_count_dict)
else:
raise ValidationError(_('No more than %(max_keys)d private keys are allowed, %(key_count)d provided.') % key_count_dict)
cert_count_dict = dict(min_certs=min_certs, max_certs=max_certs, cert_count=cert_count)
if cert_count < min_certs:
if min_certs == 1:
if max_certs == min_certs:
raise ValidationError(_('Exactly one certificate is required.'))
else:
raise ValidationError(_('At least one certificate is required.'))
else:
raise ValidationError(_('At least %(min_certs)d certificates are required, only %(cert_count)d provided.') % cert_count_dict)
elif max_certs is not None and cert_count > max_certs:
if max_certs == 1:
raise ValidationError(_('Only one certificate is allowed, %(cert_count)d provided.') % cert_count_dict)
else:
raise ValidationError(_('No more than %(max_certs)d certificates are allowed, %(cert_count)d provided.') % cert_count_dict)
return pem_objects
def validate_private_key(data):
"""
Validate that data contains exactly one private key.
"""
return validate_pem(data, min_keys=1, max_keys=1, max_certs=0)
def validate_certificate(data):
"""
Validate that data contains one or more certificates. Adds BEGIN/END lines
if necessary.
"""
if 'BEGIN CERTIFICATE' not in data:
data = '-----BEGIN CERTIFICATE-----\n{}\n-----END CERTIFICATE-----\n'.format(data)
return validate_pem(data, max_keys=0, min_certs=1)
def validate_ssh_private_key(data):
"""
Validate that data contains at least one private key and optionally
certificates; should handle any valid options for ssh_private_key on a
credential.
"""
return validate_pem(data, min_keys=1)