External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-08-04 03:50:01 -02:30
Code Issues Packages Projects Releases Wiki Activity

Configure Tower in Tower:

Browse Source 
* Add separate Django app for configuration: awx.conf.
* Migrate from existing main.TowerSettings model to conf.Setting.
* Add settings wrapper to allow get/set/del via django.conf.settings.
* Update existing references to tower_settings to use django.conf.settings.
* Add a settings registry to allow for each Django app to register configurable settings.
* Support setting validation and conversion using Django REST Framework fields.
* Add /api/v1/settings/ to display a list of setting categories.
* Add /api/v1/settings/<slug>/ to display all settings in a category as a single object.
* Allow PUT/PATCH to update setting singleton, DELETE to reset to defaults.
* Add "all" category to display all settings across categories.
* Add "changed" category to display only settings configured in the database.
* Support per-user settings via "user" category (/api/v1/settings/user/).
* Support defaults for user settings via "user-defaults" category (/api/v1/settings/user-defaults/).
* Update serializer metadata to support category, category_slug and placeholder on OPTIONS responses.
* Update serializer metadata to handle child fields of a list/dict.
* Hide raw data form in browsable API for OPTIONS and DELETE.
* Combine existing licensing code into single "TaskEnhancer" class.
* Move license helper functions from awx.api.license into awx.conf.license.
* Update /api/v1/config/ to read/verify/update license using TaskEnhancer and settings wrapper.
* Add support for caching settings accessed via settings wrapper.
* Invalidate cached settings when Setting model changes or is deleted.
* Preload all database settings into cache on first access via settings wrapper.
* Add support for read-only settings than can update their value depending on other settings.
* Use setting_changed signal whenever a setting changes.
* Register configurable authentication, jobs, system and ui settings.
* Register configurable LDAP, RADIUS and social auth settings.
* Add custom fields and validators for URL, LDAP, RADIUS and social auth settings.
* Rewrite existing validator for Credential ssh_private_key to support validating private keys, certs or combinations of both.
* Get all unit/functional tests working with above changes.
* Add "migrate_to_database_settings" command to determine settings to be migrated into the database and comment them out when set in Python settings files.
* Add support for migrating license key from file to database.
* Remove database-configuable settings from local_settings.py example files.
* Update setup role to no longer install files for database-configurable settings.

f 94ff6ee More settings work.
f af4c4e0 Even more db settings stuff.
f 96ea9c0 More settings, attempt at singleton serializer for settings.
f 937c760 More work on singleton/category views in API, add code to comment out settings in Python files, work on command to migrate settings to database.
f 425b0d3 Minor fixes for sprint demo.
f ea402a4 Add support for read-only settings, cleanup license engine, get license support working with DB settings.
f ec289e4 Rename migration, minor fixmes, update setup role.
f 603640b Rewrite key/cert validator, finish adding social auth fields, hook up signals for setting_changed, use None to imply a setting is not set.
f 67d1b5a Get functional/unit tests passing.
f 2919b62 Flake8 fixes.
f e62f421 Add redbaron to requirements, get file to database migration working (except for license).
f c564508 Add support for migrating license file.
f 982f767 Add support for regex in social map fields.
This commit is contained in:
Chris Church
2016-09-26 22:14:47 -04:00
parent 609a3e6f2f
commit 6ebe45b1bd
92 changed files with 4401 additions and 1791 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
awx/conf/__init__.py Normal file
View File

@@ -0,0 +1,18 @@
# Copyright (c) 2016 Ansible, Inc.
# All Rights Reserved.
# Django
from django.utils.module_loading import autodiscover_modules
# Tower
from .registry import settings_registry
default_app_config = 'awx.conf.apps.ConfConfig'
def register(setting, **kwargs):
settings_registry.register(setting, **kwargs)
def autodiscover():
autodiscover_modules('conf', register_to=settings_registry)

45
awx/conf/access.py Normal file
View File

@@ -0,0 +1,45 @@
# Copyright (c) 2016 Ansible, Inc.
# All Rights Reserved.
# Django
from django.db.models import Q
# Tower
from awx.main.access import BaseAccess, register_access
from awx.conf.models import Setting
class SettingAccess(BaseAccess):
'''
- I can see settings when I am a super user or system auditor.
- I can edit settings when I am a super user.
- I can clear settings when I am a super user.
- I can always see/edit/clear my own user settings.
'''
model = Setting
# For the checks below, obj will be an instance of a "Settings" class with
# an attribute for each setting and a "user" attribute (set to None unless
# it is a user setting).
def get_queryset(self):
if self.user.is_superuser or self.user.is_system_auditor:
return self.model.objects.filter(Q(user__isnull=True) | Q(user=self.user))
else:
return self.model.objects.filter(user=self.user)
def can_read(self, obj):
return bool(self.user.is_superuser or self.user.is_system_auditor or (obj and obj.user == self.user))
def can_add(self, data):
return False # There is no API endpoint to POST new settings.
def can_change(self, obj, data):
return bool(self.user.is_superuser or (obj and obj.user == self.user))
def can_delete(self, obj):
return bool(self.user.is_superuser or (obj and obj.user == self.user))
register_access(Setting, SettingAccess)

16
awx/conf/apps.py Normal file
View File

@@ -0,0 +1,16 @@
# Django
from django.apps import AppConfig
# from django.core import checks
from django.utils.translation import ugettext_lazy as _
class ConfConfig(AppConfig):
name = 'awx.conf'
verbose_name = _('Configuration')
def ready(self):
self.module.autodiscover()
from .settings import SettingsWrapper
SettingsWrapper.initialize()
# checks.register(SettingsWrapper._check_settings)

95
awx/conf/conf.py Normal file
View File

@@ -0,0 +1,95 @@
# Django
from django.conf import settings
from django.utils.translation import ugettext_lazy as _
# Tower
from awx.conf import fields, register
from awx.conf import settings_registry
# Define a conf.py file within your app and register each setting similarly to
# the example below. Any field class from Django REST Framework or subclass
# thereof can be used for validation/conversion of the setting. All keyword
# arguments to the register function (except field_class, category,
# category_slug, depends_on, placeholder) will be used to initialize
# the field_class.
register(
'ANSIBLE_COW_SELECTION',
field_class=fields.ChoiceField,
choices=[
('bud-frogs', _('Bud Frogs')),
('bunny', _('Bunny')),
('cheese', _('Cheese')),
('daemon', _('Daemon')),
('default', _('Default Cow')),
('dragon', _('Dragon')),
('elephant-in-snake', _('Elephant in Snake')),
('elephant', _('Elephant')),
('eyes', _('Eyes')),
('hellokitty', _('Hello Kitty')),
('kitty', _('Kitty')),
('luke-koala', _('Luke Koala')),
('meow', _('Meow')),
('milk', _('Milk')),
('moofasa', _('Moofasa')),
('moose', _('Moose')),
('ren', _('Ren')),
('sheep', _('Sheep')),
('small', _('Small Cow')),
('stegosaurus', _('Stegosaurus')),
('stimpy', _('Stimpy')),
('supermilker', _('Super Milker')),
('three-eyes', _('Three Eyes')),
('turkey', _('Turkey')),
('turtle', _('Turtle')),
('tux', _('Tux')),
('udder', _('Udder')),
('vader-koala', _('Vader Koala')),
('vader', _('Vader')),
('www', _('WWW')),
],
default='default',
label=_('Cow Selection'),
help_text=_('Select which cow to use with cowsay when running jobs.'),
category=_('Cows'),
# Optional; category_slug will be slugified version of category if not
# explicitly provided.
category_slug='cows',
)
def _get_read_only_ansible_cow_selection_default():
return getattr(settings, 'ANSIBLE_COW_SELECTION', 'No default cow!')
register(
'READONLY_ANSIBLE_COW_SELECTION',
field_class=fields.CharField,
# read_only must be set via kwargs even if field_class sets it.
read_only=True,
# default can be a callable to dynamically compute the value; should be in
# the plain JSON format stored in the DB and used in the API.
default=_get_read_only_ansible_cow_selection_default,
label=_('Example Read-Only Setting'),
help_text=_('Example setting that cannot be changed.'),
category=_('Cows'),
category_slug='cows',
# Optional; list of other settings this read-only setting depends on. When
# the other settings change, the cached value for this setting will be
# cleared to require it to be recomputed.
depends_on=['ANSIBLE_COW_SELECTION'],
)
register(
'EXAMPLE_USER_SETTING',
field_class=fields.CharField,
allow_blank=True,
label=_('Example Setting'),
help_text=_('Example setting which can be different for each user.'),
category=_('User'),
category_slug='user',
default='',
)
# Unregister the example settings above.
settings_registry.unregister('ANSIBLE_COW_SELECTION')
settings_registry.unregister('READONLY_ANSIBLE_COW_SELECTION')
settings_registry.unregister('EXAMPLE_USER_SETTING')

55
awx/conf/fields.py Normal file
View File

@@ -0,0 +1,55 @@
# Python
import json
import logging
import os
import urlparse
# Django
from django.core.validators import URLValidator
from django.utils.translation import ugettext_lazy as _
# Django REST Framework
from rest_framework.fields import * # noqa
logger = logging.getLogger('awx.conf.fields')
# Use DRF fields to convert/validate settings:
# - to_representation(obj) should convert a native Python object to a primitive
# serializable type. This primitive type will be what is presented in the API
# and stored in the JSON field in the datbase.
# - to_internal_value(data) should convert the primitive type back into the
# appropriate Python type to be used in settings.
class StringListField(ListField):
child = CharField()
class URLField(CharField):
def __init__(self, **kwargs):
schemes = kwargs.pop('schemes', None)
self.allow_plain_hostname = kwargs.pop('allow_plain_hostname', False)
super(URLField, self).__init__(**kwargs)
validator_kwargs = dict(message=_('Enter a valid URL'))
if schemes is not None:
validator_kwargs['schemes'] = schemes
self.validators.append(URLValidator(**validator_kwargs))
def run_validators(self, value):
if self.allow_plain_hostname:
try:
url_parts = urlparse.urlsplit(value)
if url_parts.hostname and '.' not in url_parts.hostname:
netloc = '{}.local'.format(url_parts.hostname)
if url_parts.port:
netloc = '{}:{}'.format(netloc, port)
if url_parts.username:
if url_parts.password:
netloc = '{}:{}@{}' % (url_parts.username, url_parts.password, netloc)
else:
netloc = '{}@{}' % (url_parts.username, netloc)
value = urlparse.urlunsplit([url_parts.scheme, netloc, url_parts.path, url_parts.query, url_parts.fragment])
except:
raise # If something fails here, just fall through and let the validators check it.
super(URLField, self).run_validators(value)

52
awx/conf/license.py Normal file
View File

@@ -0,0 +1,52 @@
# Copyright (c) 2016 Ansible, Inc.
# All Rights Reserved.
# Django
from django.core.cache import cache
from django.core.signals import setting_changed
from django.dispatch import receiver
from django.utils.translation import ugettext_lazy as _
# Django REST Framework
from rest_framework.exceptions import APIException
# Tower
from awx.main.task_engine import TaskEnhancer
from awx.main.utils import memoize
__all__ = ['LicenseForbids', 'get_license', 'feature_enabled', 'feature_exists']
class LicenseForbids(APIException):
status_code = 402
default_detail = _('Your Tower license does not allow that.')
@memoize(cache_key='_validated_license_data')
def _get_validated_license_data():
return TaskEnhancer().validate_enhancements()
@receiver(setting_changed)
def _on_setting_changed(sender, **kwargs):
# Clear cached result above when license changes.
if kwargs.get('setting', None) == 'LICENSE':
cache.delete('_validated_license_data')
def get_license(show_key=False):
"""Return a dictionary representing the active license on this Tower instance."""
license_data = _get_validated_license_data()
if not show_key:
license_data.pop('license_key', None)
return license_data
def feature_enabled(name):
"""Return True if the requested feature is enabled, False otherwise."""
return _get_validated_license_data().get('features', {}).get(name, False)
def feature_exists(name):
"""Return True if the requested feature name exists, False otherwise."""
return bool(name in _get_validated_license_data().get('features', {}))

0
awx/conf/management/__init__.py Normal file
View File

0
awx/conf/management/commands/__init__.py Normal file
View File

328
awx/conf/management/commands/migrate_to_database_settings.py Normal file
View File

@@ -0,0 +1,328 @@
# Copyright (c) 2016 Ansible, Inc.
# All Rights Reserved.
# Python
import collections
import difflib
import json
import os
import shutil
# Django
from django.conf import settings
from django.core.management.base import BaseCommand, CommandError
from django.db import transaction
from django.utils.text import slugify
from django.utils.timezone import now
# Tower
from awx import MODE
from awx.conf import settings_registry
from awx.conf.fields import empty, SkipField
from awx.conf.models import Setting
from awx.conf.utils import comment_assignments
class Command(BaseCommand):
def add_arguments(self, parser):
parser.add_argument(
'category',
nargs='*',
type=str,
)
parser.add_argument(
'--dry-run',
action='store_true',
dest='dry_run',
default=False,
help='Only show which settings would be commented/migrated.',
)
parser.add_argument(
'--no-comment',
action='store_true',
dest='no_comment',
default=False,
help='Skip commenting out settings in files.',
)
parser.add_argument(
'--backup-suffix',
dest='backup_suffix',
default=now().strftime('.%Y%m%d%H%M%S'),
help='Backup existing settings files with this suffix.',
)
@transaction.atomic
def handle(self, *args, **options):
self.verbosity = int(options.get('verbosity', 1))
self.dry_run = bool(options.get('dry_run', False))
self.no_comment = bool(options.get('no_comment', False))
self.backup_suffix = options.get('backup_suffix', '')
self.categories = options.get('category', None) or ['all']
self.style.HEADING = self.style.MIGRATE_HEADING
self.style.LABEL = self.style.MIGRATE_LABEL
self.style.OK = self.style.SQL_FIELD
self.style.SKIP = self.style.WARNING
self.style.VALUE = self.style.SQL_KEYWORD
# Determine if any categories provided are invalid.
category_slugs = []
invalid_categories = []
for category in self.categories:
category_slug = slugify(category)
if category_slug in settings_registry.get_registered_categories():
if category_slug not in category_slugs:
category_slugs.append(category_slug)
else:
if category not in invalid_categories:
invalid_categories.append(category)
if len(invalid_categories) == 1:
raise CommandError('Invalid setting category: {}'.format(invalid_categories[0]))
elif len(invalid_categories) > 1:
raise CommandError('Invalid setting categories: {}'.format(', '.join(invalid_categories)))
# Build a list of all settings to be migrated.
registered_settings = []
for category_slug in category_slugs:
for registered_setting in settings_registry.get_registered_settings(category_slug=category_slug, read_only=False):
if registered_setting not in registered_settings:
registered_settings.append(registered_setting)
self._migrate_settings(registered_settings)
def _get_settings_file_patterns(self):
if MODE == 'development':
return [
'/etc/tower/settings.py',
'/etc/tower/conf.d/*.py',
os.path.join(os.path.dirname(__file__), '..', '..', '..', 'settings', 'local_*.py')
]
else:
return [
os.environ.get('AWX_SETTINGS_FILE', '/etc/tower/settings.py'),
os.path.join(os.environ.get('AWX_SETTINGS_DIR', '/etc/tower/conf.d/'), '*.py'),
]
def _get_license_file(self):
return os.environ.get('AWX_LICENSE_FILE', '/etc/tower/license')
def _comment_license_file(self, dry_run=True):
license_file = self._get_license_file()
diff_lines = []
if os.path.exists(license_file):
try:
raw_license_data = open(license_file).read()
license_data = json.loads(raw_license_data)
except Exception as e:
raise CommandError('Error reading license from {0}: {1!r}'.format(license_file, e))
if self.backup_suffix:
backup_license_file = '{}{}'.format(license_file, self.backup_suffix)
else:
backup_license_file = '{}.old'.format(license_file)
diff_lines = list(difflib.unified_diff(
raw_license_data.splitlines(),
[],
fromfile=backup_license_file,
tofile=license_file,
lineterm='',
))
if not dry_run:
if self.backup_suffix:
shutil.copy2(license_file, backup_license_file)
os.remove(license_file)
return diff_lines
def _check_if_needs_comment(self, patterns, setting):
files_to_comment = []
try:
# If any diffs are returned, this setting needs to be commented.
diffs = comment_assignments(patterns, setting, dry_run=True)
if setting == 'LICENSE':
diffs.extend(self._comment_license_file(dry_run=True))
for diff in diffs:
for line in diff.splitlines():
if line.startswith('+++ '):
files_to_comment.append(line[4:])
except Exception as e:
raise CommandError('Error commenting {0}: {1!r}'.format(setting, e))
return files_to_comment
def _check_if_needs_migration(self, setting):
# Check whether the current value differs from the default.
default_value = settings.DEFAULTS_SNAPSHOT.get(setting, empty)
if default_value is empty and setting != 'LICENSE':
field = settings_registry.get_setting_field(setting, read_only=True)
try:
default_value = field.get_default()
except SkipField:
pass
current_value = getattr(settings, setting, empty)
if current_value != default_value:
if current_value is empty:
current_value = None
return current_value
return empty
def _display_tbd(self, setting, files_to_comment, migrate_value):
if self.verbosity >= 1:
if files_to_comment:
if migrate_value is not empty:
action = 'Migrate + Comment'
else:
action = 'Comment'
self.stdout.write(' {}: {}'.format(
self.style.LABEL(setting),
self.style.OK(action),
))
if self.verbosity >= 2:
if migrate_value is not empty:
self.stdout.write(' - Migrate value: {}'.format(
self.style.VALUE(repr(migrate_value)),
))
for file_to_comment in files_to_comment:
self.stdout.write(' - Comment in: {}'.format(
self.style.VALUE(file_to_comment),
))
else:
if self.verbosity >= 2:
self.stdout.write(' {}: {}'.format(
self.style.LABEL(setting),
self.style.SKIP('No Migration'),
))
def _display_migrate(self, setting, action, display_value):
if self.verbosity >= 1:
if action == 'No Change':
action = self.style.SKIP(action)
else:
action = self.style.OK(action)
self.stdout.write(' {}: {}'.format(
self.style.LABEL(setting),
action,
))
if self.verbosity >= 2:
for line in display_value.splitlines():
self.stdout.write(' {}'.format(
self.style.VALUE(line),
))
def _display_diff_summary(self, filename, added, removed):
self.stdout.write(' {} {}{} {}{}'.format(
self.style.LABEL(filename),
self.style.ERROR('-'),
self.style.ERROR(int(removed)),
self.style.OK('+'),
self.style.OK(str(added)),
))
def _display_comment(self, diffs):
for diff in diffs:
if self.verbosity >= 2:
for line in diff.splitlines():
display_line = line
if line.startswith('--- ') or line.startswith('+++ '):
display_line = self.style.LABEL(line)
elif line.startswith('-'):
display_line = self.style.ERROR(line)
elif line.startswith('+'):
display_line = self.style.OK(line)
elif line.startswith('@@'):
display_line = self.style.VALUE(line)
if line.startswith('--- ') or line.startswith('+++ '):
self.stdout.write(' ' + display_line)
else:
self.stdout.write(' ' + display_line)
elif self.verbosity >= 1:
filename, lines_added, lines_removed = None, 0, 0
for line in diff.splitlines():
if line.startswith('+++ '):
if filename:
self._display_diff_summary(filename, lines_added, lines_removed)
filename, lines_added, lines_removed = line[4:], 0, 0
elif line.startswith('+'):
lines_added += 1
elif line.startswith('-'):
lines_removed += 1
if filename:
self._display_diff_summary(filename, lines_added, lines_removed)
def _migrate_settings(self, registered_settings):
patterns = self._get_settings_file_patterns()
# Determine which settings need to be commented/migrated.
if self.verbosity >= 1:
self.stdout.write(self.style.HEADING('Discovering settings to be migrated and commented:'))
to_migrate = collections.OrderedDict()
to_comment = collections.OrderedDict()
for name in registered_settings:
files_to_comment = self._check_if_needs_comment(patterns, name)
if files_to_comment:
to_comment[name] = files_to_comment
migrate_value = empty
if files_to_comment:
migrate_value = self._check_if_needs_migration(name)
if migrate_value is not empty:
to_migrate[name] = migrate_value
self._display_tbd(name, files_to_comment, migrate_value)
if self.verbosity == 1 and not to_migrate and not to_comment:
self.stdout.write(' No settings found to migrate or comment!')
# Now migrate those settings to the database.
if self.verbosity >= 1:
if self.dry_run:
self.stdout.write(self.style.HEADING('Migrating settings to database (dry-run):'))
else:
self.stdout.write(self.style.HEADING('Migrating settings to database:'))
if not to_migrate:
self.stdout.write(' No settings to migrate!')
for name, value in to_migrate.items():
field = settings_registry.get_setting_field(name)
assert not field.read_only
try:
data = field.to_representation(value)
setting_value = field.run_validation(data)
db_value = field.to_representation(setting_value)
except Exception as e:
raise CommandError('Unable to assign value {0!r} to setting "{1}: {2!s}".'.format(value, name, e))
display_value = json.dumps(db_value, indent=4)
# Always encode "raw" strings as JSON.
if isinstance(db_value, basestring):
db_value = json.dumps(db_value)
setting = Setting.objects.filter(key=name, user__isnull=True).order_by('pk').first()
action = 'No Change'
if not setting:
action = 'Migrated'
if not self.dry_run:
Setting.objects.create(key=name, user=None, value=db_value)
elif setting.value != db_value or type(setting.value) != type(db_value):
action = 'Updated'
if not self.dry_run:
setting.value = db_value
setting.save(update_fields=['value'])
self._display_migrate(name, action, display_value)
# Now comment settings in settings files.
if self.verbosity >= 1:
if bool(self.dry_run or self.no_comment):
self.stdout.write(self.style.HEADING('Commenting settings in files (dry-run):'))
else:
self.stdout.write(self.style.HEADING('Commenting settings in files:'))
if not to_comment:
self.stdout.write(' No settings to comment!')
if to_comment:
to_comment_patterns = []
license_file_to_comment = None
for files_to_comment in to_comment.values():
for file_to_comment in files_to_comment:
if file_to_comment == self._get_license_file():
license_file_to_comment = file_to_comment
elif file_to_comment not in to_comment_patterns:
to_comment_patterns.append(file_to_comment)
# Run once in dry-run mode to catch any errors from updating the files.
diffs = comment_assignments(to_comment_patterns, to_comment.keys(), dry_run=True, backup_suffix=self.backup_suffix)
# Then, if really updating, run again.
if not self.dry_run and not self.no_comment:
diffs = comment_assignments(to_comment_patterns, to_comment.keys(), dry_run=False, backup_suffix=self.backup_suffix)
if license_file_to_comment:
diffs.extend(self._comment_license_file(dry_run=False))
self._display_comment(diffs)

30
awx/conf/migrations/0001_initial.py Normal file
View File

@@ -0,0 +1,30 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations, models
import jsonfield.fields
from django.conf import settings
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name='Setting',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('created', models.DateTimeField(default=None, editable=False)),
('modified', models.DateTimeField(default=None, editable=False)),
('key', models.CharField(max_length=255)),
('value', jsonfield.fields.JSONField(null=True)),
('user', models.ForeignKey(related_name='settings', default=None, editable=False, to=settings.AUTH_USER_MODEL, null=True)),
],
options={
'abstract': False,
},
),
]

79
awx/conf/migrations/0002_v310_copy_tower_settings.py Normal file
View File

@@ -0,0 +1,79 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
import json
from django.db import migrations
def copy_tower_settings(apps, schema_editor):
TowerSettings = apps.get_model('main', 'TowerSettings')
Setting = apps.get_model('conf', 'Setting')
for tower_setting in TowerSettings.objects.all().iterator():
try:
value = tower_setting.value
# LICENSE is stored as a string; convert it to a dict.
if tower_setting.key == 'LICENSE':
value = json.loads(value)
# Anything else (e.g. TOWER_URL_BASE) that is stored as a string
# needs to be converted to a JSON-encoded string to work with the
# JSON field.
elif tower_setting.value_type == 'string':
value = json.dumps(value)
setting, created = Setting.objects.get_or_create(
key=tower_setting.key,
user=tower_setting.user,
defaults=dict(value=value),
)
if not created and setting.value != value:
setting.value = value
setting.save(update_fields=['value'])
except Setting.MultipleObjectsReturned:
pass
def revert_tower_settings(apps, schema_editor):
TowerSettings = apps.get_model('main', 'TowerSettings')
Setting = apps.get_model('conf', 'Setting')
for setting in Setting.objects.all().iterator():
value = setting.value
# LICENSE is stored as a JSON object; convert it back to a string.
if setting.key == 'LICENSE':
value = json.dumps(value)
defaults = dict(
value=value,
value_type='string',
description='',
category='',
)
try:
tower_setting, created = TowerSettings.objects.get_or_create(
key=setting.key,
user=setting.user,
defaults=defaults,
)
if not created:
update_fields = []
for k, v in defaults.items():
if getattr(tower_setting, k) != v:
setattr(tower_setting, k, v)
update_fields.append(k)
if update_fields:
tower_setting.save(update_fields=update_fields)
except TowerSettings.MultipleObjectsReturned:
pass
class Migration(migrations.Migration):
dependencies = [
('conf', '0001_initial'),
('main', '0035_v310_jobevent_uuid'),
]
run_before = [
('main', '0036_v310_remove_tower_settings'),
]
operations = [
migrations.RunPython(copy_tower_settings, revert_tower_settings),
]

0
awx/conf/migrations/__init__.py Normal file
View File

59
awx/conf/models.py Normal file
View File

@@ -0,0 +1,59 @@
# Copyright (c) 2016 Ansible, Inc.
# All Rights Reserved.
# Python
import json
# Django
from django.db import models
# Django-JSONField
from jsonfield import JSONField
# Tower
from awx.main.models.base import CreatedModifiedModel
__all__ = ['Setting']
class Setting(CreatedModifiedModel):
key = models.CharField(
max_length=255,
)
value = JSONField(
null=True,
# FIXME: Enable when we upgrade to JSONField with support:
# load_kwargs={'object_pairs_hook': collections.OrderedDict},
)
user = models.ForeignKey(
'auth.User',
related_name='settings',
default=None,
null=True,
editable=False,
on_delete=models.CASCADE,
)
def __unicode__(self):
try:
json_value = json.dumps(self.value)
except ValueError:
# In the rare case the DB value is invalid JSON.
json_value = u'<Invalid JSON>'
if self.user:
return u'{} ({}) = {}'.format(self.key, self.user, json_value)
else:
return u'{} = {}'.format(self.key, json_value)
@classmethod
def get_cache_key(self, key):
return key
import awx.conf.signals # noqa
from awx.main.registrar import activity_stream_registrar # noqa
activity_stream_registrar.connect(Setting)
import awx.conf.access # noqa

121
awx/conf/registry.py Normal file
View File

@@ -0,0 +1,121 @@
# Copyright (c) 2016 Ansible, Inc.
# All Rights Reserved.
# Python
from collections import OrderedDict
import logging
# Django
from django.core.exceptions import ImproperlyConfigured
from django.utils.text import slugify
from django.utils.translation import ugettext_lazy as _
logger = logging.getLogger('awx.conf.registry')
__all__ = ['settings_registry']
class SettingsRegistry(object):
"""Registry of all API-configurable settings and categories."""
def __init__(self):
self._registry = OrderedDict()
self._dependent_settings = {}
def register(self, setting, **kwargs):
if setting in self._registry:
raise ImproperlyConfigured('Setting "{}" is already registered.'.format(setting))
category = kwargs.setdefault('category', None)
category_slug = kwargs.setdefault('category_slug', slugify(category or '') or None)
if category_slug in {'all', 'changed', 'user-defaults'}:
raise ImproperlyConfigured('"{}" is a reserved category slug.'.format(category_slug))
if 'field_class' not in kwargs:
raise ImproperlyConfigured('Setting must provide a field_class keyword argument.')
self._registry[setting] = kwargs
# Normally for read-only/dynamic settings, depends_on will specify other
# settings whose changes may affect the value of this setting. Store
# this setting as a dependent for the other settings, so we can know
# which extra cache keys to clear when a setting changes.
depends_on = kwargs.setdefault('depends_on', None) or set()
for depends_on_setting in depends_on:
dependent_settings = self._dependent_settings.setdefault(depends_on_setting, set())
dependent_settings.add(setting)
def unregister(self, setting):
self._registry.pop(setting, None)
for dependent_settings in self._dependent_settings.values():
dependent_settings.discard(setting)
def get_dependent_settings(self, setting):
return self._dependent_settings.get(setting, set())
def get_registered_categories(self):
categories = {
'all': _('All'),
'changed': _('Changed'),
'user': _('User'),
'user-defaults': _('User Defaults'),
}
for setting, kwargs in self._registry.items():
category_slug = kwargs.get('category_slug', None)
if category_slug is None or category_slug in categories:
continue
categories[category_slug] = kwargs.get('category', None) or category_slug
return categories
def get_registered_settings(self, category_slug=None, read_only=None):
setting_names = []
if category_slug == 'user-defaults':
category_slug = 'user'
if category_slug == 'changed':
category_slug = 'all'
for setting, kwargs in self._registry.items():
if category_slug not in {None, 'all', kwargs.get('category_slug', None)}:
continue
if read_only in {True, False} and kwargs.get('read_only', False) != read_only:
# Note: Doesn't catch fields that set read_only via __init__;
# read-only field kwargs should always include read_only=True.
continue
setting_names.append(setting)
return setting_names
def get_setting_field(self, setting, mixin_class=None, for_user=False, **kwargs):
from django.conf import settings
from rest_framework.fields import empty
field_kwargs = {}
field_kwargs.update(self._registry[setting])
field_kwargs.update(kwargs)
field_class = original_field_class = field_kwargs.pop('field_class')
if mixin_class:
field_class = type(field_class.__name__, (mixin_class, field_class), {})
category_slug = field_kwargs.pop('category_slug', None)
category = field_kwargs.pop('category', None)
depends_on = frozenset(field_kwargs.pop('depends_on', None) or [])
placeholder = field_kwargs.pop('placeholder', empty)
if getattr(field_kwargs.get('child', None), 'source', None) is not None:
field_kwargs['child'].source = None
field_instance = field_class(**field_kwargs)
field_instance.category_slug = category_slug
field_instance.category = category
field_instance.depends_on = depends_on
if placeholder is not empty:
field_instance.placeholder = placeholder
original_field_instance = field_instance
if field_class != original_field_class:
original_field_instance = original_field_class(**field_kwargs)
if category_slug == 'user' and for_user:
try:
field_instance.default = original_field_instance.to_representation(getattr(settings, setting))
except:
logger.warning('Unable to retrieve default value for user setting "%s".', setting, exc_info=True)
elif not field_instance.read_only or field_instance.default is empty:
try:
field_instance.default = original_field_instance.to_representation(settings._awx_conf_settings._get_default(setting))
except AttributeError:
pass
except:
logger.warning('Unable to retrieve default value for setting "%s".', setting, exc_info=True)
return field_instance
settings_registry = SettingsRegistry()

79
awx/conf/serializers.py Normal file
View File

@@ -0,0 +1,79 @@
# Django REST Framework
from rest_framework import serializers
# Tower
from awx.api.fields import VerbatimField
from awx.api.serializers import BaseSerializer
from awx.conf.models import Setting
from awx.conf import settings_registry
class SettingSerializer(BaseSerializer):
"""Read-only serializer for activity stream."""
value = VerbatimField(allow_null=True)
class Meta:
model = Setting
fields = ('id', 'key', 'value')
readonly_fields = ('id', 'key', 'value')
def __init__(self, instance=None, data=serializers.empty, **kwargs):
if instance is None and data is not serializers.empty and 'key' in data:
try:
instance = Setting.objects.get(key=data['key'])
except Setting.DoesNotExist:
pass
super(SettingSerializer, self).__init__(instance, data, **kwargs)
class SettingCategorySerializer(serializers.Serializer):
"""Serialize setting category """
url = serializers.CharField(
read_only=True,
)
slug = serializers.CharField(
read_only=True,
)
name = serializers.CharField(
read_only=True,
)
class SettingFieldMixin(object):
"""Mixin to use a registered setting field class for API display/validation."""
def to_representation(self, obj):
return obj
def to_internal_value(self, value):
obj = super(SettingFieldMixin, self).to_internal_value(value)
return super(SettingFieldMixin, self).to_representation(obj)
class SettingSingletonSerializer(serializers.Serializer):
"""Present a group of settings (by category) as a single object."""
def __init__(self, instance=None, data=serializers.empty, **kwargs):
# Instance (if given) should be an object with attributes for all of the
# settings in the category; never an actual Setting model instance.
assert instance is None or not hasattr(instance, 'pk')
super(SettingSingletonSerializer, self).__init__(instance, data, **kwargs)
def get_fields(self):
fields = super(SettingSingletonSerializer, self).get_fields()
try:
category_slug = self.context['view'].kwargs.get('category_slug', 'all')
except (KeyError, AttributeError):
category_slug = ''
for key in settings_registry.get_registered_settings(category_slug=category_slug):
if self.instance and not hasattr(self.instance, key):
continue
extra_kwargs = {}
# Make LICENSE read-only here; update via /api/v1/config/ only.
if key == 'LICENSE':
extra_kwargs['read_only'] = True
field = settings_registry.get_setting_field(key, mixin_class=SettingFieldMixin, for_user=bool(category_slug == 'user'), **extra_kwargs)
fields[key] = field
return fields

273
awx/conf/settings.py Normal file
View File

@@ -0,0 +1,273 @@
# Python
import contextlib
import json
import logging
import threading
import time
# Django
from django.conf import settings, UserSettingsHolder
from django.core.cache import cache
from django.core import checks
from django.core.exceptions import ImproperlyConfigured
from django.db import ProgrammingError, OperationalError
# Django REST Framework
from rest_framework.fields import empty, SkipField
# Tower
from awx.conf import settings_registry
from awx.conf.models import Setting
# FIXME: Gracefully handle when settings are accessed before the database is
# ready (or during migrations).
logger = logging.getLogger('awx.conf.settings')
# Store a special value to indicate when a setting is not set in the database.
SETTING_CACHE_NOTSET = '___notset___'
# Cannot store None in memcached; use a special value instead to indicate None.
# If the special value for None is the same as the "not set" value, then a value
# of None will be equivalent to the setting not being set (and will raise an
# AttributeError if there is no other default defined).
# SETTING_CACHE_NONE = '___none___'
SETTING_CACHE_NONE = SETTING_CACHE_NOTSET
# Cannot store empty list/tuple in memcached; use a special value instead to
# indicate an empty list.
SETTING_CACHE_EMPTY_LIST = '___[]___'
# Cannot store empty dict in memcached; use a special value instead to indicate
# an empty dict.
SETTING_CACHE_EMPTY_DICT = '___{}___'
# Expire settings from cache after this many seconds.
SETTING_CACHE_TIMEOUT = 60
# Flag indicating whether to store field default values in the cache.
SETTING_CACHE_DEFAULTS = True
__all__ = ['SettingsWrapper']
@contextlib.contextmanager
def _log_database_error():
try:
yield
except (ProgrammingError, OperationalError) as e:
logger.warning('Database settings are not available, using defaults (%s)', e, exc_info=True)
finally:
pass
class SettingsWrapper(UserSettingsHolder):
@classmethod
def initialize(cls):
if not getattr(settings, '_awx_conf_settings', False):
settings_wrapper = cls(settings._wrapped)
settings._wrapped = settings_wrapper
@classmethod
def _check_settings(cls, app_configs, **kwargs):
errors = []
# FIXME: Warn if database not available!
for setting in Setting.objects.filter(key__in=settings_registry.get_registered_settings(), user__isnull=True):
field = settings_registry.get_setting_field(setting.key)
try:
field.to_internal_value(setting.value)
except Exception as e:
errors.append(checks.Error(str(e)))
return errors
def __init__(self, default_settings):
self.__dict__['default_settings'] = default_settings
self.__dict__['_awx_conf_settings'] = self
self.__dict__['_awx_conf_preload_expires'] = None
self.__dict__['_awx_conf_preload_lock'] = threading.RLock()
def _get_supported_settings(self):
return settings_registry.get_registered_settings()
def _get_writeable_settings(self):
return settings_registry.get_registered_settings(read_only=False)
def _get_cache_value(self, value):
if value is None:
value = SETTING_CACHE_NONE
elif isinstance(value, (list, tuple)) and len(value) == 0:
value = SETTING_CACHE_EMPTY_LIST
elif isinstance(value, (dict,)) and len(value) == 0:
value = SETTING_CACHE_EMPTY_DICT
return value
def _preload_cache(self):
# Ensure we're only modifying local preload timeout from one thread.
with self._awx_conf_preload_lock:
# If local preload timeout has not expired, skip preloading.
if self._awx_conf_preload_expires and self._awx_conf_preload_expires > time.time():
return
# Otherwise update local preload timeout.
self.__dict__['_awx_conf_preload_expires'] = time.time() + SETTING_CACHE_TIMEOUT
# If local preload timer has expired, check to see if another process
# has already preloaded the cache and skip preloading if so.
if cache.get('_awx_conf_preload_expires', empty) is not empty:
return
# Initialize all database-configurable settings with a marker value so
# to indicate from the cache that the setting is not configured without
# a database lookup.
settings_to_cache = dict([(key, SETTING_CACHE_NOTSET) for key in self._get_writeable_settings()])
# Load all settings defined in the database.
for setting in Setting.objects.filter(key__in=settings_to_cache.keys(), user__isnull=True).order_by('pk'):
if settings_to_cache[setting.key] != SETTING_CACHE_NOTSET:
continue
settings_to_cache[setting.key] = self._get_cache_value(setting.value)
# Load field default value for any settings not found in the database.
if SETTING_CACHE_DEFAULTS:
for key, value in settings_to_cache.items():
if value != SETTING_CACHE_NOTSET:
continue
field = settings_registry.get_setting_field(key)
try:
settings_to_cache[key] = self._get_cache_value(field.get_default())
except SkipField:
pass
# Generate a cache key for each setting and store them all at once.
settings_to_cache = dict([(Setting.get_cache_key(k), v) for k, v in settings_to_cache.items()])
settings_to_cache['_awx_conf_preload_expires'] = self._awx_conf_preload_expires
logger.debug('cache set_many(%r, %r)', settings_to_cache, SETTING_CACHE_TIMEOUT)
cache.set_many(settings_to_cache, SETTING_CACHE_TIMEOUT)
def _get_local(self, name):
self._preload_cache()
cache_key = Setting.get_cache_key(name)
value = cache.get(cache_key, empty)
logger.debug('cache get(%r, %r) -> %r', cache_key, empty, value)
if value == SETTING_CACHE_NOTSET:
value = empty
elif value == SETTING_CACHE_NONE:
value = None
elif value == SETTING_CACHE_EMPTY_LIST:
value = []
elif value == SETTING_CACHE_EMPTY_DICT:
value = {}
field = settings_registry.get_setting_field(name)
if value is empty:
setting = None
if not field.read_only:
setting = Setting.objects.filter(key=name, user__isnull=True).order_by('pk').first()
if setting:
value = setting.value
# If None implies not set, convert when reading the value.
if value is None and SETTING_CACHE_NOTSET == SETTING_CACHE_NONE:
value = SETTING_CACHE_NOTSET
else:
value = SETTING_CACHE_NOTSET
if SETTING_CACHE_DEFAULTS:
try:
value = field.get_default()
except SkipField:
pass
logger.debug('cache set(%r, %r, %r)', cache_key, self._get_cache_value(value), SETTING_CACHE_TIMEOUT)
cache.set(cache_key, self._get_cache_value(value), SETTING_CACHE_TIMEOUT)
if value == SETTING_CACHE_NOTSET and not SETTING_CACHE_DEFAULTS:
try:
value = field.get_default()
except SkipField:
pass
if value not in (empty, SETTING_CACHE_NOTSET):
try:
if field.read_only:
internal_value = field.to_internal_value(value)
field.run_validators(internal_value)
return internal_value
else:
return field.run_validation(value)
except:
logger.warning('The current value "%r" for setting "%s" is invalid.', value, name, exc_info=True)
return empty
def _get_default(self, name):
return getattr(self.default_settings, name)
def __getattr__(self, name):
value = empty
if name in self._get_supported_settings():
with _log_database_error():
value = self._get_local(name)
if value is not empty:
return value
return self._get_default(name)
def _set_local(self, name, value):
field = settings_registry.get_setting_field(name)
if field.read_only:
logger.warning('Attempt to set read only setting "%s".', name)
raise ImproperlyConfigured('Setting "%s" is read only.'.format(name))
try:
data = field.to_representation(value)
setting_value = field.run_validation(data)
db_value = field.to_representation(setting_value)
except Exception as e:
logger.exception('Unable to assign value "%r" to setting "%s".', value, name, exc_info=True)
raise e
# Always encode "raw" strings as JSON.
if isinstance(db_value, basestring):
db_value = json.dumps(db_value)
setting = Setting.objects.filter(key=name, user__isnull=True).order_by('pk').first()
if not setting:
setting = Setting.objects.create(key=name, user=None, value=db_value)
# post_save handler will delete from cache when added.
elif setting.value != db_value or type(setting.value) != type(db_value):
setting.value = db_value
setting.save(update_fields=['value'])
# post_save handler will delete from cache when changed.
def __setattr__(self, name, value):
if name in self._get_supported_settings():
with _log_database_error():
self._set_local(name, value)
else:
setattr(self.default_settings, name, value)
def _del_local(self, name):
field = settings_registry.get_setting_field(name)
if field.read_only:
logger.warning('Attempt to delete read only setting "%s".', name)
raise ImproperlyConfigured('Setting "%s" is read only.'.format(name))
for setting in Setting.objects.filter(key=name, user__isnull=True):
setting.delete()
# pre_delete handler will delete from cache.
def __delattr__(self, name):
if name in self._get_supported_settings():
with _log_database_error():
self._del_local(name)
else:
delattr(self.default_settings, name)
def __dir__(self):
keys = []
with _log_database_error():
for setting in Setting.objects.filter(key__in=self._get_supported_settings(), user__isnull=True):
# Skip returning settings that have been overridden but are
# considered to be "not set".
if setting.value is None and SETTING_CACHE_NOTSET == SETTING_CACHE_NONE:
continue
if setting.key not in keys:
keys.append(str(setting.key))
for key in dir(self.default_settings):
if key not in keys:
keys.append(key)
return keys
def is_overridden(self, setting):
set_locally = False
if setting in self._get_supported_settings():
with _log_database_error():
set_locally = Setting.objects.filter(key=setting, user__isnull=True).exists()
set_on_default = getattr(self.default_settings, 'is_overridden', lambda s: False)(setting)
return (set_locally or set_on_default)

69
awx/conf/signals.py Normal file
View File

@@ -0,0 +1,69 @@
# Python
import logging
# Django
from django.conf import settings
from django.core.cache import cache
from django.core.signals import setting_changed
from django.db.models.signals import post_save, pre_delete, post_delete
from django.dispatch import receiver
# Tower
import awx.main.signals
from awx.conf import settings_registry
from awx.conf.models import Setting
from awx.conf.serializers import SettingSerializer
logger = logging.getLogger('awx.conf.signals')
awx.main.signals.model_serializer_mapping[Setting] = SettingSerializer
__all__ = []
def handle_setting_change(key, for_delete=False):
# When a setting changes or is deleted, remove its value from cache along
# with any other settings that depend on it.
setting_keys = [key]
for dependent_key in settings_registry.get_dependent_settings(key):
# Note: Doesn't handle multiple levels of dependencies!
setting_keys.append(dependent_key)
cache_keys = set([Setting.get_cache_key(k) for k in setting_keys])
logger.debug('cache delete_many(%r)', cache_keys)
cache.delete_many(cache_keys)
# Send setting_changed signal with new value for each setting.
for setting_key in setting_keys:
setting_changed.send(
sender=Setting,
setting=setting_key,
value=getattr(settings, setting_key, None),
enter=not bool(for_delete),
)
@receiver(post_save, sender=Setting)
def on_post_save_setting(sender, **kwargs):
instance = kwargs['instance']
# Skip for user-specific settings.
if instance.user:
return
handle_setting_change(instance.key)
@receiver(pre_delete, sender=Setting)
def on_pre_delete_setting(sender, **kwargs):
instance = kwargs['instance']
# Skip for user-specific settings.
if instance.user:
return
# Save instance key (setting name) for post_delete.
instance._saved_key_ = instance.key
@receiver(post_delete, sender=Setting)
def on_post_delete_setting(sender, **kwargs):
instance = kwargs['instance']
key = getattr(instance, '_saved_key_', None)
if key:
handle_setting_change(key, True)

15
awx/conf/urls.py Normal file
View File

@@ -0,0 +1,15 @@
# Copyright (c) 2016 Ansible, Inc.
# All Rights Reserved.
# Django
from django.conf.urls import patterns
# Tower
from awx.api.urls import url
urlpatterns = patterns(
'awx.conf.views',
url(r'^$', 'setting_category_list'),
url(r'^(?P<category_slug>[a-z0-9-]+)/$', 'setting_singleton_detail'),
)

110
awx/conf/utils.py Executable file
View File

@@ -0,0 +1,110 @@
#!/usr/bin/env python
# Python
import difflib
import glob
import os
import shutil
# RedBaron
from redbaron import RedBaron, indent
__all__ = ['comment_assignments']
def comment_assignments(patterns, assignment_names, dry_run=True, backup_suffix='.old'):
if isinstance(patterns, basestring):
patterns = [patterns]
diffs = []
for pattern in patterns:
for filename in sorted(glob.glob(pattern)):
filename = os.path.abspath(os.path.normpath(filename))
if backup_suffix:
backup_filename = '{}{}'.format(filename, backup_suffix)
else:
backup_filename = None
diff = comment_assignments_in_file(filename, assignment_names, dry_run, backup_filename)
if diff:
diffs.append(diff)
return diffs
def comment_assignments_in_file(filename, assignment_names, dry_run=True, backup_filename=None):
if isinstance(assignment_names, basestring):
assignment_names = [assignment_names]
else:
assignment_names = assignment_names[:]
current_file_data = open(filename).read()
for assignment_name in assignment_names[:]:
if assignment_name in current_file_data:
continue
if assignment_name in assignment_names:
assignment_names.remove(assignment_name)
if not assignment_names:
return ''
replace_lines = {}
rb = RedBaron(current_file_data)
for assignment_node in rb.find_all('assignment'):
for assignment_name in assignment_names:
# Only target direct assignments to a variable.
name_node = assignment_node.find('name', value=assignment_name)
if not name_node:
continue
if assignment_node.target.type != 'name':
continue
# Build a new node that comments out the existing assignment node.
indentation = '{}# '.format(assignment_node.indentation or '')
new_node_content = indent(assignment_node.dumps(), indentation)
new_node_lines = new_node_content.splitlines()
# Add a pass statement in case the assignment block is the only
# child in a parent code block to prevent a syntax error.
if assignment_node.indentation:
new_node_lines[0] = new_node_lines[0].replace(indentation, '{}pass # '.format(assignment_node.indentation or ''), 1)
new_node_lines[0] = '{0}This setting is now configured via the Tower API.\n{1}'.format(indentation, new_node_lines[0])
# Store new node lines in dictionary to be replaced in file.
start_lineno = assignment_node.absolute_bounding_box.top_left.line
end_lineno = assignment_node.absolute_bounding_box.bottom_right.line
for n, new_node_line in enumerate(new_node_lines):
new_lineno = start_lineno + n
assert new_lineno <= end_lineno
replace_lines[new_lineno] = new_node_line
if not replace_lines:
return ''
# Iterate through all lines in current file and replace as needed.
current_file_lines = current_file_data.splitlines()
new_file_lines = []
for n, line in enumerate(current_file_lines):
new_file_lines.append(replace_lines.get(n + 1, line))
new_file_data = '\n'.join(new_file_lines)
new_file_lines = new_file_data.splitlines()
# If changed, syntax check and write the new file; return a diff of changes.
diff_lines = []
if new_file_data != current_file_data:
compile(new_file_data, filename, 'exec')
if backup_filename:
from_file = backup_filename
else:
from_file = '{}.old'.format(filename)
to_file = filename
diff_lines = list(difflib.unified_diff(current_file_lines, new_file_lines, fromfile=from_file, tofile=to_file, lineterm=''))
if not dry_run:
if backup_filename:
shutil.copy2(filename, backup_filename)
with open(filename, 'wb') as fileobj:
fileobj.write(new_file_data)
return '\n'.join(diff_lines)
if __name__ == '__main__':
pattern = os.path.join(os.path.dirname(__file__), '..', 'settings', 'local_*.py')
diffs = comment_assignments(pattern, ['AUTH_LDAP_ORGANIZATION_MAP'])
for diff in diffs:
print(diff)

129
awx/conf/views.py Normal file
View File

@@ -0,0 +1,129 @@
# Copyright (c) 2016 Ansible, Inc.
# All Rights Reserved.
# Python
import collections
import json
import sys
# Django
from django.core.urlresolvers import reverse
from django.http import Http404
from django.utils.translation import ugettext_lazy as _
# Django REST Framework
from rest_framework.exceptions import PermissionDenied
from rest_framework.response import Response
from rest_framework import serializers
from rest_framework import status
# Tower
from awx.api.generics import * # noqa
from awx.main.utils import * # noqa
from awx.conf.models import Setting
from awx.conf.serializers import SettingCategorySerializer, SettingSingletonSerializer
from awx.conf import settings_registry
SettingCategory = collections.namedtuple('SettingCategory', ('url', 'slug', 'name'))
class SettingCategoryList(ListAPIView):
model = Setting # Not exactly, but needed for the view.
serializer_class = SettingCategorySerializer
filter_backends = []
new_in_310 = True
view_name = _('Setting Categories')
def get_queryset(self):
setting_categories = []
if self.request.user.is_superuser or self.request.user.is_system_auditor:
categories = settings_registry.get_registered_categories()
else:
categories = {'user': _('User')}
for category_slug in sorted(categories.keys()):
url = reverse('api:setting_singleton_detail', args=(category_slug,))
setting_categories.append(SettingCategory(url, category_slug, categories[category_slug]))
return setting_categories
class SettingSingletonDetail(RetrieveUpdateDestroyAPIView):
model = Setting # Not exactly, but needed for the view.
serializer_class = SettingSingletonSerializer
filter_backends = []
new_in_310 = True
view_name = _('Setting Detail')
def get_queryset(self):
self.category_slug = self.kwargs.get('category_slug', 'all')
all_category_slugs = settings_registry.get_registered_categories().keys()
if self.request.user.is_superuser or getattr(self.request.user, 'is_system_auditor', False):
category_slugs = all_category_slugs
else:
category_slugs = {'user'}
if self.category_slug not in all_category_slugs:
raise Http404
if self.category_slug not in category_slugs:
raise PermissionDenied()
registered_settings = settings_registry.get_registered_settings(category_slug=self.category_slug)
if self.category_slug == 'user':
return Setting.objects.filter(key__in=registered_settings, user=self.request.user)
else:
return Setting.objects.filter(key__in=registered_settings, user__isnull=True)
def get_object(self):
settings_qs = self.get_queryset()
registered_settings = settings_registry.get_registered_settings(category_slug=self.category_slug)
all_settings = {}
for setting in settings_qs:
all_settings[setting.key] = setting.value
for key in registered_settings:
if key in all_settings or self.category_slug == 'changed':
continue
try:
field = settings_registry.get_setting_field(key, for_user=bool(self.category_slug == 'user'))
all_settings[key] = field.get_default()
except serializers.SkipField:
all_settings[key] = None
all_settings['user'] = self.request.user if self.category_slug == 'user' else None
obj = type('Settings', (object,), all_settings)()
self.check_object_permissions(self.request, obj)
return obj
def perform_update(self, serializer):
settings_qs = self.get_queryset()
user = self.request.user if self.category_slug == 'user' else None
for key, value in serializer.validated_data.items():
setattr(serializer.instance, key, value)
# Always encode "raw" strings as JSON.
if isinstance(value, basestring):
value = json.dumps(value)
setting = settings_qs.filter(key=key).order_by('pk').first()
if not setting:
setting = Setting.objects.create(key=key, user=user, value=value)
elif setting.value != value or type(setting.value) != type(value):
setting.value = value
setting.save(update_fields=['value'])
def destroy(self, request, *args, **kwargs):
instance = self.get_object()
self.perform_destroy(instance)
return Response(status=status.HTTP_204_NO_CONTENT)
def perform_destroy(self, instance):
for setting in self.get_queryset():
setting.delete()
# Create view functions for all of the class-based views to simplify inclusion
# in URL patterns and reverse URL lookups, converting CamelCase names to
# lowercase_with_underscore (e.g. MyView.as_view() becomes my_view).
this_module = sys.modules[__name__]
for attr, value in locals().items():
if isinstance(value, type) and issubclass(value, APIView):
name = camelcase_to_underscore(attr)
view = value.as_view()
setattr(this_module, name, view)