django-auth-ldap recently changed its behavior at login to *delay* the
user.save() call:
b777321fb4
our current process of discovering and setting up the system auditor
role at LDAP login *relies* on the user having a primary key, so this
code now manually calls .save() to enforce one
This has a few benefits:
1. It makes adding new fields to built-in CredentialTypes _much_
simpler. In the past, we've had to write a migration every time we
want to modify an existing type (changing a label/help text,
changing options like the recent become_method changes) or
when adding a new field entirely
2. It paves the way for third party credential plugins support, where
importable libraries will define their own source code-based schema
* introduces three new models: `ProjectUpdateEvent`,
`InventoryUpdateEvent`, and `SystemJobEvent`
* simplifies the stdout callback management in `tasks.py` - now _all_
job run types capture and emit events to the callback receiver
* supports stdout reconstruction from events for stdout downloads for
_all_ job types
* configures `ProjectUpdate` runs to configure the awx display callback
(so we can capture real playbook events for `project_update.yml`)
* ProjectUpdate, InventoryUpdate, and SystemJob runs no longer write
text blobs to the deprecated `main_unifiedjob.result_stdout_text` column
see: https://github.com/ansible/awx/issues/200
This data often (in the case of inventory updates) represents large data
blobs (5+MB per job run). Storing it on the polymorphic base class
table, `main_unifiedjob`, causes it to be automatically fetched on every
query (and every polymorphic join) against that table, which can result
in _very_ poor performance for awx across the board. Django offers
`defer()`, but it's quite complicated to sprinkle this everywhere (and
easy to get wrong/introduce side effects related to our RBAC and usage
of polymorphism).
This change moves the field definition to a separate unmanaged model
(which references the same underlying `main_unifiedjob` table) and adds
a proxy for fetching the data as needed
see https://github.com/ansible/awx/issues/200
django-polymorphic itself generates queries for polymorphic object
lookups, and these queries for UnifiedJob are *not* properly defering the
`result_stdout_text` column, resulting in more very slow queries. This
solution is _very_ hacky, and very specific to this specific
version of Django and django-polymorphic, but it works until we can
solve this problem the proper way in 3.3 (by removing large stdout blobs
from the database).
see: https://github.com/ansible/ansible-tower/issues/7568
* Add separate Django app for configuration: awx.conf.
* Migrate from existing main.TowerSettings model to conf.Setting.
* Add settings wrapper to allow get/set/del via django.conf.settings.
* Update existing references to tower_settings to use django.conf.settings.
* Add a settings registry to allow for each Django app to register configurable settings.
* Support setting validation and conversion using Django REST Framework fields.
* Add /api/v1/settings/ to display a list of setting categories.
* Add /api/v1/settings/<slug>/ to display all settings in a category as a single object.
* Allow PUT/PATCH to update setting singleton, DELETE to reset to defaults.
* Add "all" category to display all settings across categories.
* Add "changed" category to display only settings configured in the database.
* Support per-user settings via "user" category (/api/v1/settings/user/).
* Support defaults for user settings via "user-defaults" category (/api/v1/settings/user-defaults/).
* Update serializer metadata to support category, category_slug and placeholder on OPTIONS responses.
* Update serializer metadata to handle child fields of a list/dict.
* Hide raw data form in browsable API for OPTIONS and DELETE.
* Combine existing licensing code into single "TaskEnhancer" class.
* Move license helper functions from awx.api.license into awx.conf.license.
* Update /api/v1/config/ to read/verify/update license using TaskEnhancer and settings wrapper.
* Add support for caching settings accessed via settings wrapper.
* Invalidate cached settings when Setting model changes or is deleted.
* Preload all database settings into cache on first access via settings wrapper.
* Add support for read-only settings than can update their value depending on other settings.
* Use setting_changed signal whenever a setting changes.
* Register configurable authentication, jobs, system and ui settings.
* Register configurable LDAP, RADIUS and social auth settings.
* Add custom fields and validators for URL, LDAP, RADIUS and social auth settings.
* Rewrite existing validator for Credential ssh_private_key to support validating private keys, certs or combinations of both.
* Get all unit/functional tests working with above changes.
* Add "migrate_to_database_settings" command to determine settings to be migrated into the database and comment them out when set in Python settings files.
* Add support for migrating license key from file to database.
* Remove database-configuable settings from local_settings.py example files.
* Update setup role to no longer install files for database-configurable settings.
f 94ff6ee More settings work.
f af4c4e0 Even more db settings stuff.
f 96ea9c0 More settings, attempt at singleton serializer for settings.
f 937c760 More work on singleton/category views in API, add code to comment out settings in Python files, work on command to migrate settings to database.
f 425b0d3 Minor fixes for sprint demo.
f ea402a4 Add support for read-only settings, cleanup license engine, get license support working with DB settings.
f ec289e4 Rename migration, minor fixmes, update setup role.
f 603640b Rewrite key/cert validator, finish adding social auth fields, hook up signals for setting_changed, use None to imply a setting is not set.
f 67d1b5a Get functional/unit tests passing.
f 2919b62 Flake8 fixes.
f e62f421 Add redbaron to requirements, get file to database migration working (except for license).
f c564508 Add support for migrating license file.
f 982f767 Add support for regex in social map fields.
