Commit Graph
100 Commits
Author SHA1 Message Date
Jeff Bradberry 79d7179c72 Fix the persistent breakage when cleaning up branches
The github workflow that we have set up for branch deletion doesn't work:

- the `on: delete` event does not support the `branches:` filter
- the `mode` flag for the aws_s3 module does not have `delete` as one
  of the options.  The proper option appears to be `delobj`.
2024-02-15 15:16:18 -05:00
Jeff Bradberry 38424487f1 Unbreak the pip-compile command when multiple files are passed in (#14875) 2024-02-13 15:59:04 -05:00
Jeff Bradberry 36f3b46726 Avoid using SmartFilter during migrations (#14786)
Our migrations that touch roles tend to bring in our real models via
migration_utils.set_current_apps_for_migrations, and that can have
some undesirable side-effects.
2024-02-12 15:48:58 +00:00
Jeff Bradberry b3aeb962ce Fix the test_export_system_auditor collection test 2024-02-07 15:55:19 -05:00
Jeff Bradberry 2300b8fddf Fix linting problem 2024-02-07 15:55:19 -05:00
Jeff Bradberry 7fc13a0569 Write tests around the two special instance serializer fields
and all of the cases that they might be in.
2024-02-02 10:37:41 -05:00
Jeff Bradberry 92c693f14e Break out peer validation into its own method 2024-02-02 10:37:41 -05:00
Jeff Bradberry f2417f0ed2 Make the peer validation more compact
and reuse information.
2024-02-02 10:37:41 -05:00
Jeff Bradberry 8f22188116 Use a select_related to build the peers queryset in the install bundle
Since the relationship is ReceptorAddress -> Instance,
prefetch_related isn't necessary.
2024-02-02 10:37:41 -05:00
Jeff Bradberry 05502c0af8 Placeholder FIXMEs for things of concern 2024-02-02 10:37:41 -05:00
Jeff Bradberry 957ce59bf7 Use Counter to find duplicate peer relationships
Gives a bit more readability.
2024-02-02 10:37:41 -05:00
Jeff Bradberry cc4cc37d46 'managed' is a read-only field on InstanceSerializer
so we don't need complex logic to compare an incoming to existing value.
2024-02-02 10:37:41 -05:00
Jeff Bradberry 1e254c804c The listener port cannot be disabled when setting peers_from_control_nodes
If the port is explicitly set to null (causing any ReceptorAddress to
be deleted), then that's a validation error.

If the port is left off but a ReceptorAddress doesn't already exist,
we should not infer a port number and that is also a validation error.
2024-02-02 10:37:41 -05:00
Jeff Bradberry f98493aa61 Support wss as ws-listener in the Receptor config 2024-02-02 10:37:41 -05:00
Jeff Bradberry 8ddb604bf1 Template the listener protocol into the receptor install bundle (#14792)
Previously we were hard-coding tcp, now we need to also support ws/wss.
2024-02-02 10:37:41 -05:00
Jeff Bradberry 82323390a7 Reconstitute migration file
Do the things as a proper 3-step process -- add new stuff, copy the data, drop the old.
2024-02-02 10:37:41 -05:00
Jeff Bradberry 8c7c00451a Join across the InstanceLink.target to the underlying Instance
to avoid having to do a lookup for every instance in the list for the
mesh visualizer endpoint.
2024-02-02 10:37:41 -05:00
Jeff Bradberry 42701f32fe Build the source distribution bundle to also upload to PyPI
This has been broken since 20.0.1.
2024-01-11 15:49:07 -05:00
Jeff Bradberry 4822dd79fc Revert "Improve performance for awx cli export (#13182)" 2023-08-15 15:55:10 -04:00
Jeff Bradberry 14992cee17 Add in an async task to migrate the data over 2023-08-10 13:48:58 -04:00
Jeff Bradberry 6db663eacb Modify main/0185 to set aside the json fields that might be a problem
Rename them, then create a new clean field of the new jsonb type.
We'll use a task to do the data conversion.
2023-08-10 13:48:58 -04:00
Jeff Bradberry 8f6849fc22 Include listener_port in the defaults for Instance.objects.register (#13328) 2022-12-19 14:16:05 -03:00
Jeff Bradberry e87e041a2a Break up and conditionally add the RBAC checks for ActivityStream (#13279)
This should vastly improve the queries executed when accessing any of
the activity stream endpoints as a normal user, in many cases.
2022-12-16 15:11:14 -03:00
Jeff Bradberry 7b8dcc98e7 Merge pull request #13308 from jbradberry/rebuild-org-ee-admin-roles
Ensure that the Organization.execution_environment_admin_role always gets built
2022-12-16 11:29:20 -05:00
Jeff Bradberry e524d3df3e Replace the role fixup post_migrate handler with a data migration 2022-12-12 10:20:56 -05:00
Jeff Bradberry 15b7ad3570 Add a post_migrate signal handler to rebuild the Org roles
particularly, the execution_environment_admin_role.
2022-12-07 15:57:20 -05:00
Jeff Bradberry adbcb5c5e4 Merge pull request #13289 from jbradberry/improve-psql-paging
Make sure that the psql pager does not clear the screen afterwards
2022-12-06 13:17:24 -05:00
Jeff Bradberry 8054c6aedc Make sure that the psql pager does not clear the screen afterwards
Also, avoid paging if there is a single page.
2022-12-06 10:46:47 -05:00
Jeff Bradberry bb047baeba Merge pull request #13277 from jbradberry/psql-pager
Configure dbshell to use the 'less' pager
2022-12-02 13:27:46 -05:00
Jeff Bradberry 57430afc55 Configure dbshell to use the 'less' pager 2022-12-02 12:31:49 -05:00
Jeff Bradberry 95e4b2064f Merge pull request #13264 from jbradberry/fix-awxkit-field-removals
Remove update_on_project_update from awxkit
2022-12-01 14:11:03 -05:00
Jeff Bradberry 48eba60be4 Remove update_on_project_update from awxkit
This was removed from the InventorySource model a bit ago.
2022-12-01 13:44:10 -05:00
Jeff Bradberry 65c3db8cb8 Merge pull request #13250 from jbradberry/fix-inventorysource-migration
Fix a problem with migration main/0164
2022-11-30 15:07:54 -05:00
Jeff Bradberry 7fa9dcbc2a Merge pull request #13168 from ArtsiomMusin/feature_13055_add_multiple_assert_export
Add multiple asset export for awx cli
2022-11-30 15:01:42 -05:00
Jeff Bradberry 7cfb957de3 Add the elements: str type to the lists 2022-11-30 14:15:57 -05:00
Jeff Bradberry d0d467e863 Fix a problem with migration main/0164
Since the original version of the migration a) invoked the .save()
method, and b) involved a model with a custom field that had a
post_save handler attached, this migration had a side-effect that
caused the codebase's version of the model to be used when the table
involved wasn't yet up to date.  This triggers an UndefinedColumn error.

This change works around the problem by making use of queryset
.update() methods instead, which should avoid the post_save signal
trigger.
2022-11-30 11:06:08 -05:00
Jeff Bradberry 721e19e1c8 Merge pull request #13181 from jbradberry/remove-qsstats
Replace the querysets provided by django-qsstats-magic
2022-11-11 10:58:51 -05:00
Jeff Bradberry e029cf7196 Remove the django-qsstats-magic dependency 2022-11-10 15:37:44 -05:00
Jeff Bradberry 477a63d1b4 Replace the querysets provided by django-qsstats-magic
with ones that are constructed to do the same thing, without any
extraneous functionality.
2022-11-10 11:00:39 -05:00
Jeff Bradberry 86b0a3d4f1 Merge pull request #13174 from jbradberry/fix-3.11-awxkit
Avoid adding the 'config' subparser twice
2022-11-09 13:37:03 -05:00
Jeff Bradberry 3d3e4ad150 Avoid adding the 'config' subparser twice
Once since it is defined as a CustomCommand subclass, and once because
it is an endpoint at the /api/v2/ level.  With Python 3.11 argparse
has become more strict and will raise an exception when you try to
inject duplicate subparsers.
2022-11-08 15:13:35 -05:00
Jeff Bradberry c5cf39abb7 Merge pull request #13132 from jbradberry/import-project-schedules
Expand the dependencies of the Schedule page type
2022-10-31 16:29:04 -04:00
Jeff Bradberry b57ae592ed Expand the dependencies of the Schedule page type
Really these could get any of the unified job template types, not just
system job templates, so importing e.g. a project with a schedule was
doing them in the wrong order.

Also, bump the timeout of the project update and make sure that we
stash it in the page cache even if it doesn't finish in 5 minutes.
2022-10-28 12:56:35 -04:00
Jeff Bradberry 153a197fad Merge pull request #13125 from jbradberry/improve-staticfile-management
UI static files are now directly copied to the proper static dir
2022-10-27 09:25:09 -04:00
Jeff Bradberry 8f4c329c2a UI static files are now directly copied to the proper static dir
when running `make ui-devel`.  Previously they were going to
/awx_devel/awx/public/static, but that directory is no longer being
served up by nginx, which forced us to have to run `make
collectstatic` (or equivalent) to get the files to the right place.
2022-10-26 17:51:29 -04:00
Jeff Bradberry eeb9d61488 Merge pull request #13069 from jbradberry/hostname-validation-regression
Add back in the uniqueness validation on Instance.hostname
2022-10-17 11:40:54 -04:00
Jeff Bradberry 234ce529fc Add back in the uniqueness validation on Instance.hostname 2022-10-17 10:55:38 -04:00
Jeff Bradberry 5b5aac675b Merge pull request #12959 from ansible/new-health-check-started
Add a new Instance.health_check_started field
2022-09-28 10:58:43 -04:00
Jeff Bradberry 6b0618b244 Merge pull request #12968 from ansible/instance-serializer-defaults
Make sure to include field defaults for Instance node_type and node_state
2022-09-28 10:53:31 -04:00
Jeff Bradberry 5aa55d7347 Make sure to include field defaults for Instance node_type and node_state 2022-09-27 17:15:45 -04:00
Jeff Bradberry 65179d9cd0 Add a new Instance.health_check_started field
This will enable us to provide more useful information for the user,
now that all user-triggered health checks are async.

Also, de-bounce the health check endpoint to not allow additional
health check tasks to be triggered when one is already in progress.
2022-09-27 17:09:41 -04:00
Jeff Bradberry b879cbc2ec Prevent any edits to hop nodes
to retain the behavior that they had pre-mesh-scaling.
2022-09-23 09:46:15 -04:00
Jeff Bradberry 0e578534fa Update the instance install bundle requirements.yml
to point to the 0.1.0 release of ansible.receptor.
2022-09-23 09:46:14 -04:00
Jeff Bradberry 08c18d71bf Move InstanceLink creation and updating to the async tasks
So that they get applied in situations that do not go through the API.
2022-09-23 09:46:14 -04:00
Jeff Bradberry 68a44529b6 Register pages for the Instance peers and install bundle endpoints
This includes exposing a new interface for Page objects, Page.bytes,
to return the full bytestring contents of the response.
2022-09-23 09:46:14 -04:00
Jeff Bradberry f3a9d4db07 Assign a default queue to wait_for_jobs() 2022-09-23 09:46:14 -04:00
Jeff Bradberry ebd200380a Resolve a deadlock in write_receptor_config() 2022-09-23 09:46:13 -04:00
Jeff Bradberry 1b650d6927 When deprovisioning a node, kick off a task that waits on running jobs
After all jobs on the node are complete, delete the node then
broadcast the write_receptor_config task.

Also, make sure that write_receptor_config updates the state of links
that are in 'adding' state.
2022-09-23 09:46:13 -04:00
Jeff Bradberry b6946c7e35 Update API to support setting instances to Deprovisioning
- allow the node_state to be set to deprovisioning
- set the links that touch the instance to removing
- only allow on K8S
- only allow to be done to execution nodes
2022-09-23 09:46:13 -04:00
Jeff Bradberry 3bc86ca8cb Follow up on new execution node creation
- hop nodes are descoped
- links need to be created on execution node creation
- expose the 'edit' capabilities on the instance serializer
2022-09-23 09:46:13 -04:00
Jeff Bradberry 0465a10df5 Deal with exceptions when running execution_node_health_check (#12733) 2022-09-23 09:46:12 -04:00
Jeff Bradberry e4518f7b13 Changes in posting constraints due to rescoping to OCP/K8S-only
- node_state is now read only
- node_state gets set automatically to Installed in the create view
- raise a validation error when creating on non-K8S
- allow SystemAdministrator the 'add' permission for Instances
- expose the new listener_port field
2022-09-23 09:46:12 -04:00
Jeff Bradberry 604fac2295 Update task management to only do things with ready instances 2022-09-23 09:46:11 -04:00
Jeff Bradberry 24bfacb654 Check state when processing receptorctl advertisements
Nodes that show up and were in one of the unready states need to be
transitioned to ready, even if the logic in Instance.is_lost was not
met.
2022-09-23 09:46:11 -04:00
Jeff Bradberry 3bcd539b3d Make sure that the health checks handle the state transitions properly
- nodes with states Provisioning, Provisioning Fail, Deprovisioning,
  and Deprovisioning Fail should bypass health checks and should never
  transition due to the existing machinery
- nodes with states Unavailable and Installed can transition to Ready
  if they check out as healthy
- nodes in the Ready state should transition to Unavailable if they
  fail a check
2022-09-23 09:46:11 -04:00
Jeff Bradberry 81e68cb9bf Update node and link registration to put them in the right state
'Installed' for the nodes, 'Established' for the links.
2022-09-23 09:46:11 -04:00
Jeff Bradberry a575f17db5 Add the state fields and the peer relationships to the serializers 2022-09-23 09:46:11 -04:00
Jeff Bradberry 2fba3db48f Add state fields to Instance and InstanceLink
Also, listener_port to Instance.
2022-09-23 09:46:11 -04:00
Jeff Bradberry 5290c692c1 Merge pull request #12620 from jbradberry/even-narrower-reload
Restrict files that trigger a reload
2022-08-04 09:21:31 -04:00
Jeff Bradberry 90a19057d5 Restrict files that trigger a reload
to files explicitly ending in '.py' that do not start with a dot.
This will avoid Emacs lockfiles from triggering the restart.
2022-08-03 18:23:48 -04:00
Jeff Bradberry 1bcfc8f28e Merge pull request #12544 from jbradberry/awxkit-fix-no-content
Suppress 204 No Content results causing an error during import
2022-07-20 10:48:02 -04:00
Jeff Bradberry fe1a767f4f Suppress 204 No Content results causing an error during import 2022-07-19 12:25:24 -04:00
Jeff Bradberry d59b6f834c Merge pull request #12431 from jbradberry/fix-ugettext-deprecation
Fix a ugettext deprecation that snuck back in
2022-06-27 13:58:07 -04:00
Jeff Bradberry ae7be525e1 Fix a ugettext deprecation that snuck back in
at some point after the Django 3.2 upgrade.
2022-06-27 13:27:35 -04:00
Jeff Bradberry b44442c460 Merge pull request #12351 from AlexSCorey/5673-t-importExportSchedules
Adds import export to awx cli for schedules as a top level object
2022-06-22 10:13:56 -04:00
Jeff Bradberry 48a2ebd48c Merge pull request #12271 from HampusLundqvist/gitlab-webhooks-fixes-#12268
return event_status on push, tag push, and merge gitlab webhook events
2022-06-14 17:12:27 -04:00
Jeff Bradberry 3bc0c53e37 Merge pull request #12368 from jbradberry/narrower-autoreload
Narrow down the inotifywait criteria for reloading the dev environment
2022-06-14 10:13:41 -04:00
Jeff Bradberry e8b2998578 Narrow down the inotifywait criteria for reloading the dev environment
- listen specifically within awx/awx, so that changes in awxkit or
  awx_collection don't trigger spurious reloads
- expand the exclude pattern to ignore the test directories
2022-06-13 16:08:20 -04:00
Jeff Bradberry 9191079dda Merge pull request #11921 from jbradberry/fix-export-reconstruct-endpoint
Look up the correct top-level resource name when reconstructing foreign keys
2022-06-06 17:08:02 -04:00
Jeff Bradberry faa5df19ca Merge pull request #12252 from jbradberry/fix-analytics-unicode
Double escape all unicode escape sequences in job events data
2022-06-06 16:41:06 -04:00
Jeff Bradberry 127016d36b Allow for multiple --extra_vars or --variables flags in awx-cli
This is particularly useful when you are using the @filepath version
of the flag, since otherwise there would be no way to issue the
command with multiple vars files.

Also, add `-e` as an alias to `--extra_vars`
2022-06-01 13:24:24 -04:00
Jeff Bradberry 973facebba Double escape all unicode escape sequences in job events data
when collecting it for analytics.
2022-05-18 12:00:03 -04:00
Jeff Bradberry 4ca9e9577b Merge pull request #12216 from jangel97/devel
add param all_pages to method export_assets
2022-05-11 14:51:50 -04:00
Jeff Bradberry f3725c714a Merge pull request #12119 from kimbernator/devel
Remove hardcoded public schema in cleanup_jobs.py
2022-05-10 17:14:11 -04:00
Jeff Bradberry 2bef5ce09b Merge pull request #12099 from jbradberry/add-content-type-option-header
Add the X-Content-Type-Options nosniff header
2022-04-28 14:41:02 -04:00
Jeff Bradberry 68221cdcbe Merge pull request #12106 from jbradberry/django-bump
Bump Django to 3.2.13
2022-04-26 15:07:52 -04:00
Jeff Bradberry d64c457b3d Bump Django to 3.2.13 2022-04-26 10:34:28 -04:00
Jeff Bradberry 47d5a89f40 Add the X-Content-Type-Options nosniff header 2022-04-25 13:45:16 -04:00
Jeff Bradberry 3a1268de1e Merge pull request #12068 from jbradberry/fix-event-partition-alignment-devel
Fix the job event partition alignment
2022-04-19 10:36:48 -04:00
Jeff Bradberry 11890f0eee Fix the job event partition alignment
it really should be always aligned to the hour, so that real job
events don't slip through the cracks.
2022-04-18 14:54:06 -04:00
Jeff Bradberry 6c1adade25 Merge pull request #11947 from jbradberry/django-3.2-upgrade
Remove the out-of-band JSONField migration
2022-03-28 12:02:53 -04:00
Jeff Bradberry d54838cd94 Remove the out-of-band migration
that was turning all old JSONFields into a jsonb type database column.
The use of JSONBlob makes this unnecessary.
2022-03-24 15:21:59 -04:00
Jeff Bradberry e3f3ab224a Replace all previously text-based json fields with JSONBlob
This JSONBlob field type is a wrapper around Django's new generic
JSONField, but with the database column type forced to be text.  This
should behave close enough to our old wrapper around
django-jsonfield's JSONField and will avoid needing to do the
out-of-band database migration.
2022-03-24 15:21:54 -04:00
Jeff Bradberry aa46a7fe06 Merge pull request #11924 from jbradberry/awxkit-support-inventory-labels
Support the new Inventory.labels in awxkit / awx-cli
2022-03-21 10:19:42 -04:00
Jeff Bradberry 51021f380b Support the new Inventory.labels in awxkit / awx-cli 2022-03-18 14:24:49 -04:00
Jeff Bradberry b562d5cc88 Look up the correct top-level resource name when reconstructing foreign keys
during an awx-cli export.
2022-03-18 10:32:33 -04:00
Jeff Bradberry ac6a82eee4 Merge pull request #11654 from jbradberry/django-3.2-upgrade
Django 3.2 upgrade
2022-03-17 10:34:22 -04:00
Jeff Bradberry 574e3ed6ef Fix the signature of RADIUSBackend.get_django_user
to match what django-radius is now expecting.
2022-03-14 13:19:57 -04:00
Jeff Bradberry 584514766d Disable our wrapping of the Django settings during migrations
This will hopefully get us past the unfortunate check against the
HostMetric table, which doesn't exist when you are upgrading from 3.8
to 4.x.

Additionally, guard against AUTH_LDAP_GROUP_TYPE not being in settings
for conf migration 0006.
2022-03-14 13:19:57 -04:00
Jeff Bradberry eb8a1fec49 Make API queries involving json fields cast the fields to text
This restores compatibility with the way it previously worked -- we
never supported proper json operator queries through the API.
2022-03-14 13:19:57 -04:00