External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-10-31 06:26:15 -02:30
Code Issues Packages Projects Releases Wiki Activity
29,850 Commits 73 Branches 102 Tags
0eddd5ce7f44bcedba38307a4a2db2a3363beef2
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jim Ladd 0eddd5ce7f Enable partition pruning when fetching job's events 2021-06-04 09:17:07 -07:00
Jim Ladd a7cabec3d0 bump partition migration to 0132 2021-06-04 09:17:07 -07:00
Jim Ladd b98b3ced1c update tests to reflect new job_event schema 2021-06-04 09:17:07 -07:00
Jim Ladd 8501a45531 lint fixes 2021-06-04 09:17:07 -07:00
Jim Ladd 14b610dabf bump partition migration 2021-06-04 09:17:07 -07:00
Jim Ladd a1d1e70e43 correct constraint name 2021-06-04 09:17:07 -07:00
Jim Ladd 0fa0a517ac create tmp schema to reference when creating partitioned table 
* if we use the actual old job events table
  and make tweaks to its schema
  namely, dropping the pkey constraint,
  then when we go to migrate the old job events
  we will be forcing postgres to do a sequential scan
  on the old table, which effectively causes the migration to hang
 2021-06-04 09:17:07 -07:00
Jim Ladd 28f9c0be0b Do not cascade delete job_events 
* want to drop job_event _partitions_
  .. instead of having the job events associated with a job
     automatically get cleaned up for us
 2021-06-04 09:17:07 -07:00
Jim Ladd 373edbf8c0 Update reference to partition migration 2021-06-04 09:17:07 -07:00
Jim Ladd 1e45e2ab9b Rev migration number 2021-06-04 09:17:07 -07:00
Jim Ladd 67046513ae Push changes before rebasing 2021-06-04 09:17:07 -07:00
Jim Ladd f9b439ae82 include job_created field in all job event types 2021-06-04 09:17:07 -07:00
Jim Ladd 80b08d17e3 Continue updating job event views 2021-06-04 09:17:07 -07:00
Jim Ladd f642c520bd Update fields for job event models to match raw sql operations 
* raw sql commands were in migration to partition table
* .. just needed to add FakeAddField entries for the new
     job_created field added to each job event model
* .. and also needed to actually list the new field on the model classes
 2021-06-04 09:17:07 -07:00
Jim Ladd 221ddeb915 Fix migration for _all_ job event tables 
- each job event table has a different name for the
  fk referencing the unified job id
- create a mapping so that we get the col name correct
  for each table
 2021-06-04 09:17:07 -07:00
Jim Ladd d90d0fb503 fix initial partition creation 
* call create_partition correctly (include tblname)
* reference 'awx_epoch'
 2021-06-04 09:17:07 -07:00
Jim Ladd 2c529f50af Update querysets for remaining job event views 
- search should use job creation date
  so that only the table partition with the relevant job events
  is searched
 2021-06-04 09:17:07 -07:00
Jim Ladd acfa1c4d1d Drop todo / question / conditional 
* can safely assume job_created is set
* .. and if it isn't, we want to expose that bug
 2021-06-04 09:17:07 -07:00
Jim Ladd ea2afeec1f Drop todo / answered question 2021-06-04 09:17:07 -07:00
Jim Ladd a5cfc3036f create_partition needs tblname 2021-06-04 09:17:07 -07:00
Jim Ladd ec484f81cf Partition *all* job event tables 2021-06-04 09:17:06 -07:00
Jim Ladd 2ffa22e38f Misc doc clean-up 2021-06-04 09:17:06 -07:00
Jim Ladd 8fb313638c get job events based on job creation date 2021-06-04 09:17:06 -07:00
Jim Ladd 0eb1984b22 Only create partitions for regular jobs 2021-06-04 09:17:06 -07:00
Jim Ladd f259b0a71b Move partition start date to 2000 2021-06-04 09:17:06 -07:00
Jim Ladd 82df3ebddb add option to create partitions that span a single minute 
* turned on by default currently for testing
 2021-06-04 09:17:06 -07:00
Jim Ladd c87d7b0d79 fix import 2021-06-04 09:17:06 -07:00
Jim Ladd 612e91263c auto-create partition 2021-06-04 09:17:06 -07:00
Jim Ladd 445042c0f4 Create partition only if it doesn't exist 2021-06-04 09:17:06 -07:00
Jim Ladd 0c289205de Give new primary key constraint unique name, create first live partition 2021-06-04 09:17:06 -07:00
Jim Ladd ba45592d93 create helper method to create partitions 
* create_partition()
 2021-06-04 09:17:06 -07:00
Jim Ladd 7e0f2b0f08 first partition is _not_ a special case after all 2021-06-04 09:17:06 -07:00
Jim Ladd fb30528197 Dynamically create initial partitions 
* First partition holds all events up to this very moment
* And second partition starts where first left off and runs
  .. through rest of current hour
* From there, dynamically generated partitions will cover
  one hour at a time
 2021-06-04 09:17:06 -07:00
Jim Ladd 48f1910075 Remove temporary catch-all partition 2021-06-04 09:17:06 -07:00
Jim Ladd 0cb2d79889 Only save job_created field on JobEvent for the time being 
* Once other job event tables are migrated, remove this
 2021-06-04 09:17:06 -07:00
Jim Ladd 1af1a5e9da Convert job_created to string for serialization 2021-06-04 09:17:06 -07:00
Jim Ladd c0d38e91f5 When saving JobEvents, include job_created 
* this is the partition key
* .. used to determine which partition job event rows are sent to
 2021-06-04 09:17:06 -07:00
Jim Ladd 2f737f644f Drop primary key index before creating partition table 
* Partition tables require unique contstraints to include the
  partition key (uniqueness can only be enforced _inside_
  of a given partition table)
 2021-06-04 09:17:06 -07:00
Jim Ladd 0574baf7f7 Create fake partition 
* Just to get things working
* Will implement dynamic creation of partitions later
 2021-06-04 09:17:06 -07:00
Jim Ladd f70473dc0b When copying main_jobevent, include all table metadata 
* copy the table just like we do in the bigint migration
* without this we lose sequences and very likely other things
  as well
* we want the new table to be identical to the old table,
  so 'including all' makes sense
 2021-06-04 09:17:06 -07:00
Jim Ladd de0b25862b By default, save job_created as null in db 
* Want to avoid saving date as empty string
* Should default to null so that it's obvious the field is empty
 2021-06-04 09:17:06 -07:00
Jim Ladd 6ff15a928a Register new column created by SQL 
* .. using FakeAddField model type
* .. without doing this, Django won't know
     about the field we created using raw SQL
 2021-06-04 09:17:06 -07:00
Jim Ladd d10d1963c1 Rename / remove old main_jobevent table 2021-06-04 09:17:06 -07:00
Jim Ladd c6acca08d5 first draft of db partitioning 2021-06-04 09:17:06 -07:00
Jim Ladd a6f3817488 verify notification errors included in NT list view 2021-05-03 13:17:34 -07:00
Jim Ladd e4eb03259b include error field in notification template's list of recent notifs 2021-05-03 13:17:34 -07:00
Jim Ladd fb97a79aca add clarifying comment re: AUTOMATION_ANALYTICS_LAST_GATHER being set regardless 2021-04-21 21:28:32 -07:00
Jim Ladd a1aec29e48 let jupyter install ipython 2021-03-25 13:17:21 -07:00
Jim Ladd 5b3bb1e81d set schedule.next_run to datetime.datetime 
* Update JobNotificationMixin.context_stub
* .. TestJobNotificationMixin.CONTEXT_STRUCTURE
 2021-02-05 13:56:56 -08:00
Jim Ladd 60dee83481 revert jinja2 dep changes for ansible venv 2021-01-11 11:48:39 -08:00
Jim Ladd 197d50bc44 patch test_inventory_update_injected_content 
* proot now enabled at task-level
  since tasks are no longer calling
  awx-manage (which would set up its own proot)
* dropping proot env var since it's not
  relevant to the test
 2020-11-30 11:35:32 -05:00
Jim Ladd 5ad60a3ed4 use inventory_id to get advisory_lock 2020-11-30 11:35:32 -05:00
Jim Ladd 38638b4a6b add note to remove private_dir when proot removed 2020-11-30 11:35:32 -05:00
Jim Ladd 232801e0ba cache end_line for RunInventoryUpdate jobs 2020-11-30 11:35:32 -05:00
Jim Ladd d55f36eb90 add clarifying comment 
* noting that the inv update task only uses the
  inventory update management command to
  save the inv to the database
  (it doesn't do the work of fetching hosts / groups)
 2020-11-30 11:35:31 -05:00
Jim Ladd 277c47ba4e add TODO reminders to remove proot / venv code 
* won't be needed once we move to container-based
  execution
 2020-11-30 11:35:31 -05:00
Jim Ladd 12cbc9756b inventory updates should use custom venv 2020-11-30 11:35:31 -05:00
Jim Ladd 72df8723f6 lint 2020-11-30 11:35:31 -05:00
Jim Ladd a8710bf2f1 restore proot for inventory updates 
- in the past, inv. update jobs called `awx-manage inventory_update`
  which took care of setting up process isolation
- at this point, though, inv. update jobs call runner / ansible-inventory
  directly, so we need another way to put process isolation in place
- thankfully, there was already support for providing process isolation
  for other types of jobs (namely JT Jobs, Project Updates and Ad Hoc
  commands)
- so, we do what those other jobs do and override the stub for should_use_proot
  (which by default returns false) so that it keys off of the
  `AWX_PROOT_ENABLED` setting
 2020-11-30 11:35:31 -05:00
Jim Ladd 4bdc488fe7 restore proot code 
* add TODOs to note where proot-related code
  can be removed in the future
  (after moving to container-based execution)
 2020-11-30 11:35:31 -05:00
Jim Ladd 9633714c49 create lock for perform_update 
* perform_update can be called from either awx-manage
  or the RunInventoryUpdate task
* need to make sure that the inventory updates
  that happen with perform_update are atomic
 2020-11-30 11:35:31 -05:00
Jim Ladd 05ac2c1ec2 Updates to a given inventory should happen serially 2020-10-12 09:34:37 -07:00
Jim Ladd 71cfeda08f unpin collections used for inventory plugins 
* unpins collections in requirements/collections_requirements.yml
 2020-06-19 09:33:51 -07:00
Jim Ladd 99b3997084 Merge pull request #4412 from jladdjr/foreman_backwards_compat_for_ssl_verify_3.7.2 
add backwards support for ssl_verify in foreman
 2020-06-18 18:25:53 -07:00
Jim Ladd 35fe255455 add backwards support for ssl_verify in foreman 
* plugin changed option name from
  ssl_verify to validate_cert
 2020-06-18 17:02:48 -07:00
Jim Ladd 91cd5b359a add backwards support for ssl_verify in foreman 
* plugin changed option name from
  ssl_verify to validate_cert
 2020-06-17 12:18:31 -07:00
Jim Ladd e1f7a7619f Merge pull request #4398 from jladdjr/instance_id_fallback 
Instance id fallback
 2020-06-11 12:19:23 -07:00
Jim Ladd 43ab6acb84 revert EC2_INSTANCE_ID_VAR 2020-06-11 09:53:50 -07:00
Jim Ladd 8eee0d40dd revert EC2_INSTANCE_ID_VAR 2020-06-11 09:52:54 -07:00
Jim Ladd 3b61b82616 wrap --instance-id-var in quotes 2020-06-10 14:06:57 -07:00
Jim Ladd 9eaee801a7 update VMWARE_INSTANCE_ID_VAR 
* Favor instanceUuid
* .. but fall back to instanceuuid if necessary
 2020-06-10 14:06:54 -07:00
Jim Ladd 9514adaf3a wrap --instance-id-var in quotes 2020-06-10 13:55:54 -07:00
Jim Ladd 17eaeb28a8 update VMWARE_INSTANCE_ID_VAR 
* Favor instanceUuid
* .. but fall back to instanceuuid if necessary
 2020-06-10 13:30:05 -07:00
Jim Ladd 18d09f892d disable reports option for foreman 2020-06-09 22:37:35 -07:00
Jim Ladd 0a0735344e Merge pull request #4342 from jladdjr/foreman_group_prefix_should_apply_to_all_groups_3.7.1 
foreman: use group_prefix for all groups (release_3.7.1)
 2020-06-05 15:25:22 -07:00
Jim Ladd 9ae344b772 foreman: use group_prefix for all groups 
* awx's "compatibility layer" for the foreman plugin had the
  group_prefix hard-coded to 'foreman_'
 2020-06-05 14:44:55 -07:00
Jim Ladd 0c4adfb32e Merge pull request #4387 from jladdjr/bump_foreman_collection_0.8.1 
Bump foreman collection to 0.8.1
 2020-06-05 12:10:22 -07:00
Jim Ladd 5df08711e9 Bump foreman collection to 0.8.1 
* New release includes:
  'add host_filters and want_ansible_ssh_host like script used to have'
 2020-06-05 11:25:11 -07:00
Jim Ladd 4ce37ec849 Bump foreman collection to 0.8.1 
* New release includes:
  'add host_filters and want_ansible_ssh_host like script used to have'
 2020-06-05 11:13:01 -07:00
Jim Ladd d242932837 foreman: use group_prefix for all groups 
* awx's "compatibility layer" for the foreman plugin had the
  group_prefix hard-coded to 'foreman_'
 2020-05-15 11:59:29 -07:00
Jim Ladd e64f9c6963 Merge pull request #4298 from jladdjr/add_foreman_options 
pass along all foreman options to plugin, add support for group_patterns
 2020-05-06 11:28:02 -07:00
Jim Ladd 84c854bdf3 add support for keyed_groups 2020-05-06 00:29:55 -07:00
Jim Ladd e243513a0d pass along all foreman options to plugin 2020-05-05 19:36:55 -07:00
Jim Ladd 45f8f0f412 Merge pull request #4264 from jladdjr/satellite6_want_ansible_ssh_host 
Add support for satellite6_want_ansible_ssh_host
 2020-04-27 11:39:03 -07:00
Jim Ladd 616e7082b3 Add support for satellite6_want_ansible_ssh_host 
* defaults to IPv6, falls back to IP
 2020-04-27 11:04:56 -07:00
Jim Ladd 6b015c9d81 Call parent get_plugin_env for foreman 2020-04-16 20:56:01 -04:00
Jim Ladd 6807878e2d I think this is the right place for iam_role_arn in the tests? 2020-04-16 20:56:01 -04:00
Jim Ladd 96c6cf9f05 pass iam_role_arn through to aws inv. plugin 2020-04-16 20:56:01 -04:00
Jim Ladd 8a20b5225b enable aws_ec2 plugin 2020-04-16 20:56:00 -04:00
Jim Ladd 0e2786d1f1 compat layer for foreman 2020-04-16 20:56:00 -04:00
Jim Ladd f38437b6bc foreman plugin updates 2020-04-16 20:56:00 -04:00
Jim Ladd 71d7bac261 Rename job_summary_dict to job_metadata 
* Clarifies purpose of notification template variable
 2019-10-29 11:24:14 -04:00
Jim Ladd fca9245536 Update unit tests 2019-10-29 11:24:14 -04:00
Jim Ladd 47031da65b Return full webhook dict when serializing notif. 2019-10-29 11:24:14 -04:00
Jim Ladd b024d91c66 Use correct notif. bodies when sending test notifs 
* Notification backends now handle body of notifications differently
* .. depending on their type (webhook, email, and pagerduty) are
  currently the only three notification types that use body
* email and pagerduty expect a string
* webhooks expects a dict in string format
 2019-10-29 11:24:14 -04:00
Jim Ladd da7002cf0c Don't use i18n for NT body string 2019-10-29 11:24:14 -04:00
Jim Ladd 12d735ec8f NotificationSerializer should gracefully handle webhook/pagerduty bodies 2019-10-29 11:24:13 -04:00
Jim Ladd 1e9173e8ef In awxkit, add support for wf approval notification templates 2019-10-29 11:24:13 -04:00
Jim Ladd 4809c40f3c Render WF approval notifications w/ custom templates 2019-10-29 11:24:13 -04:00
Jim Ladd 4e9ec271c5 Refactor notification backends to use CustomNotificationBase 2019-10-29 11:24:13 -04:00