External-Mirrors/awx
2
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2026-09-16 17:00:12 -02:30
Code Issues Packages Projects Releases Wiki Activity
30,899 Commits 63 Branches 102 Tags
a28c023cf1972c285e85703853d42d78c014e0c3
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jim Ladd 112111c7f9 Merge pull request #10904 from jladdjr/do_not_collect_artifact_data 
do not collect artifact_data when gathering analytics
 2021-10-07 22:46:00 -07:00
Jim Ladd 815a45cf2f call 'work cancel' on inactive controller jobs 2021-10-01 12:55:06 -07:00
Jim LaddandAlan Rominger 1b50db26b6 Explicitly pass in UUID to get_or_register 
Co-authored-by: Alan Rominger <arominge@redhat.com>
 2021-09-14 10:58:29 -07:00
Jim Ladd f39834ad82 pass uuid to Instance.create 2021-09-03 10:05:15 -07:00
Jim Ladd bdb13343bb remove unused import 2021-09-03 10:05:15 -07:00
Jim Ladd 262cd3c695 set default uuid 2021-09-03 10:05:15 -07:00
Jim Ladd f02099e8b7 provision_instance should create new uuid if needed 
.. instead of default to current system's UUID

related: #10990
 2021-09-03 10:05:15 -07:00
Jim Ladd 467a37f8fe use settings.DEFAULT_EXECUTION_QUEUE_NAME in lieu of default 2021-08-26 11:15:14 -07:00
Jim Ladd 88a6412b54 only need to update IG's policy_instance_list field 2021-08-26 11:15:14 -07:00
Jim Ladd 502eaf9fb9 handle case where default IG does not exist 
* also, only add discovered execution node to default group
  if `register`-ing the node actually resulted in a confirmed
  change
 2021-08-26 11:15:14 -07:00
Jim Ladd de8eab0434 inspect_execution_nodes should *not* block when retreiving lock 
* would otherwise hold up cluster heartbeat task
* furthermore, only really need one node to run through
  `inspect_execution_nodes` each interval
 2021-08-26 11:15:14 -07:00
Jim Ladd f317fca9e4 add auto-discovered nodes to default IG 
* add advisory_lock to avoid IG update race logic
* update IG by way of policy_instance_list
 2021-08-26 11:15:14 -07:00
Jim Ladd 561fc289fb disable discovered instances by default 2021-08-26 11:15:14 -07:00
Jim Ladd 77933e97c0 create default IG when bringing up dev env 2021-08-26 11:15:14 -07:00
Jim Ladd a79d7444e5 set userid to 1000 (#10714) 2021-08-24 08:40:20 -04:00
Jim Ladd e3893b1887 do not collect artifact_data when gathering analytics 
- also, store event_data in jsonb object
- .. in order to have data structure that supports '-' operator
 2021-08-17 14:55:16 -07:00
Jim Ladd 31fe500921 move get_queryset handling to child view 2021-06-04 09:17:09 -07:00
Jim Ladd c429563126 update view to handle hosts/N/ad_hoc_command_events 2021-06-04 09:17:09 -07:00
Jim Ladd 2a23b4c719 bump migration 2021-06-04 09:17:09 -07:00
Jim Ladd f7d2f7a5e6 lint 2021-06-04 09:17:09 -07:00
Jim Ladd e371de38ed update job cleanup tests for sqlite-based execution 2021-06-04 09:17:09 -07:00
Jim Ladd 84af610a1f remove rebase cruft 2021-06-04 09:17:09 -07:00
Jim Ladd ef9f9129ba bump migration 2021-06-04 09:17:09 -07:00
Jim Ladd 7b188aafea lint 2021-06-04 09:17:09 -07:00
Jim Ladd 0f9f3f58e2 bump migration 2021-06-04 09:17:08 -07:00
Jim Ladd 6123b8e148 query for jobevents based on table location 
* pre-migration jobevents live in unpartitioned table
  where only created field has index
* post-migration jobevents live in partitions
  where modified field has index
  (and should be used to ensure no events are missing)
 2021-06-04 09:17:08 -07:00
Jim Ladd b86d365dde collect job events based on job event modified time 
* when collecting job events by creation time
  it is possible to miss events that were created
  at one point, but actually committed to the db
  much later.
* since events' modified time is set when they are
  committed to the db, we shouldn't miss any job events
* selecting job events by modified time wasn't possible
  beforehand because we didn't have an index for
  jobevent's modified field
 2021-06-04 09:17:08 -07:00
Jim Ladd fb97687d14 lint 2021-06-04 09:17:08 -07:00
Jim Ladd 0f53d9b911 bump db partition migration 2021-06-04 09:17:08 -07:00
Jim Ladd 5a785798b0 is_partitioned should default to true when pk not set 2021-06-04 09:17:08 -07:00
Jim Ladd 14168297bd set event horizon to -1 for empty tables 2021-06-04 09:17:08 -07:00
Jim Ladd 7e1814e234 mock has_unpartitioned_events in collection tests 
.. just like we do with the main awx tests
 2021-06-04 09:17:08 -07:00
Jim Ladd bdf11aa962 add migrations for Unpartitioned{Job}Event proxy models 2021-06-04 09:17:08 -07:00
Jim Ladd 46807205f8 Move created kwargs to right place 2021-06-04 09:17:08 -07:00
Jim Ladd d749c172eb Ensure View.model exists before checking type 2021-06-04 09:17:08 -07:00
Jim Ladd 81db8091ea test updates 
* when tests create a UnifiedJob and JobEvent,
  the two need to have the same value for job creation time
* some view validation was skipped due to `model` being
  a property in some cases now
 2021-06-04 09:17:08 -07:00
Jim Ladd 5c1a33382c update mocks to reflect new migration 
* instead of mocking `created_or_epoch` (which no longer exists)
* .. mock `Unified_Job.has_unpartitioned_events`
 2021-06-04 09:17:08 -07:00
Jim Ladd db6f565dca black formatting 2021-06-04 09:17:08 -07:00
Jim Ladd 6b4effc85a bump partition migration to 135 2021-06-04 09:17:08 -07:00
Jim Ladd 74a0c5bac5 Minor rebase fallout 
* fix import
* remove dropped url
 2021-06-04 09:17:08 -07:00
Jim Ladd 661cf0afb3 short-circuit event_processing_finished for wf jobs 
* wf jobs are a wrapper for other jobs
* they do not process their own job events
 2021-06-04 09:17:08 -07:00
Jim Ladd 0eddd5ce7f Enable partition pruning when fetching job's events 2021-06-04 09:17:07 -07:00
Jim Ladd a7cabec3d0 bump partition migration to 0132 2021-06-04 09:17:07 -07:00
Jim Ladd b98b3ced1c update tests to reflect new job_event schema 2021-06-04 09:17:07 -07:00
Jim Ladd 8501a45531 lint fixes 2021-06-04 09:17:07 -07:00
Jim Ladd 14b610dabf bump partition migration 2021-06-04 09:17:07 -07:00
Jim Ladd a1d1e70e43 correct constraint name 2021-06-04 09:17:07 -07:00
Jim Ladd 0fa0a517ac create tmp schema to reference when creating partitioned table 
* if we use the actual old job events table
  and make tweaks to its schema
  namely, dropping the pkey constraint,
  then when we go to migrate the old job events
  we will be forcing postgres to do a sequential scan
  on the old table, which effectively causes the migration to hang
 2021-06-04 09:17:07 -07:00
Jim Ladd 28f9c0be0b Do not cascade delete job_events 
* want to drop job_event _partitions_
  .. instead of having the job events associated with a job
     automatically get cleaned up for us
 2021-06-04 09:17:07 -07:00
Jim Ladd 373edbf8c0 Update reference to partition migration 2021-06-04 09:17:07 -07:00
Jim Ladd 1e45e2ab9b Rev migration number 2021-06-04 09:17:07 -07:00
Jim Ladd 67046513ae Push changes before rebasing 2021-06-04 09:17:07 -07:00
Jim Ladd f9b439ae82 include job_created field in all job event types 2021-06-04 09:17:07 -07:00
Jim Ladd 80b08d17e3 Continue updating job event views 2021-06-04 09:17:07 -07:00
Jim Ladd f642c520bd Update fields for job event models to match raw sql operations 
* raw sql commands were in migration to partition table
* .. just needed to add FakeAddField entries for the new
     job_created field added to each job event model
* .. and also needed to actually list the new field on the model classes
 2021-06-04 09:17:07 -07:00
Jim Ladd 221ddeb915 Fix migration for _all_ job event tables 
- each job event table has a different name for the
  fk referencing the unified job id
- create a mapping so that we get the col name correct
  for each table
 2021-06-04 09:17:07 -07:00
Jim Ladd d90d0fb503 fix initial partition creation 
* call create_partition correctly (include tblname)
* reference 'awx_epoch'
 2021-06-04 09:17:07 -07:00
Jim Ladd 2c529f50af Update querysets for remaining job event views 
- search should use job creation date
  so that only the table partition with the relevant job events
  is searched
 2021-06-04 09:17:07 -07:00
Jim Ladd acfa1c4d1d Drop todo / question / conditional 
* can safely assume job_created is set
* .. and if it isn't, we want to expose that bug
 2021-06-04 09:17:07 -07:00
Jim Ladd ea2afeec1f Drop todo / answered question 2021-06-04 09:17:07 -07:00
Jim Ladd a5cfc3036f create_partition needs tblname 2021-06-04 09:17:07 -07:00
Jim Ladd ec484f81cf Partition *all* job event tables 2021-06-04 09:17:06 -07:00
Jim Ladd 2ffa22e38f Misc doc clean-up 2021-06-04 09:17:06 -07:00
Jim Ladd 8fb313638c get job events based on job creation date 2021-06-04 09:17:06 -07:00
Jim Ladd 0eb1984b22 Only create partitions for regular jobs 2021-06-04 09:17:06 -07:00
Jim Ladd f259b0a71b Move partition start date to 2000 2021-06-04 09:17:06 -07:00
Jim Ladd 82df3ebddb add option to create partitions that span a single minute 
* turned on by default currently for testing
 2021-06-04 09:17:06 -07:00
Jim Ladd c87d7b0d79 fix import 2021-06-04 09:17:06 -07:00
Jim Ladd 612e91263c auto-create partition 2021-06-04 09:17:06 -07:00
Jim Ladd 445042c0f4 Create partition only if it doesn't exist 2021-06-04 09:17:06 -07:00
Jim Ladd 0c289205de Give new primary key constraint unique name, create first live partition 2021-06-04 09:17:06 -07:00
Jim Ladd ba45592d93 create helper method to create partitions 
* create_partition()
 2021-06-04 09:17:06 -07:00
Jim Ladd 7e0f2b0f08 first partition is _not_ a special case after all 2021-06-04 09:17:06 -07:00
Jim Ladd fb30528197 Dynamically create initial partitions 
* First partition holds all events up to this very moment
* And second partition starts where first left off and runs
  .. through rest of current hour
* From there, dynamically generated partitions will cover
  one hour at a time
 2021-06-04 09:17:06 -07:00
Jim Ladd 48f1910075 Remove temporary catch-all partition 2021-06-04 09:17:06 -07:00
Jim Ladd 0cb2d79889 Only save job_created field on JobEvent for the time being 
* Once other job event tables are migrated, remove this
 2021-06-04 09:17:06 -07:00
Jim Ladd 1af1a5e9da Convert job_created to string for serialization 2021-06-04 09:17:06 -07:00
Jim Ladd c0d38e91f5 When saving JobEvents, include job_created 
* this is the partition key
* .. used to determine which partition job event rows are sent to
 2021-06-04 09:17:06 -07:00
Jim Ladd 2f737f644f Drop primary key index before creating partition table 
* Partition tables require unique contstraints to include the
  partition key (uniqueness can only be enforced _inside_
  of a given partition table)
 2021-06-04 09:17:06 -07:00
Jim Ladd 0574baf7f7 Create fake partition 
* Just to get things working
* Will implement dynamic creation of partitions later
 2021-06-04 09:17:06 -07:00
Jim Ladd f70473dc0b When copying main_jobevent, include all table metadata 
* copy the table just like we do in the bigint migration
* without this we lose sequences and very likely other things
  as well
* we want the new table to be identical to the old table,
  so 'including all' makes sense
 2021-06-04 09:17:06 -07:00
Jim Ladd de0b25862b By default, save job_created as null in db 
* Want to avoid saving date as empty string
* Should default to null so that it's obvious the field is empty
 2021-06-04 09:17:06 -07:00
Jim Ladd 6ff15a928a Register new column created by SQL 
* .. using FakeAddField model type
* .. without doing this, Django won't know
     about the field we created using raw SQL
 2021-06-04 09:17:06 -07:00
Jim Ladd d10d1963c1 Rename / remove old main_jobevent table 2021-06-04 09:17:06 -07:00
Jim Ladd c6acca08d5 first draft of db partitioning 2021-06-04 09:17:06 -07:00
Jim Ladd a6f3817488 verify notification errors included in NT list view 2021-05-03 13:17:34 -07:00
Jim Ladd e4eb03259b include error field in notification template's list of recent notifs 2021-05-03 13:17:34 -07:00
Jim Ladd fb97a79aca add clarifying comment re: AUTOMATION_ANALYTICS_LAST_GATHER being set regardless 2021-04-21 21:28:32 -07:00
Jim Ladd a1aec29e48 let jupyter install ipython 2021-03-25 13:17:21 -07:00
Jim Ladd 5b3bb1e81d set schedule.next_run to datetime.datetime 
* Update JobNotificationMixin.context_stub
* .. TestJobNotificationMixin.CONTEXT_STRUCTURE
 2021-02-05 13:56:56 -08:00
Jim Ladd 60dee83481 revert jinja2 dep changes for ansible venv 2021-01-11 11:48:39 -08:00
Jim Ladd 197d50bc44 patch test_inventory_update_injected_content 
* proot now enabled at task-level
  since tasks are no longer calling
  awx-manage (which would set up its own proot)
* dropping proot env var since it's not
  relevant to the test
 2020-11-30 11:35:32 -05:00
Jim Ladd 5ad60a3ed4 use inventory_id to get advisory_lock 2020-11-30 11:35:32 -05:00
Jim Ladd 38638b4a6b add note to remove private_dir when proot removed 2020-11-30 11:35:32 -05:00
Jim Ladd 232801e0ba cache end_line for RunInventoryUpdate jobs 2020-11-30 11:35:32 -05:00
Jim Ladd d55f36eb90 add clarifying comment 
* noting that the inv update task only uses the
  inventory update management command to
  save the inv to the database
  (it doesn't do the work of fetching hosts / groups)
 2020-11-30 11:35:31 -05:00
Jim Ladd 277c47ba4e add TODO reminders to remove proot / venv code 
* won't be needed once we move to container-based
  execution
 2020-11-30 11:35:31 -05:00
Jim Ladd 12cbc9756b inventory updates should use custom venv 2020-11-30 11:35:31 -05:00
Jim Ladd 72df8723f6 lint 2020-11-30 11:35:31 -05:00
Jim Ladd a8710bf2f1 restore proot for inventory updates 
- in the past, inv. update jobs called `awx-manage inventory_update`
  which took care of setting up process isolation
- at this point, though, inv. update jobs call runner / ansible-inventory
  directly, so we need another way to put process isolation in place
- thankfully, there was already support for providing process isolation
  for other types of jobs (namely JT Jobs, Project Updates and Ad Hoc
  commands)
- so, we do what those other jobs do and override the stub for should_use_proot
  (which by default returns false) so that it keys off of the
  `AWX_PROOT_ENABLED` setting
 2020-11-30 11:35:31 -05:00