29841 Commits

Author SHA1 Message Date
Ryan Petrello
b19bcdd882 remove the global /api/v2/job_events/ endpoint 2021-06-04 09:17:07 -07:00
Ryan Petrello
08b96a0bd7 correct filter events on the migration event horizon
events that existed *prior* to the partition migration will have
`job_created=1970-01-01` auto-applied at migration time; as such,
queries for these events e.g., /api/v2/job/N/job_events/
use 1970-01-01 in related event searche

events created *after* the partition migration (net-new playbook runs
will have `job_created` values that *exactly match* the related
`UnifiedJob.created` field.
2021-06-04 09:17:07 -07:00
Jim Ladd
1e45e2ab9b Rev migration number 2021-06-04 09:17:07 -07:00
Ryan Petrello
2a58605727 some more events view cleanup 2021-06-04 09:17:07 -07:00
Ryan Petrello
c7ab3ea86e move the partition data migration to be a post-upgrade async process
this copies the approach we took with the bigint migration
2021-06-04 09:17:07 -07:00
Jim Ladd
67046513ae Push changes before rebasing 2021-06-04 09:17:07 -07:00
Jim Ladd
f9b439ae82 include job_created field in all job event types 2021-06-04 09:17:07 -07:00
Jim Ladd
80b08d17e3 Continue updating job event views 2021-06-04 09:17:07 -07:00
Jim Ladd
f642c520bd Update fields for job event models to match raw sql operations
* raw sql commands were in migration to partition table
* .. just needed to add FakeAddField entries for the new
     job_created field added to each job event model
* .. and also needed to actually list the new field on the model classes
2021-06-04 09:17:07 -07:00
Jim Ladd
221ddeb915 Fix migration for _all_ job event tables
- each job event table has a different name for the
  fk referencing the unified job id
- create a mapping so that we get the col name correct
  for each table
2021-06-04 09:17:07 -07:00
Jim Ladd
d90d0fb503 fix initial partition creation
* call create_partition correctly (include tblname)
* reference 'awx_epoch'
2021-06-04 09:17:07 -07:00
Jim Ladd
2c529f50af Update querysets for remaining job event views
- search should use job creation date
  so that only the table partition with the relevant job events
  is searched
2021-06-04 09:17:07 -07:00
Jim Ladd
acfa1c4d1d Drop todo / question / conditional
* can safely assume job_created is set
* .. and if it isn't, we want to expose that bug
2021-06-04 09:17:07 -07:00
Jim Ladd
ea2afeec1f Drop todo / answered question 2021-06-04 09:17:07 -07:00
Jim Ladd
a5cfc3036f create_partition needs tblname 2021-06-04 09:17:07 -07:00
Jim Ladd
ec484f81cf Partition *all* job event tables 2021-06-04 09:17:06 -07:00
Jim Ladd
2ffa22e38f Misc doc clean-up 2021-06-04 09:17:06 -07:00
Jim Ladd
8fb313638c get job events based on job creation date 2021-06-04 09:17:06 -07:00
Jim Ladd
0eb1984b22 Only create partitions for regular jobs 2021-06-04 09:17:06 -07:00
Jim Ladd
f259b0a71b Move partition start date to 2000 2021-06-04 09:17:06 -07:00
Jim Ladd
82df3ebddb add option to create partitions that span a single minute
* turned on by default currently for testing
2021-06-04 09:17:06 -07:00
Jim Ladd
c87d7b0d79 fix import 2021-06-04 09:17:06 -07:00
Jim Ladd
612e91263c auto-create partition 2021-06-04 09:17:06 -07:00
Jim Ladd
445042c0f4 Create partition only if it doesn't exist 2021-06-04 09:17:06 -07:00
Jim Ladd
0c289205de Give new primary key constraint unique name, create first live partition 2021-06-04 09:17:06 -07:00
Jim Ladd
ba45592d93 create helper method to create partitions
* create_partition()
2021-06-04 09:17:06 -07:00
Jim Ladd
7e0f2b0f08 first partition is _not_ a special case after all 2021-06-04 09:17:06 -07:00
Jim Ladd
fb30528197 Dynamically create initial partitions
* First partition holds all events up to this very moment
* And second partition starts where first left off and runs
  .. through rest of current hour
* From there, dynamically generated partitions will cover
  one hour at a time
2021-06-04 09:17:06 -07:00
Jim Ladd
48f1910075 Remove temporary catch-all partition 2021-06-04 09:17:06 -07:00
Jim Ladd
0cb2d79889 Only save job_created field on JobEvent for the time being
* Once other job event tables are migrated, remove this
2021-06-04 09:17:06 -07:00
Jim Ladd
1af1a5e9da Convert job_created to string for serialization 2021-06-04 09:17:06 -07:00
Jim Ladd
c0d38e91f5 When saving JobEvents, include job_created
* this is the partition key
* .. used to determine which partition job event rows are sent to
2021-06-04 09:17:06 -07:00
Jim Ladd
2f737f644f Drop primary key index before creating partition table
* Partition tables require unique contstraints to include the
  partition key (uniqueness can only be enforced _inside_
  of a given partition table)
2021-06-04 09:17:06 -07:00
Jim Ladd
0574baf7f7 Create fake partition
* Just to get things working
* Will implement dynamic creation of partitions later
2021-06-04 09:17:06 -07:00
Jim Ladd
f70473dc0b When copying main_jobevent, include all table metadata
* copy the table just like we do in the bigint migration
* without this we lose sequences and very likely other things
  as well
* we want the new table to be identical to the old table,
  so 'including all' makes sense
2021-06-04 09:17:06 -07:00
Jim Ladd
de0b25862b By default, save job_created as null in db
* Want to avoid saving date as empty string
* Should default to null so that it's obvious the field is empty
2021-06-04 09:17:06 -07:00
Jim Ladd
6ff15a928a Register new column created by SQL
* .. using FakeAddField model type
* .. without doing this, Django won't know
     about the field we created using raw SQL
2021-06-04 09:17:06 -07:00
Jim Ladd
d10d1963c1 Rename / remove old main_jobevent table 2021-06-04 09:17:06 -07:00
Jim Ladd
c6acca08d5 first draft of db partitioning 2021-06-04 09:17:06 -07:00
softwarefactory-project-zuul[bot]
12f2975809
Merge pull request #10358 from jakemcdermott/fix-credlink
Make IG credential linkable

SUMMARY
Make the instance group credential detail link to the cred

Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
2021-06-04 15:12:03 +00:00
softwarefactory-project-zuul[bot]
945125454b
Merge pull request #10357 from jakemcdermott/fix-inventory-host-search-config
Add search config for inventory hosts

SUMMARY
Adds search config for inventory hosts.
(includes default icontains, etc.)

Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
2021-06-04 15:05:25 +00:00
softwarefactory-project-zuul[bot]
ea67c70437
Merge pull request #10257 from mabashian/6073-typeahead-lookup
Adds support for typing values into single select lookups

SUMMARY
link #6073
This adds support for typing values into single select lookups.  Multi-select lookups (credentials on the JT form, instance groups, etc) will remain unchanged.
An input is now rendered next to the lookup button which will allow users to type a value in.
Here's the button (which opens the modal):

Here's the input (which lets users type a value in):

There's a debounce on the input so that we only make a request to check to see if the name is valid after a second of no typing.
The tricky part of this implementation was handling validation on the lookup fields.  If a user types in a string that does not exactly match a value that they can use then we want to show an error indicating that no matching value was found:

but this was tricky because some of these fields are required so the validation functions would need to change a bit.
Since the typed text and the actual underlying value of the lookup are different the validation function would need to have access to both values whenever it runs for this to work.  This meant either storing the typed text up at the form level or pushing the validation down into the lookup(s).  After talking with @keithjgrant we decided to go down the route of pushing the validation down to the lookups.
You can now pass an optional validation function down to the various lookups that will get combined with the typeahead validation via the useField hook.
Here's the whole thing in action:

ISSUE TYPE

Feature Pull Request

COMPONENT NAME

UI

Reviewed-by: Kersom <None>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Michael Abashian <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
2021-06-04 15:03:53 +00:00
Jake McDermott
88ff68295b
Make IG credential linkable 2021-06-04 10:27:14 -04:00
Jake McDermott
d93f62c030
Add search config for inventory hosts 2021-06-04 09:51:43 -04:00
softwarefactory-project-zuul[bot]
c34fa30ea7
Merge pull request #10342 from mabashian/10338-checkbox-double-select
Fixes bug where checkbox list item was selecting things twice

SUMMARY
Resolves #10338
There was a click event on the row and the underlying checkbox.  I got rid of the underlying click event so now its only handled at the row level.

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

UI

Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
2021-06-03 21:37:05 +00:00
softwarefactory-project-zuul[bot]
3e5ee9d57a
Merge pull request #10322 from jbradberry/show-ee-resolver
Expose the EE resolver to the API

SUMMARY
This change exposes the output of .resolve_execution_environment() for job templates as a new summary field in the API.  Note that this is only exposed for the detail views, not the list views.  Also, there is the caveat for job templates that may be run under workflows that the workflow EE might override the results of this value.
related #10210
ISSUE TYPE

Feature Pull Request

COMPONENT NAME


API

AWX VERSION

Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Jeff Bradberry <None>
2021-06-03 21:11:33 +00:00
softwarefactory-project-zuul[bot]
c7dd3996df
Merge pull request #10351 from AlanCoding/sliced_ee
Skip sliced jobs from the workflow EE logic

Addressing some unanticipated fallout from #10305
Sliced jobs rely on creating a workflow job, but they do not have a workflow_job_template, so in those cases, that was causing a traceback.
2021-06-03 20:10:52,319 ERROR    [a17ebd7f] awx.main.dispatch Worker failed to run task awx.main.tasks.RunJob(*[341], **{}
Traceback (most recent call last):
  File "/awx_devel/awx/main/dispatch/worker/task.py", line 90, in perform_work
    result = self.run_callable(body)
  File "/awx_devel/awx/main/dispatch/worker/task.py", line 65, in run_callable
    return _call(*args, **kwargs)
  File "/awx_devel/awx/main/tasks.py", line 759, in _wrapped
    return f(self, *args, **kwargs)
  File "/awx_devel/awx/main/tasks.py", line 1264, in run
    self.instance = self.update_model(self.instance.pk, execution_environment=self.instance.resolve_execution_environment())
  File "/awx_devel/awx/main/models/mixins.py", line 477, in resolve_execution_environment
    if wf_template.execution_environment is not None:
AttributeError: 'NoneType' object has no attribute 'execution_environment'

that is fixed with this, at least for my one simple test case
This left jobs hanging out in "waiting" status, which is really not good.

Reviewed-by: Jeff Bradberry <None>
2021-06-03 21:02:01 +00:00
softwarefactory-project-zuul[bot]
e342919735
Merge pull request #10323 from wenottingham/look-over-there
Add additional controller directory for collections for inventory update

SUMMARY
This may be present in some scenarios with additional collections for inventory usage.

Reviewed-by: Alan Rominger <arominge@redhat.com>
2021-06-03 20:42:04 +00:00
Jeff Bradberry
d0991bab9e Expose the EE resolver to the job template detail API endpoints 2021-06-03 16:35:50 -04:00
Alan Rominger
efcbea1fc5
Skip sliced jobs from the workflow EE logic 2021-06-03 16:20:47 -04:00