Commit Graph

25033 Commits

Author SHA1 Message Date
Alan Rominger
7d30dff075 Feature indirect host counting (#15802)
* AAP-37282 Add parse JQ data and test it for a `job` object in isolation (#15774)

* Add jq dependency

* Add file in progress

* Add license for jq

* Write test and get it passing

* Successfully test collection of `event_query.yml` data (#15761)

* Callback plugin method from cmeyers adapted to global collection list

Get tests passing

Mild rebranding

Put behind feature flag, flip true in dev

Add noqa flag

* Add missing wait_for_events

* feat: try grabbing query files from artifacts directory (#15776)

* Contract changes for the event_query collection callback plugin (#15785)

* Minor import changes to collection processing in callback plugin

* Move agreed location of event_query file

* feat: remaining schema changes for indirect host audits (#15787)

* Re-organize test file and move artifacts processing logic to callback (#15784)

* Rename the indirect host counting test file

* Combine artifacts saving logic

* Connect host audit model to jq logic via new task

* Add unit tests for indirect host counting (#15792)

* Do not get django flags from database (#15794)

* Document, implement, and test remaining indirect host audit fields (#15796)

* Document, implement, and test remaining indirect host audit fields

* Fix hashing

* AAP-39559 Wait for all event processing to finish, add fallback task (#15798)

* Wait for all event processing to finish, add fallback task

* Add flag check to periodic task

* feat: cleanup of old indirect host audit records (#15800)

* By default, do not count indirect hosts (#15801)

* By default, do not count indirect hosts

* Fix copy paste goof

* Fix linter issue from base branch

* prevent multiple tasks from processing the same job events, prevent p… (#15805)

prevent multiple tasks from processing the same job events, prevent periodic task from spawning another task per job

* Fix typos and other bugs found by Pablo review

* fix: rely on resolved_action instead of task, adapt to proposed query… (#15815)

* fix: rely on resolved_action instead of task, adapt to proposed query structure

* tests: update indirect host tests

* update remaining queries to new format

* update live test

* Remove polling loop for job finishing event processing (#15811)

* Remove polling loop for job finishing event processing

* Make awx/main/tests/live dramatically faster (#15780)

* AAP-37282 Add parse JQ data and test it for a `job` object in isolation (#15774)

* Add jq dependency

* Add file in progress

* Add license for jq

* Write test and get it passing

* Successfully test collection of `event_query.yml` data (#15761)

* Callback plugin method from cmeyers adapted to global collection list

Get tests passing

Mild rebranding

Put behind feature flag, flip true in dev

Add noqa flag

* Add missing wait_for_events

* feat: try grabbing query files from artifacts directory (#15776)

* Contract changes for the event_query collection callback plugin (#15785)

* Minor import changes to collection processing in callback plugin

* Move agreed location of event_query file

* feat: remaining schema changes for indirect host audits (#15787)

* Re-organize test file and move artifacts processing logic to callback (#15784)

* Rename the indirect host counting test file

* Combine artifacts saving logic

* Connect host audit model to jq logic via new task

* Document, implement, and test remaining indirect host audit fields (#15796)

* Document, implement, and test remaining indirect host audit fields

* Fix hashing

* AAP-39559 Wait for all event processing to finish, add fallback task (#15798)

* Wait for all event processing to finish, add fallback task

* Add flag check to periodic task

* feat: cleanup of old indirect host audit records (#15800)

* prevent multiple tasks from processing the same job events, prevent p… (#15805)

prevent multiple tasks from processing the same job events, prevent periodic task from spawning another task per job

* Remove polling loop for job finishing event processing (#15811)

* Remove polling loop for job finishing event processing

* Make awx/main/tests/live dramatically faster (#15780)

* temp

* remove test

* reorder migrations to allow indirect instances backport

* cleanup for rebase and merge into devel

---------

Co-authored-by: Peter Braun <pbraun@redhat.com>
Co-authored-by: jessicamack <jmack@redhat.com>
Co-authored-by: Peter Braun <pbranu@redhat.com>
2025-02-24 16:39:51 +00:00
Andrea Restle-Lay
70ea0a785b revert change made to allow UI to accept x-access-token, just use htt… (#15851)
revert change made to allow UI to accept x-access-token, just use https:// instead
2025-02-21 21:10:22 +00:00
Jake Jackson
fa099fe737 Add Github dep for new cred support if used (#15850)
* Add pygithub for new app token support

* fixed git requirements file with new
* added new github dep and relevant deps it needs

* add required licenses

* Add artifacts to satisfy license check

* Remove duplicated license

---------

Co-authored-by: Andrea Restle-Lay <arestlel@redhat.com>
Co-authored-by: Alan Rominger <arominge@redhat.com>
2025-02-20 21:16:02 +00:00
Andrea Restle-Lay
bf4d45452c feat: 38589 GitHub App Authentication (#15807)
* feat: 38589 GitHub App Authentication

Allows both git@<personal-token> and x-access-token@<github-access-token> when authenticating using git.
This allows GitHub App tokens to work without interfering with existing authentication types.

---------

Co-authored-by: Jake Jackson <thedoubl3j@Jakes-MacBook-Pro.local>
2025-02-19 23:13:45 +00:00
jessicamack
e56752d55b Ship analytics data using service account token (#15812)
Use oidc client to ship analytics data
2025-02-19 16:38:47 -05:00
Alan Rominger
3495c421c1 Fix: do not use current apps in migrations (#15839) 2025-02-19 07:58:24 -05:00
Hao Liu
69a60493a3 Update feature flag list test (#15830) 2025-02-12 18:29:52 +00:00
Alan Rominger
7d2b2d672c Make awx/main/tests/live dramatically faster (#15780)
* Make awx/main/tests/live dramatically faster

* Add new setting to exclude list
2025-02-08 21:07:56 -05:00
Alan Rominger
26346d237d Fix rsyslog permission error in github ubuntu tests from apparmor (#15717)
* Add test to detect rsyslog config problems

* Get dmesg output

* Disable apparmor for rsyslogd
2025-02-05 08:38:55 -05:00
Seth Foster
c2e5425d93 Fix rrule fast forwarding across DST boundaries (#15809)
Fixes an issue where schedules were not running at
the correct time.

Details:

DST is Daylights Saving Time

If the rrule dtstart is "in" a DST period (i.e., March to November)
and the current date is outside of the DST, then the fast forwarding
is not correct.

This is because datetime timedeltas do not honor DST boundaries

The Fix:

Convert the rrule dtstart to UTC before doing operations. Then,
convert back to the original timezone at the end.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2025-02-04 15:19:49 -05:00
Hao Liu
15932e3f7c Set feature flag base on setting (#15808) 2025-02-04 11:00:05 -05:00
Zack Kayyali
a74e7301cd [AAP-39138] - Add DAB Feature Flag common API (#15786)
* Add DAB Feature Flag common API

* Use updated API /feature_flags_state/

* fix git reference

* organization updates
2025-02-03 11:40:16 +01:00
Chris Meyers
b53c576944 Add helper to proxy analytics requests
* Handles OIDC token creation and usage transparently
2025-01-30 19:15:41 -05:00
Alan Rominger
c418bc034f Put duplicate plugin location in error message (#15781) 2025-01-29 17:15:02 -05:00
Alan Rominger
d639953a4c Removing some (but not all) dead pytest fixtures (#15782)
* Try removing dead fixtures

* Add back in EE fixture
2025-01-29 17:13:31 -05:00
Jake Jackson
c6930bdf32 Address Lookup Plugins AttributeError (#15770)
* fix backend attribute error

* managedcredential may now contain 2 different classes
* managedcredentialType and one that represents a lookup plugin

* conditionalize creation params

* added a conditional statement to filter our external types

* all external credentials are managed by awx/aap
2025-01-29 10:27:51 -05:00
Peter Braun
d36cd6c6ab fix: compatibility with black v25+ (#15789) 2025-01-29 15:06:14 +00:00
Alan Rominger
1f503645fd Move some more tests out of root functional folder (#15753) 2025-01-24 15:04:59 -05:00
Chris Meyers
ad706d67c2 Add ee cleanup tests
* Adds cleanup tests to the live test.
2025-01-24 10:55:30 -05:00
jessicamack
4eed454ed7 Establish a feature flag for indirect host counting feature (#15759)
* add feature flag for indirect node counting

* fix name of flag
2025-01-21 20:00:57 +01:00
Alan Rominger
c43dfde45a Create test for using manual & file projects (#15754)
* Create test for using a manual project

* Chang default project factory to git, remove project files monkeypatch

* skip update of factory project

* Initial file scaffolding for feature

* Fill in galaxy and names

* Add README, describe project folders and dependencies
2025-01-20 17:06:15 -05:00
Adrià Sala
46403e4312 fix: invalid f-string and oidc url for insights plugin 2025-01-20 17:48:19 +01:00
Adrià Sala
492c7a1af6 feat: support insights service account credentials for project update 2025-01-17 16:30:07 +01:00
Adrià Sala
a19e1ba28f feat: update insights action plugin to handle oauth (#15742) 2025-01-16 10:44:40 +01:00
Jake Jackson
f05173cb65 Add new credential entry point discovery (#15685)
* - add new entry points
- add logic to check what version of the project is running

* remove former discovery method

* update custom_injectors and remove unused import

* fix how  we load external creds

* remove stale code to match devel

* fix cloudforms test and move credential loading

* add load credentials method to get tests passing

* Conditionalize integration tests if the cred is present

* remove inventory source test

* inventory source is covered in the workflow job template target
2025-01-15 16:10:28 -05:00
Alan Rominger
f57a9863d6 Use advisory_lock from DAB (#15676)
* Use advisory_lock from DAB

* Remove the django-pglocks dep

* Re-run updater script

* Move the import in new location
2025-01-15 14:06:59 -05:00
Alan Rominger
fb12c834eb Add test to ensure bootstrap reqs are good (#15733)
* Add test to ensure bootstrap reqs are good

* Give full diff list in assert
2025-01-13 14:31:19 -05:00
Alan Rominger
3e50b019e0 Delete test file that should have been removed and fix checks (#15739)
* Delete test file that should have been removed

* Add more insights env variables
2025-01-10 13:10:24 -05:00
Alan Rominger
2d7bbc4ec8 AAP-37080 Delete the cleanup_tokens system job template (#15711)
Delete the cleanup_tokens system job template
2025-01-10 10:33:35 -05:00
Roman Petrakov
56079612c8 Fix API documentation rendering (#15116) (#15726)
* Fix API documentation rendering related #15116

* Fix tests and formatting issues #15116
2025-01-07 15:09:18 -05:00
Alan Rominger
2186c24c8f General upgrade of dependencies (#15705)
* General upgrade of dependencies

* adjust licenses to match requirements

* add missing licenses

* another pass to fix licenses

* Try easy for for psycopg encoding pattern change

---------

Co-authored-by: jessicamack <jmack@redhat.com>
2025-01-07 15:03:43 -05:00
Alan Rominger
9a5ed20ed5 AAP-37989 Tests for exclude list with multiple jobs (#15722)
* Tests for exclude list with multiple jobs
2025-01-02 16:38:09 -05:00
Sasa Jovicic
7835e39bac Bugfix: adjust incorrectly passed keywords with exclude-strings argument (#15721)
* Fix incorrectly passed keywords with exclude-strings arg to ansible-runner worker cleanup command

Signed-off-by: Sasa Jovicic <jovicic.sasa@hotmail.com>

* Keep the quotes for each arg and adjust test_receptor

---------

Signed-off-by: Sasa Jovicic <jovicic.sasa@hotmail.com>
2025-01-02 16:16:14 -05:00
Alan Rominger
14808cb99b Move RBAC functional tests into folder (#15723) 2024-12-20 14:54:52 -05:00
Chris Meyers
cf9e6796ea Move cred type unite tests to awx-plugins 2024-12-19 14:03:27 -05:00
Chris Meyers
bd96000494 Remove inject_credential from awx
* Consume inject_credential from its new home, awx_plugins.interfaces
2024-12-19 09:48:47 -05:00
Andrea Restle-Lay
1b418f75e6 AAP-36604 (analytics) Thousands of zombie/orphaned Slow/Stuck DB queries in controller querying active host count (#15715)
* lint

* change timeout to 5 minutes

* change timeout to 5 minutes
2024-12-18 22:12:52 +00:00
Alan Rominger
288e8d78d3 Cleanup in-memory data from test that randomly causes other failures (#15716) 2024-12-18 16:59:42 -05:00
Alan Rominger
c0158181c3 Fix test warnings that escaped somehow (#15714) 2024-12-18 15:21:53 -05:00
Elijah DeLee
29f36793de Min value should be Decimal (#15413)
This hopefully resolves error message seen in logs sometimes about "should be Decimal type"
2024-12-17 12:45:39 -05:00
Alan Rominger
36c75a2c62 AAP-36536 Send job_lifecycle logs to external loggers (#15701)
* Send job_lifecycle logs to external loggers

* Include structured data in message

* Attach the organization_id of the job
2024-12-16 15:49:16 -05:00
Andrea Restle-Lay
86d202456a host_metrics date fix to make summary dates (datetime.datetime) comparable to month: datetime.date (#15704)
* host_metrics date fix

* AAP-36839 Remove excess comments

* fix extra date() conversion

* actual fix

* datetime is a library, use datetime.datetime

---------

Co-authored-by: Andrea Restle-Lay <arestlel@arestlel-thinkpadx1carbongen9.rht.csb>
2024-12-16 12:10:59 -05:00
jessicamack
c1f0a831ff Pull the correct collection plugin for the product (#15658)
* pull the correct collection plugin for the product

* remove unused import and logging line

* refactor code to load entry points

* reformat method

* lint fix

* renames for clarity and a lint fix

* move function to utils

* move the rest of the code into load_inventory_plugins

* temp - confirm that tests will pass

* revert change caught in merge

* change back requirement

the related PR has been merged
2024-12-16 11:05:21 -05:00
Seth Foster
e605883592 Do not fast forward rrule if count is set (#15696)
Fixes a bug where a schedule that was created
to run only once will continue to run repeatedly.

e.g. an rrule with
dtstart 20240730; count 1; freq MINUTELY

This job will run on 20240730, and should never
run again.

However, the next time the schedule
update_computed_fields runs, the dtstart
will fast forward to today's date, and
next_run will be computed from that. This will trigger
the job to run again, which is not intended.

If count is set, we just should not fast forward the
rrule and always calculate next_run based on original
dtstart.

Signed-off-by: Seth Foster <fosterbseth@gmail.com>
2024-12-11 11:14:58 -05:00
Alan Rominger
f377b5fdde Use runtime log utility moved to DAB (#15675)
* Use runtime log utility moved to DAB
2024-12-11 10:38:24 -05:00
Alan Rominger
32122e6822 Fix misused project cache identifier (#15690)
Fix project cache identifiers for new updates

Finish test and discover viable solution

Add comment on related task code
2024-12-10 15:26:17 -05:00
Chris Meyers
a129bc860b Flake8 fix 2024-12-10 13:02:09 -05:00
Chris Meyers
c82a8f4b9c Add custom_injectors to test code path
* Unit tests do not create CredentialType records for Credential
  plugins. Instead, they explicitly instantiate CredentialType(s) for
  Credential plugins. They rely on CredentialType.defaults[key] to do
  so. This change makes sure custom_injectors get bolted onto the
  created CredentialType.
2024-12-10 13:02:09 -05:00
Chris Meyers
99c18b681d Load all plugins in order to test them 2024-12-10 13:02:09 -05:00
Chris Meyers
aeca9db470 Rename post_injectors to custom_injectors 2024-12-10 13:02:09 -05:00