- enable schema upload to s3 bucket for feature branch
- add workflow to delete schema from s3 bucket when feature branch is deleted
Signed-off-by: Hao Liu <haoli@redhat.com>
Previously, in some cases, an InventoryUpdate sourced by an SCM project
would still run and be successful even after the project it is sourced
from failed to update. This would happen because the InventoryUpdate
would revert the project back to its last working revision. This
behavior is confusing and inconsistent with how we handle jobs (which
just refuse to launch when the project is failed).
This change pulls out the logic that the job launch serializer and
RunJob#pre_run_hook had implemented (independently) to check if the
project is in a failed state, and puts it into a method on the Project
model. This is then checked in the project launch serializer as well as
the inventory update serializer, along with
SourceControlMixin#sync_and_copy as a fallback for things that don't run
the serializer validation (such as scheduled jobs and WFJT jobs).
Signed-off-by: Rick Elrod <rick@elrod.me>
Really these could get any of the unified job template types, not just
system job templates, so importing e.g. a project with a schedule was
doing them in the wrong order.
Also, bump the timeout of the project update and make sure that we
stash it in the page cache even if it doesn't finish in 5 minutes.
when running `make ui-devel`. Previously they were going to
/awx_devel/awx/public/static, but that directory is no longer being
served up by nginx, which forced us to have to run `make
collectstatic` (or equivalent) to get the files to the right place.
More fun in the grafana dashboard. The rows organize the panels and are
collapsable. Also, tested with multiple nodes and fixed some
labeling issues when there are more than one node.
Update grafana alerting readme info and some fun prose about one of the
alerts as well as some reorganizing of the code for clarity.
finally, drop the time to fire for alerts because it's better to have them be a bit touchy so users can verify they work vs. not being sure.
* initial commit of hostname validation to InstanceSerializer
Co-authored-by: Cesar Francisco San Nicolas Martinez <cesarfsannicolasmartinez@gmail.com>
- Firstly -- add a bunch of unit tests for `update_scm_url`, because it
previously had none and desperately needed them.
- Secondly -- fix#12992 by adding back in IPv6 address brackets if they
existed in the first place when the function was called.
- Thirdly -- fix a related case where we disallowed IPv6 in URLs that
did not include the scheme.
Signed-off-by: Rick Elrod <rick@elrod.me>
Extrapolating reconciliation of desired and actual states to a function
Converting heave prefect related methods to user focus for query optimization
Converting from get_or_create to simply create
Added memory calculations for query optimization
* fix name to be consistent
this is not a mean, its the last value
so say that in the name
* add remaining capacity to dashboard
also make legends pretty with nice names
- stdout output on events was being double HTML entity encoded meaning
that all output with < and > was shown as literal "<" and ">"
Signed-off-by: Rick Elrod <rick@elrod.me>
The initial check performed case insensitive searches and the new method was case sensitive
The optimization of the new method is likely not going to contribute noticable slowness
This will enable us to provide more useful information for the user,
now that all user-triggered health checks are async.
Also, de-bounce the health check endpoint to not allow additional
health check tasks to be triggered when one is already in progress.
We don't specify defaults in the module (because it messes up Instance
updates because AWX things we are trying to change things to be the
default).
- Update the docs to remove the defaults that no longer exist
- Update tests to make them pass (oops)
- Fix tangentially related typo in Kind development docs
Signed-off-by: Rick Elrod <rick@elrod.me>
We were disabling the field when a user did not have sufficient permissions to create an Inventory. I updated this logic to check if a user has use permissions on the selected inventory before disabling the field.
typo in URL and in grafana alert rule
Important learning: no newlines in rules/equations
turns out datasourceUid can be set in prometheus_source.yml, and it can be anything we want. So I have set it to awx_alert, the PBFAnumbersetc value it was set to before was an autogenerated UID, and it would actually work just with that generated value, but because we want it to make sense, we're setting the value in prometheus_source.yml
finally, update the docs to be reflective of grafana docs and how to export new rules a user might want to add.
Co-authored-by: Elijah DeLee <kdelee@redhat.com>
- Prevents changing hostname, listener_port, or node_type for instances
that already exist
- API default node_type is execution
- API default node_state is installed
awx-web container does not have access to receptor socket, and the
execution node health check requires receptorctl.
This change runs the health check asynchronously in the task container.
After all jobs on the node are complete, delete the node then
broadcast the write_receptor_config task.
Also, make sure that write_receptor_config updates the state of links
that are in 'adding' state.
- allow the node_state to be set to deprovisioning
- set the links that touch the instance to removing
- only allow on K8S
- only allow to be done to execution nodes
- use dotted circles to represent `enabled: false`
- use solid circle stroke to represent `enabled: true`
- excise places where `Unavailable` node state is used in the UI.
add scaffolding for instance install_bundle endpoint
- add instance_install_bundle view (does not do anything yet)
- add `instance_install_bundle` related field to serializer
- add `/install_bundle` to instance URL
- `/install_bundle` only available for execution and hop node
- `/install_bundle` endpoint response contain a downloadable tgz with moc data
TODO: add actual data to the install bundle response
Signed-off-by: Hao Liu <haoli@redhat.com>
when a new remote execution/hop node is added
regenerate the receptor.conf for all control node to
peer out to the new remote execution node
Signed-off-by: Hao Liu <haoli@redhat.com>
Co-Authored-By: Seth Foster <fosterseth@users.noreply.github.com>
Co-Authored-By: Shane McDonald <me@shanemcd.com>
- node_state is now read only
- node_state gets set automatically to Installed in the create view
- raise a validation error when creating on non-K8S
- allow SystemAdministrator the 'add' permission for Instances
- expose the new listener_port field
- nodes with states Provisioning, Provisioning Fail, Deprovisioning,
and Deprovisioning Fail should bypass health checks and should never
transition due to the existing machinery
- nodes with states Unavailable and Installed can transition to Ready
if they check out as healthy
- nodes in the Ready state should transition to Unavailable if they
fail a check
Remove corresponding views for job instance_groups
Validate job_slice_count in API
Remove defaults from some job launch view prompts
the null default is preferable
Additionally, move the inventory-specific hacks of yesteryear
into the prompts_dict method of the WorkflowJob model
try to make it clear exactly what this is hacking and why
Correctly summarize label prompts, and add missing EE
Expand unit tests to apply more fields
adding missing fields to preserve during copy to workflow.py
Fix bug where empty workflow job vars blanked node vars (#12904)
* Fix bug where empty workflow job vars blanked node vars
* Fix bug where workflow job has no extra_vars, add test
* Add empty workflow job extra vars to assure fix
Removing try/except around instance_groups
Removing redefined execution_environment
Reordering labels/creds/igs/ee/etc
Removing special treatment for EEs when doing setattrs
Adding help_text to execution environments
Adding EE serializer on JobCreateScheduleSerializer
Remove if-not-data conditional from WFJTnode.can_change
these are cannonical for can_add, but this looks like a bug
Change JTaccess.can_unattach to call same method in super()
previously called can_attach, which is problematic
Better consolidate launch config m2m related checks
Test and fix pre-existing WFJT node RBAC bug
recognize not-provided instance group list on launch, avoiding bug where it fell back to default
fix bug where timeout field was saved on WFJT nodes after creating approval node
remove labels from schedule serializer summary_fields
remove unnecessary prefetch of credentials from WFJT node queryset
Fixes bug where Forks showed up in both default values and prompted values in launch summary
Fixes prompting IGs with defaults on launch
Make job tags and skip tags full width on workflow form
Fixes bug where we attempted to fetch instance groups for workflows
Fetch default instance groups from jt/schedule for schedule form prompt
Grab default IGs when adding a node that prompts for them
Adds support for saving labels on a new wf node
Fix linting errors
Fixes for various prompt on launch related issues
Adds support for saving instance groups on a new node
Adds support for saving instance groups when editing an existing node
Fix workflowReducer test
Updates useSelected to handle a non-empty starting state
Fixes visualizerNode tests
Fix visualizer test
Second batch of prompt related ui issues:
Fixes bug saving existing node when instance groups is not promptable
Fixes bug removing newly added label
Adds onError function to label prompt
Fixes tooltips on the other prompts step
Properly fetch all labels to show on schedule details
This removes a loop that ran on import
the loop was giving the wrong behavior
and it initialized too many fields as char_prompts fields
With this, we will now enumerate the char_prompts type fields manually
Adds support for prompting labels on launch in the UI
Fix execution environment prompting in UI
Round out support for prompting all the things on JT launch
Adds timeout to job details
Adds fetchAllLabels to JT/WFJT data models
Moves labels methods out to a mixin so they can be shared across JTs/WFJTs/Schedules
Fixes bug where ee was not being sent on launch
Adds the ability to prompt for ee's, ig's, labels, timeout and job slicing to schedules
Fixes bug where saving schedule form without opening the prompt would throw errors
Adds support for IGs and labels to workflow node prompting
Adds support for label prompting to node modal
Fix job template form tests
* Making almost all fields promptable on job templates and config models
* Adding EE, IG and label access checks
* Changing jobs preferred instance group function to handle the new IG cache field
* Adding new ask fields to job template modules
* Address unit/functional tests
* Adding migration file
Adds prompt on launch buttons to labels, forks, job slicing, timeout, and instance groups
Adds prompting for labels on workflow job template
Updates flags that denote when prompting is necessary in various places
Adds prompting support for timeout, job slicing, forks, labels, instance groups and execution environments to the prompt details
Show prompted ee, forks, job slice and labels on schedule details
Adds support for ee, labels, forks, job slicing and timeout prompting to the node view modal
Add default values when prompting for ee's, forks, job slicing and timeout
Adds launch prompt step for execution environments
Adds fields for timeout, job slicing and forks to other prompts step of launch
- Fix out of scope variable in error message in the action plugin
- Rename action plugin from playbook_integrity to verify_project
Refs #12887 which pointed out the out of scope variable
Signed-off-by: Rick Elrod <rick@elrod.me>
This rule alerts if the redis queue is larger than what the rolling
average event insertion rate/second * 120. In other words, if the redis
queue is larger than it appears we can process events in two minutes.
It appears it has to meet this condition for 60 seconds to start firing.
Future commits will address how to configure contact points like slack.
shout out to @jainnikhil30 and @rebeccahhh who figured this out in jam
session this morning.
When launching a job template, if the last project update failed due to
signature validation, show an error that actually says that.
Signed-off-by: Rick Elrod <rick@elrod.me>
This was missed when we landed #12813. Adds cryptography
kind to the CredentialType allowed kinds list, which now
produces the proper error message when attempting to PUT
to modify the managed credential type.
Signed-off-by: Rick Elrod <rick@elrod.me>
Rather than only allowing the signature credential to be specified on
project using git, allow it to be specified on any project at all.
This moves the field to always show, and moves it out of the git
subform.
Signed-off-by: Rick Elrod <rick@elrod.me>
add new managed credential type for gpg pub key
add migration file to setup managed credential types to add the new credential type
Signed-off-by: Hao Liu <haoli@redhat.com>
- Extract how slicing is done from Inventory#get_script_data and pull it
into a new method, Inventory#get_sliced_hosts
- Make use of this method in Inventory#get_script_data
- Make use of this method in Job#_get_inventory_hosts (used by
Job#start_job_fact_cache and Job#finish_job_fact_cache).
This fixes an issue (namely in Tower 4.1) where job slicing with fact
caching enabled doesn't save facts for all hosts.
Signed-off-by: Rick Elrod <rick@elrod.me>
If canceled attempted before, still allow attempting another cancel
in this case, attempt to send the sigterm signal again.
Keep clicking, you might help!
Replace other cancel_callbacks with sigterm watcher
adapt special inventory mechanism for this too
Get rid of the cancel_watcher method with exception in main thread
Handle academic case of sigterm race condition
Process cancelation as control signal
Fully connect cancel method and run_dispatcher to control
Never transition workflows directly to canceled, add logs
We should be consistent about this. Also this takes us from doing a as
many queries to the UnifiedJob table as we have instances to doing 1
query to the UnifiedJob table (and both do 1 query to Instances table)
Reasoning:
- This is breaking the UI in official image builds of devel
- This is always being overridden in our packaging
- PROJECTS_ROOT and JOBOUTPUT_ROOT also hardcode /var/lib/awx
Make it so that submitting a task to the dispatcher happens as part of the transaction.
this applies to dispatcher task "publishers" which NOTIFY the pg_notify queue
if the transaction is not successful, it will not be sent, as per postgres docs
This keeps current behavior for pg_notify listeners
practically, this only applies for the awx-manage run_dispatcher service
this requires creating a separate connection and keeping it long-lived
arbitrary code will occasionally close the main connection, which would stop listening
Stop sending the waiting status websocket message
this is required because the ordering cannot be maintained with other changes here
the instance group data is moved to the running websocket message payload
Move call to create_partition from task manager to pre_run_hook
mock this in relevant unit tests
Avoid running jobs that have already been reapted
Co-authored-by: Elijah DeLee <kdelee@redhat.com>
Remove unnecessary extra actions
Fix waiting jobs in other cases of reaping
add `make help`
that prints all available make targets
help text generated from comments above the make target starting with `##`
Signed-off-by: Hao Liu <haoli@redhat.com>
When on the screen in the UI that loads the job events, the ui includes
a filter to exclude job events where stdout = ''. Because this is a
TextField and was not in the allow list, we were applying DISTINCT to
the query. This made it very unperformant for large jobs, especially
on the query that gets the count and cannot put a LIMIT on the query.
Also correctly prefetch the related job_template data on the view to
cut down the number of queries we make from around 50 to under 10.
We need to analyze other similar views for other prefetch type
optimizations we should make.
* refactor ScheduleFormFields into own file
* refactor ScheduleForm
* wip complex schedules form
* build rruleset from inputs
* update schedule form validation for multiple repeat frequencies
* add basic rrule set parsing when opening schedule form
* complex schedule bugfixes, handle edge cases, etc
* fix schedule saving/parsing for single-occurrence schedules
* working with timezone issues
* fix rrule until times to be in UTC
* update tests for new schedule form format
* update ouiaIds
* tweak schedules spacing
* update ScheduleForm tests
* show message for unsupported schedule types
* default schedules to browser timezone
* show error type/message in ErrorDetail
* shows frequencies on ScheduleDetails view
* handles nullish values
* Forcing an unbind for a django-auth-ldap sticky session to the LDAP server
* Focring _connection_bound to false after closing and modifying exceptino logging
Introduce build_project_dir method
the base method will create an empty project dir for workdir
Share code between job and inventory tasks with new mixin
combine rest of pre_run_hook logic
structure to hold lock for entire sync process
force sync to run for inventory updates due to UI issues
Remove reference to removed scm_last_revision field
When creating unified job, stash the list of pk values from the
instance groups returned from preferred_instance_groups so that the
task management system does not need to call out to this method
repeatedly.
.preferred_instance_groups_cache is the new field
Instead of loading all pending Workflow Approvals in the task manager,
run a query that will only return the expired apporovals
directly expire all which are returned by that query
Cache expires time as a new field in order to simplify WorkflowApproval filter
We always add the job to the graph right before calling start task.
Reduce complexity of proper operation by just doing this in start_task,
because if you call start_task, you need to add it to the dependency
graph
we had tried doing this in the WorkflowManager, but we decided that
we want to handle ALL pending jobs and "soft blockers" to jobs with the
TaskManager/DependencyGraph and not duplicate that logic in the
WorkflowManager.
add settings to define task manager timeout and grace period
This gives us still TASK_MANAGER_TIMEOUT_GRACE_PERIOD amount of time to
get out of the task manager.
Also, apply start task limit in WorkflowManager to starting pending
workflows
more than saving the loop, we save building the WorkflowDag twice which
makes LOTS of queries!!!
Also, do a bulk update on the WorkflowJobNodes instead of saving in a
loop :fear:
implement https://github.com/ansible/awx/issues/12446
in development environment, enable set of views that run
the task manager(s).
Also introduce a setting that disables any calls to schedule()
that do not originate from the debug views when in the development
environment. With guards around both if we are in the development
environment and the setting, I think we're pretty safe this won't get
triggered unintentionally.
use MODE to determine if we are in devel env
Also, move test for skipping task managers to the tasks file
- DependencyManager spawns dependencies if necessary
- WorkflowManager processes running workflows to see if a new job is
ready to spawn
- TaskManager starts tasks if unblocked and has execution capacity
Change:
- Case-insensitive search only makes sense on strings, so check the
type of the field we are searching and ensure it is a string field
(TextField, CharField, or some subclass thereof).
- This prevents a 500 error when a user uses iexact on, e.g., an
integer field. Now, a 400 Bad Request is returned instead.
Test Plan:
- Added simple unit tests for iexact
Tickets:
- Fixes#9222
Signed-off-by: Rick Elrod <rick@elrod.me>
* Modifying SAML adapter to not auto-add default galaxy creds to orgs on login
* Adding test, fixing old tests and moving add_default_galaxy_credential to pipeline
* Enhanced detail component to handle cases with no values, and refactored components that use detail component.
* Add optional chaining operators where necessary to pass test cases
* add test cases to test suites of modified files
Co-authored-by: Veda Periwal <vperiwal@vperiwal-mac.attlocal.net>
This optimizes the ActivityStreamSerializer by only getting many-to-many
relationships that are speculatively non-empty
based on information we have in other fields
We run this every time we create an object as an on_commit action
so it is expected this will have a major impact on response times for launching jobs
fixes https://github.com/ansible/awx/issues/7946
- added WorkflowApprovalTemplate page type to allow URL registration
- added resources regex that’s associated resource URL with WorkflowApprovalTemplate
- registered the new resource regex with WorkflowApprovalTemplate page type
- modified `DEPENDENT_EXPORT` handling (insisted by @jbradberry)
- added special case handling for WorkflowApprovalTemplate due to its unique nature
unique nature of WorkflowApprovalTemplate
- when exporting WorkflowJobTemplate with approval node the WorkflowJobTemplateNode need to contain a related "create_approval_template" the POST data for "create_approval_template" need to come from the "workflow_approval_template"
- during the export of a WorkflowJobTemplateNode that is an approval node we need to get the data from "workflow_approval_template" and use that to populate the "create_approval_template"
Co-Authored-By: Jeff Bradberry <685957+jbradberry@users.noreply.github.com>
Signed-off-by: Hao Liu <haoli@redhat.com>
* Added help text to schedule form and detail with link to documentation
* Added test cases for help text in schedule form and detail
* Add help text to schedule form and detail with link to documentation
Co-authored-by: Veda Periwal <vperiwal@vperiwal-mac.attlocal.net>
* Reap jobs on dispatcher startup to increase clarity, replace existing reaping logic
* Exit jobs if receiving SIGTERM signal
* Fix unwanted reaping on shutdown, let subprocess close out
* Add some sanity tests for signal module
* Add a log for an unhandled dispatcher error
* Refine wording of error messages
Co-authored-by: Elijah DeLee <kdelee@redhat.com>
* add database connection to the metrics endpoint
* bump the counts collector version to 1.2
* check for postgresql as database so to not break the tests
* Track combined artifacts on workflow jobs
* Avoid schema change for passing nested workflow artifacts
* Basic support for nested workflow artifacts, add test
* Forgot that only does not work with polymorphic
* Remove incorrect field
* Consolidate logic and prevent recursion with UJ artifacts method
* Stop trying to do precedence by status, filter for obvious ones
* Review comments about sets
* Fix up bug with convergence node paths and artifacts
Adding standard subject line to triage_replies.md
Removing PR commit generated change log in favor of github auto-commit log
Updating some images
Adding AWX matrix chanel to IRC notifications
Adding references between operator and AWX releases
we've observed this in development and some users have reported experiencing 500's on /api/v2/metrics because of a key error here where a metric is missing from a certain instance
- listen specifically within awx/awx, so that changes in awxkit or
awx_collection don't trigger spurious reloads
- expand the exclude pattern to ignore the test directories
* Removing old awxbot files
* Removing security bug report as GitHub now shows the security piolicy from /SECURITY.md
* Changing feature_request from md to yml
* Adding additional options to bug report components andinstall method
* Removing old ISSUE_TEMPLATE.md
* Changing issue type and adding additional components
* Removing auto-generated change log
* Adding awx_collection and cli components
* Changing content search pattern for type labels
* Changing from collection to awx_collection tag and adding dependencies tag
* Adding unicode bug to bug repot to match feature unicode character
* Changing bug to bug or docs
* Remove docker on * and boot2docker infavor of docker development environmnet
* Create top level issue with: CoC, Enterprise, Top level help
* Remove old CODEOWNERS file
* Show add access button if it is a system admin
* Hide access button if the user is credential admin, org admin, but the
credential does not belong to any org.
* Show access button if the user is a credential admin, org admin, and
the credential is associated to an org.
* Show access button if the user is an org admin and the credential is
associated to the org.
All those permutations are allowed by the API RBAC.
This PR update UX to not allow the user to attempt to perform any
action that will raise an error when modifying access to the
credentials.
Update project status to reflect project update sync related to job
template that was launched with branch override.
We were displaying status of project sync itself, not from the project
update job as expected.
Also, rename `Project Status` to be `Project Update Status`.
See: https://github.com/ansible/awx/issues/11987
* Logout as User A and Login as User B redirects to `/home'
* Logout as User A and Login as User A redirects to `/home'
* Allow session to timeout as User A and Login as User A redirects to User A's last location
See: https://github.com/ansible/awx/issues/11167
grafana via prometheus.
This metric is a good indicator of how far behind the callback receiver
is. The higher the load the further behind/the greater the number of
seconds the metric will display.
This number being high may indicate the need for horizontal scaling in
the control plane or vertically scaling the number of callback
receivers.
This is particularly useful when you are using the @filepath version
of the flag, since otherwise there would be no way to issue the
command with multiple vars files.
Also, add `-e` as an alias to `--extra_vars`
- scm based inventory sources should launch project updates prior to
running inventory updates for that source.
- fixes scenario where a job is based on projectA, but the inventory
source is based on projectB. Running the job will likely trigger a
sync for projectA, but not projectB.
comments
Add details related workflow job on the work flow approval details
Remove not used prop isLoading, fix, and expand unit-tests related to
workflow approval details.
* Only use in-memory cache for database settings
Make necessary adjustments to monkeypatch
as it is very vunerable to recursion
Remove migration exception that is now redundant
Clear cache if a setting is changed
* Use dedicated middleware for setting cache stuff
Clear cache for each request
* Add tests for in-memory cache
* We observed daphne giving tracebacks when accessing logging settings.
Originally, configure tower in tower settings was no a suspect because
daphne is not multi-process. We've had issues with configure tower in
tower settings and multi-process before. We later learned that Daphne
is multi-threaded. Configure tower in tower was back to being a
suspect. We constructed a minimal reproducer to show that multiple
threads accessing settings can cause the same traceback that we saw in
daphne. See
https://gist.github.com/chrismeyersfsu/7aa4bdcf76e435efd617cb078c64d413
for that recreator. These fixes stop the recreation.
- The `z` option indicates that the bind mount content is shared among multiple containers.
- The `Z` option indicates that the bind mount content is private and unshared.
If multiple container attempt to mount the same directory `Z` option will cause a raise condition where only the last container started will have access to the file.
Ref: https://docs.docker.com/storage/bind-mounts/#configure-the-selinux-label
Signed-off-by: Hao Liu <haoli@redhat.com>
Now that we are adding popovers for details pages, I noticed a couple of
strings wrapping in odd places, update css to avoid that.
Also `word-break: break-word` was deprecated.
* Delay update of artifacts until final job save
Save tracebacks from receptor module to callback object
Move receptor traceback check up to be more logical
Use new mock_me fixture to avoid DB call with me method
Update the special runner message to the delay_update pattern
* Move special runner message into post-processing of callback fields
* Track host_status_counts and use that to process notifications
* Remove now unused setting
* Back out changes to callback class not needed after all
* Skirt the need for duck typing by leaning on the cached field
* Delete tests for deleted task
* Revert "Back out changes to callback class not needed after all"
This reverts commit 3b8ae350d218991d42bffd65ce4baac6f41926b2.
* Directly hardcode stats_event_type for callback class
* Fire notifications if stats event was never sent
* Remove test content for deleted methods
* Add placeholder for when no hosts matched
* Make field default be None, denote events processed with empty dict
* Make UI process null value for host_status_counts
* Fix tracking of EOF dispatch for system jobs
* Reorganize EVENT_MAP into class properties
* Consolidate conditional I missed from EVENT_MAP refactor
* Give up on the null condition, also applies for empty hosts
* Remove cls position argument not being used
* Move wrapup method out of class, add tests
* Added schedule_rruleset lookup plugin for awx.awx
* Added DB migration for rrule size
* Updated schedule docs
* The schedule API endpoint will now return an array of errors on rule validation to try and inform the user of all errors instead of just the first
* Remove committed_capacity field, delete supporting code
* Track consumed capacity to solve the negatives problem
* Use more verbose name for IG queryset
* move static methods used by task manager
These static methods were being used to act on Instance-like objects
that were SimpleNamespace objects with the necessary attributes.
This change introduces dedicated classes to replace the SimpleNamespace
objects and moves the formerlly staticmethods to a place where they are
more relevant instead of tacked onto models to which they were only
loosly related.
Accept in-memory data structure in init methods for tests
* initialize remaining capacity AFTER we built map of instances
* create a singular page with listed replies that can be copy and pasted for mailing list and bug scrub purposes
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
I verified what Seth found in https://github.com/ansible/awx/pull/12052, but would really hate to lose this functionality. Curious if folks on the API team can try this and see if it works for them.
By using .only we select fewer columns, avoiding potentially large
fields that we never reference.
Also, small tweak to eliminate what was a duplicate dictionary of
hostname:instance, because we don't need build and carry two copies of
the same data.
Update when deleted is show on job details.
Some job types should not display inventory or projects, update when
showing those fields.
Also, update when displaying information when
those fields where deleted.
See: https://github.com/ansible/awx/issues/12008
and then switch from using order by ID as a fallback for all ordering and instead
just set instances ordering to ID as default to prevent
OrderedManyToMany fields ordering from being interrupted.
Escape name__regex and name__iregex. Escaping the value for those
keys when creating a smart inventory is a work around for the
pyparsing code on the API side for special characters. This will just
display an extra escape when showing the host_filter on details page.
* use new children-summary endpoint data to traverse job event tree
* update job output tests for new children summary data
* force flat mode if event child summary fails to load
* update childrenSummary data for endpoint changes
* don't add jobs to job tree until children summary loaded
* force job output into flat mode if job processing not complete
- returns a special view to output the total number of children (and
grandchildren) events for all parents events for a job
value is the number of total children of that event
- intended to be consumed by the UI, as an efficient way to get the
number of children for a particular event
- see api/templates/api/job_job_events_children_summary.md for more info
* Grafana notifications: Fix panel/dashboardId type
Latest grafana fails with
Error sending notification grafana: 400
[{"classification":"DeserializationError",
"message":"json: cannot unmarshal string into Go struct
field PostAnnotationsCmd.dashboardId of type int64"}]
So ensure the IDs are really int and not strings.
* Fix the dashboard/panelId=0 case
0 is avlaid valid for the ID's, so ensure to allow them.
* Update tests to new behavior
Panel/Dashboard Id fields are not sent if they where not requested.
Alos add tests for the ID=0 case.
* Simple patches to make jobs robust to database restarts
* Add some wait time before retrying loop due to DB error
* Apply dispatcher downtime setting to job updates, fix dispatcher bug
This resolves a bug where the pg_is_down property
never had the right value
the loop is normally stuck in the conn.events() iterator
so it never recognized successful database interactions
this lead to serial database outages terminating jobs
New setting for allowable PG downtime is shared with task code
any calls to update_model will use _max_attempts parameter
to make it align with the patience time that the dispatcher
respects when consuming new events
* To avoid restart loops, handle DB errors on startup with prejudice
* If reconnect consistently fails, exit with non-zero code
* Update runtime.yml
* Extending test_completness to include meta/runtime.yml and adding remaining missing modules from runtime.yml
Co-authored-by: quasd <quasd@users.noreply.github.com>
There was a race condition because the callback reciever tried to run this code:
File "/awx_devel/awx/main/management/commands/run_callback_receiver.py", line 31, in handle
CallbackBrokerWorker(),
File "/awx_devel/awx/main/dispatch/worker/callback.py", line 49, in __init__
self.subsystem_metrics = s_metrics.Metrics(auto_pipe_execute=False)
File "/awx_devel/awx/main/analytics/subsystem_metrics.py", line 156, in __init__
self.instance_name = Instance.objects.me().hostname
Before get_or_register was being called by the dispatcher.
Occasionally the create_partition will error with,
relation "main_projectupdateevent_20220323_19" already exists
This change wraps the db command into a try except block with its
own transaction
This JSONBlob field type is a wrapper around Django's new generic
JSONField, but with the database column type forced to be text. This
should behave close enough to our old wrapper around
django-jsonfield's JSONField and will avoid needing to do the
out-of-band database migration.
* We trigger notifications when the callback receiver processes the
playbook_on_stats event. This is the last event in ansible-playbook and
the process should exist very shortly after this event is emitted. The
trouble comes in with the isolated node feature. There is a management
playbook that runs periodically that pulls the events from the remote
node. It's possible that the management playbooks runs, gets the
playbook_on_stats event, but does not see that the playbook is finished
running. Therefore the job status is still seen as 'running' BUT we have
kicked of the notification for the job. The notification worker will
enter a loop waiting on the job to enter the finished state. In this
case the time it takes for the job to enter the finished state can be
long, roughly 2 * the management playbook run time.
* This new setting allows the user to increase the time that the
notification spends waiting for the job to enter the finished state.
Add related job templates to a couple of screens. Credential and
Inventory.
Also refactor the component already in place for Projects to be in sync
with the Job Templates screen.
See: https://github.com/ansible/awx/issues/5867
Add several changes to API and UI related to Instance Groups.
* Update summary_fields for DEFAULT_CONTROL_PLANE_QUEUE_NAME, and
DEFAULT_EXECUTION_QUEUE_NAME. Rely on API validation for those fields.
* Fix Instance Group list RBAC
* Add validation for a couple of fields on the Instance Groups endpoint
1. is_container_group
2. policy_instance_percentage
3. policy_instance_list
See: https://github.com/ansible/awx/issues/11130
Also: https://github.com/ansible/awx/issues/11718
This will hopefully get us past the unfortunate check against the
HostMetric table, which doesn't exist when you are upgrading from 3.8
to 4.x.
Additionally, guard against AUTH_LDAP_GROUP_TYPE not being in settings
for conf migration 0006.
- the default auto-increment primary key field type is now
configurable, and Django's check command issues a warning if you are
just assuming the historical behavior of using AutoField.
- Django 3.2 brings in automatic AppConfig discovery, so all of our
explicit `default_app_config = ...` assignments in __init__.py
modules are no longer needed, and raise a RemovedInDjango41Warning.
- upgrades
- Django 3.2.12
- pytz 2021.3 (from 2019.3)
- oauthlib 3.2.0 (from 3.1.0)
- requests-oauthlib 1.3.1 (from 1.3.0)
- django-guid 3.2.1 (from 2.2.1)
- django-solo 2.0.0 (from 1.1.3)
- django-taggit 2.1.0 (from 1.2.0)
- netaddr 0.8.0 (from 0.7.19)
- pyrad 2.4 (from 2.3)
- django-radius devel (from 1.3.3)
- future devel (from 0.16.0)
- django-guid, django-solo, and django-taggit are upgraded to fix the
AppConfig deprecation warning. FIXME: django-guid devel has the
fix, but it hasn't been released yet.
- Released versions of django-radius have a hard-coded pin to
future==0.16.0, which has a Python warning due to an improperly
escaped character. This is fixed in future devel, so for now we are
pinning to references to the git repos.
- netaddr had a bunch of Python syntax and deprecation warnings
* Process unresponsive and newly responsive hop nodes
* Use more natural way to zero hop node capacity, add test
* Use warning as opposed to warn for log messages
It previously depended on a private Django internal class that changed
with Django 3.1.
I've switched here instead to disabling the django-polymorphic
accessors to get the underlying UnifiedJob object for a Job, which due
to the way they implement those was resulting in N+1 behavior on
deletes. This gets us back most of the way to the performance gains
we achieved with the custom collector class. See
https://github.com/django-polymorphic/django-polymorphic/issues/198.
- FieldDoesNotExist now has to be imported from django.core.exceptions
- Django docs specifically say not to import
django.conf.global_settings, which now has the side-effect of
triggering one of the check errors
The event_data field on event models, however, is getting an
overridden version that retains the underlying text data type for the
column, to avoid a heavy data migration on those tables.
Also, certain of the larger tables are getting these fields with the
NOT NULL constraint turned off, to avoid a long migration.
Remove the django.utils.six monkey patch we did at the beginning of
the upgrade.
- inspect.getargspec() -> inspect.getfullargspec()
- register pytest.mark.fixture_args
- replace use of DRF's deprecated NullBooleanField
- fix some usage of naive datetimes in the tests
- fix some strings with backslashes that ought to be raw strings
- Django's PostgreSQL JSONField wraps values in a JsonAdapter, so deal
with that when it happens. This goes away in Django 3.1.
- Setting related *_id fields clears the actual relation field, so
trying to fake objects for tests is a problem
- Instance.objects.me() was inappropriately creating stub objects
every time while running tests, but some of our tests now create
real db objects. Ditch that logic and use a proper fixture where needed.
- awxkit tox.ini was pinned at Python 3.8
- upgrades
- Django 3.0.14
- django-jsonfield 1.4.1 (from 1.2.0)
- django-oauth-toolkit 1.4.1 (from 1.1.3)
- Stopping here because later versions have changes to the
underlying model to support OpenID Connect. Presumably this can
be dealt with via a migration in our project.
- django-guid 2.2.1 (from 2.2.0)
- django-debug-toolbar 3.2.4 (from 1.11.1)
- python3-saml 1.13.0 (from 1.9.0)
- xmlsec 1.3.12 (from 1.3.3)
- Remove our project's use of django.utils.six in favor of directly
using six, in awx.sso.fields.
- Temporarily monkey patch six back in as django.utils.six, since
django-jsonfield makes use of that import, and is no longer being
updated. Hopefully we can do away with this dependency with the new
generalized JSONField brought in with Django 3.1.
- Force a json decoder to be used with all instances of JSONField
brought in by django-jsonfield. This deals with the 'cast to text'
problem noted previously in our UPGRADE_BLOCKERS.
- Remove the validate_uris validator from the OAuth2Application in
migration 0025, per the UPGRADE_BLOCKERS, and remove that note.
- Update the TEMPLATES setting to satisfy Django Debug Toolbar. It
requires at least one entry that has APP_DIRS=True, and as near as I
can tell our custom OPTIONS.loaders setting was effectively doing
the same thing as Django's own machinery if this setting is set.
The Member role can derive from e.g. the Org Admin role, so basically
all organization and team roles should be assigned first, so that RBAC
conditions are met when assigning later roles.
This is to avoid references to settings in threads,
this is known to create problems when caches expire
this leads to KeyError in environments with heavy load
* Fix integer/float errors in survey
* Add SURVEY_TYPE_MAPPING to constants
Add SURVEY_TYPE_MAPPING to constants, and replace usage in a couple of
files.
Co-authored-by: Alexander Komarov <akomarov.me@gmail.com>
* Changing session cookie name and added a way for clients to know what the key name is
* Adding session information to docs
* Fixing how awxkit gets the session id header
Right now, without this, we end up with a different number for max_workers than max_forks. For example, on a control node with 16 Gi of RAM,
max_mem_capacity w/ 100 MB/fork = (16*1024)/100 --> 164
max_workers = 5 * 16 --> 80
This means we would allow that control node to control up to 164 jobs, but all jobs after the 80th job will be stuck in `waiting` waiting for a dispatch worker to free up to run the job.
Sharing the /etc/redhat-access-insights is no longer
required for EEs. Furthermore, this fixes a SELinux issue
when launching multiple jobs with concurrency and fact_caching enabled.
i.e:
lsetxattr /etc/redhat-access-insights: operation not permitted
fix memory and cpu settings to suport k8s resource request format
* fix conversion of memory setting to bytes
This setting has not been getting set by default, and needed some fixing
up to be compatible with setting the memory in the same way as we set it
in the operator, as well as with other changes from last year which
assume that ansible runner is returning memory in bytes.
This way we can start setting this setting in the operator, and get a
more accurate reflection of how much memory is available to the control
pod in k8s.
On platforms where services are all sharing memory, we deduct a
penalty from the memory available. On k8s we don't need to do this
because the web, redis, and task containers each have memory
allocated to them.
* Support CPU setting expressed in units used by k8s
This setting has not been getting set by default, and needed some fixing
up to be compatible with setting the CPU resource request/limits in the
same way as we set it in the resource requests/limits.
This way we can start setting this setting in the
operator, and get a more accurate reflection of how much cpu is
available to the control pod in k8s.
Because cpu on k8s can be partial cores, migrate cpu field to decimal.
k8s does not allow granularity of less than 100m (equivalent to 0.1 cores), so only
store up to 1 decimal place.
fix analytics to deal with decimal cpu
need to use DjangoJSONEncoder when Decimal fields in data passed to
json.dumps
There is no current need or use to keep a seperate dependency graph for
each instance group. In the interest of making it clearer what the
current code does, eliminate this superfluous complication.
We are no longer ever referencing any accounting of instance group
capacity, instead we only look
at capacity on intances.
* Select control node before start task
Consume capacity on control nodes for controlling tasks and consider
remainging capacity on control nodes before selecting them.
This depends on the requirement that control and hybrid nodes should all
be in the instance group named 'controlplane'. Many tests do not satisfy that
requirement. I'll update the tests in another commit.
* update tests to use controlplane
We don't start any tasks if we don't have a controlplane instance group
Due to updates to fixtures, update tests to set node type and capacity
explicitly so they get expected result.
* Fixes for accounting of control capacity consumed
Update method is used to account for currently consumed capacity for
instance groups in the in-memory capacity tracking data structure we initialize in
after_lock_init and then update via calculate_capacity_consumed (both in
task_manager.py)
Also update fit_task_to_instance to consider control impact on instances
Trust that these functions do the right thing looking for a
node with capacity, and cut out redundant check for the whole group's
capacity per Alan's reccomendation.
* Refactor now redundant code
Deal with control type tasks before we loop over the preferred instance
groups, which cuts out the need for some redundant logic.
Also, fix a bug where I was missing assigning the execution node in one case!
* set job explanation on tasks that need capacity
move the job explanation for jobs that need capacity to a function
so we can re-use it in the three places we need it.
* project updates always run on the controlplane
Instance group ordering makes no sense on project updates because they
always need to run on the control plane.
Also, since hybrid nodes should always run the control processes for the
jobs running on them as execution nodes, account for this when looking for a
execution node.
* fix misleading message
the variables and wording were both misleading, fix to be more accurate
description in the two different cases where this log may be emitted.
* use settings correctly
use settings.DEFAULT_CONTROL_PLANE_QUEUE_NAME instead of a hardcoded
name
cache the controlplane_ig object during the after lock init to avoid
an uneccesary query
eliminate mistakenly duplicated AWX_CONTROL_PLANE_TASK_IMPACT and use
only AWX_CONTROL_NODE_TASK_IMPACT
* add test for control capacity consumption
add test to verify that when there are 2 jobs and only capacity for one
that one will move into waiting and the other stays in pending
* add test for hybrid node capacity consumption
assert that the hybrid node is used for both control and execution and
capacity is deducted correctly
* add test for task.capacity_type = control
Test that control type tasks have the right capacity consumed and
get assigned to the right instance group
Also fix lint in the tests
* jobs_running not accurate for control nodes
We can either NOT use "idle instances" for control nodes, or we need
to update the jobs_running property on the Instance model to count
jobs where the node is the controller_node.
I didn't do that because it may be an expensive query, and it would be
hard to make it match with jobs_running on the InstanceGroup which
filters on tasks assigned to the instance group.
This change chooses to stop considering "idle" control nodes an option,
since we can't acurrately identify them.
The way things are without any change, is we are continuing to over consume capacity on control nodes
because this method sees all control nodes as "idle" at the beginning
of the task manager run, and then only counts jobs started in that run
in the in-memory tracking. So jobs which last over a number of task
manager runs build up consuming capacity, which is accurately reported
via Instance.consumed_capacity
* Reduce default task impact for control nodes
This is something we can experiment with as far as what users
want at install time, but start with just 1 for now.
* update capacity docs
Describe usage of the new setting and the concept of control impact.
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Rebeccah <rhunter@redhat.com>
Modify usage of ansible_facts on advanced search, once `ansible_facts`
key is selected render a text input allowing the user to type special
query expected for ansible_facts.
This change will add more flexibility to the usage of ansible_facts when
creating a smart inventory.
See: https://github.com/ansible/awx/issues/11017
--- Removed all callback functions from 'jobs.py' and put them in a new file '/awx/main/tasks/callback.py'
--- Modified Unit tests unit moved
--- Moved 'update_model' from jobs.py to /awx/main/utils/update_model.py
Bump react scripts to 5.0
See: https://github.com/ansible/awx/issues/11543
Bump eslint
Bump eslint and related plugins
Add @babe/core
Add @babe/core remove babel/core.
Rename .eslintrc to .eslintrc.json
Rename .eslintrc to .eslintrc.json
Add extra plugin
Move babe-plugin-macro as dev dependencies
Move babe-plugin-macro as dev dependencies
Add preset-react
Add preset-react
Fixing lint errors
Fixing lint errors
Run eslint --fix
Run eslint --fix
Turn no-restricted-exports off
Turn no-restricted-exports off
Revert "Run eslint --fix"
This reverts commit e760885b6c199f2ca18091088cb79bfa77c1d3ed.
Run --fix
Run --fix
Fix lint errors
Also bump specificity of Select CSS border component to avoid bug of
missing borders.
Also update API tests related to lincenses.
Modifying workflows to install python for make commands
Squashing CI tasks to remove repeated steps
Modifying pre-commit.sh to not fail if there are no python file changes
The most notable change here is the removal of the conditional in
process_request. I don't know why we were preferring REQUEST_URI over
PATH_INFO. When the app is running at /, they are always the same as far as I
can tell. However, when using SCRIPT_NAME, this was incorrectly setting path and
path_info to /myprefix/myprefix/.
* Update saml.md
- Updated link to python documentation
- Added instructions for superadmin permissions
Co-authored-by: John Westcott IV <john.westcott.iv@redhat.com>
vars in ansible/instantiate-awx-deployment.yml in awx-operator repo appear to have been updated, because when we used the `tower_...` vars, they did not apply
It has no way of knowing whether a later command will fix the
situation, and this will come up in the installer. Let's just trust
the pre-flight checks.
Django ORM method get_or_create() does not call save() directly,
but it calls the create() [1].
The create method ignores the skip_update=True option, which then
will trigger a project update, however the EE was not yet created
in the database.
To avoid this problem, we just check the existence of the default
project and creates it with save(skip_update=True) manually.
* update StatusLabels on job detail
* change StatusIcon to use PF circle icons
* change status icon to status label on host event modal
* update status label on wf job output
* update tests for status label changes
* fix default status icon color
Extend the timeout, assuming that we want to let the kubernetes scheduler
start containers when it wants to start them. This allows us to make
resource requests knowing that when some jobs queue up waiting for
resources, they will not get reaped in as short of a
timeout.
This requires corresponding ansible-runner changes
which are only available in devel branch
to do this, requirements are changed
to install ansible-runner devel as it did before
Revert "Use ansible-runner 2.1.1 build"
This reverts commit f0ede01017.
Add back in change from updater.sh that we want to keep
--- Added 3 new sub-package : awx.main.tasks.system , awx.main.tasks.jobs , awx.main.tasks.receptor
--- Modified the functional tests and unit tests accordingly
* Adding SAML option in SAML configuration to specify system auditor and system superusers by role or attribute
* Adding keycloak container and documentation on how to start keycloak alongside AWX (including configuration of both)
Events were passed to `handleRelaunch` and those events structure were
not parseable to JSON - breaking the relaunch of jobs. React 17 changes
made this bug visible.
Also, remove withRouter from LaunchButton.
See: https://github.com/ansible/awx/issues/11452
- the list, detail, and health check API views should not include them
- the Instance-InstanceGroup association views should not allow them
to be changed
- the ping view excludes them
- list_instances management command excludes them
- Instance.set_capacity_value sets hop nodes to 0 capacity
- TaskManager will exclude them from the nodes available for job execution
- TaskManager.reap_jobs_from_orphaned_instances will consider hop nodes
to be an orphaned instance
- The apply_cluster_membership_policies task will not manipulate hop nodes
- get_broadcast_hosts will ignore hop nodes
- active_count also will ignore hop nodes
This will allow us to control the default container group created via settings, meaning
we could set this in the operator and the default container group would get created with it applied.
We need this for https://github.com/ansible/awx-operator/issues/242
Deepmerge the default podspec and the override
With out this, providing the `spec` for the podspec would override everything
contained, which ends up including the container used, which is not desired
Also, use the same deepmerge function def, as the code seems to be copypasted from
the utils
* Correct syntax errors & add back lost last line for messages.po
* Manually sort through es & nl translated strings
* Mnaually sort through french strings and correct syntax errors
Signed-off-by: Christian M. Adams <chadams@redhat.com>
Load ansible.cfg from the branch specified on job template (i.e. the same branch that the playbook exists), not from the branch set in the "project".
Signed-off-by: notok <noto.kazufumi@gmail.com>
* Allow for customizing the receptor image
* Hook in receptor image to docker-compose template
* Fix missing -e to pass into Dockerfile playbook
* Add some docs
Make sure to use a different session cookie name then the default, to
avoid overlapping cookies with other django apps that might be running.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
I broke everything in https://github.com/ansible/awx/pull/11242.
These changes were necessary in order to run `awx-manage collectstatic` without a running database.
* Note in the job lifecycle when the controller_node and execution_node
are chosen. This event occurs most commonly in the task manager with a
couple of exceptions that happen when we dynamically create dependenct
jobs on the fly in tasks.py
Allows the user to visit login page when the login redirect url is set.
Also, redirects to login page once logging out and there is session from
a SAML available.
See: https://github.com/ansible/awx/issues/11012
* Unsure exactly why this happens but there seems to be a race condition
related to the time window between Receptor work_results and work
release. This sleep extends that window and hopefully avoids the race
condition.
Add new logic to cleanup orphaned work units
from administrative tasks
Remove noisy log which is often irrelevant
about running-cleanup-on-execution-nodes
we already have other logs for this
Skip checking the health of a mesh instance when the instance is not registered
with the application. This prevents encountering an 'UnbouncLocalError' when
running the application attached to a multi-use Receptor mesh network
Signed-off-by: Ethan Paul <24588726+enpaul@users.noreply.github.com>
For running `make docker-compose` a working version of openssl is
required for successfully generating Private RSA key for signing work.
Signed-off-by: Daniel Ziegenberg <daniel@ziegenberg.at>
Use the slack_sdk instead of the deprecated slackclient. Because according to the official documentation:
> The slackclient PyPI project is in maintenance mode now and slack-sdk project is the successor.
With this commit one UPGRADE BLOCKER from requirements/requirements.in is removed. Als the license for slack_sdk
is updated and unit tests for slack notifications backend are added.
Signed-off-by: Daniel Ziegenberg <daniel@ziegenberg.at>
The eslint and jsconfig files are needed to start the dev server.
Without the jsconfig, the ui development server can't resolve src
modules and will fail to start.
Primary changes are:
- Generalized variable names (remove "docker")
- Add explicit "push" variable rather than checking if the "registry" variable is defined.
- Allow for passing in version as build arg
In 4.1+ / AAP 2.1+, isolated groups should be converted into plain
instance groups, and it's desirable for the old ones to stick around
since they'll likely be tied to a bunch of job templates. We do not
want to make the users have to reconstruct those relationships.
* Primary development of integrating runner cleanup command
* Fixup image cleanup signals and their tests
* Use alphabetical sort to solve the cluster coordination problem
* Update test to new pattern
* Clarity edits to interface with ansible-runner cleanup method
* Another change corresponding to ansible-runner CLI updates
* Fix incomplete implementation of receptor remote cleanup
* Share receptor utils code between worker_info and cleanup
* Complete task logging from calling runner cleanup command
* Wrap up unit tests and some contract changes that fall out of those
* Fix bug in CLI construction
* Fix queryset filter bug
-- Updated devel build to take most recent receptor binary
-- Added signWork parameter when sedning job to receptor
-- Modified docker-compose tasks to generate RSA key pair to use for work-signing
-- Modified docker-compose templates and jinja templates for implementing work-sign
-- Modified Firewall rules on the receptor jinja config
Add firewall rules to dev env
When user attempts to access `/jobs/system/66` redirect to
`/jobs/management/66`.
This will catch management jobs notifications, for instance, and redirect to
the proper URL.
See:#11113
Do not display EE if a job was canceled. Since the API is returning null
for the value of the EE for this particular scenario.
See: https://github.com/ansible/awx/issues/11127
For project updates jobs triggered due a job template run,
we must ensure that project_update job to run on at the same
controller which dispatched the original job template, otherwise
the job might fail for being unable to find the playbook YAML file.
Break lines for long strings on main lists to avoid horizontal
scrolling.
Main goal of this change is to avoid actions items to be hidden on the
main lists.
See: https://github.com/ansible/awx/issues/6302
* Write tests to assure air-tightness of overwrite
* Candidate fix for group overwrite air-tightness
* Another proposed fix for the id mapping
* Further double down on tracking old instance_id
* Separate unchanging data case and fix some test issues
* parametrize final confusion test
* cut down some more on test cases and fix bug with prior fix
* Rewrite of _delete_host code sharing with update method
This is a start-to-finish rewrite of the host overwrite bug fix
this method is much more conservative,
it does this by keeping the overall code structure where hosts
are deleted before host updates are made
To fix the bug, we share code between the method that deletes hosts
and the method that updates the hosts
A data structure is created and passed to both methods
By having both methods use the same data structure which maps
the in-memory hosts to DB hosts, we assure that the deletions
will ONLY delete hosts that will not be updated
* Full finalize the planned work for health checks of execution nodes
* Implementation of instance health_check endpoint
* Also do version conditional to node_type
* Do not use receptor mesh to check main cluster nodes health
* Fix bugs from testing health check of cluster nodes, add doc
* Add a few fields to health check serializer missed before
* Light refactoring of error field processing
* Fix errors clearing error, write more unit tests
* Update health check info in docs
* Bump migration of health check after rebase
* Mark string for translation
* Add related health_check link for system auditors too
* Handle health_check cluster node timeout, add errors for peer judgement
see 9d000a76de
This change works around the fact that the presumed correct python3 for rhel8 (which the EE is based on)
is not the python3 that ansible-playbook is using, and is not where the python dependencies are installed.
* Add wording about expections for enabled status and default group
* fix pluralization
Co-authored-by: Alex Corey <acorey@redhat.com>
* Correct grammar mistake
Co-authored-by: Alex Corey <acorey@redhat.com>
* Change the location of the receptor socket
to /var/run/awx-receptor, to match what the installer is currently doing.
* Sync awx and receptor nodes for control socket
Co-authored-by: Jeff Bradberry <jeff.bradberry@gmail.com>
* Exclude control-only nodes from IG policy calculations
Also, as a reverse to this, exclude execution-only nodes from
the calculations if the group in question is the controlplane
* Incorporate review comments
* Always use controlplane as project update backup IG
Before, this was done conditionally to container_group jobs
this logic changes it so that controlgroup will always be a
firm backstop for project updates
* Code a little more defensively to make unit tests pass
* Fix unit tests
To avoid calling the user.save() on every single login (PR#9703)
we can check if the user.profile is available. For new users,
accessing the user.profile throws an ValueError exception which
is capture on this fix.
Example:
----
>>> _ = user.profile
*** ValueError: save() prohibited to prevent data loss due to unsaved related object 'user'.
>>> User.objects.filter(username=user.username).count()
0
This way, the user.save() gets called for brand users and will get the
ldap_dn associated as expected.
* Clean up added work_type processing for mesh_code branch
* track both execution and control capacity
* Remove unused execution_capacity property
* Count all forms of capacity to make test pass
* Force jobs to be on execution nodes, updates on control nodes
* Introduce capacity_type property to abstract some details out
* Update test to cover all job types at same time
* Register OpenShift nodes as control types
* Remove unqualified consumed_capacity from task manager and make unit tests work
* Remove unqualified consumed_capacity from task manager and make unit tests work
* Update unit test to execution vs control TM logic changes
* Fix bug, else handling for work_type method
* Model changes for instance last_seen field to replace modified
* Break up refresh_capacity into smaller units
* Rename execution node methods, fix last_seen clustering
* Use update_fields to make it clear save only affects capacity
* Restructing to pass unit tests
* Fix bug where a PATCH did not update capacity value
* Introduce utilities for --worker-info health check integration
* Handle case where ansible-runner is not installed
* Add ttl parameter for health check
* Reformulate return data structure and add lots of error cases
* Move up the cleanup tasks, close sockets
* Integrate new --worker-info into the execution node capacity check
* Undo the raw value override from the PoC
* Additional refinement to execution node check frequency
* Put in more complete network diagram
* Followup on comment to remove modified from from health check responsibilities
* Add developer docs for incoming receptor mesh features
* Additional wording about the receptor mesh process
* Wrap up docs feedback changes and polishing
* Add in way more terminology introductions, delete statement about past
* Fix typo around OCP-incluster type
* Update docker-compose
- Deploys 1 control and 1 execution node
* Add a new Receptor cluster configuration file
* update receptor peer to awx_1
to match how hop node is configured in cluster (Jim Ladd's commit)
* Move receptor_1 instantiation in the docker-compose setup
* Hard code receptor_1 name
* Update execution node name, move standalone conf file to docker-compose directory
* Reformat docker-compose file, mount another volume, change privileges
This requires swapping out the container images
for the execution nodes from awx-ee to the awx image
For completeness, the hop node image is switched to the raw
receptor image
A few outright bugs are fixed here
memory calculation just was not right at all
the execution_capacity calculation was reverse of intention
Drop in a few TODOs about error handling from debugging
Without this patch, you get:
```
tools/docker-compose-minikube/_sources/minikube delete
make: tools/docker-compose-minikube/_sources/minikube: No such file or directory
```
Renders Command Module in job detail view and improves Ad Hoc Commands execution permissions
SUMMARY
This closes#10440 and #8660.
On the job details view we now render the command module name, and the arguments. We also remove the run command button for user that do not have permission to launch a ad hoc command.
ISSUE TYPE
-enhancement
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Adds Ad Hoc Preview step and adds workflow similar to prompt on launch
SUMMARY
This closes#9399 it also introduce the same workflow that prompt on launch uses in the Ad Hoc Commands, where the user to go between steps as they wish.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Fix inaccurate translation
SUMMARY
Fix some inaccurate translation in zh meesage.po
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
devel branch
ADDITIONAL INFORMATION
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: weidong <None>
Properly compile translated API strings
SUMMARY
Fixes: #10681
Now when make ui-devel is run, it will compile the API translated strings and display them to the client browser.
[chadams@chadams-work awx]$ make ui-devel
make[1]: Entering directory '/home/chadams/awx'
python3.8 tools/scripts/compilemessages.py
processing file django.po in /home/chadams/awx/awx/locale/fr/LC_MESSAGES
processing file django.po in /home/chadams/awx/awx/locale/zh/LC_MESSAGES
processing file django.po in /home/chadams/awx/awx/locale/en-us/LC_MESSAGES
processing file django.po in /home/chadams/awx/awx/locale/ja/LC_MESSAGES
processing file django.po in /home/chadams/awx/awx/locale/es/LC_MESSAGES
processing file django.po in /home/chadams/awx/awx/locale/nl/LC_MESSAGES
npm --prefix awx/ui_next --loglevel warn run compile-strings
...
ISSUE TYPE
Bugfix Pull Request
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Fix relative imports in src code, relocate src
(Lots of files modified but the changes are low-risk and automated)
This PR is a follow-up to #6954, specifically the part where we removed webpack aliases to pick up react-scripts.
SUMMARY
Add the jsconfig.json w/ baseUrl to enable absolute imports from src. Replaces all the ../../../../../ importing
jsx -> js extension renaming
hooks directory
[ ] ui_next -> ui see: #10676
For posterity, the script used to fix the relative imports is included in the commit history.
python3 tools/fixrelative.py awx/ui_next/src
npm --prefix=awx/ui_next run lint -- --fix
npm --prefix=awx/ui_next run prettier
import argparse
import glob
import os
from shutil import move, copymode
from tempfile import mkstemp
def get_new_import_string(old_import_str, root):
if not root.startswith("./"):
raise Exception("root must start with './'")
name = root.replace("./", "")
left, right = old_import_str.split("from")
left += "from '"
_, trailing = right.split(name)
return left + name + trailing
roots = [
"./api",
"./components",
# "./constants",
"./contexts",
"./screens",
"./types",
"./util",
]
def get_root(line):
matched_root = None
for root in roots:
if root in line:
matched_root = root
break
if "jest" in line:
matched_root = None
return matched_root
def find_and_replace_roots(file_path, root_dir, preview):
fh, temp_path = mkstemp()
has_logged_file_name = False
with os.fdopen(fh, "w") as new_file:
with open(file_path) as old_file:
for (line_number, line) in enumerate(old_file):
matched_root = get_root(line)
if matched_root:
new_line = get_new_import_string(line, matched_root)
if not preview:
new_file.write(new_line)
if not has_logged_file_name:
log_file_replacement(root_dir, file_path)
has_logged_file_name = True
log_line_replacement(line_number, line, new_line)
elif not preview:
new_file.write(line)
if not preview:
copymode(file_path, temp_path)
os.remove(file_path)
move(temp_path, file_path)
def log_line_replacement(line_number, line, new_line):
display_line = line.replace(os.linesep, "")
display_new_line = new_line.replace(os.linesep, "")
print(f"\t (line {line_number}): {display_line} --> {display_new_line}")
def log_file_replacement(root_dir, file_path):
display_path = os.path.relpath(file_path, root_dir)
print("")
print(f"{display_path}:")
def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument("root_dir", help="Root directory")
parser.add_argument("--preview", help="Preview (no write)", action="store_true")
args = parser.parse_args()
return args
def run():
args = parse_args()
search_path = args.root_dir + "**/**/*.js*"
for file_path in glob.iglob(search_path, recursive=True):
find_and_replace_roots(file_path, args.root_dir, args.preview)
if __name__ == "__main__":
run()
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fix conditional to check that minikube should start
SUMMARY
make docker-compose tries to start container group without this fix
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.2.2
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Update User screens
Add modified field to user endpoints. The API returns the last login
date as the modified field to the users.
Modify user form layout to be as per mockups.
Remove last name and first name from user row, since it is
already on the column title.
closes: #5687closes: #9564
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Sarah Akus <sakus@redhat.com>
* Add `modified` field to user endpoints. The API returns the last login
date as the `modified` field to the users.
* Add modified to user details.
* Modify user form layout to be as per mockups.
* Remove `last name` and `first name` from user row, since it is
already on the column title.
closes: https://github.com/ansible/awx/issues/5687
closes: https://github.com/ansible/awx/issues/9564
Adds functionality for expand all
SUMMARY
Closes#4208. Adds basically the same work as useSelected, in a useExpanded hook.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Update login redirect to a be text input field
Update login redirect to a be text input field. Allowing values like
/sso/login/saml/?idp=SSO
closes: #10612
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Disable remove button if row is being dragged
Disable remove button if row is being dragged.
Also, disable drag button there is just one item selected.
closes: #10548
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Update RBAC sidebar
Update RBAC sidebar. Do the following:
Expose read-only applications to normal users in the UI
Expose read-only credential types to normal users in the UI
closes: #10432closes: #10433
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Point REST API favicon at the one the UI uses
SUMMARY
Don't ship one that is only used for the API browser.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
UI
AWX VERSION
devel
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Identify sliced jobs on Job List and Job Details
Identify sliced jobs on Job List and Job details - for workflow jobs.
closes: #2479closes: #10593
Jobs List
Job Details - Workflow Job
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Incorporates Minikube to devel environment
SUMMARY
Incorporates Minikube to devel environment
This PR allows creating a smooth devel environment deploying Minikube connected to the AWX devel network interface using Docker.
Furthermore, both CredentialType and Container Group gets created automatically.
Steps performed by the target make docker-compose-container-group
Downloads Minikube (currently supported Linux and MacOS)
Downloads kubectl (currently supported Linux and MacOS)
Starts Minikube using docker as backend
Creates a Service Account, Role and RoleBinding on Minikube
Exports Minikube ServiceAccount token to be used on AWX Credential
Creates Minikube AWX Credential
Creates Container Group to use Minikube backend
Starts regular AWX devel instances and connected with Minikube network
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
devel
ADDITIONAL INFORMATION
Quick step-by-step to use this PR:
$ make docker-compose-container-group-clean # this will remove your database and any old Minikube already deployed
$ make docker-compose-container-group
Verifying some work:
$ kubectl get serviceaccount awx-devel
NAME SECRETS AGE
awx-devel 1 130m
$ kubectl get roles
NAME CREATED AT
awx-devel 2021-07-15T00:47:37Z
$ kubectl get rolebindings
NAME ROLE AGE
awx-devel Role/awx-devel 131m
Credential
Container Instance Group
After updating a Job Template and assigning to the Container Instance Group
kubectl get pods -w
NAME READY STATUS RESTARTS AGE
automation-job-6-w6rmh 0/1 Pending 0 0s
automation-job-6-w6rmh 0/1 Pending 0 0s
automation-job-6-w6rmh 0/1 ContainerCreating 0 0s
automation-job-6-w6rmh 1/1 Running 0 5s
automation-job-6-w6rmh 1/1 Terminating 0 6s
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Marcelo Moreira de Mello <tchello.mello@gmail.com>
Reviewed-by: Sarabraj Singh <singh.sarabraj@gmail.com>
Update kubernetes pod doc url
SUMMARY
Just a little change allowing users to find the Kubernetes pod core documentation.
ISSUE TYPE
Docs Pull Request
COMPONENT NAME
UI
AWX VERSION
N/A
ADDITIONAL INFORMATION
N/A
Reviewed-by: Shane McDonald <me@shanemcd.com>
Update defaults.py
we missed this...need it for container groups to work out of the box in the dev environment
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Fixes missing key values in ad hoc credental step advanced search
SUMMARY
Resolves#10547
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Removes tabs from survey form
SUMMARY
Resolves#10499
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Display error message if schedule does not exist
Display error message if schedule does not exist when redirecting to
details page.
closes: #9550
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Update EE lookup title
Upated EE lookup name to be singular. Also, another reference that to EE
that should be singular.
Closes: #10497
Reviewed-by: Marliana Lara <marliana.lara@gmail.com>
Reviewed-by: Sarah Akus <sakus@redhat.com>
Test cleanup
SUMMARY
Makes a unit test fail if any errors or warnings are logged (see setupTest.js)
Updates/fixes all tests that were logging errors
Fixes a key cause of test flake caused by Jobs.test.jsx (I have still seen a rogue "Network error" cause a flaky failure once; which I will hopefully take care of in a subsequent PR)
Removes all unnecessary calls to wrapper.unmount() (this also surfaced several instances of tests leaving asynchronous code running after test completion, which are now fixed)
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Conditional polymorphic setnull
SUMMARY
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Wait until inventory is fully deleted in silent_delete
The silent_delete method does not appear to be used by the AWX CLI. I believe the only use is from:
awx/awxkit/awxkit/api/pages/base.py
Lines 185 to 187
in
eec4f8d
def cleanup(self):
log.debug('{0.endpoint} cleaning up.'.format(self))
return self._cleanup(self.delete)
This is used in integration tests to cleanup at the end of a test. Whenever inventory gets deleted, this leads to various conflict scenarios and deadlocks. This tends to play out by:
DELETE request for the inventory object
DELETE request for the organization
within the request-response cycle, this cascade deletes other objects, or takes SET_NULL action
Because the actual inventory deletion happens in a separate task, this means that inventory and organization deletion cascades are happening simultaneously (dependent on timing and resources).
This is low-priority on our list of practical concerns, because the inventory object accurately reports that its deletion is in progress. Clients can workaround if needed - and that's what I'm trying to do here. In an effort to reduce the flakiness and erroneous errors in integration tests, I propose that this will basically serialize the teardown process (for a given test agent), and that will eliminate a large cluster of flaky errors.
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Prevents submission of jtform if name is > 512 characters
SUMMARY
This addresses #10544 by adding a maxLength validator to the name field on the Job Template form
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Delete some old code related to the reencryption migration
We've moved past the point where this code would still be live.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
current
ADDITIONAL INFORMATION
We could probably squash more of these?
Reviewed-by: Alan Rominger <arominge@redhat.com>
Add job status on expand row for hosts
Add job status on expand row for hosts
See: #5236
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Run flake8 on the AWX collection in CI
We kind of assumed that linter issues like this would be picked up by ansible-test sanity, but they're not.
This seems to provide meaningful value. Unused variables are reflecting a lot of history of changes, and it's good to move on to leave those in the past.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
No result_traceback is blank, not null
I kept staring at a failure where:
cannot create resource
was expected in result_traceback, but it was blank. The job had "error" status, sure enough.
Then I re-created, sure enough, the job was right there - error status, no explanation, no traceback, no indication of what happened. We did have logs though (which I still can't fully disambiguate).
After that initial confusion, I realized this is a server-side bug. The test is happy with what happens after this change.
I can re-organize this diff if needed.
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
bumped django version to 2.2.20 in devel
SUMMARY
Bumped Django version from 2.2.16 -> 2.2.20 for the devel environment. All builds, tests and tests consuming the AWX_USE_FIPS flag passed in Jenkins.
AWX Issue - ansible/tower#4930
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.2.2
Reviewed-by: Alan Rominger <arominge@redhat.com>
At the request via webhook just uses default pre configured headers (with Agent/ContentType) by ignoring at all customer provided headers (self.headers).
Thus the OAuth authentication via Bearer cannot be implemented since custom headers are not sent to service so receiving Unauthorized response.
Solution is merging the default headers with custom ones (custom headers will override default ones in case of key clash).
Fix pylint 2.9 errors in awx collection
SUMMARY
fixes errors like these
ERROR: plugins/modules/workflow_launch.py:131:4: consider-using-dict-items: Consider iterating with .items()
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.2.2
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Make quay.io the registry URL default
SUMMARY
It's unclear what format the registry "Authentication URL" should be.
It's too late to change the strings because of translations, ping @rooftopcellist, but this change would clarify the format without adding wording.
ISSUE TYPE
Feature Pull Request
Docs Pull Request
COMPONENT NAME
API
ADDITIONAL INFORMATION
Seems to behave pretty good in the UI.
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
reload settings on fork or new thread
Reloading settings will re-import cache. Django cache is thread aware,
but keeping an old reference to the cache avoids the awareness.
The performance team that was hitting this every time before. With this change they are unable to recreated.
Reviewed-by: Shane McDonald <me@shanemcd.com>
Add job status on expand row for hosts.
Also, update HostDetail to remove the addition of `type=job` since it is
already present on the API hosts endpoint.
See: https://github.com/ansible/awx/issues/5236
Close file before returning
Resolves
/home/alancoding/repos/tower-qa/tests/lib/plugins/pytest_restqa/plugin.py:122: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/alancoding/repos/tower-qa/scripts/resource_loading/data_latest_loading.yml' mode='r' encoding='UTF-8'>
qe_config.resources = PseudoNamespace(yaml_file.load_file(config.option.resource_file))
We have this same pattern earlier in the file.
Reviewed-by: Shane McDonald <me@shanemcd.com>
Fix awx collections copy_item warning keyword error
SUMMARY
warn() does not take keyword 'msg'
add testing around the copy_from parameter for project creation
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.2.1
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Remove plus button job output since this feature was not implemented yet.
And it will be evaluated for a future release.
See: https://github.com/ansible/awx/issues/6613
- the task container needs to wait longer for migrations to complete for fresh installs before starting services
- otherwise, services start prematurely and clutter the logs with errors because migrations are mid-flight
--- Removed reference to tower in InventorySource and InventoryUpdate model
--- Added a migration for above change
--- Added new CONTROLLER* variables in awx/main/models/credentials/__init__.py
--- Migrated awxkit to new CONTROLLER* variables
--- Updated the tests to use new CONTROLLER* variables
--- Fix some issues with upgrade path, rename more cases
* Our tests could consistently get awx jobs into a deadlocked state
whenever the parallelism was high. Even podman ps would hang when the
system was in this state. We don't know exactly where in runc the bug is
but the deadlocks stopped happening when we changed the OCI runtime
environment to crun.
The logic to disable certain fields is just valid for `encrypted`
fields. Since the multiline field - no secret, does not have a button to `replace`
the content to null - it was always set to disabled.
Fix: https://github.com/ansible/awx/issues/10079
Remove more freenode from docs
SUMMARY
Removing more mention of Freenode, since we moved to Libera.
ISSUE TYPE
Docs Pull Request
Reviewed-by: Shane McDonald <me@shanemcd.com>
Getting all connections that have been alive > 5 minutes is noisy. Instead, we want only queries that are active and have been alive for more than 5 minutes. This will filter out any idle connections.
Update Dockerfile.j2
SUMMARY
Jobs unable to start because podman trying to use systemd cgroup manager. See error below :
WARN[0000] Failed to add conmon to systemd sandbox cgroup: dial unix /run/systemd/private: connect: no such file or directory
Error: OCI runtime error: systemd cgroup flag passed, but systemd support for managing cgroups is not available
related #10099
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.0.0
ADDITIONAL INFORMATION
According to this PR containers/podman#7009, podman switch references from libpod.conf to containers.conf.
According to containers.conf man (https://github.com/containers/common/blob/main/docs/containers.conf.5.md), configuration file is a TOML file but engine section declaration is missing.
thanks to @Siorde too 👍
Reviewed-by: Chris Meyers <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
List platforms that developers regularly run the dev env on
SUMMARY
Other platforms are generally untested.
ISSUE TYPE
Docs Pull Request
Reviewed-by: Julen Landa Alustiza <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Update and expand docs/ folder for EEs
This has some new content about EE precedence, which I don't think we've documented anywhere else, thinking of @tvo318 here, content was developed by @jbradberry
(I think the numbers 2 and 3 in the global job default EE may still be subject to revision, just a warning)
@shanemcd #10324 is incorporated into this
This mentions @rebeccahhh's venv migration stuff, but I'm trying to write the absolute minimum possible while still mentioning migration.
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jeff Bradberry <None>
so, remove them from the API endpoints for workflows. Also, tear out
the WFJT.execution_environment step in the resolver. If we want that
to be a thing, it ought to be a .default_environment instead.
* Show resolved EE for job template details.
* Do not show EE if a certain job status still running, since EE on the
API is not set yet. It was causing the bug to temporarily show `missing
resource`.
* Fix discrepancy about job types for list and details.
See: https://github.com/ansible/awx/issues/10327
Also: https://github.com/ansible/awx/issues/10399
Add RBAC rules to the side-nav
System Admin
System Auditor
Org Admin
Notification Admin
Execution Environment Admin
Normal User
Those are the user profiles taken in consideration when displaying the
side-nav.
See: https://github.com/ansible/awx/issues/4426
Handle inventory types where Automation Hub collection names differ
Some collections will be moving, which is #10323
Yet, other collections will change name, which this is intended to address.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Sarabraj Singh <singh.sarabraj@gmail.com>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Remove isolated instances and groups
SUMMARY
before we remove the fields that allow us to correctly identify them.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Chris Meyers <None>
Add validation for EE image field name
related #10431
SUMMARY
Adds validation for the EE image name
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.2.1
Reviewed-by: Seth Foster <None>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Jobs unable to start because podman trying to use systemd cgroup manager. See error below :
```
WARN[0000] Failed to add conmon to systemd sandbox cgroup: dial unix /run/systemd/private: connect: no such file or directory
Error: OCI runtime error: systemd cgroup flag passed, but systemd support for managing cgroups is not available
```
* According to this PR https://github.com/containers/podman/pull/7009, podman switch references from libpod.conf to containers.conf.
* According to containers.conf man (https://github.com/containers/common/blob/main/docs/containers.conf.5.md), configuration file is a TOML file but engine section declaration is missing.
introduced a pre-flight check for postgres 12
SUMMARY
Introduced a pre-flight check to check the version of Postgres 12 before the application starts - ansible/tower#4940
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.2.0
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Sarabraj Singh <singh.sarabraj@gmail.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Prefer installer defaults over user-defined global EEs
The installer knows better than the user
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Add Thycotic DevOps Secrets Vault support
SUMMARY
Add a Credential Plugin for Thycotic DevOps Secrets Vault
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
UI
AWX VERSION
awx: 19.1.0
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Adam C. Migus <adam@migus.org>
Reviewed-by: Matthew Jones <bsdmatburt@gmail.com>
Make ExecutionEnvironment awxkit class able to handle null values in …
This is how the Project class does it, and I can confirm it makes factories.execution_environment(organization=None) work correctly.
Reviewed-by: Jeff Bradberry <None>
Fixes bug where source list page would crash if first sync was running
SUMMARY
link #10411
The ID being passed to the cancel button was incorrect as well. We want to reference current_job not last_job.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Kersom <None>
Reviewed-by: Sarah Akus <sarah.akus@gmail.com>
Addresses bug where advanced search by groups wasn't working on host list
SUMMARY
link #10400
OK so this got a bit more complicated than I wanted it to but I think on the whole this is a nice improvement. In the initial bug report it was noted that we were ignoring the or/not operators when searching on a related key (in this case it was groups). We were also ignoring the modifier at the end (icontains, exact, startswith, etc). Since we were always defaulting to __search for these types of keys this was actually valid. The API doesn't like us attempting to do something like ?or__groups__search=foo or ?groups__search__icontains because they aren't valid. So, I changed the UX a little bit. A user can still do a fuzzy search on a related key but the prefix is disabled in this case:
I changed the third dropdown (specifically for related keys) to contain the following options: search, id, name__icontains which I think would be the three most general cases. id and name__icontains do allow for prefixing (or, not, and).
This should make searching on related keys a little bit easier and a little bit more flexible since name and id weren't possible in the UI before.
Once place where this is a little different is the host filter in the smart inventory form. Using __search currently throws an error when the user attempts to save. I believe this is an API bug and will file it as such but for now I prevent users from attempting to use __search by removing it as an option for this particular list. When this bug gets fixed in the API we can remove this logic.
I also noticed that there was a bug where or__ search terms were being converted to and when using the Smart Inventory button on the host list so I fixed that.
Here's the fix in action:
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
fixes next button and internationalizes nav buttons on wizard
SUMMARY
Resolves#10404 and it internationalizes the next button.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Pull brand name from default.strings.json
SUMMARY
We were pulling this string from variables.js.
There were several places where the brand name was not being pulled dynamically - this PR addresses that. This changes the way we set the document title a little bit. Since we needed access to a dynamic string, I went down the JS route to set this.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Sarah Akus <sarah.akus@gmail.com>
Changes settings revert all to send DELETE on individual endpoint rather than PATCHing
SUMMARY
Resolves#3831#3831 (comment) is a good explanation of the state of the new UI as well as the old. The Misc System settings were the most problematic part of this since that form pulled fields from both /api/v2/settings/system and /api/v2/settings/authentication. In order to revert these fields with a DELETE request I needed to split this form up into Miscellaneous System and Miscellaneous Authentication. The Activity Stream Settings also needed to be absorbed by the Miscellaneous System Settings.
All settings forms (except noted below) will now issue a DELETE request to revert all the fields for a particular settings subsection.
There was one form that I did not change the Revert All functionality on and that was the LDAP Settings form(s). The UI splits these fields out into 5 different subforms and reverting one (issuing a DELETE on /api/v2/settings/ldap) would revert all the ldap settings which I think is undesirable.
This fix has the added benefit of cleaning up future activity stream entries for reverting an entire settings category.
I also consulted with @wenottingham and @gamuniz on some fields that were present in the api but missing in the ui and I added those.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add support for Insights as an inventory source
SUMMARY
Use the insights inventory plugin as an inventory source.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
ADDITIONAL INFORMATION
Requires at minimum RedHatInsights/ansible-collections-insights#14
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Chris Meyers <None>
Control plane EE
SUMMARY
Per discussion, we are now going to have privileged execution environments for use on the control plane, and these will not be allowed to be edited or deleted.
related ansible/tower#5016
TODO
reinstate the restrictive RBAC for managed_by_tower=True EEs
convert any EEs automatically installed prior into managed_by_tower=False EEs
change the resolver so that ordinary jobs do not get a managed EE, and project updates only get a managed EE
allow sysadmin users to edit the pull field for managed EEs
automatically disassociate EEs that get deleted from the DEFAULT_EXECUTION_ENVIRONMENT setting #10363
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Seth Foster <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Resolves issue where FireFox users could not schedule a job template
SUMMARY
This addresses #10152 and #6214
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Michael Abashian <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
DRF, when using this field, short-circuits the call to
.to_representation() when the value is None, since clearly you aren't
going to be able to get the .pk attribute off of it in that case. We
were previously unconditionally calling .to_representation() which
throws an error when we try to clear the value of
DEFAULT_EXECUTION_ENVIRONMENT.
Adds version context processor back in to fix api browser doc link
SUMMARY
Here's what it looks like on devel now (note the URL in the bottom left):
Here's what it looks like after the change (note the URL in the bottom left):
I dropped this in as it was before the removal of the old UI. I believe the new UI needs access to some of these variables as well to force assets to be refetched after upgrade.
Also of note: I have no idea what I'm doing with django so please help me to become educated if I've done something silly here.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Stops infinte REST calls on ResourcesAccessAdd workflow
SUMMARY
resolves#10350
There is no functionality changes, I just moved the location of the variables so they are defined outside of the component.
E2E test triggered
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Do not show `not` as choice for Advanced Search on Smart Inventory
Do not show not set type as a choice for Advanced Search on Smart Inventory since
this feature is not implemented on the API side yet.
See: #2817
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Update translation make target for ui_next
SUMMARY
This adds the --clean flag to the extract-strings command (cleans up obsolete strings) and separates out the extract-template command.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
Translation Automation
AWX VERSION
19.2.0
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Extend launched by details to handle scheduled JTs and WFJTs
SUMMARY
Issue: #5057
Launched By detail should link to the schedule that launched the job.
ISSUE TYPE
Enhancement Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Sarah Akus <sarah.akus@gmail.com>
Add OPTIONS documentation for new job limit feature
Looking at the docs and stuff from #10023
I'm sure this is somewhere else too, but this is the place that users should naturally expect it to be.
Reviewed-by: Chris Meyers <None>
Intentionally drop job event websocket messages in excess of 30 per second (configurable)
SUMMARY
The UI no longer follows the latest job events from websocket messages. Because of that, there's no reason to send messages for all events if the job event rate is high.
I used 30 because this is the number of events that I guesstimate will show in one page in the UI.
Needs the setting added in the UI.
This adds skip_websocket_message to event event_data. We could promote it to a top-level key for job events, if that is preferable aesthetically. Doing this allows us to test this feature without having to connect a websocket client. Ping @mabashian @chrismeyersfsu
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
UI
ADDITIONAL INFORMATION
Scenario walkthrough:
a job is producing 1,000 events per second. User launches it, the screen fills up in, say 1/4 of a second. The scrollbar indicates content beyond the bottom of the screen. Now, for 3/4ths of a second, the scrollbar stays still. After that, it updates the scrollbar to the current line number that the job is on. The scrollbar continues to update the length of the output effectively once per second.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Always send websocket messages for
high priority events like playbook_on_stats
Never send websocket messages for
events with no output
unless they are a high priority event type
Enter flake8 rules in tox config as actual config entries
I really like to do things like
flake8 awx/main/models
but this is made impractical where running flake8 for the project carries with it an assumption that you will pass a specific set of options.
https://flake8.pycqa.org/en/latest/user/configuration.html
Blessedly, flake8 is flexible in their config options so that it may weather the storm of python project config wars.
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Add a field for hosts automated across to the subscription info
SUMMARY
This is populated by the new table we've added.
Update the subs check to check against this, not imported hosts.
ISSUE TYPE
Feature Pull Request
Bugfix Pull Request
COMPONENT NAME
API
UI
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Amol Gautam <amol_gautam25@yahoo.co.in>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Do not show `not` as choice for Advanced Search on Smart Inventory since
this feature is not implemented on the API side yet.
See: https://github.com/ansible/awx/issues/2817
Only attempt to reconnect socket if connection wasn't closed cleanly
SUMMARY
link #8596
After some investigation, I believe that this error is caused by the reconnect logic that we have. When the component that references the ws hook unmounts, we disconnect the socket. There's some logic in the onclose method that attempts to reconnect the socket but if we've disconnected cleanly we shouldn't try to reconnect (because we probably meant to disconnect in the first place).
This should clean up the console errors that we've been seeing about the socket already being in a connected state since we won't have timers running past the lifecycle of the component.
cc @keithjgrant does this sound good to you?
Here's the spec for the disconnect event https://developer.mozilla.org/en-US/docs/Web/API/CloseEvent#properties
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Adds ability to refresh project revision on sync'd rows in the project list
SUMMARY
This PR also adds the revision to the project details view as well as handles updating the revision on the project details view when the project is done syncing.
Since the project revision is not included in the websocket payload, when a project is done syncing the displayed revision may be out of date. As such, we wanted to expose that information to the user and give them the ability to "refresh" and fetch the new revision.
Here's what that flow looks like:
When a particular row finishes syncing the user should see this in place of the revision:
Clicking on that refresh button goes out and fetches the updated project (and with it the potentially updated revision).
We don't do this automatically on the projects list (and force the user to click on the refresh button) is due to issues we've had in the past with the UI triggering API calls based on websocket events.
The flow when a user is on the project details view is a little different because I wasn't as worried about spamming the API with requests.
When a project finishes syncing and the user is viewing the details I do go ahead and automatically refresh the project data (and with it, the revision). Here's what that looks like:
A few other notes:
@tiagodread @akus062381 @one-t I'm almost certain this is going to break some tests because I removed the ClipboardCopyButton in favor of PF's ClipboardCopy component. This component looks and behaves slightly differently from our home grown solution. Additionally, the PF ClipboardCopy button does not expose the ouiaId prop so I had to add a data-cy on that component. You'll likely have to use that identifier to then grab the button inside in order to test out the clipboard copy functionality.
Source Control Revision is a net-new detail in the project details. I think this was simply missed on our initial build of this page.
Here are the identifiers on the various bits:
Note that the identifiers on the project rows have the id of the project appended to them for uniqueness.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Michael Abashian <None>
Reviewed-by: Sarah Akus <sarah.akus@gmail.com>
Avoid reload on manual logout
SUMMARY
#10383
Instead of reloading, set the authRedirectTo context variable to "/logout" and handle it as a special case when routing
Reviewed-by: Marliana Lara <marliana.lara@gmail.com>
Remove uses of ansible_virtualenv_path
Run a job with recent AWX, then inspect:
In [2]: Job.objects.order_by('-created').first().custom_virtualenv
Out[2]: '/var/lib/awx/venv/ansible'
We shouldn't do this. The job was ran after the custom_virtualenv field has become entirely informative / non-functional. It is very arguable whether jobs should keep this field. For pre-migration jobs, it is an accurate record of fact, so I'm not going out of my way to argue for removal.
With this change, new jobs will record a blank value for custom_virtualenv, which is more correct, because we don't do this.
I have carefully looked over @rebeccahhh's #10090, and I don't see any way that ansible_virtualenv_path is used. Model use is reported for templates, so that doesn't block removing the field on jobs either.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Disable cancel button on http 405
SUMMARY
for #9905
When a 405 is received, it means the job is already cancelled. Treat the
request as a success and don't show an error modal.
We disable the button to handle a rare scenario where we receive the 405 long before
the job status is properly updated to "cancelled" over websockets. We want to prevent
more cancel requests when this happens. We're disabling instead of hiding the button
because, assuming the status hasn't changed over sockets, the buttons that usually
replace the cancel button on the toolbar won't be shown yet and we don't want to
needlessly flicker and shift button positions around by rapidly loading and unloading
a different number of buttons onto the bar.
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Sarah Akus <sarah.akus@gmail.com>
- Add a field for hosts automated across
This is populated by the new table we've added.
- Update the subs check to check against this, not imported hosts.
- Reword messages on inventory import
Update message for missing EE for Job pages
Update message for missing EE for Job pages
See: #10355
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Kersom <None>
Reviewed-by: Michael Abashian <None>
add a new awx-manage command `custom_venvs`
add an awx-manage command that gets pip freeze data from custom_venv and outputs to command line stdout
SUMMARY
part of #7062 - this command is a glorified pip freeze + some extra stuff, people could navigate to each of their custom virtual environments themselves and run a pip freeze, but this allows them to not, and everyone likes their life to be easier. The extra stuff allows users to see the connections that their existing virtual envs have in awx to things like organizations, jobs, inventory updates, and projects.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.1.0
ADDITIONAL INFORMATION
This is built off of existing code and there is a line that gets custom venv paths from the settings module, that line does not seem to be working. I have written around that but want to make a note of it.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Db partition analytics cmeyers2
Keep old primary key based analytics gathering for unpartitioned
tables.
Use created time on new partitioned tables.
80 million partitioned + 1.5 million unpartitioned Events
Query
awx-manage gather_analytics --dry-run Time
Micro Benchmark Query Time*
Query Only Time**
sequential index scan, multiple ::json casts, 100,000 event batches
102m7.836s
6s
80 minutes
sequential index scan, optimized json cast, 100,000 event batches
48m9.276s
2.2s
30.4 minutes
sequential index scan, optimized json cast, 1,00,000 event batches
39m35.094s
10s
13.3 minutes
sequential table scan, optimized json cast, per-partition batch 600,000 ***
36m42.081s
11.5s
25.5 minutes
*micro benchmarking consists of simply copying a query, running it manually, and observing the runtime.
**micro benchmark time x (80 million / batch size)
**Note that this testing does NOT include the extra modified range query that is needed for correctness. We expect this to be quite fast and is only needed to catch edge case events.
Reviewed-by: Ladislav Smola <lsmola@redhat.com>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
* job_created is a fake field as far as Django is concerned. Under the
hood, in postgres, this is the partition key so it is real. sqlite
doesn't support partitioning so we need to fake some things.
Specifically, we need to remove job_created from being auto-added to
get_event_queryset()
* Add pagination tests for <unified_job_name>/<id>/<job_events>?limit=x
endpoint to make sure the paginator is wired up.
* Use an initial request for max event `counter` to get the total row count,
otherwise rely on websocket message counters to update remote row count
* For running jobs, request event ranges with counters to handle events getting
saved to db out of display order
* For jobs that are no longer running, continue to use page/pageSize scheme for
paging through the job events
* trigger via jobs/<id>/job_events/?limit=10
* Can and should be used in conjunction with an indexed set of fields to
generate efficient pagination queries. i.e.
jobs/<id>/job_events?limit=10&start_line__gte=10
* If limit is not specified in the query params then the default
pagination will be used.
* Before, we would get the min and max pk of the set we are to gather.
This changeset removes that.
* Before, we would, basically, know the size of the set we are to gather
and would query 100,000 of those job event records at a time. That logic
is now gone.
* Now, for unpartitioned job events we gather 4 hours at a time by
created time.
* Now, for partitioned job events we gather 4 hours at a time by
modified time.
* The order by results in an in-memory sort that COULD blow out the
worker mem buffer and result in sorting having to take place on disk.
* This WILL happen with a default postgres 4MB mem buffer. We saw as
much as 20MB used. Note that AWX defaults postgres mem worker buffer to
3% of the DB memory on external installs and 1% on same-node installs.
So for a 16GB remote DB this would not be a problem.
* We are going to avoid this problem all together by NOT doing a sort
when gathering. Instead, we will sort remotely, in analytics.
* Old, _unpartitioned_main_jobevent table does not have the job_created
column
* New, main_jobevent does.
* Always in clude the job_created column. NULL if old, job_created if
new
* Bump events_table schema version from 1.2 to 1.3 because of the
job_created field
* We found that having multiple `::json` casts in a query slows down
queries more and more by =~> 33%.
* This change coerces postgres into only casting once. Micro
benchmarking shows =~ 2-3x performance boost
* pre-migration jobevents live in unpartitioned table
where only created field has index
* post-migration jobevents live in partitions
where modified field has index
(and should be used to ensure no events are missing)
* when collecting job events by creation time
it is possible to miss events that were created
at one point, but actually committed to the db
much later.
* since events' modified time is set when they are
committed to the db, we shouldn't miss any job events
* selecting job events by modified time wasn't possible
beforehand because we didn't have an index for
jobevent's modified field
Changes in old unpartitioned cleanup logic:
* Manually cascade delete events related to job(s)
(new partitions cleanup logic) For each event type:
* Get the event partitions that are within the cleanup date range
* Get a list of jobs to delete that are in the cutoff range.
* Jobs that are running, pending, or waiting in the job list are special.
* Use the special list to further filter the partition drop list.
* Drop partitions
* delete jobs
* when tests create a UnifiedJob and JobEvent,
the two need to have the same value for job creation time
* some view validation was skipped due to `model` being
a property in some cases now
we don't need this code at all anymore - the bigint migration is long
gone, and anybody upgrading to this version of AWX has already
migrated their data
keep pre-upgrade events in an old table (instead of a partition)
- instead of creating a default partition, keep all events in special
"unpartitioned" tables
- track these tables via distinct proxy=true models
- when generating the queryset for a UnifiedJob's events, look at the
creation date of the job; if it's before the date of the migration,
query on the old unpartitioned table, otherwise use the more modern table
that provides auto-partitioning
* if we use the actual old job events table
and make tweaks to its schema
namely, dropping the pkey constraint,
then when we go to migrate the old job events
we will be forcing postgres to do a sequential scan
on the old table, which effectively causes the migration to hang
events that existed *prior* to the partition migration will have
`job_created=1970-01-01` auto-applied at migration time; as such,
queries for these events e.g., /api/v2/job/N/job_events/
use 1970-01-01 in related event searche
events created *after* the partition migration (net-new playbook runs
will have `job_created` values that *exactly match* the related
`UnifiedJob.created` field.
* raw sql commands were in migration to partition table
* .. just needed to add FakeAddField entries for the new
job_created field added to each job event model
* .. and also needed to actually list the new field on the model classes
- each job event table has a different name for the
fk referencing the unified job id
- create a mapping so that we get the col name correct
for each table
* First partition holds all events up to this very moment
* And second partition starts where first left off and runs
.. through rest of current hour
* From there, dynamically generated partitions will cover
one hour at a time
* copy the table just like we do in the bigint migration
* without this we lose sequences and very likely other things
as well
* we want the new table to be identical to the old table,
so 'including all' makes sense
Make IG credential linkable
SUMMARY
Make the instance group credential detail link to the cred
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Adds support for typing values into single select lookups
SUMMARY
link #6073
This adds support for typing values into single select lookups. Multi-select lookups (credentials on the JT form, instance groups, etc) will remain unchanged.
An input is now rendered next to the lookup button which will allow users to type a value in.
Here's the button (which opens the modal):
Here's the input (which lets users type a value in):
There's a debounce on the input so that we only make a request to check to see if the name is valid after a second of no typing.
The tricky part of this implementation was handling validation on the lookup fields. If a user types in a string that does not exactly match a value that they can use then we want to show an error indicating that no matching value was found:
but this was tricky because some of these fields are required so the validation functions would need to change a bit.
Since the typed text and the actual underlying value of the lookup are different the validation function would need to have access to both values whenever it runs for this to work. This meant either storing the typed text up at the form level or pushing the validation down into the lookup(s). After talking with @keithjgrant we decided to go down the route of pushing the validation down to the lookups.
You can now pass an optional validation function down to the various lookups that will get combined with the typeahead validation via the useField hook.
Here's the whole thing in action:
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Michael Abashian <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fixes bug where checkbox list item was selecting things twice
SUMMARY
Resolves#10338
There was a click event on the row and the underlying checkbox. I got rid of the underlying click event so now its only handled at the row level.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Expose the EE resolver to the API
SUMMARY
This change exposes the output of .resolve_execution_environment() for job templates as a new summary field in the API. Note that this is only exposed for the detail views, not the list views. Also, there is the caveat for job templates that may be run under workflows that the workflow EE might override the results of this value.
related #10210
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Jeff Bradberry <None>
Skip sliced jobs from the workflow EE logic
Addressing some unanticipated fallout from #10305
Sliced jobs rely on creating a workflow job, but they do not have a workflow_job_template, so in those cases, that was causing a traceback.
2021-06-03 20:10:52,319 ERROR [a17ebd7f] awx.main.dispatch Worker failed to run task awx.main.tasks.RunJob(*[341], **{}
Traceback (most recent call last):
File "/awx_devel/awx/main/dispatch/worker/task.py", line 90, in perform_work
result = self.run_callable(body)
File "/awx_devel/awx/main/dispatch/worker/task.py", line 65, in run_callable
return _call(*args, **kwargs)
File "/awx_devel/awx/main/tasks.py", line 759, in _wrapped
return f(self, *args, **kwargs)
File "/awx_devel/awx/main/tasks.py", line 1264, in run
self.instance = self.update_model(self.instance.pk, execution_environment=self.instance.resolve_execution_environment())
File "/awx_devel/awx/main/models/mixins.py", line 477, in resolve_execution_environment
if wf_template.execution_environment is not None:
AttributeError: 'NoneType' object has no attribute 'execution_environment'
that is fixed with this, at least for my one simple test case
This left jobs hanging out in "waiting" status, which is really not good.
Reviewed-by: Jeff Bradberry <None>
Add additional controller directory for collections for inventory update
SUMMARY
This may be present in some scenarios with additional collections for inventory usage.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Fix error msg wording and sdb docs
SUMMARY
Fixes wording in container groups error message
Updates ports for sdb debugging docs
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
Reviewed-by: Shane McDonald <me@shanemcd.com>
When a 405 is received, it means the job is already cancelled. Treat the
request as a success and don't show an error modal.
We disable the button to handle a rare scenario where we receive the 405 long before
the job status is properly updated to "cancelled" over websockets. We want to prevent
more cancel requests when this happens. We're disabling instead of hiding the button
because, assuming the status hasn't changed over sockets, the buttons that usually
replace the cancel button on the toolbar won't be shown yet and we don't want to
needlessly flicker and shift button positions around by rapidly loading and unloading
a different number of buttons onto the bar.
updated ansible-runner version from 2.0.0a2 -> 2.0.0a4 in requirement…
SUMMARY
version bump (ansible-runner 2.0.0a2 -> 2.0.0a4)
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.2.0
Reviewed-by: Shane McDonald <me@shanemcd.com>
Fixes bug where user/team role add modal state is not cleared on close
SUMMARY
link #8824
I modeled these changes after the pattern that existed between ResourceAccessList and AddResourceRole. The key to fixing this bug is that the component that renders the wizard needs to be unmounted when the wizard closes so that the state, etc get cleared out before the next time the wizard is opened. In order to achieve that I needed to decouple the add button from the wizard.
The sort of weird part of this pattern (and this exists in the ResourceAccessList as well) is error handling. We pass the error back and set that to state before rendering the modal which isn't quite as clean as having the request made out at the list level and leveraging our hooks for error handling but I decided to just get in and get out and not worry about trying to refactor too much.
Here it is in action:
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Force fully qualified image names
If we try and pull an unqualified image name, jobs hang on a podman
prompt.
I set the permissions as 644 because thats what worked for me because rootless podman needs to be able to read the file, but maybe there is another way to achieve that
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
allow singular execution_environment for cli
this makes "execution_environments" or "execution_environment" work
which is helpful
Now these both work:
awx execution_environments create --name foobar --image quay.io/ansible/awx-ee:latest
awx execution_environment create --name foobar1 --image quay.io/ansible/awx-ee:latest
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Jeff Bradberry <None>
Restore UI license checker
27219d3 removed the license checker for UI dependencies as part of removing the UI. We actually still need it, just pointed at the new UI.
This brings it back, and updates it to do so. It also removes no longer needed license files.
It doesn't pass yet, as I haven't added licenses for the new UI dependencies in this PR. Hi @jakemcdermott.
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Clear list selections on pagination/search/sort
SUMMARY
Updates nearly every list* so that URL param changes (pagination, search, or sort) clear the selection. This prevents the list of selected items in state from including items that may no longer appear on screen — preventing the user from accidentally deleting or otherwise altering an item they may not realize they still have selected.
This also updates the useSelected hook to provide selectAll and clearSelected functions. Any lists that weren't yet already using this hook have been updated to do so.
Addresses #6853 and #7509
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
*Lists that do not include this change are modals where the user is expected to paginate through screens and make several selections along the way (e.g. Multi Credential select modal), and lists that still use PaginatedDataList and are yet to be converted to PaginatedTable
Note: I originally wanted to make the clearSelected prop on PaginatedTable required, so any lists that don't have this fix applied would fail loudly. Unfortunately that wasn't possible, as there were a few lists that should not have this behavior, so I had to leave it as an optional prop.
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Add Session context and redirects
SUMMARY
Issues:
#8272#9197#8155#9311
When user is logged out due to inactivity, redirect to the Login Page with a warning alert stating that their session expired.
When a user logs in on one tab, redirect from login page to home on the other tabs.
Clear session state when it expires and/or when a user logs out.
Redirect users to direct link destination after authentication
@AlexSCorey is looking at possibly rolling #9915 (SSO login redirect override URL is not working) into this PR
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Marliana Lara <marliana.lara@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
fix namespaced url params
SUMMARY
Fixes issues when multiple lists are on the page (generally only occurs now when on a list page with an open modal that contains a second list) — Navigating within the modal list currently wipes out URL parameters for the main list, causing the page to reload, which closes the modal. The fix prevents changes to one set of namespaced URL parameters from wiping out URL parameters from another namespace
Refactors query string utils to consolidate a lot of repeated logic from components into a new util, updateQueryString. Use this function to modify query string parameters while maintaining params for multiple namespaces at the same time.
QS utils that are no longer needed have been deleted: replaceParams and encodeNonDefaultQueryString
Addresses #10181 and #9218
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: John Mitchell <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Adds ouiaId to YAML and JSON buttons in Variables Detail
small change to add a better locator to these buttons. if we do not do this, our locators for our visual tests are not good and will eventually break tests
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Sysadmins may need to modify managed_by_tower credentials, the
only known case of which is a default container registry credential
for use by the default execution environments, which are also managed_by_tower
and allow modifications.
Create EE at a seperate time and also attach a registry credential if
auth information provided
This command can be run multiple times on the same instance and should be
idempotent.
Include the EE set on a workflow template in the resolver hierarchy
SUMMARY
This step comes immediately after checking the actual job/template for
an explicitly set EE.
Note that now, because of how jobs are spawned off of workflow nodes,
the call to .resolve_execution_environment() no longer happens in
.create_unified_job(). The job instance within .create_unified_job()
doesn't yet have access to the node that it will be attached to,
making it impossible to use this information in the resolver if called
there.
related #9560
ISSUE TYPE
Feature Pull Request
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
when sharing paths use little z
AWX_ISOLATION_SHOW_PATHS will be shared between containers. Strange
file not found error can crop up when concurrently accessing shared
directories between multiple containers that are bind mounted with big
Z. So make sure we use little z.
Fixes errors like below:
fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/usr/bin/git ls-remote file:///opt/tmpawx/at_DrunkMail525450112299457413919634186288881628802211907645041298254_test/ -h refs/heads/HEAD", "msg": "fatal: '/opt/tmpawx/at_DrunkMail525450112299457413919634186288881628802211907645041298254_test/' does not appear to be a git repository\\nfatal: Could not read from remote repository.\\n\\nPlease make sure you have the correct access rights\\nand the repository exists.", "rc": 128, "stderr": "fatal: '/opt/tmpawx/at_DrunkMail525450112299457413919634186288881628802211907645041298254_test/' does not appear to be a git repository\\nfatal: Could not read from remote repository.\\n\\nPlease make sure you have the correct access rights\\nand the repository exists.\\n", "stderr_lines": ["fatal: '/opt/tmpawx/at_DrunkMail525450112299457413919634186288881628802211907645041298254_test/' does not appear to be a git repository", "fatal: Could not read from remote repository.", "", "Please make sure you have the correct access rights", "and the repository exists."], "stdout": "", "stdout_lines": []}```
Reviewed-by: Shane McDonald <me@shanemcd.com>
* AWX_ISOLATION_SHOW_PATHS will be shared between containers. Strange
file not found error can crop up when concurrently accessing shared
directories between multiple containers that are bind mounted with big
Z. So make sure we use little z.
Adds resource name and description to launch and schedule prompt wizards
SUMMARY
link #8249
link #7254
Launch with description:
Launch without description:
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Kersom <None>
Reviewed-by: Michael Abashian <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
clarify format of date for host_metric
SUMMARY
@amolgautam25 @AlanCoding let me know what you think -- I was taking a second look at the CLI and took me a bit to remember the format it wants is YYYY-MM-DD
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Amol Gautam <amol_gautam25@yahoo.co.in>
This step comes immediately after checking the actual job/template for
an explicitly set EE.
Note that now, because of how jobs are spawned off of workflow nodes,
the call to .resolve_execution_environment() no longer happens in
.create_unified_job(). The job instance within .create_unified_job()
doesn't yet have access to the node that it will be attached to,
making it impossible to use this information in the resolver if called
there.
Fixes bug where users were unable to click on text next to checkboxes/radios in modals
SUMMARY
link #8852
This should impact lists in modals where the user can select one or more of the rows. They should now be able to click on the text/row in order to select. Examples:
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add the ability to disable local authentication
SUMMARY
When an external authentication system is enabled, users would like the ability to disable local authentication for enhanced security.
related #4553
TODO
create a configure-Tower-in-Tower setting, DISABLE_LOCAL_AUTH
expose the setting in the settings UI
be able to query out all local-only users
User.objects.filter(Q(profile__isnull=True) | Q(profile__ldap_dn=''), enterprise_auth__isnull=True, social_auth__isnull=True)
see: awx/main/utils/common.py, get_external_account
write a thin wrapper around the Django model-based auth backend
update the UI tests to include the new setting
be able to trigger a side-effect when this setting changes
revoke all OAuth2 tokens for users that do not have a remote
auth backend associated with them
revoke sessions for local-only users
ultimately I did this by adding a new middleware that checks the value of this new setting and force-logouts any local-only user making a request after it is enabled
settings API endpoint raises a validation error if there are no external users or auth sources configured
The remote user existence validation has been removed, since ultimately we can't know for sure if a sysadmin-level user will still have access to the UI. This is being dealt with by using a confirmation modal, see below.
add a modal asking the user to confirm that they want to turn this setting on
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
UI
AWX VERSION
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Mat Wilson <mawilson@redhat.com>
Reviewed-by: Michael Abashian <None>
Reviewed-by: Chris Meyers <None>
Remove explicit eula acceptance
SUMMARY
Issue: ansible/tower#4948
This PR changes the entitlement process to no require explicit acceptance of the Eula, but still display it for the user to read.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
UI
AWX VERSION
19.1.0
ADDITIONAL INFO
If the eula_accepted field is passed, it will just be ignored.
Reviewed-by: Bill Nottingham <None>
Redirect documentation to awx-operator docs
SUMMARY
CC: @shanemcd
Related: ansible/awx-operator#330
The PR ansible/awx-operator#330 enhances the AWX installation process via the awx-operator.
This PR redirects the installation notes to the awx-operator page so we can simplify and easy maintain the docs in one place.
ISSUE TYPE
Docs Pull Request
Reviewed-by: Shane McDonald <me@shanemcd.com>
The list secrets role rule is no longer not needed for container groups
SUMMARY
The list secrets role rule is no longer not needed for container groups. This was simplified to just get the single matching secret using the read_namespaced_secret() method.
Reviewed-by: Seth Foster <None>
Converts Options lists and other modal lists to tables
SUMMARY
This addresses #10162. Note this issue I found while working on this #10181
Kicked Off E2E
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fixes bug where activity stream changes were displaying as [object object]
SUMMARY
Resolves#10138
Here it is working:
In this case the value passed to the VariablesDetail component is a straight up JSON object (not a stringified JSON object) which that component does not seem to like in its current state.
Rather than looking at changing the VariablesDetail implementation I decided to just stringify the object before it gets passed in. Since this object is generated by the backend I think it's safe to assume it will always be an object.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Update migration page as per latest name changes
Update migration page as per latest name changes
rename.mov
See: #8355
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Container groups registry creds
SUMMARY
Fixes#10114
In VM-based installs, the user provides image pull creds to us, then we put them in an authfile and give it to podman via --authfile.
This is not so simple with ContainerGroups because we need to use the k8s API to apply a podspec to create containers in this paradigm.
Currently, the EE pod gets created, but errors when pulling the custom EE in from the private repo:
This work will modify the init() for the AWXReceptorJob class to create a k8s secret in the given namespace, then specify that secret name in the pod-spec as an imagePullSecret
Also, the imagePullPolicy was not being enforced when running JT's in EE's using container groups, this is because the imagePullPolicy nevery got set on the pod spec.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
devel
ADDITIONAL INFORMATION
Issues that this solves:
Image pull secret now gets created in the cluster namespace specified by the user for that container group.
imagePullSecret name gets set on the pod spec
If the pull secret already exists in the namespace, delete it, then create it. (kube_api.replace_namespaced_secret did not work for this case...)
Enforce imagePullPolicy for EE's in container groups
Basic error handling
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Chris Meyers <None>
Updates PF
SUMMARY
This is a dependency bump for PF so that I can get an update that fixes a validation bug in pf date picker.
E2E test have been triggered
ISSUE TYPE
dep bump
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
- base64 encode secret values before creating the secret
- Construct valid .dockerconfigjson
- Cancel jobs where it will obviously fail & error handling
- Check if the secret exists first, then attempts to replace it if it does.
Document usage of workflow job template nodes with approvals
SUMMARY
I'm not sure if this procedure is mandatory, but it was the only way it worked for me.
It's already documented in one sentence that you can create all workflow template nodes first and then link them later. However in the Examples it has been missing to make it explicit, as it's not really intuitive when you create the nodes and link them at the same time, it won't work and you run into #8777
ISSUE TYPE
Docs Pull Request
AWX VERSION
awx: 19.0.0
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Maximilian Meister <None>
Reviewed-by: Alan Rominger <arominge@redhat.com>
tweak message about image pull option
SUMMARY
I'm not sure if it is OK to change the migration in this way.
This message in the UI didn't make sense to me. If you all prefer the existing message, that is fine...I just think it is weird and doesn't tell you what it does.
If I don't care, I can leave it on --------- in the UI which is "do whatever the default is"
If I do care, its weird to choose an option saying I don't choose and option.
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Jeff Bradberry <None>
Only initialize pendo if the key is available
SUMMARY
Don't bootstrap/init pendo if the key is missing or empty string
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Adding AWX_CONTAINER_GROUP_POD_LAUNCH_TIMEOUT in defaults.py
--- Adding a new global variable AWX_CONTAINER_GROUP_POD_LAUNCH_TIMEOUT in defaults.py
--- Passing this parameter in receptor_params in AWXReceptorJob class
SUMMARY
Related issue : ansible/receptor#315
Corresponding PR on receptor side : ansible/receptor#316
I have added a global variable AWX_CONTAINER_GROUP_POD_LAUNCH_TIMEOUT ( and set default value as 5 minutes ).
This will be passed onto Receptor in case a job is container group task.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.1.0
ADDITIONAL INFORMATION
File Name : awx/main/tasks.py
Class Name : AWXReceptorJob
Function name: receptor_params(self)
Before the change the value of receptor_params before returning was :
{'secret_kube_config': '---\n'
'apiVersion: v1\n'
'clusters:\n'
'- cluster:\n'
' insecure-skip-tls-verify: true\n'
' server: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'contexts:\n'
'- context:\n'
' cluster: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' namespace: default\n'
' user: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'current-context: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'kind: Config\n'
'preferences: {}\n'
'users:\n'
'- name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' user:\n'
' token: '
'sha256~otM0Z2vC8qfzm3eV0tfz-0V4DXKPqLC_Pp1gxXKuC-c\n',
'secret_kube_pod': '---\n'
'apiVersion: v1\n'
'kind: Pod\n'
'metadata:\n'
' labels:\n'
' ansible-awx: 04366fdb-65ec-4efc-bb01-685e9ca7e05b\n'
" ansible-awx-job-id: '21'\n"
' name: automation-job-21\n'
' namespace: default\n'
'spec:\n'
' containers:\n'
' - args:\n'
' - ansible-runner\n'
' - worker\n'
' - --private-data-dir=/runner\n'
' image: quay.io/ansible/awx-ee:0.2.0\n'
' name: worker\n'}
>>>
After the change the value of receptor_params before returning is :
{'pod_pending_timeout': '5m',
'secret_kube_config': '---\n'
'apiVersion: v1\n'
'clusters:\n'
'- cluster:\n'
' insecure-skip-tls-verify: true\n'
' server: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'contexts:\n'
'- context:\n'
' cluster: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' namespace: default\n'
' user: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'current-context: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
'kind: Config\n'
'preferences: {}\n'
'users:\n'
'- name: '
'https://api.tower.ocp4.testing.ansible.com:6443\n'
' user:\n'
' token: '
'sha256~otM0Z2vC8qfzm3eV0tfz-0V4DXKPqLC_Pp1gxXKuC-c\n',
'secret_kube_pod': '---\n'
'apiVersion: v1\n'
'kind: Pod\n'
'metadata:\n'
' labels:\n'
' ansible-awx: 04366fdb-65ec-4efc-bb01-685e9ca7e05b\n'
" ansible-awx-job-id: '21'\n"
' name: automation-job-21\n'
' namespace: default\n'
'spec:\n'
' containers:\n'
' - args:\n'
' - ansible-runner\n'
' - worker\n'
' - --private-data-dir=/runner\n'
' image: quay.io/ansible/awx-ee:0.2.0\n'
' name: worker\n'}
>>>
We can see that an extra parameter has been added : 'pod_pending_timeout': '5m',
####Additional changes :
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRIES from defaults.py as it was not being used anywhere
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRY_DELAY from defaults.py as it was not being used anywhere
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Amol Gautam <amol_gautam25@yahoo.co.in>
Reviewed-by: Seth Foster <None>
--- Passing this parameter in receptor_params in AWXReceptorJob class
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRIES from defaults.py as it was not being used anywhere
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRY_DELAY from defaults.py as it was not being used anywhere
Adding host_metrics for customers who don't have automation analytics
SUMMARY
I am trying to accomplish the following:
Create a new model that stores details about hostname
create corresponding migrations for it
Update the table (main_hostmetrics) after playbook execution
retrieve the data with custom awx-manage command : awx-manage host_metrics --since <date> --until <date>
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
ADDITIONAL INFORMATION
Reviewed-by: Ryan Petrello <ryan@ryanpetrello.com>
Reviewed-by: Amol Gautam <amol_gautam25@yahoo.co.in>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Ladislav Smola <lsmola@redhat.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Use inventory and env private_data_dir subfolders
SUMMARY
This is another part of trying to get AWX to follow ansible-runner best practices.
See docs:
https://ansible-runner.readthedocs.io/en/stable/intro.html#runner-input-directory-hierarchy
So what runner & people expect is:
.
├── env
│ ├── envvars
│ ├── extravars
│ ├── passwords
│ ├── cmdline
│ ├── settings
│ └── ssh_key
├── inventory
│ └── hosts
└── project
├── test.yml
└── roles
└── testrole
├── defaults
├── handlers
├── meta
├── README.md
├── tasks
├── tests
└── vars
What we were producing was something more like
├── cp
├── env
│ ├── cmdline
│ ├── envvars
│ ├── extravars
│ ├── passwords
│ ├── settings
│ └── ssh_key
├── project
│ ├── ansible.cfg
│ ├── ansible_env.yml
│ ├── async_tasks.yml
<snip>
│ └── vault.yml
└── tmp3evnorsu
There was no inventory folder, and inventory was stored in tmp3evnorsu as a top-level file. Any credential files were also saved in that same structure.
With this change, it's more like:
├── cp
├── env
│ ├── cmdline
│ ├── envvars
│ ├── extravars
│ ├── passwords
│ └── settings
├── inventory
│ └── tmp94xjm0d7
└── project
├── ansible.cfg
├── ansible_env.yml
├── async_tasks.yml
<snip>
└── vault.yml
and any credential files (starting with tmp) are dumped into the env folder.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
14.1.0
ADDITIONAL INFORMATION
The motivation for this is that ansible-runner may require specific mounts for every file of directory in the private_data_dir. As such, we really don't want randomized filenames in the top level directory. I don't know for sure this is the direction we are going to go, but this seems like a defensible change on its own, to better adhere to the practices someone familiar with ansible-runner would expect.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Chris Meyers <None>
Updates LingUI Again
SUMMARY
This updates ling ui to bring in their latest release so that we don't have to run npm extract-strings every time we add/update a <Plural> or a string with a variable inside it.
ISSUE TYPE
-dependency upgrade
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
close db and cache connection in new threads
It's not safe to share or re-use django db connections nor django cache connections across new threads nor processes.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Commits settings.DEFAULT_EXECUTION_ENVIRONMENTS updates into the database
SUMMARY
Fixes: ansible/awx-operator#256
cc: @shanemcd
Cannot update execution environment version in place by updating the AWX spec via awx-operator. For example, consider the scenario below:
spec:
tower_ee_images:
- image: registry.tatu.home/ansible/awx-ee:0.1.0
name: My Custom Execution Environment
The expectation would be once you modify the AWX spec to the version below, the EE should reflect this change once the awx-operator finishes its reconciliation.
After:
spec:
tower_ee_images:
- image: registry.tatu.home/ansible/awx-ee:0.2.0
name: My Custom Execution Environment
-- deployment got updated
- args:
- receptor
- --config
- /etc/receptor.conf
image: registry.tatu.home/ansible/awx-ee:0.2.0
imagePullPolicy: IfNotPresent
name: awx-devel-ee-ee
Besides the awx-operator updated the YAML deployment as expected, the change did not hit the database.
awx=# select * from main_executionenvironment;
id | created | modified | description | image | managed_by_tower | created_by_id | credential_id | modified_by_id | organization_id | name | pull
----+---------------+---------------+-------------+----------------+------------------+---------------+---------------+----------------+-----------------+----------------+------
1 | 2021-05-11 15.| 2021-05-11 15.| | registry.tatu..| t | 1 | | 1 | | My Custom Exec.|
|.:24:02.674302.|.:24:02.674327.| |.home/ansible/a.| | | | | |.ution Environm.|
|.+00 |.+00 | |.wx-ee:0.1.0 | | | | | |.ent |
(1 row)
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
devel
ADDITIONAL INFORMATION
Quick proof of concept
>>> settings.DEFAULT_EXECUTION_ENVIRONMENTS
[{'name': 'My Custom Execution Environment', 'image': 'registry.tatu.home/ansible/awx-ee:0.2.0'}]
>>> [a.name for a in ExecutionEnvironment.objects.all()]
['My Custom Execution Environment']
>>> [a.image for a in ExecutionEnvironment.objects.all()]
['registry.tatu.home/ansible/awx-ee:0.2.0']
*** updating to new but keeping name
>>> _, created = ExecutionEnvironment.objects.update_or_create(name='My Custom Execution Environment', defaults={'image': 'registry.tatu.home/ansible/awx-ee:0.3.0', 'managed_by_tower': True})
>>> [a.name for a in ExecutionEnvironment.objects.all()]
['My Custom Execution Environment']
>>> [a.image for a in ExecutionEnvironment.objects.all()]
['registry.tatu.home/ansible/awx-ee:0.3.0']
*** adding a new name
>>> _, created = ExecutionEnvironment.objects.update_or_create(name='My Custom Execution Environment2', defaults={'image': 'registry.tatu.home/ansible/awx-ee:0.3.0', 'managed_by_tower': True})
>>> [a.name for a in ExecutionEnvironment.objects.all()]
['My Custom Execution Environment2', 'My Custom Execution Environment']
>>> [a.image for a in ExecutionEnvironment.objects.all()]
['registry.tatu.home/ansible/awx-ee:0.3.0', 'registry.tatu.home/ansible/awx-ee:0.3.0']
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Add a check for the existence of pre-commit.sh
SUMMARY
so that you don't get stuck when working on branches that don't have this yet.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Have the project update playbook warn if role/collection syncing is disabled
In recent AWX, a galaxy credential (even a blank one for galaxy.ansible.com) is required to sync role/collection content. This is done so that server precedence can be properly set, and so that it can be configured to pull only from a private content host.
This does lead to bug reports where the credentials are not set, and users don't understand why their content is not syncing. This makes that more clear.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Adds support for pendo initialization across the app
SUMMARY
We were already bootstrapping pendo as part of the subscription code I just moved that code to a more general place. When the app container mounts (after login or on refresh) we check to see if the pendo flag is turned on. If it is, we initialize pendo. If it's not then we do nothing. If a user goes into settings and manually changes the pendo tracking setting then we trigger a hard reload of the browser tab (to take the new setting into account and either initialize or not). This functionality existed in the old UI as well.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Michael Abashian <None>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add default error boundary around screens
SUMMARY
Catch any unhandled non-async errors with an error boundary around screens. This will show a generic error message instead
of crashing the page.
The react docs decently summarize the general concept of error boundaries. The example code is informative, but there appears to be a few more edge cases that need to be dealt with. Rather than re-implement solutions for those, I decided to pull in a simple and well-tested library.
ADDITIONAL INFORMATION
before
after
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Update with force option if branch_override
Addresses a reported issue where force-pushing to feature branches did not work.
Connect #9371 (comment)
Reviewed-by: Bill Nottingham <None>
created a TransmitterThread class to start transmit jobs within. Surr…
SUMMARY
Surfaced this bug while fixing issue #9952.
Receptor-ctl would hang trying to read from a socket pair that is never closed, when the transmit job was executed yet threw an exception. This anomaly occurred when using a k8s cluster (Openshift in our case) as the execution environment.
FYI - for now, this only applies to container groups.
We moved the transmit function to execute in its own thread, and rejoin the calling function (_run_internal) when it finishes transmitting, and bubbling up exceptions if any occurred in the thread.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.1.0
ADDITIONAL INFORMATION
Steps to reproduce bug:
Create an ansible project with some files (or roles) and commit to a git repository
Purposely commit a broken symlink on any file in the project files to git
Create a credential set that authenticates to your k8s cluster in AWX
Create a container instance group that leverages the credentials from Step 3 in AWX
Create a project in AWX that leverages the ansible project/git repo from Step 1
Create a job template that leverages the AWX project from Step 4 and set your EE to the container instance group from Step 3
Execute the job from Step 5
Reviewed-by: Shane McDonald <me@shanemcd.com>
converts inventory groups related groups and related hosts to tables
SUMMARY
This addresses #10132
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Remove source_script from tower_inventory_source integration test
SUMMARY
source_script parameter is on longer available in the current version AWX API(#9822), so we need to remove it from the integration test of tower_inventory_source module
Fixes#10104
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
awx_collections
AWX VERSION
devel
ADDITIONAL INFORMATION
None
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Hideki Saito <saito@fgrep.org>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Use development container when updating dependencies
GOAL
When adding, updating, or removing a package, the resultant changeset to the package.json and package-lock.json files that we check into devel should be consistent and reproducible, regardless of who is doing the update or what machine they're doing the update on.
SUMMARY OF CHANGES
Even though we usually run the ui development tooling outside of the awx development container, we should standardize on using this environment to update packages to ensure some consistency.
Use npm ci instead of npm install as the default package installation command, since we generally want to modify package files only when updating deps
You can run into memory limits from docker or node when building in the container, so increase the node memory limit and add a note to the docs that you may need to increase the container memory limit in your docker-for-mac settings (or similar tool, if you use it)
Reviewed-by: Kersom <None>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add an nginx location directive for the favicon
SUMMARY
Add an nginx location directive for the favicon so that the rewrite rule that adds slashes to the ends of requested
urls doesn't get to it.
related #8618
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
UI
AWX VERSION
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Update login message to avoid duplicate brand name
Returned brandName already contain the name brand Ansible. Update to
avoid name brand repetition.
Also update brandName on document title.
See: #9126
Reviewed-by: Kersom <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fixes bug where workflow approvals page would crash if deleted username was referenced
SUMMARY
link #9163
The tooltip on the status labels for completed workflow approvals attempts to show the username of the user that acted on it. If that user has been deleted then the key is not present in the object. This would cause the page to crash.
To fix this, I conditionally show a different string when the username is not available. If a user wants to see which deleted user acted on the approval they'll have to go to the activity stream.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Mat Wilson <mawilson@redhat.com>
Add Variables field validation in launch prompt
SUMMARY
Adds validation for the Variables field during launch prompts.
Prevents unexpected expanding of YAML expressions in the default value (similar to #7506) — this was caused by the default value being parsed into a JavaScript object and then re-translated back to YAML when mounting the "Other Prompts" step
addresses #8907
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
when the DISABLE_LOCAL_AUTH setting is set. This avoids the ugliness
of getting a SuspiciousOperation error for any request/response cycles
that are in flight when a user gets bounced.
Add instance_group to inventory update serializer
SUMMARY
Inventory updates run remotely, in the execution plane. This adds the instance_group field to the inventory update serializer to help clarify this reality.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Remove all AWX_ISOLATED* fields
I believe this should address #10123. The issue mentions a worker-json.js 404 and, while we should fix that at some point, I don't think it's actually related to the save not working.
SUMMARY
I noticed the job settings form wasn't saving on devel.
After some debugging, I found that removing the AWX_ISOLATED_CHECK_INTERVAL and AWX_ISOLATED_LAUNCH_TIMEOUT made the form saveable again.
It doesn't seem like we use any of the AWX_ISOLATED* fields anymore (not to be confused with AWX_ISOLATION* fields, which we do use) so this PR removes them all.
cc @wenottingham @shanemcd @marshmalien @gamuniz @tiagodread @jbradberry
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
* Increase node memory limit
* Add note to docs that you may need to increase container memory thresholds
imposed by docker-for-mac and similar tools if you're using them
Remove the shebang line from pre-commit.sh
SUMMARY
since bash isn't available by default on OS X.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Drop the word 'name' from image field, add examples
for #10170
We call this field "Image" in other places, so we want it to match. Also, "Image name" can be confusing because people may think they just need to provide the image name instead of the full image location, which includes registry and tag version.
cc @gamuniz @nixocio @beeankha @tiagodread
Reviewed-by: Kersom <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Julen Landa Alustiza <None>
Add more fields to .only since they get referenced
SUMMARY
Fixes a performance bottleneck when saving playbook_on_stats for jobs that use --limit against a small number of hosts while it has a large number in the inventory.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
ADDITIONAL INFORMATION
'tis the problem with .only(), that only the developer who wrote the queryset remembers that the other fields cannot be referenced except at the cost of another query
https://github.com/ansible/awx/pull/7352/files
Reviewed-by: Chris Meyers <None>
Reviewed-by: Alan Rominger <arominge@redhat.com>
This avoids writing files to the top level
of the ansible-runner private_data_dir
Inventory is moved to be in the standard "inventory" folder
Credential related files are moved inside of the "env" folder
Also pre-create these folders when preparing for a job run
With this, args is the only top-level file still remaining
Always resolve launching status
SUMMARY
Resolves#10127
Always resolve the launching status after any intermediate async activity that needs to occur while loading prompts, etc.
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Display error for boolean fields
Display error when modifying Allow Override Branch on Projects.
See: #10144
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Kersom <None>
Add slider to adjust capacity_adjustment
Add slider to adjust capacity adjustment.
Also add a new custom hook, useDebounce.
See: #7777
Reviewed-by: Kersom <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Show Error if No Execution Environment is Found on Project Sync or Job Run
SUMMARY
Duplicate of PR #10147 fixing issue #10118
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Adds Job Cancel Button
SUMMARY
This addresses part of #10045 and adds a sync cancel button on the projects list. It also expands the usage of that button to the Project details page, and the Inventory Source list. It does this by introducing a new component called JobCancelButton, that basically takes the work of the job cancel button on the Output toolbar and refactors it slightly to make it useable in these other areas. This button could also be used in the Inventory Source details page once we have websockets hooked up for that view and we can track the status of the sync. (#9013)
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Kersom <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
This field is labeled as just "Image" in other places, so we want
it to match. Also, "name" can be confusing because users will think
they just need to provide the image name instead of the full image
location, which includes registry and tag version.
Make tower_license module idempotent
SUMMARY
Currently, the tower_license module always installs a license, without checking if Tower is already licensed. In general, Ansible modules are supposed to be idempotent. This PR updates the module to check whether Tower is already licensed, and do nothing in that case. A new parameter force is provided so the user can always install the license, or override an existing license with a new one.
ISSUE TYPE
Bughancement
COMPONENT NAME
awx_collection
AWX VERSION
awx: 18.0.0
(but really Tower 3.8.2)
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Graham Mainwaring <graham@mhn.org>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Allow modify scm branch override
Source Control Branch was not being displayed as part of the
JobTemplate Edit, since the project did not have the variable
allow_override as part of the summary_fields.
Add source control details for JobDetail and WorkflowJobTemplateDetail
See: #8788
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Mat Wilson <mawilson@redhat.com>
Break out the pre-commit hook into an explicit script
SUMMARY
This means that
we don't have to be always updating the underlying .git/hooks/pre-commit file
updates to the logic will just work automatically
the logic of the conditional invocation of black has been fixed so that AWX_IGNORE_BLACK=1 should work correctly now
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jeff Bradberry <None>
This means that
- we don't have to be always updating the underlying .git/hooks/pre-commit file
- updates to the logic will just work automatically
- the logic of the black logic has been fixed so that AWX_IGNORE_BLACK=1 should work right now
Data migration to remove the isolated instances from the database
SUMMARY
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Elyézer Rezende <None>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Display Edit/Delete on EE details page as per last API changes
Display Edit/Delete buttons on details page for EE managed by tower.
See: #10078
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
This refactors the cancel button on the job output page so that it can be used on the Project List page,
the Project detail page, and the Inventory Source list page. Once websockets are ready for the Inventory Source details page
and we can track the status of the source we can use this button there too.
minikube instructions adjustments from debugging
SUMMARY
The awx-operator examples expect the tower_image_version as a separate var, and can error without that.
Also, there's still an issue getting going with ansible/deploy-operator.yml's Deploy Operator task. I'm still looking into that, the associated hack might change.
ISSUE TYPE
Docs Pull Request
COMPONENT NAME
API
AWX VERSION
19.1.0
Reviewed-by: Shane McDonald <me@shanemcd.com>
Fix typos
Fix typos
I had to run npm run extract-strings since the typos were present on the Plural component.
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Avoid prop reference error when recreating survey
SUMMARY
For #9370
The survey object is undefined when recreating a survey after deleting it. Add optional chaining on survey fields to avoid prop reference error.
Reviewed-by: Kersom <None>
`console.warning` is not exists on browser
SUMMARY
fix typo error for console.warn
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
19.1.0
ADDITIONAL INFORMATION
console.warning is not exists function in browser envirment.
Reviewed-by: Kersom <None>
* Source Control Branch was not being displayed as part of the
JobTemplate Edit, since the project did not have the variable
`allow_override` as part of the summary_fields.
* Add source control details for JobDetail and WorkflowJobTemplateDetail
See: https://github.com/ansible/awx/issues/8788
Converts Teams Roles tab to tables
SUMMARY
Addresses #10088. Converts The Roles tab inside of Teams to tables view.
E2E workflows triggered
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Redesign survey multiple choice
SUMMARY
Addresses #6464.
This new design improves UI and reduces the risk to having mismatching choices and default values.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Marliana Lara <marliana.lara@gmail.com>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Sarah Akus <sarah.akus@gmail.com>
add custom_venv for backward compatibility
SUMMARY
As per #10055 , this is the proposal to add back custom virtual environments to the modules. With the understanding that they will not be tested. But to allow users of Ansible Tower 3.8 the ability to use newer features and bugfixes in awx.awx as applicable, without restorting to backporting these removals. Would expect these to remain for a time after Tower moves to Execution environments as teams/companies adopt the next version of tower.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
awx_collection
AWX VERSION
19.1.0
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
When we go to submit the form to the api we format it again in a way
the api will recognize. Allowing formik to manage updating, the choices
and the default values this way cleans up the code and removes a bunch of unnecessary
splitting and joining of the choices an default choices strings
Adds bold to some list items
SUMMARY
Some list items were not bolded during conversion to tables mistakenly
kicked off E2E tests
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Fixes for a couple of credential form bugs
SUMMARY
link #9635
This addresses the first and second bullet points in the linked issue. The third is actually covered by a separate PR.
The ace editor fix was to simply wrap it in a div and apply the previous styling to that div instead of the editor.
The secret key/details page crashing was caused by a race condition when would redirect back to the details view after editing. The details view was rendering with some stale data and a key that was expected to be there was not. To fix this I wrapped the request to fetch the credential detail in a useRequest hook and show the loading spinner while this request is outstanding. Only after the request resolves do we try to render the details view so that it always has the up to date object.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
add testability to DataListToolbar.jsx
add testability to AssociateModal.jsx HostGroupsList.jsx
fix lint
add ouiaId to InventoryGroupHostList.jsx
add ouiaId to ExecutionEnvironmentList.jsx
add ouiaId to InstanceGroupList.jsx
add ouiaId to InstanceGroupDetails.jsx
add ouiaId to ScheduleList.jsx and ContainerGroupDetails.jsx
Docker-compose requires the options to be passed before up
SUMMARY
The $(COMPOSE_UP_OPTS) in the docker-compose make target need to come before the up command or else they are not parsed correctly.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
19.0.0
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Properly adds survey default values on Schedules
SUMMARY
This addresses #9977. This also fixes a bug where the extra_data was not rendering properly in schedule details
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Make date consistent on tooltip display
Make date consistent on tooltip to display info about job.
There are 3 places that define similar function on the code basis. Make
them consistent in how to display date.
Search for generateLastJobTooltip on code basis.
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Make date consistent on tooltip to display info about job.
There are 3 places that define similar function on the code basis. Make
them consistent in how to display date.
Fix tower_job_launch tags being passed to API
SUMMARY
#10008
Fix tower_job_launch tags being passed to API
Previously the wrong field was being used on the api to pass tags, also while the module accepts lists, the API does not, so it would error. Take a users list, and converts to comma separated string, in order to maintain status quo.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
awx_collection
AWX VERSION
19.1.0
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
add org search to labels
SUMMARY
After having an issue with labels I saw that orgs were referenced, Remembering #7567 implemented the code to do the search for orgs so you can search for labels within an org, Tests include creating the same named label in two orgs, and the search not erroring out that it found two labels of that name.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
awx_collection
AWX VERSION
19.0.0
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
add notification error to notification template list view
In support of #8853
Updates /api/v2/notification_templates to include the error field for summary_fields -> recent_notifications
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Fix CSP error on Safari
This was causing:
[Error] Refused to execute a script because its hash, its nonce, or
'unsafe-inline' does not appear in the script-src directive of the Content
Security Policy. (migrations_notran, line 16)
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
This was causing:
```
[Error] Refused to execute a script because its hash, its nonce, or
'unsafe-inline' does not appear in the script-src directive of the Content
Security Policy. (migrations_notran, line 16)
```
Convert user sub-lists to tables
SUMMARY
Converts User Organizations, Teams, and Roles lists to tables
Addresses #6189
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fix projects to use the utility function get_default_execution_environment
SUMMARY
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Shane McDonald <me@shanemcd.com>
Make the check_migrations command a direct alias of makemigration
SUMMARY
Make the check_migrations command a direct alias of makemigration.
It already more or less was, but it previously did some machinations
around copying and modifying the database settings. These on-the-fly
changes to the database connection no longer work after the
rearrangement of settings files, so let's just get rid of it.
Example, from running make test:
py3: commands succeeded
congratulations :)
awx-manage check_migrations --dry-run --check -n 'missing_migration_file'
Traceback (most recent call last):
File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/django/db/backends/base/base.py", line 217, in ensure_connection
self.connect()
File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/django/db/backends/base/base.py", line 195, in connect
self.connection = self.get_new_connection(conn_params)
File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/django/db/backends/sqlite3/base.py", line 194, in get_new_connection
conn = Database.connect(**conn_params)
sqlite3.OperationalError: unable to open database file
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.1.0
Reviewed-by: Shane McDonald <me@shanemcd.com>
Removes remaining I18n props, HOCs and misc objects
SUMMARY
This Removes withI18n, the i18n object and all the places that it is passed around as a prop, or argument.
E2E Tests have been triggered. There should be no functional or visual impact. E2E results
ISSUE TYPE
-Dependency Upgrade
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Revert the code that prevents sysadmins from changing managed EEs
SUMMARY
related #10078
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.1.0
Reviewed-by: Shane McDonald <me@shanemcd.com>
It already more or less was, but it previously did some machinations
around copying and modifying the database settings. These on-the-fly
changes to the database connection no longer work after the
rearrangement of settings files, so let's just get rid of it.
Fix unit-test styling warning
Fix unit-test styling warning.
Warning: Unsupported style property white-space. Did you mean
whiteSpace?
Style still valid after change:
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
project: Add last job status as for project sync feedback feature.
SUMMARY
Fixes#9872
Add last job status as for the project sync feedback feature. When users clicked the sync button on the project page, last job status will eventually update the status to Pending, Waiting, Running, and final result (Successful, Cancelled, Failed, Error). The implementation requires the WebSocket connection to ensure synchronous status update.
In particular, the last job status is similar to the functionality of status in the project list (status in ProjectListItem.jsx). More specifically, the last job status accompanied by a job link that allowing the user to navigate to the job output page. Besides, there is also the tooltip that allowing the user to view the information related to the most recent sync, covering information like JOB ID, STATUS, FINISHED.
The rationale of having the last job status instead of redirection or toast notification is that:
User has choices on whether to redirect to the job output. If the user wishes to navigate to job output, he/she can click the link. Besides, the user might have other projects to be synced right after the current project and he/she may want to proceed back to the project list page instead of the job output page. If we implement force redirection, it would take a longer time to navigate to the project list page.
The status update on last job status is fundamentally similar to toast notification where the user can immediately be notified if he/she already clicked the sync button to launch the job.
Nevertheless, this PR requires further discussion. Any comments are welcomed!
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
awx/ui_next/src/screens/Project/Project.jsx
awx/ui_next/src/screens/Project/useWsProject.js - Added websocket implementation
awx/ui_next/src/screens/Project/ProjectDetail/ProjectDetail.jsx
AWX VERSION
awx: 19.0.0
ADDITIONAL INFORMATION
In case if users spam the sync button, we will need to ensure the fluent UI on the most recent sync tooltip and last job status. Thus, we would not want to update our last job status to Pending if there is a current running job.
For instance, we clicked sync for a particular project twice.
For the first sync, our last job status should immediately change to Pending, then Waiting, then Running, then result (which are Successful, Failed, Error, Cancelled).
For the second sync, if we have a running job, we should not update our UI to Pending, otherwise our most recent sync tooltip UI will lose our current running job and we cannot navigate to the job link through the link provided by last job status tooltip.
Issue of sync button click spam
Ideally, we should prevent any spamming on the sync button using backend logic to reduce overload on the server as we already have a similar running project. Together with backend logic, we can disable the sync button right after we start to sync a project.
However, if we only disable sync through the frontend, this seems insecure as people with bad intentions might able to change the button disable attribute.
After
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Sei Wai Lai <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Unit test warning cleanup
SUMMARY
These commits target the warnings that look like:
Can't perform a React state update on an unmounted component
The underlying problem here is that we have network requests that are being made by components that are subsequently being unmounted. When the network request resolves, we attempt to update some state but the component is no longer mounted and the warning is triggered. To address this I consolidated a lot of isMounted code into a single hook which can be used across the app to check to see whether the component in question is still mounted before attempting to update state inside of a useEffect. This primarily applies to network requests.
I think this points to a larger issue which is that we sometimes mount components prematurely. For example, when the job template edit component is mounted we actually mount:
JobTemplateForm (briefly)
ContentLoading
JobTemplateForm
Network requests triggered by the first mount of JobTemplateForm are suscepitble to attempting to update state on an unmounted component. I believe this pattern exists in many places across the app but I haven't tried to solve this in this PR.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Marliana Lara <marliana.lara@gmail.com>
Reviewed-by: Kersom <None>
1. Activate web socket once get into project detail page to ensure job status update synchronization.\n 2. Show last job status if there is no current job.\n 3. Show current job status if there is any current pending, waiting or running job.
Assorted renaming and string changes
SUMMARY
Goal was to hit user-facing strings and docs.
Didn't try and catch all comments, although did a few when in those files.
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
A couple EE bug fixes
See commit messages for more info
Reviewed-by: Seth Foster <None>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Adds an execution environment step to the ad hoc commands
SUMMARY
This addresses some of #9864 by adding a step to select an execution environment to the ad hoc commands wizard
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Isolated removal
SUMMARY
Removal of the isolated nodes feature.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Elyézer Rezende <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Remove inventory-related docs which are outdated or wrong
SUMMARY
Around 10% of this is updating docs for #9822
The rest of it is updating for changes that took place long ago, but docs still has stuff which is now wrong.
ISSUE TYPE
Docs Pull Request
COMPONENT NAME
API
Reviewed-by: Chris Meyers <None>
Make Workflow Surveys Idempotent
SUMMARY
Update the Workflow survey check to be in line with job template check.
Resolving issue #7554
Found that it was returning json/status_code, and needed to reference json output.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
awx_collection
AWX VERSION
19.0.0
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
add ouiaId to toolbar delete button
In working on fixing a broken e2e test, I discovered an element that needed a better locator in order to allow testability.
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Improve error handling / display when Ansible Runner errors
Before you would see unhelpful error messages like:
Traceback (most recent call last):
File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 1397, in run
res = receptor_job.run()
File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 2957, in run
return self._run_internal(receptor_ctl)
File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 3008, in _run_internal
raise RuntimeError(detail)
RuntimeError: exit status 0
Now you will see the underlying error:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/ansible_runner/streaming.py", line 108, in run
utils.unstream_dir(zip_data, self.private_data_dir)
File "/usr/local/lib/python3.8/site-packages/ansible_runner/utils.py", line 104, in unstream_dir
data = base64.b85decode(data)
File "/usr/lib64/python3.8/base64.py", line 463, in b85decode
raise ValueError('bad base85 character at position %d'
ValueError: bad base85 character at position 121
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Fix project update
SUMMARY
Fixing bug found in #8686
Found that the on change was not triggering due to no actual changes in the Post, Set so when project update is set to true, that the wait for update is Always triggered. Also added logic from project_update to set changed status depending on if refspec changes.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
awx_collection
AWX VERSION
19.0.0
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: John Westcott IV <None>
Reviewed-by: Sean Sullivan <None>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Hide teams option when adding access role to a team
SUMMARY
link #8835
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Log adjustments made to the analytics interval due to the 4-week limit
SUMMARY
If the user passes in the since and/or until parameters, sometimes they will be truncated to limit the data to only 4 weeks. Emit a log message when this happens.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Jim Ladd <None>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Jeff Bradberry <None>
Add docs for building and using custom awx image
SUMMARY
Docs for how to build and push a custom AWX image to be used by the awx-operator.
ISSUE TYPE
Docs Pull Request
AWX VERSION
19.0.0
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Modify formatting in response to black update
This seems to have changed sometime between versions 20.8b1 and 21.4b0
Reviewed-by: Seth Foster <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Add workflow approval and node wait modules
SUMMARY
Please see #9878 this is a clean PR after redoing my fork.
Add a module to find a workflow approval node and approve or deny it, based on Issue #8013.
Add a module to wait for a specific workflow node to complete and return information on it.
Both of these are based on tests I have been creating for testing workflows.
Scenario
Launch workflow
Wait for A node in the workflow to finish, compare output to expected output.
If it matches, approve the approval node, otherwise deny the approval node.
Workflow completes.
Even used in concert I've added the wait feature to both of these so a user can wait on either to appear.
This does require a workflow to use unique names on the job nodes they are waiting on, As the job # is created on the fly, it would be difficult for user to specify, A future update could explore searching for a specific identifier among a workflow template and then finding that job created by that identifier.
Currently without the modules this depends on generous use of the uri module, with until and retry coupled together.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
awx-collection
AWX VERSION
19.0.0
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Tower workflow schema
SUMMARY
See #9309 This is a clean PR of that, after an errant rebase
Adds a way to add entire workflow node schemas to workflows. Either through the workflow schema module or the workflow job template module.
This speeds up workflow creation vs the workflow node module by 3x.
The model for the schemas is the format used by the tower_export module.
The main difference between this and the workflow node module is that the loops are done in python. Traditionally if you have a workflow with 10 nodes, ansible tasks need to be invoked 19 times. 1x to create the workflow, 10 x to initially create the nodes, and then one time for each node that is not an endpoint in the schema. This removes the need to loop and invoke many times.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
awx-collection
AWX VERSION
17.0.1
Reviewed-by: John Westcott IV <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Properly tags Plural-able strings for translation, and removes unnecessary i18n
SUMMARY
This resolves#9891.
It also begings the process of remove i18n._(t string) in places in favor of this syntax
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Kersom <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
[devel] add clarifying comment re: AUTOMATION_ANALYTICS_LAST_GATHER being set regardless
(the comment in the changeset pretty much says it all)
Reviewed-by: Chris Meyers <None>
Fixes bug deleting the last workflow survey question
SUMMARY
link #7670
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Stop installing kubectl/oc
Realized this when looking at #9428
Reviewed-by: Chris Meyers <None>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Removes Snapshot tests
SUMMARY
Removes snapshot tests
ISSUE TYPE
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Michael Abashian <None>
Fixes SSO Redirect
SUMMARY
This fixes#9115 by simply checking if there is a redirect url in and then replacing it with the existing url in history, navigating the user to the correct login url.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Add bindep.txt file for execution environments
This will be used by ansible-builder, for people creating EEs.
Signed-off-by: Paul Belanger pabelanger@redhat.com
Reviewed-by: Shane McDonald <me@shanemcd.com>
Wait for the database migrations before starting
cc: @shanemcd @Spredzy
SUMMARY
Before starting the tower-task container, we should wait for the database schema migrations to complete to avoid misleading users.
This is much more evident on newer installations as the tower-task container will be noisy and populated with SQL errors which is a false positive due to the database being populated.
See the ansible/awx-operator#238 for more information
Fixes: ansible/awx-operator#238
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
devel
Reviewed-by: Shane McDonald <me@shanemcd.com>
Move to fileglob loop for yml requirement locations
SUMMARY
Overdue followup on #8312
I knew there was more research needed, but didn't know what form it would take until I got my hands dirty.
This entails a look and feel change.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
ADDITIONAL INFORMATION
It has a warning... but I don't dislike it.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jake Jackson <jljacks93@gmail.com>
Fixes bug where users could not delete a single inventory source
SUMMARY
link #9852
Also fixes a bug that I came across with deletion warnings. The deletion warning was showing a count for workflow nodes that referenced any inventory source with the same parent inventory. For example:
Create an inventory
Create two inventory sources invsrc1 and invsrc2
Create a workflow with a node that syncs invsrc1
Attempt to delete invsrc2
The warning will indicate that there's 1 workflow node that uses the inventory source but that's actually not true. There should be no deletion warning in this case.
This PR addresses ^^
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fixes object rendered on dom in aria-label
SUMMARY
This addresses #9910. Aria-labels that use <Plural/> won't work properly. In this case I used aria-labelledby and passed it the id of the button. I tested it with a screen reader and that fixed it. I also fixed some JobListCancelButton tooltips
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Remove custom inventory scripts from the API
SUMMARY
Connect #7775
AWX VERSION
18.0.0
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Bump migration number
Skip data migration for fresh migrations
where ContentType for custom inventory scripts
has not yet been created
no scripts will exist in this case, so no-op
Add migration to carry out corresponding schema change
Add data migration to delete inventory sources
which are the custom type
Split migration into two files
data migration touches same rows as schema migration
Fix problematic dependencies
SUMMARY
Fixes high-severity warnings for problematic dependencies.
These fixes required updating react-scripts to a later version, which broke many of our tests due to breaking changes in how test setup works. As such, this PR also updates a lot of tests.
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Shane McDonald <me@shanemcd.com>
- This file shouldnt need the executable bit
- Should have been setting permissions before writing any data
- No need to close the file since we're using open w/ a context manager
Initialize variables field/detail with formatted JSON strings
SUMMARY
When a variables detail or variables field are mounted with JSON code, this ensures the JSON is formatted with friendly whitespace, regardless how it was originally formatted when saved.
addresses #3167
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
I18n devel translations
SUMMARY
First round of translations for ui_next. I have also included:
necessary changes to import fr, nl, es, zh, and also add them to the catalog.
fixed a syntax error that was keeping strings from compiling.
ISSUE TYPE
Feature
COMPONENT NAME
Translations
AWX VERSION
devel
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Add expanded row content to project list
SUMMARY
#5070
Add the following details to expanded area:
Description
Organization
Execution Environment
Last modified
Last used
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Return distinct labels for normal Users
SUMMARY
Create a single label and add it to 2 or more JTs
Visit /api/v2/labels as superuser and you'll see a single label
Create a normal user and give admin role to each JT in step 1
Visit /api/v2/labels as this normal user and you will see duplicate entries for the label. Number of entries will be equal to the number of JTs in step 1
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.0.0
Reviewed-by: Seth Foster <None>
Reviewed-by: Jeff Bradberry <None>
Add in the missing dry-run check for csv analytics collectors
SUMMARY
Follow on bug fix for the analytics gathering feature. @chrismeyersfsu noticed that dry-run gathers of sufficiently large data was breaking out of the loop after the first csv chunk was packaged.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.0.0
Reviewed-by: Chris Meyers <None>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Julen Landa Alustiza <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
metadata: Fixes invalid model classes field.
Fixes#9441.
SUMMARY
Set type' field's filterable as True only if the model classes contain type field such as UnifiedJob, WorkflowApproval, UnifiedJobTemplate, Project and SystemJobTemplate
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API - metadata.py
AWX VERSION
awx: 19.0.0
ADDITIONAL INFORMATION
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Sei Wai Lai <None>
CredentialPluginField: Fixes bug to display invalid helper text for empty required field.
Fixes#9911. Include the HTML element of displaying helper text which is similar to FormGroup.tsx's inValidHelperText implementation
Signed-off-by: seiwailai laiseiwai@gmail.com
SUMMARY
Under normal circumstances, we initiate input fields using FormGroup element. However, for credentials purpose, we customized the FormGroup by adding another children element called CredentialPluginInput which comprised of Input Group. Thus, events related to the input fields will happen within the InputGroup logic. However, the Input Group doesn't have the functionality of rendering error at the moment.
Thus, we should explicitly render the error under FormGroup element if there is an error and needed to be displayed.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI: CredentialPluginField.jsx
AWX VERSION
awx=19.0.0
ADDITIONAL INFORMATION
After
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Prevent double-clicking/double-launching jobs
SUMMARY
Prevents double-launching a job if the user double-clicks the launch icon. This is done by disabling the button upon first launch. Applied to all instances of <LaunchButton>.
Addresses: #4249
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Updating installation documentation
Updating installation documentation due to bug. Adding this will prevent users to encounter errrors at the moment of installing awx.
SUMMARY
New users will encounter problems installing awx, as there's a bug using a specific version of minikube. Adding a note, so users are aware of this and don't struggle in the installation process.
ISSUE TYPE
Docs Pull Request
AWX VERSION
latest
ADDITIONAL INFORMATION
https://github.com/ansible/awx-operator/issues/205
Reviewed-by: Shane McDonald <me@shanemcd.com>
Fixes#9911. Include the jsx element of displaying helper text which is similar to FormGroup.tsx's inValidHelperText implementation
Signed-off-by: seiwailai <laiseiwai@gmail.com>
Define variables in the correct Ansible role
SUMMARY
This pull request is related to #9913.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
Docker
AWX VERSION
awx: 19.0.0
ADDITIONAL INFORMATION
This pull request ensures that tini is downloaded for the correct architecture.
$ grep tini Dockerfil
# Install tini
RUN curl -L -o /usr/bin/tini https://github.com/krallin/tini/releases/download/v0.19.0/tini-arm64 && \
chmod +x /usr/bin/tini
Reviewed-by: Shane McDonald <me@shanemcd.com>
Observability Metrics
SUMMARY
This adds the chart for Observability Metrics (#9019). To see the chart you need to navigate to /metrics. Also, its best if you run a build that has multiple instances. This will do that for you COMPOSE_TAG=devel CLUSTER_NODE_COUNT=2 make docker-compose.
When this feature loads the user has to select an instance (1, or all) and a metric to render data on the graph. Once they select those items, the chart appears and we start to make requests to the api every 3 seconds to get the data. (Currently the api does not support web sockets for this feature) If the user changes the values for either of the drop down items the chart resets. The chart also only show the last 50 data points.
There is a "tooltip" that is rendered at the bottom left hand side. I decided to put it there, instead of on the chart itself because this chart could get quite crowded depending the number of data points rendered and the number of instances rendering lines.
The X axis is sort of meaningless. The values below simply render the number of api requests. This isn't a value of time. Since the main goal of this feature is to show significant changes instead of tryin to pinpoint when the change occurs I felt that showing a time stamp on this axis would crowd the axis as well.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Mat Wilson <mawilson@redhat.com>
Always overwrite the contents of .git/hooks/pre-commit
SUMMARY
Set the make command to always write into the pre-commit hook file, even if it already exists. This will allow this file to be updated when changes are made without the developer having to jump through hoops or remember that it is a thing.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Ryan Petrello <None>
Accessibility fixes
SUMMARY
Fixes numerous accessibility issues, including:
updates CodeEditor so label correctly points at associated textarea
fixes issues with tabs on dashboard and details pages
adds missings ids
adds alt text to logo
removes duplicate ids on some lists
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Keith Grant <keithjgrant@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Don't unnecessarily expand YAML expressions
SUMMARY
Prevents variables fields from expanding YAML expressions when possible:
In the detail view, the user may toggle to JSON (seeing the data structure fully expanded), but toggling back to YAML will continue to display the original un-expanded value with expressions intact
In edit mode, this works the same way, UNLESS the user edits the value while in JSON mode.
Addresses #7506
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Chris Meyers <None>
Adding the scm_track_submodules option for project
SUMMARY
Adding the scm_track_submodules option which should fix the related #7846
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
UI
AWX VERSION
awx: 17.0.1
ADDITIONAL INFORMATION
This option will add the track_submodules option which is described in the ansible git module: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/git_module.html
if yes, submodules will track the latest commit on their master branch (or other branch specified in .gitmodules). If no, submodules will be kept at the revision specified by the main project. This is equivalent to specifying the --remote flag to git submodule update.
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: None <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
If the user toggles a VariablesField to JSON then back to YAML without
editing the field content, revert to the initial YAML value to maintain
any shorthand expressions
Pull with credentials from protected registries
SUMMARY
relates to #7066
if a credential is associated with an EE this will create a JSON authfile that is then passed with the pull request to the host of the registry
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
TODOs
Remove separate token field from the registry credential.
Rename the existing password field to say "password/token"
Ensure only registry credentials can be associated with an EE #9628
Write out the auth.json file to the pdd_wrapper_ directory. #9683 (comment)
Use secure permissions for auth.json #9683 (comment)
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Nana <natr@hey.com>
allow schedules to be disabled even if the associated UJT isn't valid
see: #8641
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Jeff Bradberry <None>
wip
tooltip renders with colors and disappears
scales y axis properly
adds legend without buttlets
adds legend data but needs styling
adds legend, and cleans up code
show help text
Prevent execution environment from being assigned to a new organization
SUMMARY
related #9769
ee organization can be changed to null (less restrictive)
if organization is null, cannot be assigned to org (more restrictive)
if org is assigned, it cannot be set to a different org
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.0.0
Reviewed-by: Kersom <None>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Seth Foster <None>
- ee organization can be changed to null (less restrictive)
- if organization is null, cannot be assigned to org (more restrictive)
- if org is assigned, it cannot be set to a different org
Updates Ling UI
SUMMARY
This PR updates Ling ui. One of the reasons for updating this dependency was that they were deprecating withI18n(). They changed their minds on that so we didn't need to urgently remove all those HOCs. Thus, we can now make that conversion a bit slower a couple of files at a time.
One other thing: When we are changing the string based on a count of an item (ie. Cancel Job vs. Cancel Jobs) we should use ling ui's <Plural> component. However, in order to show the update strings passed to that component the developer will have to run npm run extract-strings each time they are changed to render the updated strings properly. More info here.
ISSUE TYPE
-dependency upgrade
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Sergio Moreno <sergiomorenoalbert@gmail.com>
sprinkle back in some flake8 to catch missing imports
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Ryan Petrello <None>
Change Reference of 'ansible-tower-service' to 'automation-controller-service' in Docs File
Related to the work done in #9720
Reviewed-by: Shane McDonald <me@shanemcd.com>
Fix elapsed time on job showing incorrect value
SUMMARY
Elapsed time would always stay at zero
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
Reviewed-by: Seth Foster <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Add missing template list expanded section details
SUMMARY
Issue: #7662
Add the following detail items:
Description
Organization and link to organization details
Credentials
Move Credential and Label details to the bottom of the expanded section and make them stretch the entire width of the row.
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add VariablesField YAML/JSON validation
SUMMARY
Validates YAML or JSON syntax in the variables field on blur. Prevents formik from submitting the form while this error is present.
addresses #8906, #8907
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Fix non-translatable template string
SUMMARY
The template string for the trademark doesn't seem like it can be used for a valid translation key. This caused the raw variable to be displayed for production builds instead of a readable string. see: #9296 (comment)
ADDITIONAL INFORMATION
Screenshots, after applying this patch:
development environment
production build
Reviewed-by: Kersom <None>
Use validated prop in cred plugin field
SUMMARY
Fxes an initialization bug for cred plugin fields, introduced by this commit
This pr applies the patch described below:
cc @marshmalien @unlikelyzero
before
after
Reviewed-by: John Hill <johill@redhat.com>
Update subscription docs link to use single source of truth
SUMMARY
Related: #8428
Update subscription edit form documentation links to use the docs link utility.
Fix broken unit tests
COMPONENT NAME
UI
Reviewed-by: Keith Grant <None>
Update version and changelog for version 19
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Subscription wizard
SUMMARY
Adds subscriptions routes
Extends Config context to return an array with two items: config state and config setter
Update components that use Config with the new return pattern
Move mock config into setupTests.js
Return only the routes the user is "authorized" (valid license key) to view
Subscription Details view: /settings/subscription/details
This is our standard details view
Clicking Edit will send the user to subscription wizard view /settings/subscription/edit
Route is not accessible when license type is OPEN
Subscription Add wizard view: /settings/subscription_management
Step 1 - Subscription:
If a user does not have a Red Hat Ansible Automation Platform subscription, they can request a trial subscription via the link
Toggle between uploading a subscription manifest .zip file or retrieving subscriptions using Red Hat credentials (username and password)
Get Subscriptions button fetches subscriptions and displays them in a modal
Step 2 - Tracking and analytics:
Shows two checkboxes to enable User analytics and Automation analytics
If the user has previously selected the RH subscription manifest flow, checking the Automation Analytics box will display required RH username and password fields
If the user has previously selected the RH username/password flow, they will not see this additional username/password field if Automation Analytics is checked
Step 3 - EULA: https://tower-mockups.testing.ansible.com/patternfly/settings/settings-license-step-03/
Submission should show a success message and navigate user to dashboard if this is the initial launch and to the subscription detail view if they are editing the subscription
Failed submission should show a wizard form error
ISSUE TYPE
Feature
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Michael Abashian <None>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Adjust datetimes to be aware when using awx-manage gather_analytics
SUMMARY
Adjust datetimes to be aware when using awx-manage gather_analytics
Also, make sure that an explicit since parameter will win over
default until=now() when calculating the 4-week data limit.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Ryan Petrello <None>
Fix up request tower ps script
SUMMARY
Fixes the request_tower_configuration.ps1 script with the following:
Added PowerShell 6+ support, can now run on Linux
Removes the uneeded strict and ErrorActionPreference settings which could impact the global settings
Enables TLS 1.2 support on Windows PowerShell
Fixes -insecure handling for certificates
Works across ps versions
Original didn't actually work (at least in my testing)
Hopefully doesn't impact the global state
Fixes IE not loaded error on Windows PowerShell when running on a new profile
Changes the encoding to UTF-8 (no BOM) and made it executable
Can now be invoked on Linux like ./request_tower_configuration.ps1 ...
There are no non-ASCII chars in the script so this won't matter on Windows
Fixes#9434
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
Tools
AWX VERSION
N/A
Reviewed-by: Ryan Petrello <None>
Do not allow user to attempt to delete a running job
Do not allow user to attempt to delete a running job
See: #9187
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Kersom <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
instance groups options for modules
SUMMARY
Add ability to set instance groups in organization, job templates, and inventories.
#9788
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
awx-collection
AWX VERSION
18.0.0
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Create single source of truth for outgoing Docs link URLs
SUMMARY
Adds a utility function to generate links to the correct version of documentation and updates/adds a number of docs links throughout the app.
Addresses #8428
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Configure ace editor to not use blob worker
SUMMARY
Intended to address #9817
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Ryan Petrello <None>
Gather job event analytics by last pk instead of created datetime
SUMMARY
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 17.0.1
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Ladislav Smola <lsmola@redhat.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Jim Ladd <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Update capitalization for a few screens
Update capitalization for EE and Instance Groups screens to better align
with mock ups.
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
- `since` should not be after `until`
- neither `since` nor `until` should be in the future
- `since`, `AUTOMATION_ANALYTICS_LAST_GATHER`, and `AUTOMATION_ANALYTICS_LAST_ENTRIES[key]` should be truncated to 4 weeks prior to `until`
- an explicit `since` parameter should always take precedence over the settings values
- stop trying to ship csv slices when one breaks
- only update LAST_ENTRIES if all of the files in a time/pk slice succeed
- don't allow an explicit --until parameter to set the GATHER/ENTRIES values backwards
Previously, invoking the command with neither of the --ship or
--dry-run flags would result in effectively doing a dry run. With the
stricter checks now in place in analytics.core.gather, let's make sure
that we pass the 'dry-run' parameter in to gather() in the no-flags
case.
With the change to use pk-based interval slicing for the job events
table, we need analytics.gather to be the code that manages all of the
"expensive" collector slicing. While we are at it, let's ship each
chunked tarball file as we produce it.
Remove invocation of Ansible from launch script
This was missed in the initial EE PR. Ansible is no longer installed inside of the web & task containers, causing this to show up in the container logs:
/usr/bin/launch_awx.sh: line 18: ansible: command not found
/usr/bin/launch_awx.sh: line 19: ansible: command not found
Reviewed-by: Ryan Petrello <None>
Fix docker-compose up options
SUMMARY
Should the up options be after the up command? I'm assuming COMPOSE_UP_OPTS exists so we can run detached like so:
COMPOSE_UP_OPTS=-d make docker-compose
If I am misunderstanding the purpose of COMPOSE_UP_OPTS, is there other guidance on how to autostart AWX detached?
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
Installer
AWX VERSION
awx: 18.0.0
Reviewed-by: Shane McDonald <me@shanemcd.com>
Should the `up` options be after the `up` command? I'm assuming `COMPOSE_UP_OPTS` exists so we can run detached like so:
`COMPOSE_UP_OPTS=-d make docker-compose`
If I am misunderstanding the purpose of `COMPOSE_UP_OPTS`, is there other guidance on how to autostart AWX detached?
This was missed in the initial EE PR. Ansible is no longer installed inside of
the web & task containers, causing this to show up in the container logs:
```
/usr/bin/launch_awx.sh: line 18: ansible: command not found
/usr/bin/launch_awx.sh: line 19: ansible: command not found
```
Make logs more readable in development environment
This uses https://github.com/coderanger/supervisor-stdout to prefix process
names before log messages in the dev env
Before:
After:
Reviewed-by: Jim Ladd <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
workflow convergence
Satisfies #7669
This adds the ability to set convergence from any nodes (default) to all nodes to the workflow feature. Specifically:
There is an additional convergence dropdown located on the bottom of the first "node type" step for all node types:
This field defaults to "Any" on add node and whatever the api has the field set to on edit node. It resets to any if you change the node type dropdown (even on edit when changing and then changing back to the original type...I can update that point depending on what UX is preferred).
tvo created a new link explicitly to the explanation in documentation of what the convergence setting does here, and I link to it in the help popover shown in the screenshot below.
Consistent with the old UI, When "All" is selected, a small tab is displayed with the label "ALL" in the node visualizer. "Any" nodes do not get any sort of tab.
A slight tweak compared to the old ui...I set the tab's border and background to be the same color as the border of the node to create a consistent look for the node across various states and confirmed this behavior was good with @trahman73
OLD:
NEW:
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: John Hill <johill@redhat.com>
Add wheel in venv creation
The generated based venv from python3.8 -m venv vs. virtualenv -p python38 is different. This changes aims to address the differences.
It was introduced as part of the Python 3.8 migration.
#8778
Reviewed-by: Shane McDonald <me@shanemcd.com>
The generated based venv from `python3.8 -m venv` vs. `virtualenv -p
python38` is different. This changes aims to address the differences.
It was introduced as part of the Python 3.8 migration.
https://github.com/ansible/awx/pull/8778
Bump Up Version of iPython to be Compatible with Python 3.8.5
SUMMARY
Problem:
With Python version being upgraded, iPython now throws errors like this in the AWX shell:
bash-4.4$ echo "from datetime import timedelta; job = Job.objects.get(id=3); job.job_events.update(created=F('created') - timedelta(weeks=5))" | awx-manage shell_plus
~~snip~~
In [1]: ---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/usr/lib64/python3.8/codeop.py in __call__(self, source, filename, symbol)
134
135 def __call__(self, source, filename, symbol):
--> 136 codeob = compile(source, filename, symbol, self.flags, 1)
137 for feature in _features:
138 if codeob.co_flags & feature.compiler_flag:
TypeError: required field "type_ignores" missing from Module
TypeError: required field "type_ignores" missing from Module is mentioned in this iPython issue:
ipython/ipython#12558
Solution:
Upgrade iPython to latest version, which is 7.21.0. After doing this, the same command run in the shell works:
bash-4.4$ echo "from datetime import timedelta; job = Job.objects.get(id=3); job.job_events.update(created=F('created') - timedelta(weeks=5))" | awx-manage shell_plus
~~snip~~
In [1]: Out[1]: 16
In [2]: Do you really want to exit ([y]/n)?
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
Reviewed-by: Seth Foster <None>
Reviewed-by: Jeff Bradberry <None>
Add Expand button to variables fields
SUMMARY
Adds expand button to variables fields and details. Clicking it opens the code editor & YAML/JSON toggles in a large modal
Addresses #8031
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Mat Wilson <mawilson@redhat.com>
Update changed logic on project update
SUMMARY
Related to #8349
Found the inventory source does not have a way to track it, so did not do that part.
However scm revision is tracked for project update, as we already retrieve this data when looking for the existing item and waiting, put in logic to compare the two values when wait is true.
Also double checked the integration tests, and believe this should not change them, as its looking for result is success not changed in all but the first update.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
awx_collection
AWX VERSION
18.0.0
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Fix inventories-from-projects when running in Kubernetes
Related: #9704
Will also require a new release of the operator which will contain ansible/awx-operator#155
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Add managed by tower as part of the EE details page
Add managed by tower as part of the EE details page.
See: #8171
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Enable ?page_size=1 to Fetch Correct Objects on JT Endpoint
SUMMARY
Problem:
When multiple schedules are made from a single JT and a URL sort is done via:
api/v2/job_templates/[id]/schedules/?page_size=1
... the first half of the results come back with the same ID. 🤔
Solution:
Implementing a backup sort via ID on the schedules model fixes this problem, and now all schedules on the job template endpoint are being fetched properly.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
ADDITIONAL INFORMATION
This might fix#9632
Reviewed-by: Seth Foster <None>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Update the versioning on the docker-compose template
SUMMARY
Some versions of docker-compose will break with the new addition of
name parameters without this.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
Installer
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Hide commands that are being run for `make black`
Before:
$ make black
command -v black >/dev/null 2>&1 || { echo "could not find black on your PATH, you may need to \`pip install black\`, or set AWX_IGNORE_BLACK=1" && exit 1; }
could not find black on your PATH, you may need to `pip install black`, or set AWX_IGNORE_BLACK=1
make: *** [Makefile:275: black] Error 1
After:
$ make black
could not find black on your PATH, you may need to `pip install black`, or set AWX_IGNORE_BLACK=1
make: *** [Makefile:275: black] Error 1
Reviewed-by: Ryan Petrello <None>
Remove/Modify Usage of tower-cli (Deprecated) in Collections Test
tower-cli is no longer being maintained; this PR removes three test tasks that are no longer necessary from awx_collection/tests/integration/targets/tower_project_manual/tasks/create_project_dir.yml, and it modifies the last task in that test file to use awx-cli instead.
I also noticed this warning while running Collections tests:
awx/main/tasks.py:2109
/awx_devel/awx/main/tasks.py:2109: DeprecationWarning: invalid escape sequence \s
d['Password:\s*?$'] = 'scm_password' # noqa
This PR modifies that part of /awx_devel/awx/main/tasks.py into a raw string to fix this issue.
FYI, some deprecated Collections modules depend on the usage of tower-cli, but those tests are currently getting skipped:
awx_collection/test/awx/test_send_receive.py::test_receive_send_jt SKIPPED (The tower-cli library is needed to run th...) [ 56%]
awx_collection/test/awx/test_workflow_template.py::test_create_workflow_job_template SKIPPED (The tower-cli library i...) [ 73%]
awx_collection/test/awx/test_workflow_template.py::test_with_nested_workflow SKIPPED (The tower-cli library is needed...) [ 74%]
awx_collection/test/awx/test_workflow_template.py::test_schema_with_branches SKIPPED (The tower-cli library is needed...) [ 74%]
awx_collection/test/awx/test_workflow_template.py::test_with_missing_ujt SKIPPED (The tower-cli library is needed to ...) [ 75%]
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Ryan Petrello <None>
fix up a bug in rsyslogd error handling
@kdelee I'm unclear on why this is actually happening, so I'm just gonna cheat and generate a datestring
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Add EE to the settings page
Allow a system admin to set the global default execution environment.
See: #9088
This PR is also addressing the issue: #9669
Edit:
Details
Reviewed-by: Kersom <None>
Reviewed-by: Marliana Lara <marliana.lara@gmail.com>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
remove ansible_version from the API config and metrics endpoints
AWX no longer includes Ansible on the control plane and there is no
"global" version of Ansible aside from what's configured at the
Execution Environment level
see: #9472
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Ladislav Smola <lsmola@redhat.com>
Reviewed-by: Yanis Guenane <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Set a custom name for Docker volumes
SUMMARY
This pull request is related to #9695 and will make sure that the command make docker-compose does not create additional Docker volumes with the wrong prefix.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
Installer
AWX VERSION
awx: 18.0.0
ADDITIONAL INFORMATION
The current version of docker-compose.yml.j2 does not care about the Docker volumes created during the execution of migrate.yml. This means new Docker volumes will be created with the wrong prefix, which only contain the preloaded data.
This does not make sense, because I want to keep my data and according to the docs, the following command is necessary to load the demo data.
$ docker exec tools_awx_1 awx-manage create_preload_data
Reviewed-by: Shane McDonald <me@shanemcd.com>
let jupyter install ipython
running akit with ipython on py38:
In [1]: jt = v2.job_templates.create()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/usr/lib64/python3.8/codeop.py in __call__(self, source, filename, symbol)
141
142 def __call__(self, source, filename, symbol):
--> 143 codeob = compile(source, filename, symbol, self.flags, 1)
144 for feature in _features:
145 if codeob.co_flags & feature.compiler_flag:
TypeError: required field "type_ignores" missing from Module
Looks like we need a newer version of ipython:
ipython/ipython#12558 (comment)
.. because we pinned ipython in 2017
a39b1e8
If I try to install the newer ipython alongside jupyter, pip's dep resolver mentions a conflict
.. but if we just install jupyter there are no conflicts and a newer ipython (that avoids the original issue) gets installed.
pip freeze | grep "ipython\|jupyter"
ipython==7.21.0
ipython-genutils==0.2.0
jupyter==1.0.0
jupyter-client==6.1.12
jupyter-console==6.4.0
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.0
In [1]: jt = v2.job_templates.create()
In [2]: jt
Out[2]:
{
"id": 12,
"type": "job_template",
"url": "/api/v2/job_templates/12/",
Reviewed-by: Shane McDonald <me@shanemcd.com>
Add subsystem metrics that propagate through Redis
SUMMARY
#9019 -- list of metrics and their purpose / description
#9012#9056#8629
Use Redis to store metrics pertaining to the performance and health of subsystems such as the callback receiver and task manager. It is thread / multiprocess safe and should be fast enough to handle a high volume of data.
This data shows up at the /api/v2/metrics endpoint
You can filter down nodes using /api/v2/metrics/?subsystemonly=1&node=awx-1
You can also filter down to a specific metric,
/api/v2/metrics/?subsystemonly=1&metrics=callback_receiver_events_insert_db_seconds&node=awx-1
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 17.0.1
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Chris Meyers <None>
Reviewed-by: Seth Foster <None>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
- Adds a Metrics() class that can track data such as number of
events the callback receiver inserted into database
- Exposes this metric data at the api/v2/metrics/ endpoint.
This data is prometheus-friendly
- Metric data is stored in memory, then periodically saved to Redis.
- Metric data is periodically broadcast to other nodes in the cluster,
so that each node has a copy of the most recent metric data collected.
Loosen Collections vs Tower Version Check
SUMMARY
Connecting issue #9532
This change will make it so that if the major version numbers of Collections + Tower matches, a warning will not get activated.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
Collections
AWX VERSION
awx: 17.0.1
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: John Westcott IV <None>
Fix api/v2/metrics data displaying incorrect value
SUMMARY
How to reproduce bug
Disable all instances
Queue up 5 jobs, (if using same JT, enable concurrent jobs)
Enable instance so jobs will start running
Refresh /api/v2/metrics endpoint. You should see a metric that says awx_status_total{status="pending"} 5.0. Once you see this, don't refresh again.
Wait for all jobs to finish.
Now start refreshing the api/v2/metrics endpoint. Every once in a while you will see this pending jobs metric show (awx_status_total{status="pending"} 5.0) even though there are no jobs in pending state
Fix
Use a locally defined prometheus registry instead of a global registry. Each time the endpoint is refreshed, a new, local registry is set up with prometheus objects (Gauge, Info). Since we aren't actually incrementing these metrics across api calls, we should be safe. That is to say, we just lookup the values from the database and set the prometheus values explicitly.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
Reviewed-by: Ryan Petrello <None>
Fixes a bug with ResourceAccess Add
SUMMARY
This addresses #9456 and #9466
The user can now properly add roles to a resource.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
ADDITIONAL INFORMATION
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Support job cancellation through details panel
SUMMARY
for #7657
edit: also addresses #8838 cc @nixocio
Reviewed-by: Kersom <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Adds delete warnings on lists
SUMMARY
This addresses #5546.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Keith Grant <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
Reviewed-by: Kersom <None>
AWX no longer includes Ansible on the control plane and there is no
"default" version of Ansible aside from what's configured at the
Execution Environment level
see: https://github.com/ansible/awx/issues/9472
Python: Bump to python 3.8
SUMMARY
Bumping default python used in awx and ansible venv to python 3.8
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
devel
ADDITIONAL INFORMATION
N/A
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Bill Nottingham <None>
Reviewed-by: Yanis Guenane <None>
Properly set launched_by for jobs launched by scheduled workflows : Issue 5057
SUMMARY
with respect to the issue 5057, there should be a way by which the UI should know , if a job was launched from a workflow , and that workflow was launched from a schedule.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 17.0.1
ADDITIONAL INFORMATION
This PR tries to solve the issue 5057.
There is an edge case , where the API response is not very clear. When a JT is invoked from a workflow , and that workflow is invoked from a schedule, the API response does not convey how the job was launched.
So, I have added the schedule id of the schedule that invoked the workflow and which in turn invokes the JT. The new key in API response is 'launched_by', and it has the schedule id. In all the 5 previous cases mentioned in the initial issue , the 'launched_by' field is blank ( that is empty string ( "" )).
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Amol Gautam <amol_gautam25@yahoo.co.in>
Reviewed-by: Nana <natr@hey.com>
'launched_by’ property returns summary { id,type,name,url } of object that launched the current UnifiedJob
'ancestor_job’ property returns summary { id,type,name,url } of the first workflow in case the current UnifiedJob was started by a workflow or a workflow chain
Added ‘launched_by’ field and ‘get_launched_by’ function in ‘UnifiedJobSerializer’ , to expose the ‘launched_by’ field in GET ‘api/v2/unified_job/id’ response
Added ‘ancestor_job’ field in the summary field of UnifiedJob in the GET ‘api/v2/unified_job/id’ response
fix the large file parsing in project sync
SUMMARY
Fixes the issue of scm update stuck on reading large files because of islice trying to read the whole file.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
Reviewed-by: Ryan Petrello <None>
Reviewed-by: None <None>
Add the expected launch and run methods to the different job type
models. Include a new helper function to look up the right model
given a job type and use it in place of switch statements or
passing the type in to build a url.
Filter credential type by Container Registry for EE
Filter credential type by Container Registry for EE form.
See: #9628
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fix service reference error for cred plugin dev env
SUMMARY
Intended to fix the error I see when running COMPOSE_TAG=devel make docker-compose-credential-plugins:
Reviewed-by: Ryan Petrello <None>
move code linting to a stricter pep8-esque auto-formatting tool, black
black (https://pypi.org/project/black/#description) is a strict superset of PEP8
It's also a tool that auto-formats your code on the fly for you based on its ruleset.
With this PR, you can run make check, and any style issues will automatically be applied.
Additionally, with this PR, if you spin up the development environment using our make targets, you'll automatically get a pre-commit hook installed that automatically runs linting prior to commit.
If you don't like this behavior, or don't want it locally, you can:
~ export AWX_IGNORE_BLACK=1
...but it's important to note that we won't merge your PR if it doesn't adhere to our style guidelines (which will run automatically as part of pre-merge CI).
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
Reviewed-by: Seth Foster <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Update some links and notes in the changelog
SUMMARY
Fix typo, remove duplicate change note, fix a wrong link, add link to the ui virtualenv removal
Reviewed-by: Ryan Petrello <None>
Allow one to select non-global execution environments for organizations
Allow one to select non-global EE when editing an Organization.
See: #9592
All those EE should be present as a choice when editing the Default organization.
Editing Default organization.
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
fix port conflict in dev cluster
SUMMARY
problem: loop adds 100 to ports 7899 and 7999, which would yield 7999 to 8099 on the next iteration, so the 7999 is conflicting
fix: add 1000 instead
Also, haproxy was being defined twice, now it renders once.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 17.0.1
Reviewed-by: Seth Foster <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Instruct git to ignore the .vscode/ directory
SUMMARY
Instruct git to ignore the .vscode/ directory
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 18.0.0
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
add ouiaID to select and cancel buttons on modals
SUMMARY
Add ouiaId prop to select and cancel button within modals
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Fixes silent error on SCM subform
SUMMARY
This addresses #9373. It disallows the user to select both Update on launch and update on project update. It also adds a bit of info to the tool tip including a link to the project in question so the user can edit the project to allow them to update on launch and on project update
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Create a wrapper directory for the private data dir
Reviewed-by: None <None>
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Fixes Several Bugs
SUMMARY
This address #9485 (Job template project field validate), #9319 (Job Details view only would show job type run, even if it was a job type check, #7516 (changes the Completed Jobs tab for a JT or WFJT to show Jobs since it show completed and pending/running jobs).
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: John Mitchell <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Prevent users from selecting credentials that prompt for passwords on workflow nodes and schedules
SUMMARY
link #8921
If a user selects a job template with default credentials that prompt for passwords (but does not prompt for credentials) then the user should not be allowed to create the node and a different JT must be selected:
If a user selects a credential that prompts for passwords when creating/editing a workflow node or schedule then we show this error:
If a user removes a credential that exists in the default collection of credentials on the JT then it must be replaced. This is the error we show:
If a user attempts to create a schedule for a job template with default credentials that prompt (but does not prompt for credentials) then the API responds with this error:
I believe this UX is consistent with the old UI but I am double checking that now.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Fixes crashing wizard, and adds error handle on adding role
SUMMARY
This addresses #8769. It also adds error handling if there is some sort of request error during the submit request.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
AWX VERSION
ADDITIONAL INFORMATION
Reviewed-by: John Mitchell <None>
Adds ouiaId's to various buttons
SUMMARY
@tiagodread @unlikelyzero @one-t @akus062381 this will likely break something because I changed some existing ouia-id's so that they are a consistent structure.
^^ Let's let one of them merge this
I also removed an unused component
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: John Hill <johill@redhat.com>
Adds support for html in custom login text
SUMMARY
link #7603
I couldn't come up with a way to do this without breaking up the component and discontinuing use of the LoginPage PF component. This is because LoginPage expects the textContent component (what we use to display the custom login text) to be a string. By using the underlying LoginPage components I reconstructed the login page and got more control over that prop.
The custom message in the old UI supported both strings and HTML:
So we need to support rendering HTML but we need to do it in a safe way. Our solution to that was https://docs.angularjs.org/api/ngSanitize. React doesn't seem to have anything like this built in so I went looking for outside help. html-entities is already included in our project but as best as I can tell that lib is mainly focused on swapping special characters out for html entities. I wanted something that was going to strip the HTML of bits that could be exploited by a malicious actor.
I settled on https://www.npmjs.com/package/sanitize-html because it was a) small and b) actively maintained. The API was simple and let me sanitize the HTML before setting it using dangerouslySetInnerHTML. If we need to tweak the configuration away from the default values then we can certainly do that.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Only attempt to fetch event options on non workflow jobs
SUMMARY
link #9640
This was fallout from output search filtering. We need this request for non workflow jobs so that we can build the search options.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
UI
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add support for replace/revert on secret credential fields
SUMMARY
link #7256
Note that this only applies to editing an existing credential. You should not see this button on fields when adding a new credential.
When editing an existing credential the replace button should show up on fields where secret is true and the field has an existing value that is not an external credential. Examples:
Fields with external credentials should look the same:
Initially the button tooltip should say Replace. Clicking Replace will clear out the previously saved value and enable the form field:
The tooltip will change to Revert. Clicking Revert will take the field back to it's original state.
I also noticed a race condition which would result in the input fields (subform) not being populated due to the form rendering before the request(s) were completed. I fixed this.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Kersom <None>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
Add support for filtering and pagination on job output
SUMMARY
link #6612
link #5906
This PR adds the ability to filter job events and also includes logic to handle fetching filtered job events across different pages.
Note that the verbosity dropdown included in #5906 is not included in this work. I don't think that's possible without api changes.
As part of this work, I converted JobOutput.jsx from a class based component to a functional component. I've tried my best to make sure that all existing functionality has remained the same by comparing the experience of this branch to devel.
Like the old UI, the output filter is disabled while the job is running.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
UI
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Marliana Lara <marliana.lara@gmail.com>
Enable Ansible version to be collected from EEs
SUMMARY
Connecting issue #9473
This PR, along with this Ansible-Runner PR, enables us to obtain the Ansible (core) version for each execution environment that is utilized. This info can be gathered from the new ansible_version column on the main_unifiedjobs table.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 17.0.1
ADDITIONAL INFORMATION
Screenshot/example of the DB output:
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Ladislav Smola <lsmola@redhat.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Do not allow user to modify EE managed by tower
Do not allow user to attempt to modify EE managed by tower.
See: #9250
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Tiago Góes <tiago.goes2009@gmail.com>
refactor payload construction for awxkit
This fixes container_group creation to allow passing
"is_container_group" and "credential" to the "create" method
on instance groups, and refactors other page objects
to use a common utility function to eliminate copy-pasted code
This will help us update to set is_container_group correctly as is now needed since de52ade
Reviewed-by: Ryan Petrello <None>
Remove custom virtual env
Remove custom virtual from the UI.
Also, surface missing-resource warnings on list items for UJTs that were using
custom virtualenvs. And related details page.
See: #9190
Also: #9207
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Mat Wilson <mawilson@redhat.com>
Add support for Centrify Vault as a credential plugin
replaces #8952
cc @surbhijain1502 @Asharma-bhavna @badrogh
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
Reviewed-by: Chris Meyers <None>
This fixes container_group creation to allow passing
"is_container_group" and "credential" to the "create" method
on instance groups, and refactors other page objects
to use a common utility function to eliminate copy-pasted code
Dont require is_container_group in payload when creating InstanceGroups
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
Reviewed-by: Ryan Petrello <None>
Hashicorp Vault Credential Plugin : Support for namespace
SUMMARY
Added the support for Vault Namespace (Enterprise feature)
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
credential_plugins/hashivault.py
AWX VERSION
1.7.0
ADDITIONAL INFORMATION
Adding specific X-Vault-Namespace header when Namespace option is set.
Reviewed-by: Ryan Petrello <None>
* fix typing space character
* hide cursor when editor doesn't have user focus
* show help text any time editor is in focus
* fix content shifting when help text appears/disappears
* remove 80 character "print limit" line
- remove requirements_ansible logic from the update script
- removed the need for py2-specific system dependencies
- update to the latest pip-tools and move to the new long format
(https://github.com/jazzband/pip-tools/pull/1237)
- fixed a few busted references to receptorctl @ devel
- Organization.default_environment
- Project.default_environment
- JobTemplate.execution_environment
- WorkflowJobTemplate.execution_environment
System jobs are not editable by anyone other than a system admin, so
we don't need to check. It appears that unified job templates can't
be created or edited outside of the endpoints for the specific types.
They've made breaking changes that is going to take
some deeper investigation to update awxkit to use
This is only used for development purposes, and should
have not impact on the "awx" cli entry point
Add EE to the following screens:
* Job Template
* Organization
* Project
* Workflow Job Template
Also, add a new lookup component - ExecutionEnvironmentLoookup.
See: https://github.com/ansible/awx/issues/9189
moved AWXKit pull additions to separate PR and made some changes that were causing linting errors in tests and add copy to show_capabilities for the ee serializer
can_add: gets an explicit role to check against, `'execution_environment_admin_role'`
can_change: leverages `self.check_related()` for the case where the Org is not changing, but also adds an explicit check for the EE Admin Role when the Org is changing to an explicit different Org.
* add ee option on factories for organizations
* add new lines
* remove inventory from the options
* remove line
* remove line from projects
* fix the tuple
* fix lint problems
- In K8S-based installs, only container groups are intended to be used
for playbook execution (JTs, adhoc, inventory updates), so in this
scenario, other job types have a task impact of zero.
- In K8S-based installs, traditional instances have *zero* capacity
(because they're only members of the control plane where services
- http/s, local control plane execution - run)
- This commit also includes some changes that allow for the task manager
to launch tasks with task_impact=0 on instances that have capacity=0
(previously, an instance with zero capacity would never be selected
as the "execution node"
This means that when IS_K8S=True, any Job Template associated with an
Instance Group will never actually go from pending -> running (because
there's no capacity - all playbooks must run through Container Groups).
For an improved ux, our intention is to introduce logic into the
operator install process such that the *default* group that's created at
install time is a *Container Group* that's configured to point at the
K8S cluster where awx itself is deployed.
The part where we pass in the runner params to the processor phase is
legit. Need to investigate why the fact_cache directory is no longer nested
under job.id.
- a new unique name field to EE
- a new configure-Tower-in-Tower setting DEFAULT_EXECUTION_ENVIRONMENT
- an Org-level execution_environment_admin_role
- a default_environment field on Project
- a new Container Registry credential type
- order EEs by reverse of the created timestamp
- a method to resolve which EE to use on jobs
Add organization as part of creating/editing an execution environments
If one is a `system admin` the Organization is an optional field. Not
providing an Organization makes the execution environment globally
available.
If one is a `org admin` the Organization is a required field.
See: https://github.com/ansible/awx/issues/7887
Update to AWX execution environment
use the special 2.9 container image
revert setting back for merge
Fix another permission error by mapping 2 folders
also create folders before running
* Migate management jobs list to tables
* Use cancel link variant for consistency with other prompts
* Add basic test coverage for sysjobs
* Remove select-all from mgmt jobs
* Remove unneeded component variables
* Fix missing schedule breadcrumb
* Optimize data fetching with useCallback
- removes local_docker installer and points community users to our development environment (make docker-compose)
- provides a migration path from Local Docker Compose installations --> the dev environment
- the dev env can now be configured to use an external database
- consolidated the Local Docker and dev env docker-compose.yml files into one template file, used by the dockerfile role
- added a 'sources' role to template out config files
- the postgres data dir is no longer a bind-mount, it is a docker volume
- the redis socket is not longer a bind-mount, it is a docker volume
- the local_settings.py.docker-compose file no longer needs to be copied over in the dev env
- Create tmp rsyslog.conf in rsyslog volume to avoid cross-linking. Previously, the tmp code-generated rsyslog.conf was being written to /tmp (by default). As a result, we were attempting to shutil.move() across volumes.
- move k8s image build and push roles under tools/ansible
- See tools/docker-compose/README.md for usage of these changes
this middleware allready existed, and we were trying to log this
data but it was not working.
Hope is these logs will be able to be shipped via external logging
and we could use kibana to track response time of different endpoints
Fixed the customized notification returning incorrect values for host_status_counts
Update notifications.py
Removed if condition
Added exception handling
A nitpick
This fixes the issue addressed in 9273
The symlinks are created on the build container as opposed to the final
container causing awx-manage command to break.
Signed-off-by: Siva Renganathan <siva.rg@protonmail.com>
Various points (e.g. created, running, processing events), are
structured into json format and output to /var/log/tower/job_lifecycle.log
As part of this work, the DependencyGraph is reworked to return
which job object is doing the blocking, rather than a boolean.
* The cron ran logrotate will now rotate our log files instead of python
* If not error log file is specified in the config then do not include
it as a paremter to rsyslog omhttp module. This is useful for
containers.
related: https://github.com/ansible/awx/issues/6010
as noted in the comment removed from this diff, it's probably time
to stop calling this function on every dispatcher service restart
team roles title crumb missing
various inventory crums missing
make it so inventories and templates don't get rid of data needed to generate the crumb config
Fix Inventory/Project rbac broken on JT form.
Also, update ProjectLookup to filter using `role_level: 'use_role'` as
per old UI implementation.
Also, update InventoryLookup to filter using `role_level: 'use_role'` as
per old UI implementation.
See: https://github.com/ansible/awx/issues/8194
Credential access tab should be shown when cred doesn't belong to an organization.
Also, update unit-tests to reflect change.
See: https://github.com/ansible/awx/issues/7708
a new version of pip-tools changed the format of dependency annotations
in generated requirements.txt files
we should probably change to the new format at some point, but maybe
*after* we merge a few of our long-running branches that touch these
files (otherwise, managing conflicts could be pretty hellish)
This does a few things:
- Removes need for awx_sdist_builder image
- Reorders Dockerfile steps to optimize image cache between prod and dev builds
- Unifies VENV_BASE and COLLECTION_BASE in prod and dev builds
* Fix repeated api calls from useEffect hook by wrapping the breadcrumb
setter with useCallback
* Rework the top-level routes to remove some old patterns and bring it more
into alignment with how it's done on the projects screen
Intercept all http(s) responses and store expiration time from headers
in local storage. Drive expiration timers in app container across all
tabs with browser storage events and accompanying react hooks
integration. Show a warning with logout countdown and continue button
when session is nearly expired.
* The namespace for isolated logging was not enabled. Add a handler and
logger so that it's enabled. This is particularly useful when the
logging level is switched to DEBUG
* Knowing how long check_isolated.yml ran can be helpful in debuging the
isolated execution path. Especially if you suspect the connection speed
or reliability of the control node -> execution node
* It's hard/impossible to know what job a check_isolated.yml playbook
runs for by just looking at the logs.
* Forward the job id for which an iso management playbook is running for
and output that job id so it can be found in the logs.
* We batch logging isolated management playbook output. This results in
the timestamp of the log being useless when trying to determine when
each task in the playbook ran.
* To fix this, we enable timestamp logging at the playbook level via
ansible `profile_tasks` callback plugin.
This restores some of the original files and routes from the migration
view of the classic ui with the eventual goal of fully reintegrating this
system with the new ui.
See: b39db745d4
* The exported field shows total quantity exported to a manifest for a given sub. We want to sum the quantities of each sub allocation in a manifest instead.
We _always_ want INLINE_RUNTIME_CHUNK to be false when building the ui,
even if someone happens to unexpectedly make a production build without
using the top-level make targets for some reason.
Hide instance group for Inventory Details if the data is not available.
This is the the same approach used in other details screens.
See: https://github.com/ansible/awx/issues/8620
The fixes and issue where the timestaps in the stdout for
inventory updates gave the time since the start of the dispatcher
instead of the time since the start of the update.
This commit also moves the handler into the utils module where
other custom AWX handlers live, instead of tasks.py
this is to keep tasks.py relatively clean, as best as possible
* Change handling of error cases to global post_run_hook
* handle license errors correctly again
* Fix some issues with line ordering from the custom logger thing
* Remove debug log statement
* Use PermissionDenied for license errors
* More elegant handling of line initialization
Update tests to new exception type
Catch all save errors, fix timing offset bug
Fix license error handling inside import command
* proot now enabled at task-level
since tasks are no longer calling
awx-manage (which would set up its own proot)
* dropping proot env var since it's not
relevant to the test
* noting that the inv update task only uses the
inventory update management command to
save the inv to the database
(it doesn't do the work of fetching hosts / groups)
- in the past, inv. update jobs called `awx-manage inventory_update`
which took care of setting up process isolation
- at this point, though, inv. update jobs call runner / ansible-inventory
directly, so we need another way to put process isolation in place
- thankfully, there was already support for providing process isolation
for other types of jobs (namely JT Jobs, Project Updates and Ad Hoc
commands)
- so, we do what those other jobs do and override the stub for should_use_proot
(which by default returns false) so that it keys off of the
`AWX_PROOT_ENABLED` setting
* perform_update can be called from either awx-manage
or the RunInventoryUpdate task
* need to make sure that the inventory updates
that happen with perform_update are atomic
This commit makes the needed changes to inventory update
post_save_hook logic so that the historic log lines that
inventory updates write will be written to stdout,
but this hack bypasses the ansible-runner verbose event
logic and dispatches verbose events directly.
Fix the venv application with the ansible-inventory system
(note: much of this is undone in a later commit)
Deal with some minor test updates for
the ansible-inventory interface changes
Add updates related to smart inventories.
* Add popover for `Smart host filter`.
* Add popover for `Instance Groups` on Smart Inventory screen.
* Rename `Host filter` to `Smart host filter` per mockup.
* Add inventory as part of dynamic host filter.
See: https://github.com/ansible/awx/issues/8581
Also: https://github.com/ansible/awx/issues/8548
we've seen evidence of a race condition on fork for awx.conf.Setting
access; in the past, we've attempted to solve this by explicitly closing
connections pre-fork, but we've seen evidence that this isn't always
good enough
this patch is an attempt to close connections post-fork so that sockets
aren't inherited post fork, leading to bizarre race conditions in
setting access
Add feature to associate teams to users. For the time being when
associating Users to a team, the User will be associated with `member_role` only. And when `diassociating` the User from a team all related roles - member, read, and admin will be removed.
Also, fix a bug related to search not being cleared after closing/cancel
the `AssociateModal`.
Hide max hosts field on org form.
Also, simplify the usage of context API to read the value of me
parameter.
Hide max hosts field on org form.
See: https://github.com/ansible/awx/issues/4950
Remove related resources groups/hosts when deleting inventory sources.
The current UI deletes `groups` and `hosts` once the inventory source is
deleted. Add this behavior to the new UI.
See: https://github.com/ansible/awx/issues/8098
- Write a deepmerge() implementation, keeping only the test suite of
https://stackoverflow.com/a/20666342/435004
- Use it to deep-merge pod['metadata'] with user input,
instead of replacing fields in it
Fix username as a required field. `UserForm` is used for adding and
editing an user. When adding an user, the initial user value is a `{}`
update logic to cover this case.
Also, add unit-tests to cover this particular case.
See: https://github.com/ansible/awx/issues/8453
cast create_preload_data to boolean with `create_preload_data | bool` in launch_awx_task.sh.j2
Reviewed-by: Ryan Petrello
https://github.com/ryanpetrello
* Task manager fit_ optimization code caused problems with container
group code.
* Note that we don't actually get the benefit of the optimization for
container groups. We just make it so that the code doesn't blow up. It
will take another pass to apply optimizations to the container group
task manager path.
- output a profiling disabled message when appropriate
- specify that we are doing SQL profiling in the enabled case
- treat negative thresholds the same as zero, disabling profiling
This was problematic because it was overwriting the original values that had been defined in the other serializers. Additionally, there are no other dunders for other capabilities prefetch
this was likely added because UnifiedJobTemplateSerializer does not have it's own capabilities, but rather derives them from JTSerializer and WFJTSeralizer, but it worked better without the dunder once I removed the data that was overwriting the data from the WFJT and JT serializers.
* Tried to fill in application_name in awx/__init__.py but I think that
is too late
* Fill in database application_name with enough information to easily
trace the connection from postgres back to the node and pid that
initiated the connection.
* Set application_name in django settings so that application_name is
set _before_ the first postgres connection is established.
Annotations are only supported for ingress and service accounts
This PR will allow you now to specify annotations for Kubernetes Deployment
resources by defining `kubernetes_deployment_annotations` var list
* Do not query the database for the set of Instance that belong to the
group for which we are trying to fit a job on, for each job.
* Instead, cache the set of instances per-instance group.
* We update the parent unified job template to point at new jobs
created. We also update a similar foreign key when the job finishes
running. This causes lock contention when the job template is
allow_simultaneous and there are a lot of jobs from that job template
running in parallel. I've seen as bad as 5 minutes waiting for the lock
when a job finishes.
* This change moves the parent->child update to OUTSIDE of the
transaction if the job is allow_simultaneous (inherited from the parent
unified job). We sacrafice a bit of correctness for performance. The
logic is, if you are launching 1,000 parallel jobs do you really care
that the job template contains a pointer to the last one you launched?
Probably not. If you do, you can always query jobs related to the job
template sorted by created time.
* Do not query the database for the set of Instance that belong to the
group for which we are trying to fit a job on, for each job.
* Instead, cache the set of instances per-instance group.
* We update the parent unified job template to point at new jobs
created. We also update a similar foreign key when the job finishes
running. This causes lock contention when the job template is
allow_simultaneous and there are a lot of jobs from that job template
running in parallel. I've seen as bad as 5 minutes waiting for the lock
when a job finishes.
* This change moves the parent->child update to OUTSIDE of the
transaction if the job is allow_simultaneous (inherited from the parent
unified job). We sacrafice a bit of correctness for performance. The
logic is, if you are launching 1,000 parallel jobs do you really care
that the job template contains a pointer to the last one you launched?
Probably not. If you do, you can always query jobs related to the job
template sorted by created time.
* fixes#8347
* Rename inventory_source to name in the tower_inventory_source_update
* Allow to specify both name or id for `name` and `inventory` params
* Add type of login used as part of UserListItem.
* Add type of login used as part of UserDetail.
* Hide password field, UserForm, in case login method is LDAP or Social.
* Make username field, UserForm, not required in case login is LDAP or
Social.
See: https://github.com/ansible/awx/issues/5685
Add missing words for translation.
`...more`, and `Show Less` were already marked for translation in a
previous PR, since this code is shared as part of the `ChipGroup` code.
See: https://github.com/ansible/awx/issues/6857
This changeset introduces two changes:
1. Update the API representation of Workflow Job Templates to use the
natural key of the Inventory type instead of its id;
2. Override the related property of the CLI's WorkflowJobTemplate page
type to patch the related references during the export process,
allowing the resource to be serialised using the natural key of the
Inventory type instead of the id.
Change n.2 is a workaround that is used when exporting resources from
AWX/Tower instances that don't have change n.1. It can be removed in the
future.
In order to create a container group is necessary to provide a
credential.
See: https://github.com/ansible/awx/issues/8184
This change makes the code related to display the credential as part of
the container group details a bit more robust. Avoiding to attempt to
show a non-existent credential - what is not supposed to exist.
Closes: https://github.com/ansible/awx/issues/8199
the bigint migration removed the foreign key constraints for:
- host_id
- job_id (and projectupdate_id, etc...)
because of this, we don't really need to check explicitly for a host_id
IntegrityError anymore (because it won't occur)
additionally, while it's possible to insert an event with a mismatched
job_id now (for example, you can totally start a long-running job, and
delete the job record in the background using the ORM or psql), doing
so results in DoesNotExist errors in the code that handles the
playbook_on_stats events
instead, just have each worker connect directly to redis
this has a few benefits:
- it's simpler to explain and debug
- back pressure on the queue keeps messages around in redis (which is
observable, and survives the restart of Python processes)
- it's likely notably more performant at high loads
make the --status flag work by fetching a periodically recorded snapshot
of internal process state; additionally, update the callback receiver to
*also* record these statistics so we can gain more insight into any
performance issues
The analytics change PR adjusted the logging for awx.analytics,
which solved the issue, but should have used the targeted awx.main.analytics.
Also flip a couple of loggers to use the regular awx.analytics (awx analytics)
logger instead of awx.main.analytics (the automation anayltics task system).
I'm not sure that this function is actually in use anywhere anymore, but
it shouldn't be a top-level import because it represents an optional
dependency.
Add quotes around volume value for posgres data. I installed via docker without changing any values and the UI was stuck in upgrading for long time. Browsed around and figured out that issue was due to postgres volume as a query was getting error. Inspected the template and found that there was no quotes around volume, unlike volumes for others.
I added the quotes and docker compose was working
Errors/warnings when gathering analytics are about 50/50 split between
the gathering code in analytics and the task code that calls it, so
they should be in the same place for debugging sanity.
Collect expensive collectors separately, and in a loop
where we make smaller intermediate dumps.
Don't return a table dump if there are no records, and
don't put that CSV in the manifest.
Fix up unit tests.
doing this in the migration *before* any Organizations actually exist
is stirring up RBAC dragons that I don't have time to fight
this commit meanst that *new* installs will pre-create the default
Galaxy (public) credential in create_preload_data, while
*upgraded/migrations* installs will do so via the migration
* Do not write out inventory source_vars to a file on disk as they _may_
contain sensitive information. This also removes support for backwards
migrations. This is fine, backwards migration is really only useful
during development.
* Before, we were re-writing `plugin:` when users updated the
InventorySource via the API. Now, we just override at run-time. This
makes for a more sane API interaction
Options which are not in the API POST and are marked in the module as deprecated are ignored
If an option is not in the API POST but is marked as a list we assume its a relation
The auth_path is used with the approle auth method
It's not linked to the secret we are reading but to the auth method,
this parameter has to be moved to inputs
Changed library structure
Origional TowerModule becomes TowerLegacyModule
TowerModule from tower_api becomes TowerAPIModule
A real base TowerModule is created in tower_module.py
A new TowerAWXKitModule is created in tower_awxkit
TowerAWXKitModule and TowerAPIModule are child classes of TowerModule
This changset allows the import of YAML formatted resources. The CLI
user can indicate which format to use with the `-f, --format` option.
The CLI help text has been amended to reflect the new feature.
The AWX CLI `export` subcommand offers the option of formatting the output
as YAML or JSON, so it makes sense that the `import` subcommand reflects
this.
A simple test is also provided. In order to ease the task of testing
commands that import resources by reading the stdin, the CLI has been
extended to allow specifying an alternative file descriptor for stdin,
similarly to stdout and stderr.
This change adds related Labels to the Workflow Job Template document that is
exported by the AWX CLI.
Previously, exporting and then importing Workflow Job Templates would
not retain their related Labels.
This change fixes an erroneus early return in a private method that was
preventing more than one type of related object from being correctly
assigned to the parent object, and therefore imported.
Also, a minor spelling mistake was corrected.
Remove showExpandCollapse prop from the DataListToolbar calls. This is
not an expected prop to be passed to this component.
Inside DataListToolbar.
```
const showExpandCollapse = onCompact && onExpand;
```
In order to use this feature, `onCompact` and `onExpand` props should
be passed.
...
* upgrade `chromedriver` for ARM support
* upgrade `pynacl` to fix `libsodium` build issue on ARM
* remove unnecessary i686-specific `libstdc++.so.6` package
* install `kubectl` and `tini` from upstream binaries for ARM support
* use upstream `postgres` and `alpine` docker images for `postgresql` helm chart
Fixes#7051
Populate the cache the first time the job is run for a revision
that needs them, and for future runs for that revision just
copy it into the private directory.
Delete the cache on project deletion.
Invalidate the cache on a new project revision
Also download roles/collections during the sync job
Since we're writing into a per-revision cache, we can do this easily now.
Don't try and install content if there aren't any requirements expecting it
Adjust pathing to the proper location.
Force install if doing a manual sync.
Requirements may be unversioned.
Remove the cache when delete-on-update is set
Integrate content caching with existing task logic
Revert the --force flags
use the update id as metric for role caching
Shift the movement of cache to job folder from rsync task to python
Only install roles and collections if needed
Deal with roles and collections for jobs without sync
Skip local copy if roles or collections turned off
update docs for content caching
Design pivot - use empty cache dir to indicate lack of content
Do not cache content if we did not install content
Test changes to allay concerns about reliability of local_path
Do not blow away cache for SCM inventory updates
Remove project update vars no longer used
Remove job pre-creation of content folders
code style edit, always use cache_id as property in tasks
Fix log message
Situations have come up where the 5+ minute kill signal for
run_task_manager is emitted to the worker process running it, but
since the worker improperly inherited the AWXConsumerBase().stop()
handler a deadlock ultimately was triggered on the database
connection.
The docker_registry_password var isn't interpolated by the shell, so
it shouldn't be quoted
Fixes: #7695
Signed-off-by: Philip DOUGLASS <philip.douglass@amadeus.com>
Revert the --force flags
use the update id as metric for role caching
Shift the movement of cache to job folder from rsync task to python
Only install roles and collections if needed
Deal with roles and collections for jobs without sync
Skip local copy if roles or collections turned off
update docs for content caching
Design pivot - use empty cache dir to indicate lack of content
Do not cache content if we did not install content
Test changes to allay concerns about reliability of local_path
Do not blow away cache for SCM inventory updates
Populate the cache the first time the job is run for a revision
that needs them, and for future runs for that revision just
copy it into the private directory.
Delete the cache on project deletion.
* Use more selective route matching when determining if a nav item is
active
* Don't automatically collapse nav groups when user navigates to a
different group
this resolves an issue that causes an endless hang on with Cyberark AIM
lookups when a certificate *and* key are specified
the underlying issue here is that we can't rely on the underyling Python
ssl implementation to *only* read from the fifo that stores the pem data
*only once*; in reality, we need to just use *actual* tempfiles for
stability purposes
see: https://github.com/ansible/awx/issues/6986
see: https://github.com/urllib3/urllib3/issues/1880
When we used ints and passed this data into a nother call like:
- name: Create a job template with a looked up credential from a folded lookup
tower_job_template:
name: "{{ job_template_name }}"
credentials: >-
{{ lookup(
'awx.awx.tower_api',
'credentials',
query_params={ 'name' : credential_name },
return_ids=True,
expect_one=True,
wantlist=True
) }}
project: "{{ project_name }}"
inventory: Demo Inventory
playbook: hello_world.yml
job_type: run
state: present
register: create_jt
Ansible would raise this warning:
[WARNING]: The value 30 (type int) in a string field was converted to '30' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.
Returning a list of strings prevents that.
Custom credentials can have input fields named 'name', 'organization',
'description', etc. Underscore these variables to make collisions
less likely to occur.
this tool looks at the most recent jobs for a specific job template and
attempts to discover the _slowest_ tasks and hosts
$ awx-manage bottleneck --template N
$ awx-manage bottleneck --template N --threshold 1 --ignore yum
$ awx-manage bottleneck --template N --ignore pause --ignore yum
this change fixes a bug introduced in the optimization at https://github.com/ansible/awx/pull/7352
1. Create inventory with multiple hosts
2. Run a playbook with a limit to match only one host
3. Run job, verify that it only acts on the one host
4. Go to inventory host list and see that all the hosts have last_job updated to point to the job that only acted on one host.
There is some history here.
https://github.com/ansible/awx/pull/7190 <- This PR was an attempt at fixing a
bug notting ran into where some jobs on k8s installs would get stuck in Waiting
forever.
The PR mentioned above introduced a bug where there are no instance groups on a
fresh k8s-based install. This is because this process currently happens in the
launch scripts, before the database is up.
With this patch, queue / instance group registration happens in the heartbeat,
right after auto-registering the instance.
When targeting, ../workflow_job_templates/id#/workflow_nodes/ endpoint,
user could not set all_parents_must_converge to true.
3.7.1 backport for awx issue #7063
* broadcast websockets have stats tracked (i.e. connection status,
number of messages total, messages per minute, etc). Previous to this
change, stats were tracked by ip address, if it was defined on the
instance, XOR hostname. This changeset tracks stats by hostname.
* The websocket backplane interconnect is done via ip address for
Kubernetes and OpenShift. On init run_wsbroadcast reads all Instances
from the DB and makes a decision to use the ip address or the hostname
based, with preference given to the ip address if defined. For
Kubernetes and OpenShift the nodes can load the Instance before the
ip_address is set. This would cause the connection to be tried by
hostname rather than ip address. This changeset ensures that an ip
address set after an Instance record is created will be detected and
used.
This was causing our EL8 Brew builds to break, because it wasn't being
vendored. This is in fact required for python3. It was being resolved as a
dependency of other things (see list at end of line), so it was being downloaded
on-the-fly since our normal builds have internet access. It only broke when it
wasn't vendored for offline builds.
2020-05-15 17:52:08 -04:00
5333 changed files with 620868 additions and 379786 deletions
- For the roundup of all the different mailing lists available from AWX, Ansible, and beyond visit: https://docs.ansible.com/ansible/latest/community/communication.html
- Hello, we think your question is answered in our FAQ. Does this: https://www.ansible.com/products/awx-project/faq cover your question?
- You can find the latest documentation here: https://docs.ansible.com/automation-controller/latest/html/userguide/index.html
## PRs/Issues
### Visit our mailing list
- Hello, this appears to be less of a bug report or feature request and more of a question. Could you please ask this on our mailing list? See https://github.com/ansible/awx/#get-involved for information for ways to connect with us.
### Denied Submission
- Hi! \
\
Thanks very much for your submission to AWX. It means a lot to us that you have taken time to contribute. \
\
At this time we do not want to merge this PR. Our reasons for this are: \
\
(A) INSERT ITEM HERE \
\
Please know that we are always up for discussion but this project is very active. Because of this, we're unlikely to see comments made on closed PRs, and we lock them after some time. If you or anyone else has any further questions, please let us know by using any of the communication methods listed in the page below: \
\
https://github.com/ansible/awx/#get-involved \
\
In the future, sometimes starting a discussion on the development list prior to implementing a feature can make getting things included a little easier, but it is not always necessary. \
\
Thank you once again for this and your interest in AWX!
### No Progress Issue
- Hi! \
\
Thank you very much for for this issue. It means a lot to us that you have taken time to contribute by opening this report. \
\
On this issue, there were comments added but it has been some time since then without response. At this time we are closing this issue. If you get time to address the comments we can reopen the issue if you can contact us by using any of the communication methods listed in the page below: \
\
https://github.com/ansible/awx/#get-involved \
\
Thank you once again for this and your interest in AWX!
### No Progress PR
- Hi! \
\
Thank you very much for your submission to AWX. It means a lot to us that you have taken time to contribute. \
\
On this PR, changes were requested but it has been some time since then. We think this PR has merit but without the requested changes we are unable to merge it. At this time we are closing your PR. If you get time to address the changes you are welcome to open another PR or we can reopen this PR upon request if you contact us by using any of the communication methods listed in the page below: \
\
https://github.com/ansible/awx/#get-involved \
\
Thank you once again for this and your interest in AWX!
## Common
### Give us more info
- Hello, we'd love to help, but we need a little more information about the problem you're having. Screenshots, log outputs, or any reproducers would be very helpful.
### Code of Conduct
- Hello. Please keep in mind that Ansible adheres to a Code of Conduct in its community spaces. The spirit of the code of conduct is to be kind, and this is your friendly reminder to be so. Please see the full code of conduct here if you have questions: https://docs.ansible.com/ansible/latest/community/code_of_conduct.html
### EE Contents / Community General
- Hello. The awx-ee contains the collections and dependencies needed for supported AWX features to function. Anything beyond that (like the community.general package) will require you to build your own EE. For information on how to do that, see https://ansible-builder.readthedocs.io/en/stable/ \
\
The Ansible Community is looking at building an EE that corresponds to all of the collections inside the ansible package. That may help you if and when it happens; see https://github.com/ansible-community/community-topics/issues/31 for details.
## Mailing List Triage
### Create an issue
- Hello, thanks for reaching out on list. We think this merits an issue on our Github, https://github.com/ansible/awx/issues. If you could open an issue up on Github it will get tagged and integrated into our planning and workflow. All future work will be tracked there. Issues should include as much information as possible, including screenshots, log outputs, or any reproducers.
### Create a Pull Request
- Hello, we think your idea is good! Please consider contributing a PR for this following our contributing guidelines: https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md
### Receptor
- You can find the receptor docs here: https://receptor.readthedocs.io/en/latest/
- Hello, your issue seems related to receptor. Could you please open an issue in the receptor repository? https://github.com/ansible/receptor. Thanks!
### Ansible Engine not AWX
- Hello, your question seems to be about Ansible development, not about AWX. Try asking on the Ansible-devel specific mailing list: https://groups.google.com/g/ansible-devel
- Hello, your question seems to be about using Ansible, not about AWX. https://groups.google.com/g/ansible-project is the best place to visit for user questions about Ansible. Thanks!
### Ansible Galaxy not AWX
- Hey there. That sounds like an FAQ question. Did this: https://www.ansible.com/products/awx-project/faq cover your question?
We'd be happy to help if you can reproduce this with AWX since we do not have Oracle's Linux Automation Manager. If you need help with this specific version of Oracles Linux Automation Manager you will need to contact your Oracle for support.
### AWX Release
Subject: Announcing AWX Xa.Ya.za and AWX-Operator Xb.Yb.zb
- Hi all, \
\
We're happy to announce that the next release of AWX, version <b>`Xa.Ya.za`</b> is now available! \
In addition AWX Operator version <b>`Xb.Yb.zb`</b> has also been released! \
This is a list of high-level changes for each release of AWX. A full list of commits can be found at `https://github.com/ansible/awx/releases/tag/<version>`.
**Note:** This file is deprecated and will be removed at some point in a future release.
## 13.0.0 (Jun 23, 2020)
- Added import and export subcommands to the awx-cli tool, replacing send and receive from the old tower-cli (https://github.com/ansible/awx/pull/6125).
- Removed scripts as a means of running inventory updates of built-in types (https://github.com/ansible/awx/pull/6911)
- Ansible 2.8 is now partially unsupported; some inventory source types are known to no longer work.
- Fixed an issue where the vmware inventory source ssl_verify source variable was not recognized (https://github.com/ansible/awx/pull/7360)
- Fixed a bug that caused redis' listen socket to have too-permissive file permissions (https://github.com/ansible/awx/pull/7317)
- Fixed a bug that caused rsyslogd's configuration file to have world-readable file permissions, potentially leaking secrets (CVE-2020-10782)
Starting with AWX 20, release notes are published to [GitHub Releases](https://github.com/ansible/awx/releases).
## 12.0.0 (Jun 9, 2020)
- Removed memcached as a dependency of AWX (https://github.com/ansible/awx/pull/7240)
- Moved to a single container image build instead of separate awx_web and awx_task images. The container image is just `awx` (https://github.com/ansible/awx/pull/7228)
- Official AWX container image builds now use a two-stage container build process that notably reduces the size of our published images (https://github.com/ansible/awx/pull/7017)
- Removed support for HipChat notifications ([EoL announcement](https://www.atlassian.com/partnerships/slack/faq#faq-98b17ca3-247f-423b-9a78-70a91681eff0)); all previously-created HipChat notification templates will be deleted due to this removal.
- Fixed a bug which broke AWX installations with oc version 4.3 (https://github.com/ansible/awx/pull/6948/files)
- Fixed a performance issue that caused notable delay of stdout processing for playbooks run against large numbers of hosts (https://github.com/ansible/awx/issues/6991)
- Fixed a bug that caused CyberArk AIM credential plugin looks to hang forever in some environments (https://github.com/ansible/awx/issues/6986)
- Fixed a bug that caused ANY/ALL converage settings not to properly save when editing approval nodes in the UI (https://github.com/ansible/awx/issues/6998)
- Fixed a bug that broke support for the satellite6_group_prefix source variable (https://github.com/ansible/awx/issues/7031)
- Fixed a bug that prevented changes to workflow node convergence settings when approval nodes were in use (https://github.com/ansible/awx/issues/7063)
- Fixed a bug that caused notifications to fail on newer version of Mattermost (https://github.com/ansible/awx/issues/7264)
- Fixed a bug (by upgrading to 0.8.1 of the foreman collection) that prevented host_filters from working properly with Foreman-based inventory (https://github.com/ansible/awx/issues/7225)
- Fixed a bug that prevented the usage of the Conjur credential plugin with secrets that contain spaces (https://github.com/ansible/awx/issues/7191)
- Fixed a bug in awx-manage run_wsbroadcast --status in kubernetes (https://github.com/ansible/awx/pull/7009)
- Fixed a bug that broke notification toggles for system jobs in the UI (https://github.com/ansible/awx/pull/7042)
- Fixed a bug that broke local pip installs of awxkit (https://github.com/ansible/awx/issues/7107)
- Fixed a bug that prevented PagerDuty notifications from sending for workflow job template approvals (https://github.com/ansible/awx/issues/7094)
- Fixed a bug that broke external log aggregation support for URL paths that include the = character (such as the tokens for SumoLogic) (https://github.com/ansible/awx/issues/7139)
- Fixed a bug that prevented organization admins from removing labels from workflow job templates (https://github.com/ansible/awx/pull/7143)
## 11.2.0 (Apr 29, 2020)
- Inventory updates now use collection-based plugins by default (in Ansible 2.9+):
- amazon.aws.aws_ec2
- community.vmware.vmware_vm_inventory
- azure.azcollection.azure_rm
- google.cloud.gcp_compute
- theforeman.foreman.foreman
- openstack.cloud.openstack
- ovirt.ovirt_collection.ovirt
- awx.awx.tower
- Added support for Approle and LDAP/AD mechanisms to the Hashicorp Vault credential plugin (https://github.com/ansible/awx/issues/5076)
- Added Project (Domain Name) support for the OpenStack Keystone v3 API (https://github.com/ansible/awx/issues/6831)
- Added a new setting for raising log verbosity for rsyslogd (https://github.com/ansible/awx/pull/6818)
- Added the ability to monitor stdout in the CLI for running jobs and workflow jobs (https://github.com/ansible/awx/issues/6165)
- Fixed a bug which prevented the AWX CLI from properly installing with newer versions of pip (https://github.com/ansible/awx/issues/6870)
- Fixed a bug which broke AWX's external logging support when configured with HTTPS endpoints that utilize self-signed certificates (https://github.com/ansible/awx/issues/6851)
- Fixed a local docker installer bug that mistakenly attempted to upgrade PostgreSQL when an external pg_hostname is specified (https://github.com/ansible/awx/pull/5398)
- Fixed a race condition that caused task container crashes when pods are quickly brought down and back up (https://github.com/ansible/awx/issues/6750)
- Fixed a bug that caused 404 errors when attempting to view the second page of the workflow approvals view (https://github.com/ansible/awx/issues/6803)
- Fixed a bug that prevented the use of ANSIBLE_SSH_ARGS for ad-hoc-commands (https://github.com/ansible/awx/pull/6811)
- Fixed a bug that broke AWX installs/upgrades on Red Hat OpenShift (https://github.com/ansible/awx/issues/6791)
## 11.1.0 (Apr 22, 2020)
- Changed rsyslogd to persist queued events to disk (to prevent a risk of out-of-memory errors) (https://github.com/ansible/awx/issues/6746)
- Added the ability to configure the destination and maximum disk size of rsyslogd spool (in the event of a log aggregator outage) (https://github.com/ansible/awx/pull/6763)
- Added the ability to discover playbooks in project clones from symlinked directories (https://github.com/ansible/awx/pull/6773)
- Fixed a bug that caused certain log aggregator settings to break logging integration (https://github.com/ansible/awx/issues/6760)
- Fixed a bug that caused playbook execution in container groups to sometimes unexpectedly deadlock (https://github.com/ansible/awx/issues/6692)
- Improved stability of the new redis clustering implementation (https://github.com/ansible/awx/pull/6739 https://github.com/ansible/awx/pull/6720)
- Improved stability of the new rsyslogd-based logging implementation (https://github.com/ansible/awx/pull/6796)
## 11.0.0 (Apr 16, 2020)
- As of AWX 11.0.0, Kubernetes-based deployments use a Deployment rather than a StatefulSet.
- Reimplemented external logging support using rsyslogd to improve reliability and address a number of issues (https://github.com/ansible/awx/issues/5155)
- Changed activity stream logs to include summary fields for related objects (https://github.com/ansible/awx/issues/1761)
- Added code to more gracefully attempt to reconnect to redis if it restarts/becomes unavailable (https://github.com/ansible/awx/pull/6670)
- Fixed a bug that caused REFRESH_TOKEN_EXPIRE_SECONDS to not properly be respected for OAuth2.0 refresh tokens generated by AWX (https://github.com/ansible/awx/issues/6630)
- Fixed a bug that broke schedules containing RRULES with very old DTSTART dates (https://github.com/ansible/awx/pull/6550)
- Fixed a bug that broke installs on older versions of Ansible packaged with certain Linux distributions (https://github.com/ansible/awx/issues/5501)
- Fixed a bug that caused the activity stream to sometimes report the incorrect actor when associating user membership on SAML login (https://github.com/ansible/awx/pull/6525)
- Fixed a bug in AWX's Grafana notification support when annotation tags are omitted (https://github.com/ansible/awx/issues/6580)
- Fixed a bug that prevented some users from searching for Source Control credentials in the AWX user interface (https://github.com/ansible/awx/issues/6600)
- Fixed a bug that prevented disassociating orphaned users from credentials (https://github.com/ansible/awx/pull/6554)
- Updated Twisted to address CVE-2020-10108 and CVE-2020-10109.
## 10.0.0 (Mar 30, 2020)
- As of AWX 10.0.0, the official AWX CLI no longer supports Python 2 (it requires at least Python 3.6) (https://github.com/ansible/awx/pull/6327)
- AWX no longer relies on RabbitMQ; Redis is added as a new dependency (https://github.com/ansible/awx/issues/5443)
- Altered AWX's event tables to allow more than ~2 billion total events (https://github.com/ansible/awx/issues/6010)
- Improved the performance (time to execute, and memory consumption) of the periodic job cleanup system job (https://github.com/ansible/awx/pull/6166)
- Updated Job Templates so they now have an explicit Organization field (it is no longer inferred from the associated Project) (https://github.com/ansible/awx/issues/3903)
- Updated social-auth-core to address an upcoming GitHub API deprecation (https://github.com/ansible/awx/issues/5970)
- Updated to ansible-runner 1.4.6 to address various bugs.
- Updated Django to address CVE-2020-9402
- Updated pyyaml version to address CVE-2017-18342
- Fixed a bug which prevented the new `scm_branch` field from being used in custom notification templates (https://github.com/ansible/awx/issues/6258)
- Fixed a race condition that sometimes causes success/failure notifications to include an incomplete list of hosts (https://github.com/ansible/awx/pull/6290)
- Fixed a bug that can cause certain setting pages to lose unsaved form edits when a playbook is launched (https://github.com/ansible/awx/issues/5265)
- Fixed a bug that can prevent the "Use TLS/SSL" field from properly saving when editing email notification templates (https://github.com/ansible/awx/issues/6383)
- Fixed a race condition that sometimes broke event/stdout processing for jobs launched in container groups (https://github.com/ansible/awx/issues/6280)
## 9.3.0 (Mar 12, 2020)
- Added the ability to specify an OAuth2 token description in the AWX CLI (https://github.com/ansible/awx/issues/6122)
- Added support for K8S service account annotations to the installer (https://github.com/ansible/awx/pull/6007)
- Added support for K8S imagePullSecrets to the installer (https://github.com/ansible/awx/pull/5989)
- Launching jobs (and workflows) using the --monitor flag in the AWX CLI now returns a non-zero exit code on job failure (https://github.com/ansible/awx/issues/5920)
- Improved UI performance for various job views when many simultaneous users are logged into AWX (https://github.com/ansible/awx/issues/5883)
- Updated to the latest version of Django to address a few open CVEs (https://github.com/ansible/awx/pull/6080)
- Fixed a critical bug which can cause AWX to hang and stop launching playbooks after a periodic of time (https://github.com/ansible/awx/issues/5617)
- Fixed a bug which caused delays in project update stdout for certain large SCM clones (as of Ansible 2.9+) (https://github.com/ansible/awx/pull/6254)
- Fixed a bug which caused certain smart inventory filters to mistakenly return duplicate hosts (https://github.com/ansible/awx/pull/5972)
- Fixed an unclear server error when creating smart inventories with the AWX collection (https://github.com/ansible/awx/issues/6250)
- Fixed a bug that broke Grafana notification support (https://github.com/ansible/awx/issues/6137)
- Fixed a UI bug which prevent users with read access to an organization from editing credentials for that organization (https://github.com/ansible/awx/pull/6241)
- Fixed a bug which prevent workflow approval records from recording a `started` and `elapsed` date (https://github.com/ansible/awx/issues/6202)
- Fixed a bug which caused workflow nodes to have a confusing option for `verbosity` (https://github.com/ansible/awx/issues/6196)
- Fixed an RBAC bug which prevented projects and inventory schedules from being created by certain users in certain contexts (https://github.com/ansible/awx/issues/5717)
- Fixed a bug that caused `role_path` in a project's config to not be respected due to an error processing `/etc/ansible/ansible.cfg` (https://github.com/ansible/awx/pull/6038)
- Fixed a bug that broke inventory updates for installs with custom home directories for the awx user (https://github.com/ansible/awx/pull/6152)
- Fixed a bug that broke fact data collection when AWX encounters invalid/unexpected fact data (https://github.com/ansible/awx/issues/5935)
## 9.2.0 (Feb 12, 2020)
- Added the ability to configure the convergence behavior of workflow nodes https://github.com/ansible/awx/issues/3054
- AWX now allows for a configurable global limit for fork count (per-job run). The default maximum is 200. https://github.com/ansible/awx/pull/5604
- Added the ability to specify AZURE_PUBLIC_CLOUD (for e.g., Azure Government KeyVault support) for the Azure credential plugin https://github.com/ansible/awx/issues/5138
- Added support for several additional parameters for Satellite dynamic inventory https://github.com/ansible/awx/pull/5598
- Added a new field to jobs for tracking the date/time a job is cancelled https://github.com/ansible/awx/pull/5610
- Made a series of additional optimizations to the callback receiver to further improve stdout write speed for running playbooks https://github.com/ansible/awx/pull/5677 https://github.com/ansible/awx/pull/5739
- Updated AWX to be compatible with Helm 3.x (https://github.com/ansible/awx/pull/5776)
- Optimized AWX's job dependency/scheduling code to drastically improve processing time in scenarios where there are many pending jobs scheduled simultaneously https://github.com/ansible/awx/issues/5154
- Fixed a bug which could cause SCM authentication details (basic auth passwords) to be reported to external loggers in certain failure scenarios (e.g., when a git clone fails and ansible itself prints an error message to stdout) https://github.com/ansible/awx/pull/5812
- Fixed a k8s installer bug that caused installs to fail in certain situations https://github.com/ansible/awx/issues/5574
- Fixed a number of issues that caused analytics gathering and reporting to run more often than necessary https://github.com/ansible/awx/pull/5721
- Fixed a bug in the AWX CLI that prevented JSON-type settings from saving properly https://github.com/ansible/awx/issues/5528
- Improved support for fetching custom virtualenv dependencies when AWX is installed behind a proxy https://github.com/ansible/awx/pull/5805
- Updated the bundled version of openstacksdk to address a known issue https://github.com/ansible/awx/issues/5821
- Updated the bundled vmware_inventory plugin to the latest version to address a bug https://github.com/ansible/awx/pull/5668
- Fixed a bug that can cause inventory updates to fail to properly save their output when run within a workflow https://github.com/ansible/awx/pull/5666
- Removed a number of pre-computed fields from the Host and Group models to improve AWX performance. As part of this change, inventory group UIs throughout the interface no longer display status icons https://github.com/ansible/awx/pull/5448
## 9.1.1 (Jan 14, 2020)
- Fixed a bug that caused database migrations on Kubernetes installs to hang https://github.com/ansible/awx/pull/5579
- Upgraded Python-level app dependencies in AWX virtual environment https://github.com/ansible/awx/pull/5407
- Running jobs no longer block associated inventory updates https://github.com/ansible/awx/pull/5519
- Optimized the callback receiver to drastically improve the write speed of stdout for parallel jobs (https://github.com/ansible/awx/pull/5618)
## 9.1.0 (Dec 17, 2019)
- Added a command to generate a new SECRET_KEY and rekey the secrets in the database
- Removed project update locking when jobs using it are running
- Fixed slow queries for /api/v2/instances and /api/v2/instance_groups when smart inventories are used
- Fixed a partial password disclosure when special characters existed in the RabbitMQ password (CVE-2019-19342)
- Fixed hang in error handling for source control checkouts
- Fixed an error on subsequent job runs that override the branch of a project on an instance that did not have a prior project checkout
- Fixed an issue where jobs launched in isolated or container groups would incorrectly timeout
- Fixed an incorrect link to instance groups documentation in the user interface
- Fixed editing of inventory on Workflow templates
- Fixed multiple issues with OAuth2 token cleanup system jobs
- Fixed a bug that broke email notifications for workflow approval/deny https://github.com/ansible/awx/issues/5401
- Updated SAML implementation to automatically login if authorization already exists
- Updated AngularJS to 1.7.9 for CVE-2019-10768
## 9.0.1 (Nov 4, 2019)
- Fixed a bug in the installer that broke certain types of k8s installs https://github.com/ansible/awx/issues/5205
## 9.0.0 (Oct 31, 2019)
- Updated AWX images to use centos:8 as the parent image.
- Updated to ansible-runner 1.4.4 to address various bugs.
- Added oc and kubectl to the AWX images to support new container-based execution introduced in 8.0.0.
- Added some optimizations to speed up the deletion of large Inventory Groups.
- Fixed a bug that broke webhook launches for Job Templates that define a survey (https://github.com/ansible/awx/issues/5062).
- Fixed a bug in the CLI which incorrectly parsed launch time arguments for `awx job_templates launch` and `awx workflow_job_templates launch` (https://github.com/ansible/awx/issues/5093).
- Fixed a bug that caused inventory updates using "sourced from a project" to stop working (https://github.com/ansible/awx/issues/4750).
- Fixed a bug that caused Slack notifications to sometimes show the wrong bot avatar (https://github.com/ansible/awx/pull/5125).
- Fixed a bug that prevented the use of digits in AWX's URL settings (https://github.com/ansible/awx/issues/5081).
## 8.0.0 (Oct 21, 2019)
- The Ansible Tower Ansible modules have been migrated to a new official Ansible AWX collection: https://galaxy.ansible.com/awx/AWX
Please note that this functionality is only supported in Ansible 2.9+
- AWX now supports the ability to launch jobs from external webhooks (GitHub and GitLab integration are supported).
- AWX now supports Container Groups, a new feature that allows you to schedule and run playbooks on single-use kubernetes pods on-demand.
- AWX now supports sending notifications when Workflow steps are approved, denied, or time out.
- AWX now records the user who approved or denied Workflow steps.
- AWX now supports fetching Ansible Collections from private galaxy servers.
- AWX now checks the user's ansible.cfg for paths where role/collections may live when running project updates.
- AWX now uses PostgreSQL 10 by default.
- AWX now warns more loudly about underlying AMQP connectivity issues (https://github.com/ansible/awx/pull/4857).
- Added a few optimizations to drastically improve dashboard performance for larger AWX installs (installs with several hundred thousand jobs or more).
- Updated to the latest version of Ansible's VMWare inventory script (which adds support for vmware_guest_facts).
- Deprecated /api/v2/inventory_scripts/ (this endpoint - and the Custom Inventory Script feature - will be removed in a future release of AWX).
- Fixed a bug which prevented Organization Admins from removing users from their own Organization (https://github.com/ansible/awx/issues/2979)
- Fixed a bug which sometimes caused cluster nodes to fail to re-join with a cryptic error, "No instance found with the current cluster host id" (https://github.com/ansible/awx/issues/4294)
- Fixed a bug that prevented the use of launch-time passphrases when using credential plugins (https://github.com/ansible/awx/pull/4807)
- Fixed a bug that caused notifications assigned at the Organization level not to take effect for Workflows in that Organization (https://github.com/ansible/awx/issues/4712)
- Fixed a bug which caused a notable amount of CPU overhead on RabbitMQ health checks (https://github.com/ansible/awx/pull/5009)
- Fixed a bug which sometimes caused the <return> key to stop functioning in <textarea> elements (https://github.com/ansible/awx/issues/4192)
- Fixed a bug which caused request contention when the same OAuth2.0 token was used in multiple simultaneous requests (https://github.com/ansible/awx/issues/4694)
- Fixed a bug related to parsing multiple choice survey options (https://github.com/ansible/awx/issues/4452).
- Fixed a bug that caused single-sign-on icons on the login page to fail to render in certain Windows browsers (https://github.com/ansible/awx/issues/3924)
- Fixed a number of bugs that caused certain OAuth2 settings to not be properly respected, such as REFRESH_TOKEN_EXPIRE_SECONDS.
- Fixed a number of bugs in the AWX CLI, including a bug which sometimes caused long lines of stdout output to be unexpectedly truncated.
- Fixed a number of bugs on the job details UI which sometimes caused auto-scrolling stdout to become stuck.
- Fixed a bug which caused LDAP authentication to fail if the TLD of the server URL contained digits (https://github.com/ansible/awx/issues/3646)
- Fixed a bug which broke HashiCorp Vault integration on older versions of HashiCorp Vault.
## 7.0.0 (Sept 4, 2019)
- AWX now detects and installs Ansible Collections defined in your project (note - this feature only works in Ansible 2.9+) (https://github.com/ansible/awx/issues/2534)
- AWX now includes an official command line client. Keep an eye out for a follow-up email on this mailing list for information on how to install it and try it out.
- Added the ability to provide a specific SCM branch on jobs (https://github.com/ansible/awx/issues/282)
- Added support for Workflow Approval Nodes, a new feature which allows you to add "pause and wait for approval" steps into your workflows (https://github.com/ansible/awx/issues/1206)
- Added the ability to specify a specific HTTP method for webhook notifications (POST vs PUT) (https://github.com/ansible/awx/pull/4124)
- Added the ability to specify a username and password for HTTP Basic Authorization for webhook notifications (https://github.com/ansible/awx/pull/4124)
- Added support for customizing the text content of notifications (https://github.com/ansible/awx/issues/79)
- Added the ability to enable and disable hosts in dynamic inventory (https://github.com/ansible/awx/pull/4420)
- Added the description (if any) to the Job Template list (https://github.com/ansible/awx/issues/4359)
- Added new metrics for instance hostnames and pending jobs to the /api/v2/metrics/ endpoint (https://github.com/ansible/awx/pull/4375)
- Changed AWX's on/off toggle buttons to a non-text based style to simplify internationalization (https://github.com/ansible/awx/pull/4425)
- Events emitted by ansible for adhoc commands are now sent to the external log aggregrator (https://github.com/ansible/awx/issues/4545)
- Fixed a bug which allowed a user to make an organization credential in another organization without permissions to that organization (https://github.com/ansible/awx/pull/4483)
- Fixed a bug that caused `extra_vars` on workflows to break when edited (https://github.com/ansible/awx/issues/4293)
- Fixed a slow SQL query that caused performance issues when large numbers of groups exist (https://github.com/ansible/awx/issues/4461)
- Fixed a few minor bugs in survey field validation (https://github.com/ansible/awx/pull/4509) (https://github.com/ansible/awx/pull/4479)
- Fixed a bug that sometimes resulted in orphaned `ansible_runner_pi` directories in `/tmp` after playbook execution (https://github.com/ansible/awx/pull/4409)
- Fixed a bug that caused the `is_system_auditor` flag in LDAP configuration to not work (https://github.com/ansible/awx/pull/4396)
- Fixed a bug which caused schedules to disappear from the UI when toggled off (https://github.com/ansible/awx/pull/4378)
- Fixed a bug that sometimes caused stdout content to contain extraneous blank lines in newer versions of Ansible (https://github.com/ansible/awx/pull/4391)
- Updated to the latest Django security release, 2.2.4 (https://github.com/ansible/awx/pull/4410) (https://www.djangoproject.com/weblog/2019/aug/01/security-releases/)
- Updated the default version of git to a version that includes support for x509 certificates (https://github.com/ansible/awx/issues/4362)
- Removed the deprecated `credential` field from `/api/v2/workflow_job_templates/N/` (as part of the `/api/v1/` removal in prior AWX versions - https://github.com/ansible/awx/pull/4490).
## 6.1.0 (Jul 18, 2019)
- Updated AWX to use Django 2.2.2.
- Updated the provided openstacksdk version to support new functionality (such as Nova scheduler_hints)
- Added the ability to specify a custom cacert for the HashiCorp Vault credential plugin
- Fixed a number of bugs related to path lookups for the HashiCorp Vault credential plugin
- Fixed a bug which prevented signed SSH certificates from working, including the HashiCorp Vault Signed SSH backend
- Fixed a bug which prevented custom logos from displaying on the login page (as a result of a new Content Security Policy in 6.0.0)
- Fixed a bug which broke websocket connectivity in Apple Safari (as a result of a new Content Security Policy in 6.0.0)
- Fixed a bug on the job output page that occasionally caused the "up" and "down" buttons to not load additional output
- Fixed a bug on the job output page that caused quoted task names to display incorrectly
## 6.0.0 (Jul 1, 2019)
- Removed support for "Any" notification templates and their API endpoints e.g., /api/v2/job_templates/N/notification_templates/any/ (https://github.com/ansible/awx/issues/4022)
- Fixed a bug which prevented credentials from properly being applied to inventory sources (https://github.com/ansible/awx/issues/4059)
- Fixed a bug which can cause the task dispatcher to hang indefinitely when external logging support (e.g., Splunk, Logstash) is enabled (https://github.com/ansible/awx/issues/4181)
- Fixed a bug which causes slow stdout display when running jobs against smart inventories. (https://github.com/ansible/awx/issues/3106)
- Fixed a bug that caused SSL verification flags to fail to be respected for LDAP authentication in certain environments. (https://github.com/ansible/awx/pull/4190)
- Added a simple Content Security Policy (https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP) to restrict access to third-party resources in the browser. (https://github.com/ansible/awx/pull/4167)
- Updated ovirt4 library dependencies to work with newer versions of oVirt (https://github.com/ansible/awx/issues/4138)
## 5.0.0 (Jun 21, 2019)
- Bump Django Rest Framework from 3.7.7 to 3.9.4
- Bump setuptools / pip dependencies
- Fixed bug where Recent Notification list would not appear
- Added notifications on job start
- Default to Ansible 2.8
For older release notes, see https://github.com/ansible/awx/blob/19.3.0/CHANGELOG.md.
Hi there! We're excited to have you as a contributor.
Have questions about this document or anything not covered here? Come chat with us at `#ansible-awx` on irc.freenode.net, or submit your question to the [mailing list](https://groups.google.com/forum/#!forum/awx-project).
Have questions about this document or anything not covered here? Come chat with us at `#ansible-awx` on irc.libera.chat, or submit your question to the [mailing list](https://groups.google.com/forum/#!forum/awx-project).
## Table of contents
* [Things to know prior to submitting code](#things-to-know-prior-to-submitting-code)
* [Setting up your development environment](#setting-up-your-development-environment)
* [Prerequisites](#prerequisites)
* [Docker](#docker)
* [Docker compose](#docker-compose)
* [Node and npm](#node-and-npm)
* [Build the environment](#build-the-environment)
* [Fork and clone the AWX repo](#fork-and-clone-the-awx-repo)
* [Create local settings](#create-local-settings)
* [Build the base image](#build-the-base-image)
* [Build the user interface](#build-the-user-interface)
* [Running the environment](#running-the-environment)
* [Start the containers](#start-the-containers)
* [Start from the container shell](#start-from-the-container-shell)
* [Post Build Steps](#post-build-steps)
* [Start a shell](#start-a-shell)
* [Create a superuser](#create-a-superuser)
* [Load the data](#load-the-data)
* [Building API Documentation](#build-api-documentation)
* [Accessing the AWX web interface](#accessing-the-awx-web-interface)
* [Purging containers and images](#purging-containers-and-images)
* [What should I work on?](#what-should-i-work-on)
- All code submissions are done through pull requests against the `devel` branch.
- You must use `git commit --signoff` for any commit to be merged, and agree that usage of --signoff constitutes agreement with the terms of [DCO 1.1](./DCO_1_1.md).
- Take care to make sure no merge commits are in the submission, and use `git rebase` vs `git merge` for this reason.
- If collaborating with someone else on the same branch, consider using `--force-with-lease` instead of `--force`. This will prevent you from accidentally overwriting commits pushed by someone else. For more information, see https://git-scm.com/docs/git-push#git-push---force-with-leaseltrefnamegt
- If submitting a large code change, it's a good idea to join the `#ansible-awx` channel on irc.freenode.net, and talk about what you would like to do or add first. This not only helps everyone know what's going on, it also helps save time and effort, if the community decides some changes are needed.
- If collaborating with someone else on the same branch, consider using `--force-with-lease` instead of `--force`. This will prevent you from accidentally overwriting commits pushed by someone else. For more information, see [git push docs](https://git-scm.com/docs/git-push#git-push---force-with-leaseltrefnamegt).
- If submitting a large code change, it's a good idea to join the `#ansible-awx` channel on irc.libera.chat, and talk about what you would like to do or add first. This not only helps everyone know what's going on, it also helps save time and effort, if the community decides some changes are needed.
- We ask all of our community members and contributors to adhere to the [Ansible code of conduct](http://docs.ansible.com/ansible/latest/community/code_of_conduct.html). If you have questions, or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)
## Setting up your development environment
The AWX development environment workflow and toolchain is based on Docker, and the docker-compose tool, to provide dependencies, services, and databases necessary to run all of the components. It also binds the local source tree into the development container, making it possible to observe and test changes in real time.
The AWX development environment workflow and toolchain uses Docker and the docker-compose tool, to provide dependencies, services, and databases necessary to run all of the components. It also bind-mounts the local source tree into the development container, making it possible to observe and test changes in real time.
### Prerequisites
@@ -50,241 +43,55 @@ The AWX development environment workflow and toolchain is based on Docker, and t
Prior to starting the development services, you'll need `docker` and `docker-compose`. On Linux, you can generally find these in your distro's packaging, but you may find that Docker themselves maintain a separate repo that tracks more closely to the latest releases.
For macOS and Windows, we recommend [Docker for Mac](https://www.docker.com/docker-mac) and [Docker for Windows](https://www.docker.com/docker-windows)
respectively.
For macOS and Windows, we recommend [Docker for Mac](https://www.docker.com/docker-mac) and [Docker for Windows](https://www.docker.com/docker-windows) respectively.
For Linux platforms, refer to the following from Docker:
If you're not using Docker for Mac, or Docker for Windows, you may need, or choose to, install the Docker compose Python module separately, in which case you'll need to run the following:
If you're not using Docker for Mac, or Docker for Windows, you may need, or choose to, install the `docker-compose` Python module separately.
```bash
(host)$ pip install docker-compose
(host)$ pip3 install docker-compose
```
#### Frontend Development
See [the ui development documentation](awx/ui/README.md).
### Build the environment
See [the ui development documentation](awx/ui/CONTRIBUTING.md).
#### Fork and clone the AWX repo
If you have not done so already, you'll need to fork the AWX repo on GitHub. For more on how to do this, see [Fork a Repo](https://help.github.com/articles/fork-a-repo/).
#### Create local settings
### Build and Run the Development Environment
AWX will import the file `awx/settings/local_settings.py` and combine it with defaults in `awx/settings/defaults.py`. This file is required for starting the development environment and startup will fail if it's not provided.
See the [README.md](./tools/docker-compose/README.md) for docs on how to build the awx_devel image and run the development environment.
An example is provided. Make a copy of it, and edit as needed (the defaults are usually fine):
The AWX base container image (defined in `tools/docker-compose/Dockerfile`) contains basic OS dependencies and symbolic links into the development environment that make running the services easy.
Run the following to build the image:
```bash
(host)$ make docker-compose-build
```
**NOTE**
> The image will need to be rebuilt, if the Python requirements or OS dependencies change.
Once the build completes, you will have a `ansible/awx_devel` image in your local image cache. Use the `docker images` command to view it, as follows:
```bash
(host)$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ansible/awx_devel latest ba9ec3e8df74 26 minutes ago 1.42GB
```
#### Build the user interface
Run the following to build the AWX UI:
```bash
(host) $ make ui-devel
```
See [the ui development documentation](awx/ui/README.md) for more information on using the frontend development, build, and test tooling.
### Running the environment
#### Start the containers
Start the development containers by running the following:
```bash
(host)$ make docker-compose
```
The above utilizes the image built in the previous step, and will automatically start all required services and dependent containers. Once the containers launch, your session will be attached to the *awx* container, and you'll be able to watch log messages and events in real time. You will see messages from Django and the front end build process.
If you start a second terminal session, you can take a look at the running containers using the `docker ps` command. For example:
```bash
# List running containers
(host)$ docker ps
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
44251b476f98 gcr.io/ansible-tower-engineering/awx_devel:devel "/entrypoint.sh /bin…"27 seconds ago Up 23 seconds 0.0.0.0:6899->6899/tcp, 0.0.0.0:7899-7999->7899-7999/tcp, 0.0.0.0:8013->8013/tcp, 0.0.0.0:8043->8043/tcp, 0.0.0.0:8080->8080/tcp, 22/tcp, 0.0.0.0:8888->8888/tcp tools_awx_run_9e820694d57e
40de380e3c2e redis:latest "docker-entrypoint.s…"28 seconds ago Up 26 seconds
b66a506d3007 postgres:10 "docker-entrypoint.s…"28 seconds ago Up 26 seconds 0.0.0.0:5432->5432/tcp tools_postgres_1
```
**NOTE**
> The Makefile assumes that the image you built is tagged with your current branch. This allows you to build images for different contexts or branches. When starting the containers, you can choose a specific branch by setting `COMPOSE_TAG=<branch name>` in your environment.
> For example, you might be working in a feature branch, but you want to run the containers using the `devel` image you built previously. To do that, start the containers using the following command: `$ COMPOSE_TAG=devel make docker-compose`
##### Wait for migrations to complete
The first time you start the environment, database migrations need to run in order to build the PostgreSQL database. It will take few moments, but eventually you will see output in your terminal session that looks like the following:
awx_1 | Apply all migrations: sso, taggit, sessions, sites, kombu_transport_django, social_auth, contenttypes, auth, conf, main
awx_1 | Synchronizing apps without migrations:
awx_1 | Creating tables...
awx_1 | Running deferred SQL...
awx_1 | Installing custom SQL...
awx_1 | Running migrations:
awx_1 | Rendering model states... DONE
awx_1 | Applying contenttypes.0001_initial... OK
awx_1 | Applying contenttypes.0002_remove_content_type_name... OK
awx_1 | Applying auth.0001_initial... OK
awx_1 | Applying auth.0002_alter_permission_name_max_length... OK
awx_1 | Applying auth.0003_alter_user_email_max_length... OK
awx_1 | Applying auth.0004_alter_user_username_opts... OK
awx_1 | Applying auth.0005_alter_user_last_login_null... OK
awx_1 | Applying auth.0006_require_contenttypes_0002... OK
awx_1 | Applying taggit.0001_initial... OK
awx_1 | Applying taggit.0002_auto_20150616_2121... OK
awx_1 | Applying main.0001_initial... OK
awx_1 | Applying main.0002_squashed_v300_release... OK
awx_1 | Applying main.0003_squashed_v300_v303_updates... OK
awx_1 | Applying main.0004_squashed_v310_release... OK
awx_1 | Applying conf.0001_initial... OK
awx_1 | Applying conf.0002_v310_copy_tower_settings... OK
...
```
Once migrations are completed, you can begin using AWX.
#### Start from the container shell
Often times you'll want to start the development environment without immediately starting all of the services in the *awx* container, and instead be taken directly to a shell. You can do this with the following:
```bash
(host)$ make docker-compose-test
```
Using `docker exec`, this will create a session in the running *awx* container, and place you at a command prompt, where you can run shell commands inside the container.
If you want to start and use the development environment, you'll first need to bootstrap it by running the following command:
```bash
(container)# /usr/bin/bootstrap_development.sh
```
The above will do all the setup tasks, including running database migrations, so it may take a couple minutes. Once it's done it
will drop you back to the shell.
In order to launch all developer services:
```bash
(container)# /usr/bin/launch_awx.sh
```
`launch_awx.sh` also calls `bootstrap_development.sh` so if all you are doing is launching the supervisor to start all services, you don't
need to call `bootstrap_development.sh` first.
### Post Build Steps
Before you can log in and use the system, you will need to create an admin user. Optionally, you may also want to load some demo data.
##### Start a shell
To create the admin user, and load demo data, you first need to start a shell session on the *awx* container. In a new terminal session, use the `docker exec` command as follows to start the shell session:
```bash
(host)$ docker exec -it tools_awx_1 bash
```
This creates a session in the *awx* containers, just as if you were using `ssh`, and allows you execute commands within the running container.
##### Create an admin user
Before you can log into AWX, you need to create an admin user. With this user you will be able to create more users, and begin configuring the server. From within the container shell, run the following command:
```bash
(container)# awx-manage createsuperuser
```
You will be prompted for a username, an email address, and a password, and you will be asked to confirm the password. The email address is not important, so just enter something that looks like an email address. Remember the username and password, as you will use them to log into the web interface for the first time.
##### Load demo data
You can optionally load some demo data. This will create a demo project, inventory, and job template. From within the container shell, run the following to load the data:
```bash
(container)# awx-manage create_preload_data
```
**NOTE**
> This information will persist in the database running in the `tools_postgres_1` container, until the container is removed. You may periodically need to recreate
this container, and thus the database, if the database schema changes in an upstream commit.
##### Building API Documentation
AWX includes support for building [Swagger/OpenAPI
documentation](https://swagger.io). To build the documentation locally, run:
AWX includes support for building [Swagger/OpenAPI documentation](https://swagger.io). To build the documentation locally, run:
```bash
(container)/awx_devel$ make swagger
```
This will write a file named `swagger.json` that contains the API specification
in OpenAPI format. A variety of online tools are available for translating
this data into more consumable formats (such as HTML). http://editor.swagger.io
is an example of one such service.
This will write a file named `swagger.json` that contains the API specification in OpenAPI format. A variety of online tools are available for translating this data into more consumable formats (such as HTML). http://editor.swagger.io is an example of one such service.
### Accessing the AWX web interface
You can now log into the AWX web interface at [https://localhost:8043](https://localhost:8043), and access the API directly at [https://localhost:8043/api/](https://localhost:8043/api/).
To log in use the admin user and password you created above in [Create an admin user](#create-an-admin-user).
[Create an admin user](./tools/docker-compose/README.md#create-an-admin-user) if needed.
### Purging containers and images
@@ -294,64 +101,71 @@ When necessary, remove any AWX containers and images by running the following:
(host)$ make docker-clean
```
### Pre commit hooks
When you attempt to perform a `git commit` there will be a pre-commit hook that gets run before the commit is allowed to your local repository. For example, python's [black](https://pypi.org/project/black/) will be run to test the formatting of any python files.
While you can use environment variables to skip the pre-commit hooks GitHub will run similar tests and prevent merging of PRs if the tests do not pass.
If you would like to add additional commit hooks for your own usage you can create a directory in the root of the repository called `pre-commit-user`. Any executable file in that directory will be executed as part of the pre-commit hooks. If any of the pre-commit checks fail the commit will be halted. For your convenience in user scripts, a variable called `CHANGED_FILES` will be set with any changed files present in the commit.
## What should I work on?
We have a ["good first issue" label](https://github.com/ansible/awx/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) we put on some issues that might be a good starting point for new contributors.
Fixing bugs and updating the documentation are always appreciated, so reviewing the backlog of issues is always a good place to start.
For feature work, take a look at the current [Enhancements](https://github.com/ansible/awx/issues?q=is%3Aissue+is%3Aopen+label%3Atype%3Aenhancement).
If it has someone assigned to it then that person is the person responsible for working the enhancement. If you feel like you could contribute then reach out to that person.
Fixing bugs, adding translations, and updating the documentation are always appreciated, so reviewing the backlog of issues is always a good place to start. For extra information on debugging tools, see [Debugging](https://github.com/ansible/awx/blob/devel/docs/debugging.md).
**NOTES**
**NOTE**
> Issue assignment will only be done for maintainers of the project. If you decide to work on an issue, please feel free to add a comment in the issue to let others know that you are working on it; but know that we will accept the first pull request from whomever is able to fix an issue. Once your PR is accepted we can add you as an assignee to an issue upon request.
> If you work in a part of the codebase that is going through active development, your changes may be rejected, or you may be asked to `rebase`. A good idea before starting work is to have a discussion with us in the `#ansible-awx` channel on irc.freenode.net, or on the [mailing list](https://groups.google.com/forum/#!forum/awx-project).
**NOTE**
> If you work in a part of the codebase that is going through active development, your changes may be rejected, or you may be asked to `rebase`. A good idea before starting work is to have a discussion with us in the `#ansible-awx` channel on irc.libera.chat, or on the [mailing list](https://groups.google.com/forum/#!forum/awx-project).
> If you're planning to develop features or fixes for the UI, please review the [UI Developer doc](./awx/ui/README.md).
### Translations
At this time we do not accept PRs for adding additional language translations as we have an automated process for generating our translations. This is because translations require constant care as new strings are added and changed in the code base. Because of this the .po files are overwritten during every translation release cycle. We also can't support a lot of translations on AWX as its an open source project and each language adds time and cost to maintain. If you would like to see AWX translated into a new language please create an issue and ask others you know to upvote the issue. Our translation team will review the needs of the community and see what they can do around supporting additional language.
If you find an issue with an existing translation, please see the [Reporting Issues](#reporting-issues) section to open an issue and our translation team will work with you on a resolution.
## Submitting Pull Requests
Fixes and Features for AWX will go through the Github pull request process. Submit your pull request (PR) against the `devel` branch.
Here are a few things you can do to help the visibility of your change, and increase the likelihood that it will be accepted:
* No issues when running linters/code checkers
* Python: flake8: `(container)/awx_devel$ make flake8`
* Javascript: JsHint:`(container)/awx_devel$ make jshint`
* No issues from unit tests
* Python: py.test: `(container)/awx_devel$ make test`
* JavaScript: Jasmine:`(container)/awx_devel$ make ui-test-ci`
* Write tests for new functionality, update/add tests for bug fixes
* Make the smallest change possible
* Write good commit messages. See [How to write a Git commit message](https://chris.beams.io/posts/git-commit/).
- No issues when running linters/code checkers
- Python: black: `(container)/awx_devel$ make black`
- Javascript: `(container)/awx_devel$ make ui-lint`
- No issues from unit tests
- Python: py.test: `(container)/awx_devel$ make test`
- JavaScript: `(container)/awx_devel$ make ui-test`
- Write tests for new functionality, update/add tests for bug fixes
- Make the smallest change possible
- Write good commit messages. See [How to write a Git commit message](https://chris.beams.io/posts/git-commit/).
It's generally a good idea to discuss features with us first by engaging us in the `#ansible-awx` channel on irc.freenode.net, or on the [mailing list](https://groups.google.com/forum/#!forum/awx-project).
It's generally a good idea to discuss features with us first by engaging us in the `#ansible-awx` channel on irc.libera.chat, or on the [mailing list](https://groups.google.com/forum/#!forum/awx-project).
We like to keep our commit history clean, and will require resubmission of pull requests that contain merge commits. Use `git pull --rebase`, rather than
`git pull`, and `git rebase`, rather than `git merge`.
Sometimes it might take us a while to fully review your PR. We try to keep the `devel` branch in good working order, and so we review requests carefully. Please be patient.
All submitted PRs will have the linter and unit tests run against them via Zuul, and the status reported in the PR.
## PR Checks ran by Zuul
Zuul jobs for awx are defined in the [zuul-jobs](https://github.com/ansible/zuul-jobs) repo.
Zuul runs the following checks that must pass:
1) `tox-awx-api-lint`
2) `tox-awx-ui-lint`
3) `tox-awx-api`
4) `tox-awx-ui`
5) `tox-awx-swagger`
Zuul runs the following checks that are non-voting (can not pass but serve to inform PR reviewers):
1) `tox-awx-detect-schema-change`
This check generates the schema and diffs it against a reference copy of the `devel` version of the schema.
Reviewers should inspect the `job-output.txt.gz` related to the check if their is a failure (grep for `diff -u -b` to find beginning of diff).
If the schema change is expected and makes sense in relation to the changes made by the PR, then you are good to go!
If not, the schema changes should be fixed, but this decision must be enforced by reviewers.
When your PR is initially submitted the checks will not be run until a maintainer allows them to be. Once a maintainer has done a quick review of your work the PR will have the linter and unit tests run against them via GitHub Actions, and the status reported in the PR.
## Reporting Issues
We welcome your feedback, and encourage you to file an issue when you run into a problem. But before opening a new issues, we ask that you please view our [Issues guide](./ISSUES.md).
## Getting Help
If you require additional assistance, please reach out to us at `#ansible-awx` on irc.libera.chat, or submit your question to the [mailing list](https://groups.google.com/forum/#!forum/awx-project).
For extra information on debugging tools, see [Debugging](./docs/debugging/).
* [Installing the AWX CLI](#installing-the-awx-cli)
* [Building the CLI Documentation](#building-the-cli-documentation)
# Installing AWX
This document provides a guide for installing AWX.
:warning: NOTE |
--- |
If you're installing an older release of AWX (prior to 18.0), these instructions have changed. Take a look at your version specific instructions, e.g., for AWX 17.0.1, see: [https://github.com/ansible/awx/blob/17.0.1/INSTALL.md](https://github.com/ansible/awx/blob/17.0.1/INSTALL.md)
If you're attempting to migrate an older Docker-based AWX installation, see: [Migrating Data from Local Docker](https://github.com/ansible/awx/blob/devel/tools/docker-compose/docs/data_migration.md) |
## Table of contents
## The AWX Operator
- [Installing AWX](#installing-awx)
* [Getting started](#getting-started)
+ [Clone the repo](#clone-the-repo)
+ [AWX branding](#awx-branding)
+ [Prerequisites](#prerequisites)
+ [System Requirements](#system-requirements)
+ [Choose a deployment platform](#choose-a-deployment-platform)
+ [Official vs Building Images](#official-vs-building-images)
* [Upgrading from previous versions](#upgrading-from-previous-versions)
* [OpenShift](#openshift)
+ [Prerequisites](#prerequisites-1)
+ [Pre-install steps](#pre-install-steps)
- [Deploying to Minishift](#deploying-to-minishift)
- [PostgreSQL](#postgresql)
+ [Run the installer](#run-the-installer)
+ [Post-install](#post-install)
+ [Accessing AWX](#accessing-awx)
* [Kubernetes](#kubernetes)
+ [Prerequisites](#prerequisites-2)
+ [Pre-install steps](#pre-install-steps-1)
+ [Configuring Helm](#configuring-helm)
+ [Run the installer](#run-the-installer-1)
+ [Post-install](#post-install-1)
+ [Accessing AWX](#accessing-awx-1)
+ [SSL Termination](#ssl-termination)
* [Docker-Compose](#docker-compose)
+ [Prerequisites](#prerequisites-3)
+ [Pre-install steps](#pre-install-steps-2)
- [Deploying to a remote host](#deploying-to-a-remote-host)
- [Inventory variables](#inventory-variables)
- [Docker registry](#docker-registry)
- [Proxy settings](#proxy-settings)
- [PostgreSQL](#postgresql-1)
+ [Run the installer](#run-the-installer-2)
+ [Post-install](#post-install-2)
+ [Accessing AWX](#accessing-awx-2)
- [Installing the AWX CLI](#installing-the-awx-cli)
* [Building the CLI Documentation](#building-the-cli-documentation)
## Getting started
### Clone the repo
If you have not already done so, you will need to clone, or create a local copy, of the [AWX repo](https://github.com/ansible/awx). We generally recommend that you view the releases page:
Please note that deploying from `HEAD` (or the latest commit) is **not** stable, and that if you want to do this, you should proceed at your own risk (also, see the section #official-vs-building-images for building your own image).
For more on how to clone the repo, view [git clone help](https://git-scm.com/docs/git-clone).
Once you have a local copy, run commands within the root of the project tree.
### AWX branding
You can optionally install the AWX branding assets from the [awx-logos repo](https://github.com/ansible/awx-logos). Prior to installing, please review and agree to the [trademark guidelines](https://github.com/ansible/awx-logos/blob/master/TRADEMARKS.md).
To install the assets, clone the `awx-logos` repo so that it is next to your `awx` clone. As you progress through the installation steps, you'll be setting variables in the [inventory](./installer/inventory) file. To include the assets in the build, set `awx_official=true`.
### Prerequisites
Before you can run a deployment, you'll need the following installed in your local environment:
- [Ansible](http://docs.ansible.com/ansible/latest/intro_installation.html) Requires Version 2.8+
+ This is only required if you're [building your own container images](#official-vs-building-images) with `use_container_for_build=false`
- [NPM 6.x LTS](https://docs.npmjs.com/)
+ This is only required if you're [building your own container images](#official-vs-building-images) with `use_container_for_build=false`
### System Requirements
The system that runs the AWX service will need to satisfy the following requirements
- At least 4GB of memory
- At least 2 cpu cores
- At least 20GB of space
- Running Docker, Openshift, or Kubernetes
- If you choose to use an external PostgreSQL database, please note that the minimum version is 10+.
### Choose a deployment platform
We currently support running AWX as a containerized application using Docker images deployed to either an OpenShift cluster, a Kubernetes cluster, or docker-compose. The remainder of this document will walk you through the process of building the images, and deploying them to either platform.
The [installer](./installer) directory contains an [inventory](./installer/inventory) file, and a playbook, [install.yml](./installer/install.yml). You'll begin by setting variables in the inventory file according to the platform you wish to use, and then you'll start the image build and deployment process by running the playbook.
In the sections below, you'll find deployment details and instructions for each platform:
- [OpenShift](#openshift)
- [Kubernetes](#kubernetes)
- [Docker Compose](#docker-compose).
### Official vs Building Images
When installing AWX you have the option of building your own image or using the image provided on DockerHub (see [awx](https://hub.docker.com/r/ansible/awx/))
This is controlled by the following variables in the `inventory` file
```
dockerhub_base=ansible
dockerhub_version=latest
```
If these variables are present then all deployments will use these hosted images. If the variables are not present then the images will be built during the install.
*dockerhub_base*
> The base location on DockerHub where the images are hosted (by default this pulls a container image named `ansible/awx:tag`)
*dockerhub_version*
> Multiple versions are provided. `latest` always pulls the most recent. You may also select version numbers at different granularities: 1, 1.0, 1.0.1, 1.0.0.123
*use_container_for_build*
> Use a local distribution build container image for building the AWX package. This is helpful if you don't want to bother installing the build-time dependencies as it is taken care of already.
## Upgrading from previous versions
Upgrading AWX involves rerunning the install playbook. Download a newer release from [https://github.com/ansible/awx/releases](https://github.com/ansible/awx/releases) and re-populate the inventory file with your customized variables.
For convenience, you can create a file called `vars.yml`:
To complete a deployment to OpenShift, you will need access to an OpenShift cluster. For demo and testing purposes, you can use [Minishift](https://github.com/minishift/minishift) to create a single node cluster running inside a virtual machine.
When using OpenShift for deploying AWX make sure you have correct privileges to add the security context 'privileged', otherwise the installation will fail. The privileged context is needed because of the use of [the bubblewrap tool](https://github.com/containers/bubblewrap) to add an additional layer of security when using containers.
You will also need to have the `oc` command in your PATH. The `install.yml` playbook will call out to `oc` when logging into, and creating objects on the cluster.
The default resource requests per-deployment requires:
> Memory: 6GB
> CPU: 3 cores
This can be tuned by overriding the variables found in [/installer/roles/kubernetes/defaults/main.yml](/installer/roles/kubernetes/defaults/main.yml). Special care should be taken when doing this as undersized instances will experience crashes and resource exhaustion.
For more detail on how resource requests are formed see: [https://docs.openshift.com/container-platform/latest/dev_guide/compute_resources.html#dev-compute-resources](https://docs.openshift.com/container-platform/latest/dev_guide/compute_resources.html#dev-compute-resources)
### Pre-install steps
Before starting the install, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section:
*openshift_host*
> IP address or hostname of the OpenShift cluster. If you're using Minishift, this will be the value returned by `minishift ip`.
*openshift_skip_tls_verify*
> Boolean. Set to True if using self-signed certs.
*openshift_project*
> Name of the OpenShift project that will be created, and used as the namespace for the AWX app. Defaults to *awx*.
*openshift_user*
> Username of the OpenShift user that will create the project, and deploy the application. Defaults to *developer*.
*openshift_pg_emptydir*
> Boolean. Set to True to use an emptyDir volume when deploying the PostgreSQL pod. Note: This should only be used for demo and testing purposes.
*docker_registry*
> IP address and port, or URL, for accessing a registry that the OpenShift cluster can access. Defaults to *172.30.1.1:5000*, the internal registry delivered with Minishift. This is not needed if you are using official hosted images.
*docker_registry_repository*
> Namespace to use when pushing and pulling images to and from the registry. Generally this will match the project name. It defaults to *awx*. This is not needed if you are using official hosted images.
*docker_registry_username*
> Username of the user that will push images to the registry. Will generally match the *openshift_user* value. Defaults to *developer*. This is not needed if you are using official hosted images.
#### Deploying to Minishift
Install Minishift by following the [installation guide](https://docs.openshift.org/latest/minishift/getting-started/installing.html).
The recommended minimum resources for your Minishift VM:
```bash
$ minishift start --cpus=4 --memory=8GB
```
The Minishift VM contains a Docker daemon, which you can use to build the AWX images. This is generally the approach you should take, and we recommend doing so. To use this instance, run the following command to setup your environment:
```bash
# Set DOCKER environment variable to point to the Minishift VM
$ eval$(minishift docker-env)
```
**Note**
> If you choose to not use the Docker instance running inside the VM, and build the images externally, you will have to enable the OpenShift cluster to access the images. This involves pushing the images to an external Docker registry, and granting the cluster access to it, or exposing the internal registry, and pushing the images into it.
#### PostgreSQL
By default, AWX will deploy a PostgreSQL pod inside of your cluster. You will need to create a [Persistent Volume Claim](https://docs.openshift.org/latest/dev_guide/persistent_volumes.html) which is named `postgresql` by default, and can be overridden by setting the `openshift_pg_pvc_name` variable. For testing and demo purposes, you may set `openshift_pg_emptydir=yes`.
If you wish to use an external database, in the inventory file, set the value of `pg_hostname`, and update `pg_username`, `pg_password`, `pg_admin_password`, `pg_database`, and `pg_port` with the connection information. When setting `pg_hostname` the installer will assume you have configured the database in that location and will not launch the postgresql pod.
### Run the installer
To start the install, you will pass two *extra* variables on the command line. The first is *openshift_password*, which is the password for the *openshift_user*, and the second is *docker_registry_password*, which is the password associated with *docker_registry_username*.
If you're using the OpenShift internal registry, then you'll pass an access token for the *docker_registry_password* value, rather than a password. The `oc whoami -t` command will generate the required token, as long as you're logged into the cluster via `oc cluster login`.
Run the following command (docker_registry_password is optional if using official images):
After the playbook run completes, check the status of the deployment by running `oc get pods`:
```bash
# View the running pods
$ oc get pods
NAME READY STATUS RESTARTS AGE
awx-3886581826-5mv0l 4/4 Running 0 8s
postgresql-1-l85fh 1/1 Running 0 20m
```
In the above example, the name of the AWX pod is `awx-3886581826-5mv0l`. Before accessing the AWX web interface, setup tasks and database migrations need to complete. These tasks are running in the `awx_task` container inside the AWX pod. To monitor their status, tail the container's STDOUT by running the following command, replacing the AWX pod name with the pod name from your environment:
```bash
# Follow the awx_task log output
$ oc logs -f awx-3886581826-5mv0l -c awx-celery
```
You will see the following indicating that database migrations are running:
The deployment process creates a route, `awx-web-svc`, to expose the service. How the ingres is actually created will vary depending on your environment, and how the cluster is configured. You can view the route, and the external IP address and hostname assigned to it, by running the following command:
```bash
# View available routes
$ oc get routes
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
The above example is taken from a Minishift instance. From a web browser, use `https` to access the `HOST/PORT` value from your environment. Using the above example, the URL to access the server would be [https://awx-web-svc-awx.192.168.64.2.nip.io](https://awx-web-svc-awx.192.168.64.2.nip.io).
Once you access the AWX server, you will be prompted with a login dialog. The default administrator username is `admin`, and the password is `password`.
## Kubernetes
### Prerequisites
A Kubernetes deployment will require you to have access to a Kubernetes cluster as well as the following tools:
The installation program will reference `kubectl` directly. `helm` is only necessary if you are letting the installer configure PostgreSQL for you.
The default resource requests per-pod requires:
> Memory: 6GB
> CPU: 3 cores
This can be tuned by overriding the variables found in [/installer/roles/kubernetes/defaults/main.yml](/installer/roles/kubernetes/defaults/main.yml). Special care should be taken when doing this as undersized instances will experience crashes and resource exhaustion.
For more detail on how resource requests are formed see: [https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
### Pre-install steps
Before starting the install process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section uncommenting when necessary. Make sure the openshift and standalone docker sections are commented out:
*kubernetes_context*
> Prior to running the installer, make sure you've configured the context for the cluster you'll be installing to. This is how the installer knows which cluster to connect to and what authentication to use
*kubernetes_namespace*
> Name of the Kubernetes namespace where the AWX resources will be installed. This will be created if it doesn't exist
*docker_registry_*
> These settings should be used if building your own base images. You'll need access to an external registry and are responsible for making sure your kube cluster can talk to it and use it. If these are undefined and the dockerhub_ configuration settings are uncommented then the images will be pulled from dockerhub instead
### Configuring Helm
If you want the AWX installer to manage creating the database pod (rather than installing and configuring postgres on your own). Then you will need to have a working `helm` installation, you can find details here: [https://docs.helm.sh/using_helm/#quickstart-guide](https://docs.helm.sh/using_helm/#quickstart-guide).
Newer Kubernetes clusters with RBAC enabled will need to make sure a service account is created, make sure to follow the instructions here [https://docs.helm.sh/using_helm/#role-based-access-control](https://docs.helm.sh/using_helm/#role-based-access-control)
### Run the installer
After making changes to the `inventory` file use `ansible-playbook` to begin the install
```bash
$ ansible-playbook -i inventory install.yml
```
### Post-install
After the playbook run completes, check the status of the deployment by running `kubectl get pods --namespace awx` (replace awx with the namespace you used):
```bash
# View the running pods, it may take a few minutes for everything to be marked in the Running state
$ kubectl get pods --namespace awx
NAME READY STATUS RESTARTS AGE
awx-2558692395-2r8ss 4/4 Running 0 29s
awx-postgresql-355348841-kltkn 1/1 Running 0 1m
```
### Accessing AWX
The AWX web interface is running in the AWX pod behind the `awx-web-svc` service:
The deployment process creates an `Ingress` named `awx-web-svc` also. Some kubernetes cloud providers will automatically handle routing configuration when an Ingress is created others may require that you more explicitly configure it. You can see what kubernetes knows about things with:
```bash
kubectl get ing --namespace awx
NAME HOSTS ADDRESS PORTS AGE
awx-web-svc * 35.227.x.y 80 3m
```
If your provider is able to allocate an IP Address from the Ingress controller then you can navigate to the address and access the AWX interface. For some providers it can take a few minutes to allocate and make this accessible. For other providers it may require you to manually intervene.
### SSL Termination
Unlike Openshift's `Route` the Kubernetes `Ingress` doesn't yet handle SSL termination. As such the default configuration will only expose AWX through HTTP on port 80. You are responsible for configuring SSL support until support is added (either to Kubernetes or AWX itself).
## Docker-Compose
### Prerequisites
- [Docker](https://docs.docker.com/engine/installation/) on the host where AWX will be deployed. After installing Docker, the Docker service must be started (depending on your OS, you may have to add the local user that uses Docker to the ``docker`` group, refer to the documentation for details)
+ This also installs the `docker` Python module, which is incompatible with `docker-py`. If you have previously installed `docker-py`, please uninstall it.
By default, the delivered [installer/inventory](./installer/inventory) file will deploy AWX to the local host. It is possible, however, to deploy to a remote host. The [installer/install.yml](./installer/install.yml) playbook can be used to build images on the local host, and ship the built images to, and run deployment tasks on, a remote host. To do this, modify the [installer/inventory](./installer/inventory) file, by commenting out `localhost`, and adding the remote host.
For example, suppose you wish to build images locally on your CI/CD host, and deploy them to a remote host named *awx-server*. To do this, add *awx-server* to the [installer/inventory](./installer/inventory) file, and comment out or remove `localhost`, as demonstrated by the following:
```yaml
# localhost ansible_connection=local
awx-server
[all:vars]
...
```
In the above example, image build tasks will be delegated to `localhost`, which is typically where the clone of the AWX project exists. Built images will be archived, copied to remote host, and imported into the remote Docker image cache. Tasks to start the AWX containers will then execute on the remote host.
If you choose to use the official images then the remote host will be the one to pull those images.
**Note**
> You may also want to set additional variables to control how Ansible connects to the host. For more information about this, view [Behavioral Inventory Parameters](http://docs.ansible.com/ansible/latest/intro_inventory.html#id12).
> As mentioned above, in [Prerequisites](#prerequisites-1), the prerequisites are required on the remote host.
> When deploying to a remote host, the playbook does not execute tasks with the `become` option. For this reason, make sure the user that connects to the remote host has privileges to run the `docker` command. This typically means that non-privileged users need to be part of the `docker` group.
#### Inventory variables
Before starting the install process, review the [inventory](./installer/inventory) file, and uncomment and provide values for the following variables found in the `[all:vars]` section:
*postgres_data_dir*
> If you're using the default PostgreSQL container (see [PostgreSQL](#postgresql-1) below), provide a path that can be mounted to the container, and where the database can be persisted.
*host_port*
> Provide a port number that can be mapped from the Docker daemon host to the web server running inside the AWX container. If undefined no port will be exposed. Defaults to *80*.
*host_port_ssl*
> Provide a port number that can be mapped from the Docker daemon host to the web server running inside the AWX container for SSL support. If undefined no port will be exposed. Defaults to *443*, only works if you also set `ssl_certificate` (see below).
*ssl_certificate*
> Optionally, provide the path to a file that contains a certificate and its private key. This needs to be a .pem-file
*docker_compose_dir*
> When using docker-compose, the `docker-compose.yml` file will be created there (default `/tmp/awxcompose`).
*custom_venv_dir*
> Adds the custom venv environments from the local host to be passed into the containers at install.
*ca_trust_dir*
> If you're using a non trusted CA, provide a path where the untrusted Certs are stored on your Host.
#### Docker registry
If you wish to tag and push built images to a Docker registry, set the following variables in the inventory file:
*docker_registry*
> IP address and port, or URL, for accessing a registry.
*docker_registry_repository*
> Namespace to use when pushing and pulling images to and from the registry. Defaults to *awx*.
*docker_registry_username*
> Username of the user that will push images to the registry. Defaults to *developer*.
**Note**
> These settings are ignored if using official images
#### Proxy settings
*http_proxy*
> IP address and port, or URL, for using an http_proxy.
*https_proxy*
> IP address and port, or URL, for using an https_proxy.
*no_proxy*
> Exclude IP address or URL from the proxy.
#### PostgreSQL
AWX requires access to a PostgreSQL database, and by default, one will be created and deployed in a container, and data will be persisted to a host volume. In this scenario, you must set the value of `postgres_data_dir` to a path that can be mounted to the container. When the container is stopped, the database files will still exist in the specified path.
If you wish to use an external database, in the inventory file, set the value of `pg_hostname`, and update `pg_username`, `pg_password`, `pg_admin_password`, `pg_database`, and `pg_port` with the connection information.
### Run the installer
If you are not pushing images to a Docker registry, start the install by running the following:
```bash
# Set the working directory to installer
$ cd installer
# Run the Ansible playbook
$ ansible-playbook -i inventory install.yml
```
If you're pushing built images to a repository, then use the `-e` option to pass the registry password as follows, replacing *password* with the password of the username assigned to `docker_registry_username` (note that you will also need to remove `dockerhub_base` and `dockerhub_version` from the inventory file):
After the playbook run completes, Docker starts a series of containers that provide the services that make up AWX. You can view the running containers using the `docker ps` command.
If you're deploying using Docker Compose, container names will be prefixed by the name of the folder where the docker-compose.yml file is created (by default, `awx`).
Immediately after the containers start, the *awx_task* container will perform required setup tasks, including database migrations. These tasks need to complete before the web interface can be accessed. To monitor the progress, you can follow the container's STDOUT by running the following:
Apply all migrations: sso, taggit, sessions, sites, kombu_transport_django, social_auth, contenttypes, auth, conf, main
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Installing custom SQL...
Running migrations:
Rendering model states... DONE
Applying contenttypes.0001_initial... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0001_initial... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying taggit.0001_initial... OK
Applying taggit.0002_auto_20150616_2121... OK
Applying main.0001_initial... OK
...
```
Once migrations complete, you will see the following log output, indicating that migrations have completed:
```bash
Python 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623(Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license"for more information.
(InteractiveConsole)
>>> <User: admin>
>>> Default organization added.
Demo Credential, Inventory, and Job Template added.
Successfully registered instance awx
(changed: True)
Creating instance group tower
Added instance awx to tower
(changed: True)
...
```
### Accessing AWX
The AWX web server is accessible on the deployment host, using the *host_port* value set in the *inventory* file. The default URL is [http://localhost](http://localhost).
You will prompted with a login dialog. The default administrator username is `admin`, and the password is `password`.
Starting in version 18.0, the [AWX Operator](https://github.com/ansible/awx-operator) is the preferred way to install AWX. Please refer to the [AWX Operator](https://github.com/ansible/awx-operator) documentation.
AWX can also alternatively be installed and [run in Docker](./tools/docker-compose/README.md), but this install path is only recommended for development/test-oriented deployments, and has no official published release.
# Installing the AWX CLI
@@ -651,17 +35,16 @@ Potential uses include:
* Checking on the status and output of job runs
* Managing objects like organizations, users, teams, etc...
The preferred way to install the AWX CLI is through pip directly from GitHub:
The preferred way to install the AWX CLI is through pip directly from PyPI:
Use the GitHub [issue tracker](https://github.com/ansible/awx/issues) for filing bugs. In order to save time, and help us respond to issues quickly, make sure to fill out as much of the issue template
as possible. Version information, and an accurate reproducing scenario are critical to helping us identify the problem.
Please don't use the issue tracker as a way to ask how to do something. Instead, use the [mailing list](https://groups.google.com/forum/#!forum/awx-project) , and the `#ansible-awx` channel on irc.freenode.net to get help.
Please don't use the issue tracker as a way to ask how to do something. Instead, use the [mailing list](https://groups.google.com/forum/#!forum/awx-project) , and the `#ansible-awx` channel on irc.libera.chat to get help.
Before opening a new issue, please use the issue search feature to see if what you're experiencing has already been reported. If you have any extra detail to provide, please comment. Otherwise, rather than posting a "me too" comment, please consider giving it a ["thumbs up"](https://github.com/blog/2119-add-reactions-to-pull-requests-issues-and-comment) to give us an indication of the severity of the problem.
@@ -31,7 +31,7 @@ If your issue isn't considered high priority, then please be patient as it may t
`state:needs_info` The issue needs more information. This could be more debug output, more specifics out the system such as version information. Any detail that is currently preventing this issue from moving forward. This should be considered a blocked state.
`state:needs_review` The the issue/pull request needs to be reviewed by other maintainers and contributors. This is usually used when there is a question out to another maintainer or when a person is less familar with an area of the code base the issue is for.
`state:needs_review` The issue/pull request needs to be reviewed by other maintainers and contributors. This is usually used when there is a question out to another maintainer or when a person is less familar with an area of the code base the issue is for.
`state:needs_revision` More commonly used on pull requests, this state represents that there are changes that are being waited on.
# Run to start the background task dispatcher for development.
## Run to start the background task dispatcher for development.
dispatcher:
@if ["$(VENV_BASE)"];then\
. $(VENV_BASE)/awx/bin/activate;\
@@ -308,7 +227,7 @@ dispatcher:
$(PYTHON) manage.py run_dispatcher
# Run to start the zeromq callback receiver
## Run to start the zeromq callback receiver
receiver:
@if ["$(VENV_BASE)"];then\
. $(VENV_BASE)/awx/bin/activate;\
@@ -327,20 +246,15 @@ jupyter:
reports:
mkdir -p $@
pep8:reports
@(set -o pipefail &&$@| tee reports/$@.report)
black:reports
@command -v black >/dev/null 2>&1||{echo"could not find black on your PATH, you may need to \`pip install black\`, or set AWX_IGNORE_BLACK=1"&&exit 1;}
@(set -o pipefail &&$@$(BLACK_ARGS) awx awxkit awx_collection | tee reports/$@.report)
rm -f $(shell ls -d $(VENV_BASE)/awx/lib/python* | head -n 1)/no-global-site-packages.txt
if["$(VENV_BASE)"];then\
. $(VENV_BASE)/awx/bin/activate;\
fi;\
fi&&\
pip install ansible-core &&\
py.test $(COLLECTION_TEST_DIRS) -v
# The python path needs to be modified so that the tests can find Ansible within the container
# First we will use anything expility set as PYTHONPATH
# Second we will load any libraries out of the virtualenv (if it's unspecified that should be ok because python should not load out of an empty directory)
# Finally we will add the system path so that the tests can find the ansible libraries
flake8_collection:
flake8 awx_collection/ # Different settings, in main exclude list
# This directory is bind-mounted inside of the development container and
# needs to be pre-created for permissions to be set correctly. Otherwise,
# Docker will create this directory as root.
awx/projects:
@mkdir -p $@
# Docker isolated rampart
docker-compose-isolated:awx/projects
CURRENT_UID=$(shell id -u)TAG=$(COMPOSE_TAG)DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml -f tools/docker-isolated-override.yml up
COMPOSE_UP_OPTS?=
COMPOSE_OPTS?=
CONTROL_PLANE_NODE_COUNT?=1
EXECUTION_NODE_COUNT?=2
MINIKUBE_CONTAINER_GROUP?=false
MINIKUBE_SETUP?=false# if false, run minikube separately
EXTRA_SOURCES_ANSIBLE_OPTS?=
# Docker Compose Development environment
docker-compose:docker-authawx/projects
CURRENT_UID=$(shell id -u)OS="$(shell docker info | grep 'Operating System')"TAG=$(COMPOSE_TAG)DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml up --no-recreate awx
echo -e "\033[0;31mTo generate a CyberArk Conjur API key: docker exec -it tools_conjur_1 conjurctl account create quick-start\033[0m"
CURRENT_UID=$(shell id -u)TAG=$(COMPOSE_TAG)DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose -f tools/docker-compose.yml -f tools/docker-credential-plugins-override.yml up --no-recreate awx
docker-compose -f tools/docker-compose/_sources/docker-compose.yml -f tools/docker-credential-plugins-override.yml up --no-recreate awx_1 --remove-orphans
docker-compose-test:docker-auth awx/projects
cd tools &&CURRENT_UID=$(shell id -u)OS="$(shell docker info | grep 'Operating System')"TAG=$(COMPOSE_TAG)DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE)docker-compose run --rm --service-ports awx /bin/bash
docker-compose -f tools/docker-compose/_sources/docker-compose.yml run --rm --service-ports awx_1 /bin/bash
docker-compose-runtest:awx/projects
cd tools &&CURRENT_UID=$(shell id -u)TAG=$(COMPOSE_TAG)DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE)docker-compose run --rm --service-ports awx /start_tests.sh
docker-compose -f tools/docker-compose/_sources/docker-compose.yml run --rm --service-ports awx_1 /start_tests.sh
docker-compose-build-swagger:awx/projects
cd tools &&CURRENT_UID=$(shell id -u)TAG=$(COMPOSE_TAG)DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE)docker-compose run --rm --service-ports --no-deps awx /start_tests.sh swagger
AWX provides a web-based user interface, REST API, and task engine built on top of [Ansible](https://github.com/ansible/ansible). It is the upstream project for [Tower](https://www.ansible.com/tower), a commercial derivative of AWX.
AWX provides a web-based user interface, REST API, and task engine built on top of [Ansible](https://github.com/ansible/ansible). It is one of the upstream projects for [Red Hat Ansible Automation Platform](https://www.ansible.com/products/automation-platform).
To install AWX, please view the [Install guide](./INSTALL.md).
@@ -16,31 +17,25 @@ Contributing
------------
- Refer to the [Contributing guide](./CONTRIBUTING.md) to get started developing, testing, and building AWX.
- All code submissions are done through pull requests against the `devel` branch.
- All contributors must use git commit --signoff for any commit to be merged, and agree that usage of --signoff constitutes agreement with the terms of [DCO 1.1](./DCO_1_1.md)
- Take care to make sure no merge commits are in the submission, and use `git rebase` vs `git merge` for this reason.
- If submitting a large code change, it's a good idea to join the `#ansible-awx` channel on irc.freenode.net, and talk about what you would like to do or add first. This not only helps everyone know what's going on, it also helps save time and effort, if the community decides some changes are needed.
- All code submissions are made through pull requests against the `devel` branch.
- All contributors must use git commit --signoff for any commit to be merged and agree that usage of --signoff constitutes agreement with the terms of [DCO 1.1](./DCO_1_1.md)
- Take care to make sure no merge commits are in the submission, and use `git rebase` vs.`git merge` for this reason.
- If submitting a large code change, it's a good idea to join the `#ansible-awx` channel on web.libera.chat and talk about what you would like to do or add first. This not only helps everyone know what's going on, but it also helps save time and effort if the community decides some changes are needed.
Reporting Issues
----------------
If you're experiencing a problem that you feel is a bug in AWX, or have ideas for how to improve AWX, we encourage you to open an issue, and share your feedback. But before opening a new issue, we ask that you please take a look at our [Issues guide](./ISSUES.md).
If you're experiencing a problem that you feel is a bug in AWX or have ideas for improving AWX, we encourage you to open an issue and share your feedback. But before opening a new issue, we ask that you please take a look at our [Issues guide](./ISSUES.md).
Code of Conduct
---------------
We ask all of our community members and contributors to adhere to the [Ansible code of conduct](http://docs.ansible.com/ansible/latest/community/code_of_conduct.html). If you have questions, or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)
We ask all of our community members and contributors to adhere to the [Ansible code of conduct](http://docs.ansible.com/ansible/latest/community/code_of_conduct.html). If you have questions or need assistance, please reach out to our community team at [codeofconduct@ansible.com](mailto:codeofconduct@ansible.com)
Get Involved
------------
We welcome your feedback and ideas. Here's how to reach us with feedback and questions:
- Join the `#ansible-awx` channel on irc.freenode.net
- Join the `#ansible-awx` channel on irc.libera.chat
- Join the [mailing list](https://groups.google.com/forum/#!forum/awx-project)
Make a GET request to retrieve the list of aggregated task data associated with the play given by event_id.
`event_id` is a required query parameter and must match the job event id of the parent play in order to receive the list of tasks associated with the play
## Filtering
This endpoints supports a limited filtering subset:
?event_id__in=1,2,3
Will show only the given task ids under the play given by `event_id`.
?event_id__gt=1
Will show ids greater than the given one.
?event_id__lt=3
Will show ids less than the given one.
?failed=true
Will show only failed plays. Alternatively `false` may be used.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
