Revert "Remove external script call to D3.js."

Remove external script call to D3.js.

.github/triage_replies.md

@@ -1,5 +1,5 @@
 ## General
-- For the roundup of all the different mailing lists available from AWX, Ansible, and beyond visit: https://docs.ansible.com/ansible/latest/community/communication.html 
+- For the roundup of all the different mailing lists available from AWX, Ansible, and beyond visit: https://docs.ansible.com/ansible/latest/community/communication.html
 - Hello, we think your question is answered in our FAQ. Does this: https://www.ansible.com/products/awx-project/faq cover your question?
 - You can find the latest documentation here: https://docs.ansible.com/automation-controller/latest/html/userguide/index.html
 
@@ -58,10 +58,10 @@ Thank you once again for this and your interest in AWX!
 ## Common
 
 ### Give us more info
-- Hello, we'd love to help, but we need a little more information about the problem you're having. Screenshots, log outputs, or any reproducers would be very helpful. 
+- Hello, we'd love to help, but we need a little more information about the problem you're having. Screenshots, log outputs, or any reproducers would be very helpful.
 
 ### Code of Conduct
-- Hello. Please keep in mind that Ansible adheres to a Code of Conduct in its community spaces. The spirit of the code of conduct is to be kind, and this is your friendly reminder to be so. Please see the full code of conduct here if you have questions: https://docs.ansible.com/ansible/latest/community/code_of_conduct.html 
+- Hello. Please keep in mind that Ansible adheres to a Code of Conduct in its community spaces. The spirit of the code of conduct is to be kind, and this is your friendly reminder to be so. Please see the full code of conduct here if you have questions: https://docs.ansible.com/ansible/latest/community/code_of_conduct.html
 
 ### EE Contents / Community General
 - Hello. The awx-ee contains the collections and dependencies needed for supported AWX features to function. Anything beyond that (like the community.general package) will require you to build your own EE. For information on how to do that, see https://ansible-builder.readthedocs.io/en/stable/ \
@@ -79,31 +79,31 @@ The Ansible Community is looking at building an EE that corresponds to all of th
 - Hello, we think your idea is good! Please consider contributing a PR for this following our contributing guidelines: https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md
 
 ### Receptor
-- You can find the receptor docs here: https://receptor.readthedocs.io/en/latest/ 
+- You can find the receptor docs here: https://receptor.readthedocs.io/en/latest/
 - Hello, your issue seems related to receptor. Could you please open an issue in the receptor repository? https://github.com/ansible/receptor. Thanks!
 
 ### Ansible Engine not AWX
-- Hello, your question seems to be about Ansible development, not about AWX. Try asking on the Ansible-devel specific mailing list: https://groups.google.com/g/ansible-devel 
+- Hello, your question seems to be about Ansible development, not about AWX. Try asking on the Ansible-devel specific mailing list: https://groups.google.com/g/ansible-devel
 - Hello, your question seems to be about using Ansible, not about AWX. https://groups.google.com/g/ansible-project is the best place to visit for user questions about Ansible. Thanks!
 
 ### Ansible Galaxy not AWX
 - Hey there. That sounds like an FAQ question. Did this: https://www.ansible.com/products/awx-project/faq cover your question?
 
 ### Contributing Guidelines
-- AWX: https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md 
+- AWX: https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md
 - AWX-Operator: https://github.com/ansible/awx-operator/blob/devel/CONTRIBUTING.md
 
 ### AWX Release
-Subject: Announcing AWX X.Y.z
+Subject: Announcing AWX Xa.Ya.za and AWX-Operator Xb.Yb.zb
 
 - Hi all, \
 \
-We're happy to announce that the next release of AWX, version <X> is now available! \
-In addition AWX Operator version <Y> has also been release! \
+We're happy to announce that the next release of AWX, version <b>`Xa.Ya.za`</b> is now available! \
+In addition AWX Operator version <b>`Xb.Yb.zb`</b> has also been released! \
 \
 Please see the releases pages for more details: \
-	AWX: https://github.com/ansible/awx/releases/tag/<X> \
-	Operator: https://github.com/ansible/awx-operator/releases/tag/<Y> \
+	AWX: https://github.com/ansible/awx/releases/tag/Xa.Ya.za \
+	Operator: https://github.com/ansible/awx-operator/releases/tag/Xb.Yb.zb \
 \
 The AWX team.
 

awx/main/analytics/collectors.py

@@ -396,7 +396,7 @@ def events_table_partitioned_modified(since, full_path, until, **kwargs):
     return _events_table(since, full_path, until, 'main_jobevent', 'modified', project_job_created=True, **kwargs)
 
 
-@register('unified_jobs_table', '1.3', format='csv', description=_('Data on jobs run'), expensive=four_hour_slicing)
+@register('unified_jobs_table', '1.4', format='csv', description=_('Data on jobs run'), expensive=four_hour_slicing)
 def unified_jobs_table(since, full_path, until, **kwargs):
     unified_job_query = '''COPY (SELECT main_unifiedjob.id,
                                  main_unifiedjob.polymorphic_ctype_id,
@@ -422,7 +422,8 @@ def unified_jobs_table(since, full_path, until, **kwargs):
                                  main_unifiedjob.job_explanation,
                                  main_unifiedjob.instance_group_id,
                                  main_unifiedjob.installed_collections,
-                                 main_unifiedjob.ansible_version
+                                 main_unifiedjob.ansible_version,
+                                 main_job.forks
                                  FROM main_unifiedjob
                                  JOIN django_content_type ON main_unifiedjob.polymorphic_ctype_id = django_content_type.id
                                  LEFT JOIN main_job ON main_unifiedjob.id = main_job.unifiedjob_ptr_id

awx/main/models/notifications.py

@@ -408,6 +408,7 @@ class JobNotificationMixin(object):
  'inventory': 'Stub Inventory',
  'id': 42,
  'hosts': {},
+ 'extra_vars': {},
  'friendly_name': 'Job',
  'finished': False,
  'credential': 'Stub credential',

awx/main/models/workflow.py

@@ -659,6 +659,13 @@ class WorkflowJob(UnifiedJob, WorkflowJobOptions, SurveyJobMixin, JobNotificatio
                 node_job_description = 'job #{0}, "{1}", which finished with status {2}.'.format(node.job.id, node.job.name, node.job.status)
             str_arr.append("- node #{0} spawns {1}".format(node.id, node_job_description))
         result['body'] = '\n'.join(str_arr)
+        result.update(
+            dict(
+                inventory=self.inventory.name if self.inventory else None,
+                limit=self.limit,
+                extra_vars=self.display_extra_vars(),
+            )
+        )
         return result
 
     @property
@@ -906,3 +913,12 @@ class WorkflowApproval(UnifiedJob, JobNotificationMixin):
     @property
     def workflow_job(self):
         return self.unified_job_node.workflow_job
+
+    def notification_data(self):
+        result = super(WorkflowApproval, self).notification_data()
+        result.update(
+            dict(
+                extra_vars=self.workflow_job.display_extra_vars(),
+            )
+        )
+        return result

awx/ui/docs/JobOutput.md

@@ -2,15 +2,39 @@ This document is meant to provide some guidance into the functionality of Job Ou
 
 ## Overview of the feature/screen. Summary of what it does/is
 
-1. Elapsed time / unfollow button
-2. Page up and page down buttons
-3. Unique qualities of the different job types.
+Joboutput is a feature that allows users to see how their job is doing as it is being run.
+This feature displays data sent to the UI via websockets that are connected to several
+different endpoints in the API.
 
-- Some don’t allow search by event data and thus Event is not an option in the drop down
-- Some don’t have expand, collapse
+The job output has 2 different states that result in different functionality. One state
+is when, the job is actively running. There is limited functionality because of how the
+job events are processed when they reach the UI. While the job is running, and
+output is coming into the UI, the following features turn off:
 
-4. Differences in the output from when a job is running and when a job is complete.
-5. Which features are enabled when it’s running and which aren’t.
+1. [Search](#Search)- The ability to search the output of a job.
+2. [Expand/Collapse](#Expand/Collapse)- The ability to expand and collapse job events, tasks, plays, or even the
+   job itself. The only part of the job ouput that is not collapsable is the playbook summary (only jobs that
+   are executed from a Job Template have Expand/Collapse functionality).
+
+The following features are enabled:
+
+1. Follow/unfollow - `Follow` indicates you are streaming the output on the screen
+   as it comes into the UI. If you see some output that you want to examine closer while the job is running
+   scroll to it, and click `Unfollow`, and the output will stop streaming onto the screen. This feature is only
+   enabled when the job is running and is not complete. If the user scrolls up in the output the UI will unfollow.
+2. Page up and page down buttons- Use these buttons to navigate quickly up and down the output.
+
+![Running job](images/JobOutput-running.png)
+
+After the job is complete, the Follow/Unfollow button disabled, and Expand/Collapse and Search become enabled.
+![Finished job](images/JobOutput-complete.png)
+
+Not all job types are created equal. Some jobs have a concept of parent-child events. Job events can be inside a Task,
+a Task can be inside a Play, and a Play inside a Playbook. Leveraging this concept to enable Expand/Collapse for these
+job types, allows you to collapse and hide the children of a particular line of output. This parent-child event
+relationship only exists on jobs executed from a job template. All other types of jobs do not
+have this event concept, and therefore, do not have Expand/Collapse functionality. By default all job
+events are expanded.
 
 ## How output works generally.
 
@@ -26,11 +50,13 @@ This document is meant to provide some guidance into the functionality of Job Ou
 ## Non-standard cases
 
 1. When an event comes into the output that has a parent, but the parent hasn’t arrived yet.
-2. When an event that has children arrives in output, but the children are not present yet
+2. When an event with children arrives in output, but the children are not yet present.
 
-## Expand collapse a single event- how it works and how it changes the state object
+## Expand/Collapse
 
-## Expand collapse all- how it works and how it changes the state object
+### Expand collapse a single event - how it works and how it changes the state object
+
+### Expand collapse all - how it works and how it changes the state object
 
 ## Search
 

awx/ui/src/screens/Setting/Subscription/SubscriptionDetail/SubscriptionDetail.js

@@ -35,6 +35,13 @@ function SubscriptionDetail() {
     },
   ];
 
+  const { automated_instances: automatedInstancesCount, automated_since } =
+    license_info;
+
+  const automatedInstancesSinceDateTime = automated_since
+    ? formatDateString(new Date(automated_since * 1000).toISOString())
+    : null;
+
   return (
     <>
       <RoutedTabs tabsArray={tabsArray} />
@@ -127,19 +134,23 @@ function SubscriptionDetail() {
             label={t`Hosts imported`}
             value={license_info.current_instances}
           />
-          <Detail
-            dataCy="subscription-hosts-automated"
-            label={t`Hosts automated`}
-            value={
-              <>
-                {license_info.automated_instances} <Trans>since</Trans>{' '}
-                {license_info.automated_since &&
-                  formatDateString(
-                    new Date(license_info.automated_since * 1000).toISOString()
-                  )}
-              </>
-            }
-          />
+          {typeof automatedInstancesCount !== 'undefined' &&
+            automatedInstancesCount !== null && (
+              <Detail
+                dataCy="subscription-hosts-automated"
+                label={t`Hosts automated`}
+                value={
+                  automated_since ? (
+                    <Trans>
+                      {automatedInstancesCount} since{' '}
+                      {automatedInstancesSinceDateTime}
+                    </Trans>
+                  ) : (
+                    automatedInstancesCount
+                  )
+                }
+              />
+            )}
           <Detail
             dataCy="subscription-hosts-remaining"
             label={t`Hosts remaining`}

awx/ui/src/screens/Setting/Subscription/SubscriptionDetail/SubscriptionDetail.test.js

@@ -82,4 +82,17 @@ describe('<SubscriptionDetail />', () => {
 
     expect(wrapper.find('Button[aria-label="edit"]').length).toBe(1);
   });
+
+  test('should not render Hosts Automated Detail if license_info.automated_instances is undefined', () => {
+    wrapper = mountWithContexts(<SubscriptionDetail />, {
+      context: {
+        config: {
+          ...config,
+          license_info: { ...config.license_info, automated_instances: null },
+        },
+      },
+    });
+
+    expect(wrapper.find(`Detail[label="Hosts automated"]`).length).toBe(0);
+  });
 });

awxkit/awxkit/api/pages/__init__.py

@@ -40,3 +40,4 @@ from .instance_groups import *  # NOQA
 from .credential_input_sources import *  # NOQA
 from .metrics import *  # NOQA
 from .subscriptions import *  # NOQA
+from .workflow_approval_templates import *  # NOQA

awxkit/awxkit/api/pages/api.py

@@ -36,14 +36,15 @@ EXPORTABLE_RELATIONS = ['Roles', 'NotificationTemplates', 'WorkflowJobTemplateNo
 # These are special-case related objects, where we want only in this
 # case to export a full object instead of a natural key reference.
 DEPENDENT_EXPORT = [
-    ('JobTemplate', 'labels'),
-    ('JobTemplate', 'survey_spec'),
-    ('WorkflowJobTemplate', 'labels'),
-    ('WorkflowJobTemplate', 'survey_spec'),
-    ('WorkflowJobTemplate', 'workflow_nodes'),
-    ('Inventory', 'groups'),
-    ('Inventory', 'hosts'),
-    ('Inventory', 'labels'),
+    ('JobTemplate', 'Label'),
+    ('JobTemplate', 'SurveySpec'),
+    ('WorkflowJobTemplate', 'Label'),
+    ('WorkflowJobTemplate', 'SurveySpec'),
+    ('WorkflowJobTemplate', 'WorkflowJobTemplateNode'),
+    ('Inventory', 'Group'),
+    ('Inventory', 'Host'),
+    ('Inventory', 'Label'),
+    ('WorkflowJobTemplateNode', 'WorkflowApprovalTemplate'),
 ]
 
 
@@ -57,6 +58,7 @@ DEPENDENT_NONEXPORT = [
     ('Group', 'all_hosts'),
     ('Group', 'potential_children'),
     ('Host', 'all_groups'),
+    ('WorkflowJobTemplateNode', 'create_approval_template'),
 ]
 
 
@@ -85,6 +87,7 @@ class ApiV2(base.Base):
         # Note: doing _page[key] automatically parses json blob strings, which can be a problem.
         fields = {key: _page.json[key] for key in post_fields if key in _page.json and key not in _page.related and key != 'id'}
 
+        # iterate over direct fields in the object
         for key in post_fields:
             if key in _page.related:
                 related = _page.related[key]
@@ -114,6 +117,12 @@ class ApiV2(base.Base):
                     return None
                 log.warning("Foreign key %r export failed for object %s, setting to null", key, _page.endpoint)
                 continue
+
+            # Workflow approval templates have a special creation endpoint,
+            # therefore we are skipping the export via natural key.
+            if rel_endpoint.__item_class__.__name__ == 'WorkflowApprovalTemplate':
+                continue
+
             rel_natural_key = rel_endpoint.get_natural_key(self._cache)
             if rel_natural_key is None:
                 log.error("Unable to construct a natural key for foreign key %r of object %s.", key, _page.endpoint)
@@ -121,19 +130,38 @@ class ApiV2(base.Base):
                 return None  # This foreign key has unresolvable dependencies
             fields[key] = rel_natural_key
 
+        # iterate over related fields in the object
         related = {}
         for key, rel_endpoint in _page.related.items():
-            if key in post_fields or not rel_endpoint:
+            # skip if no endpoint for this related object
+            if not rel_endpoint:
                 continue
 
             rel = rel_endpoint._create()
+
+            if rel.__item_class__.__name__ != 'WorkflowApprovalTemplate':
+                if key in post_fields:
+                    continue
+
             is_relation = rel.__class__.__name__ in EXPORTABLE_RELATIONS
-            is_dependent = (_page.__item_class__.__name__, key) in DEPENDENT_EXPORT
+
+            # determine if the parent object and the related object that we are processing through are related
+            # if this tuple is in the DEPENDENT_EXPORT than we output the full object
+            # else we output the natural key
+            is_dependent = (_page.__item_class__.__name__, rel.__item_class__.__name__) in DEPENDENT_EXPORT
+
             is_blocked = (_page.__item_class__.__name__, key) in DEPENDENT_NONEXPORT
             if is_blocked or not (is_relation or is_dependent):
                 continue
 
-            rel_post_fields = utils.get_post_fields(rel_endpoint, self._cache)
+            # if the rel is of WorkflowApprovalTemplate type, get rel_post_fields from create_approval_template endpoint
+            rel_option_endpoint = rel_endpoint
+            export_key = key
+            if rel.__item_class__.__name__ == 'WorkflowApprovalTemplate':
+                export_key = 'create_approval_template'
+                rel_option_endpoint = _page.related.get('create_approval_template')
+
+            rel_post_fields = utils.get_post_fields(rel_option_endpoint, self._cache)
             if rel_post_fields is None:
                 log.debug("%s is a read-only endpoint.", rel_endpoint)
                 continue
@@ -147,24 +175,28 @@ class ApiV2(base.Base):
                 continue
 
             rel_page = self._cache.get_page(rel_endpoint)
+
             if rel_page is None:
                 continue
 
             if 'results' in rel_page:
                 results = (x.get_natural_key(self._cache) if by_natural_key else self._export(x, rel_post_fields) for x in rel_page.results)
-                related[key] = [x for x in results if x is not None]
+                related[export_key] = [x for x in results if x is not None]
+            elif rel.__item_class__.__name__ == 'WorkflowApprovalTemplate':
+                related[export_key] = self._export(rel_page, rel_post_fields)
             else:
-                related[key] = rel_page.json
+                related[export_key] = rel_page.json
 
         if related:
             fields['related'] = related
 
-        natural_key = _page.get_natural_key(self._cache)
-        if natural_key is None:
-            log.error("Unable to construct a natural key for object %s.", _page.endpoint)
-            self._has_error = True
-            return None
-        fields['natural_key'] = natural_key
+        if _page.__item_class__.__name__ != 'WorkflowApprovalTemplate':
+            natural_key = _page.get_natural_key(self._cache)
+            if natural_key is None:
+                log.error("Unable to construct a natural key for object %s.", _page.endpoint)
+                self._has_error = True
+                return None
+            fields['natural_key'] = natural_key
 
         return utils.remove_encrypted(fields)
 

awxkit/awxkit/api/pages/workflow_approval_templates.py

@@ -0,0 +1,25 @@
+from awxkit.api.pages.unified_job_templates import UnifiedJobTemplate
+from awxkit.api.resources import resources
+from . import page
+
+
+class WorkflowApprovalTemplate(UnifiedJobTemplate):
+
+    pass
+
+
+page.register_page(
+    [
+        resources.workflow_approval_template,
+        resources.workflow_job_template_node_create_approval_template,
+    ],
+    WorkflowApprovalTemplate,
+)
+
+
+class WorkflowApprovalTemplates(page.PageList, WorkflowApprovalTemplate):
+
+    pass
+
+
+page.register_page(resources.workflow_approval_templates, WorkflowApprovalTemplates)

awxkit/awxkit/api/resources.py

@@ -82,6 +82,9 @@ class Resources(object):
     _inventory_variable_data = r'inventories/\d+/variable_data/'
     _workflow_approval = r'workflow_approvals/\d+/'
     _workflow_approvals = 'workflow_approvals/'
+    _workflow_approval_template = r'workflow_approval_templates/\d+/'
+    _workflow_approval_templates = 'workflow_approval_templates/'
+    _workflow_job_template_node_create_approval_template = r'workflow_job_template_nodes/\d+/create_approval_template/'
     _job = r'jobs/\d+/'
     _job_cancel = r'jobs/\d+/cancel/'
     _job_create_schedule = r'jobs/\d+/create_schedule/'

docs/release_process.md

@@ -2,6 +2,7 @@
 
 The release process for AWX is completely automated as of version 19.5.0.
 
+If you need to revert a release, please refer to the [Revert a Release](#revert-a-release) section.
 ## Get latest release version and list of new work
 
 1. Open the main project page for [AWX](https://github.com/ansible/awx/releases) and [AWX Operator](https://github.com/ansible/awx-operator/releases).
@@ -135,3 +136,33 @@ We're happy to announce that [AWX version 21.1.0](https://github.com/ansible/awx
 We're happy to announce that [AWX Operator version 0.22.0](https://github.com/ansible/awx-operator/releases/tag/0.22.0) is now available!
 
 ## Send the same IRC message (less the @newsbot) to #awx:ansible.com
+
+## Revert a Release
+
+Decide whether or not you can just fall-forward with a new AWX Release to fix a bad release. If you need to remove published artifacts from publically facing repositories, follow the steps below.
+
+Here are the steps needed to revert an AWX and an AWX-Operator release. Depending on your use case, follow the steps for reverting just an AWX release, an Operator release or both.
+
+
+1. Navigate to the [AWX Release Page](https://github.com/ansible/awx/releases) and delete the AWX Release that needs to be removed.
+
+![Revert-1-Image](img/revert-1.png)
+
+2. Navigate to the [AWX Tags Page](https://github.com/ansible/awx/tags) and delete the AWX Tag that got created by the Github Actions Workflow from when you originally tried to release AWX. You need delete the release in step 1 before you can do this step. The tag must not be tied to a release if you want to delete a tag.
+
+![Tag-Revert-1-Image](img/tag-revert-1.png)
+[comment]: <> (Need an image here for actually deleting an orphaned tag, place here during next release)
+
+3. Navigate to the [AWX Operator Release Page]() and delete the AWX-Operator release that needss to tbe removed.
+
+![Revert-2-Image](img/revert-2.png)
+
+4. Navigate to [quay.io](https://quay.io/repository/ansible/awx?tag=latest&tab=tags) and delete the published AWX image(s) and tags.
+
+5. Navigate to [quay.io](https://github.com/ansible/awx-operator/releases) and delete the published AWX Operator image(s) and tags.
+
+6. Navigate to the [Ansible Galaxy Collections](https://galaxy.ansible.com/awx/awx) website and remove the published AWX collection with the bad tag.
+
+7. Navigate to the [PyPi](https://pypi.org/project/awxkit/#history) and delete the bad AWX tag and release that got published.
+
+8. [Restart the Release Process](#releasing-awx-and-awx-operator)

requirements/requirements.in

@@ -38,6 +38,7 @@ prometheus_client
 psycopg2
 psutil
 pygerduty
+pyjwt>=2.4.0 # https://github.com/ansible/awx/security/dependabot/58
 pyparsing
 python3-saml==1.13.0
 python-dsv-sdk

requirements/requirements.txt

@@ -258,8 +258,9 @@ pycparser==2.20
     # via cffi
 pygerduty==0.38.2
     # via -r /awx_devel/requirements/requirements.in
-pyjwt==2.3.0
+pyjwt==2.4.0
     # via
+    #   -r /awx_devel/requirements/requirements.in
     #   adal
     #   social-auth-core
     #   twilio