Revert "Remove external script call to D3.js."

Remove external script call to D3.js.

.github/triage_replies.md

@@ -94,16 +94,16 @@ The Ansible Community is looking at building an EE that corresponds to all of th
 - AWX-Operator: https://github.com/ansible/awx-operator/blob/devel/CONTRIBUTING.md
 
 ### AWX Release
-Subject: Announcing AWX X.Y.z
+Subject: Announcing AWX Xa.Ya.za and AWX-Operator Xb.Yb.zb
 
 - Hi all, \
 \
-We're happy to announce that the next release of AWX, version <X> is now available! \
+We're happy to announce that the next release of AWX, version <b>`Xa.Ya.za`</b> is now available! \
-In addition AWX Operator version <Y> has also been release! \
+In addition AWX Operator version <b>`Xb.Yb.zb`</b> has also been released! \
 \
 Please see the releases pages for more details: \
-	AWX: https://github.com/ansible/awx/releases/tag/<X> \
+	AWX: https://github.com/ansible/awx/releases/tag/Xa.Ya.za \
-	Operator: https://github.com/ansible/awx-operator/releases/tag/<Y> \
+	Operator: https://github.com/ansible/awx-operator/releases/tag/Xb.Yb.zb \
 \
 The AWX team.
 

awx/main/analytics/collectors.py

@@ -396,7 +396,7 @@ def events_table_partitioned_modified(since, full_path, until, **kwargs):
     return _events_table(since, full_path, until, 'main_jobevent', 'modified', project_job_created=True, **kwargs)
 
 
-@register('unified_jobs_table', '1.3', format='csv', description=_('Data on jobs run'), expensive=four_hour_slicing)
+@register('unified_jobs_table', '1.4', format='csv', description=_('Data on jobs run'), expensive=four_hour_slicing)
 def unified_jobs_table(since, full_path, until, **kwargs):
     unified_job_query = '''COPY (SELECT main_unifiedjob.id,
                                  main_unifiedjob.polymorphic_ctype_id,
@@ -422,7 +422,8 @@ def unified_jobs_table(since, full_path, until, **kwargs):
                                  main_unifiedjob.job_explanation,
                                  main_unifiedjob.instance_group_id,
                                  main_unifiedjob.installed_collections,
-                                 main_unifiedjob.ansible_version
+                                 main_unifiedjob.ansible_version,
+                                 main_job.forks
                                  FROM main_unifiedjob
                                  JOIN django_content_type ON main_unifiedjob.polymorphic_ctype_id = django_content_type.id
                                  LEFT JOIN main_job ON main_unifiedjob.id = main_job.unifiedjob_ptr_id

awx/main/models/notifications.py

@@ -408,6 +408,7 @@ class JobNotificationMixin(object):
  'inventory': 'Stub Inventory',
  'id': 42,
  'hosts': {},
+ 'extra_vars': {},
  'friendly_name': 'Job',
  'finished': False,
  'credential': 'Stub credential',

awx/main/models/workflow.py

@@ -659,6 +659,13 @@ class WorkflowJob(UnifiedJob, WorkflowJobOptions, SurveyJobMixin, JobNotificatio
                 node_job_description = 'job #{0}, "{1}", which finished with status {2}.'.format(node.job.id, node.job.name, node.job.status)
             str_arr.append("- node #{0} spawns {1}".format(node.id, node_job_description))
         result['body'] = '\n'.join(str_arr)
+        result.update(
+            dict(
+                inventory=self.inventory.name if self.inventory else None,
+                limit=self.limit,
+                extra_vars=self.display_extra_vars(),
+            )
+        )
         return result
 
     @property
@@ -906,3 +913,12 @@ class WorkflowApproval(UnifiedJob, JobNotificationMixin):
     @property
     def workflow_job(self):
         return self.unified_job_node.workflow_job
+
+    def notification_data(self):
+        result = super(WorkflowApproval, self).notification_data()
+        result.update(
+            dict(
+                extra_vars=self.workflow_job.display_extra_vars(),
+            )
+        )
+        return result

awx/ui/docs/JobOutput.md

@@ -2,15 +2,39 @@ This document is meant to provide some guidance into the functionality of Job Ou
 
 ## Overview of the feature/screen. Summary of what it does/is
 
-1. Elapsed time / unfollow button
+Joboutput is a feature that allows users to see how their job is doing as it is being run.
-2. Page up and page down buttons
+This feature displays data sent to the UI via websockets that are connected to several
-3. Unique qualities of the different job types.
+different endpoints in the API.
 
-- Some don’t allow search by event data and thus Event is not an option in the drop down
+The job output has 2 different states that result in different functionality. One state
-- Some don’t have expand, collapse
+is when, the job is actively running. There is limited functionality because of how the
+job events are processed when they reach the UI. While the job is running, and
+output is coming into the UI, the following features turn off:
 
-4. Differences in the output from when a job is running and when a job is complete.
+1. [Search](#Search)- The ability to search the output of a job.
-5. Which features are enabled when it’s running and which aren’t.
+2. [Expand/Collapse](#Expand/Collapse)- The ability to expand and collapse job events, tasks, plays, or even the
+   job itself. The only part of the job ouput that is not collapsable is the playbook summary (only jobs that
+   are executed from a Job Template have Expand/Collapse functionality).
+
+The following features are enabled:
+
+1. Follow/unfollow - `Follow` indicates you are streaming the output on the screen
+   as it comes into the UI. If you see some output that you want to examine closer while the job is running
+   scroll to it, and click `Unfollow`, and the output will stop streaming onto the screen. This feature is only
+   enabled when the job is running and is not complete. If the user scrolls up in the output the UI will unfollow.
+2. Page up and page down buttons- Use these buttons to navigate quickly up and down the output.
+
+![Running job](images/JobOutput-running.png)
+
+After the job is complete, the Follow/Unfollow button disabled, and Expand/Collapse and Search become enabled.
+![Finished job](images/JobOutput-complete.png)
+
+Not all job types are created equal. Some jobs have a concept of parent-child events. Job events can be inside a Task,
+a Task can be inside a Play, and a Play inside a Playbook. Leveraging this concept to enable Expand/Collapse for these
+job types, allows you to collapse and hide the children of a particular line of output. This parent-child event
+relationship only exists on jobs executed from a job template. All other types of jobs do not
+have this event concept, and therefore, do not have Expand/Collapse functionality. By default all job
+events are expanded.
 
 ## How output works generally.
 
@@ -26,11 +50,13 @@ This document is meant to provide some guidance into the functionality of Job Ou
 ## Non-standard cases
 
 1. When an event comes into the output that has a parent, but the parent hasn’t arrived yet.
-2. When an event that has children arrives in output, but the children are not present yet
+2. When an event with children arrives in output, but the children are not yet present.
 
-## Expand collapse a single event- how it works and how it changes the state object
+## Expand/Collapse
 
-## Expand collapse all- how it works and how it changes the state object
+### Expand collapse a single event - how it works and how it changes the state object
+
+### Expand collapse all - how it works and how it changes the state object
 
 ## Search
 

awx/ui/src/screens/Setting/Subscription/SubscriptionDetail/SubscriptionDetail.js

@@ -35,6 +35,13 @@ function SubscriptionDetail() {
     },
   ];
 
+  const { automated_instances: automatedInstancesCount, automated_since } =
+    license_info;
+
+  const automatedInstancesSinceDateTime = automated_since
+    ? formatDateString(new Date(automated_since * 1000).toISOString())
+    : null;
+
   return (
     <>
       <RoutedTabs tabsArray={tabsArray} />
@@ -127,19 +134,23 @@ function SubscriptionDetail() {
             label={t`Hosts imported`}
             value={license_info.current_instances}
           />
-          <Detail
+          {typeof automatedInstancesCount !== 'undefined' &&
-            dataCy="subscription-hosts-automated"
+            automatedInstancesCount !== null && (
-            label={t`Hosts automated`}
+              <Detail
-            value={
+                dataCy="subscription-hosts-automated"
-              <>
+                label={t`Hosts automated`}
-                {license_info.automated_instances} <Trans>since</Trans>{' '}
+                value={
-                {license_info.automated_since &&
+                  automated_since ? (
-                  formatDateString(
+                    <Trans>
-                    new Date(license_info.automated_since * 1000).toISOString()
+                      {automatedInstancesCount} since{' '}
-                  )}
+                      {automatedInstancesSinceDateTime}
-              </>
+                    </Trans>
-            }
+                  ) : (
-          />
+                    automatedInstancesCount
+                  )
+                }
+              />
+            )}
           <Detail
             dataCy="subscription-hosts-remaining"
             label={t`Hosts remaining`}

awx/ui/src/screens/Setting/Subscription/SubscriptionDetail/SubscriptionDetail.test.js

@@ -82,4 +82,17 @@ describe('<SubscriptionDetail />', () => {
 
     expect(wrapper.find('Button[aria-label="edit"]').length).toBe(1);
   });
+
+  test('should not render Hosts Automated Detail if license_info.automated_instances is undefined', () => {
+    wrapper = mountWithContexts(<SubscriptionDetail />, {
+      context: {
+        config: {
+          ...config,
+          license_info: { ...config.license_info, automated_instances: null },
+        },
+      },
+    });
+
+    expect(wrapper.find(`Detail[label="Hosts automated"]`).length).toBe(0);
+  });
 });

awxkit/awxkit/api/pages/__init__.py

@@ -40,3 +40,4 @@ from .instance_groups import *  # NOQA
 from .credential_input_sources import *  # NOQA
 from .metrics import *  # NOQA
 from .subscriptions import *  # NOQA
+from .workflow_approval_templates import *  # NOQA

awxkit/awxkit/api/pages/api.py

@@ -36,14 +36,15 @@ EXPORTABLE_RELATIONS = ['Roles', 'NotificationTemplates', 'WorkflowJobTemplateNo
 # These are special-case related objects, where we want only in this
 # case to export a full object instead of a natural key reference.
 DEPENDENT_EXPORT = [
-    ('JobTemplate', 'labels'),
+    ('JobTemplate', 'Label'),
-    ('JobTemplate', 'survey_spec'),
+    ('JobTemplate', 'SurveySpec'),
-    ('WorkflowJobTemplate', 'labels'),
+    ('WorkflowJobTemplate', 'Label'),
-    ('WorkflowJobTemplate', 'survey_spec'),
+    ('WorkflowJobTemplate', 'SurveySpec'),
-    ('WorkflowJobTemplate', 'workflow_nodes'),
+    ('WorkflowJobTemplate', 'WorkflowJobTemplateNode'),
-    ('Inventory', 'groups'),
+    ('Inventory', 'Group'),
-    ('Inventory', 'hosts'),
+    ('Inventory', 'Host'),
-    ('Inventory', 'labels'),
+    ('Inventory', 'Label'),
+    ('WorkflowJobTemplateNode', 'WorkflowApprovalTemplate'),
 ]
 
 
@@ -57,6 +58,7 @@ DEPENDENT_NONEXPORT = [
     ('Group', 'all_hosts'),
     ('Group', 'potential_children'),
     ('Host', 'all_groups'),
+    ('WorkflowJobTemplateNode', 'create_approval_template'),
 ]
 
 
@@ -85,6 +87,7 @@ class ApiV2(base.Base):
         # Note: doing _page[key] automatically parses json blob strings, which can be a problem.
         fields = {key: _page.json[key] for key in post_fields if key in _page.json and key not in _page.related and key != 'id'}
 
+        # iterate over direct fields in the object
         for key in post_fields:
             if key in _page.related:
                 related = _page.related[key]
@@ -114,6 +117,12 @@ class ApiV2(base.Base):
                     return None
                 log.warning("Foreign key %r export failed for object %s, setting to null", key, _page.endpoint)
                 continue
+
+            # Workflow approval templates have a special creation endpoint,
+            # therefore we are skipping the export via natural key.
+            if rel_endpoint.__item_class__.__name__ == 'WorkflowApprovalTemplate':
+                continue
+
             rel_natural_key = rel_endpoint.get_natural_key(self._cache)
             if rel_natural_key is None:
                 log.error("Unable to construct a natural key for foreign key %r of object %s.", key, _page.endpoint)
@@ -121,19 +130,38 @@ class ApiV2(base.Base):
                 return None  # This foreign key has unresolvable dependencies
             fields[key] = rel_natural_key
 
+        # iterate over related fields in the object
         related = {}
         for key, rel_endpoint in _page.related.items():
-            if key in post_fields or not rel_endpoint:
+            # skip if no endpoint for this related object
+            if not rel_endpoint:
                 continue
 
             rel = rel_endpoint._create()
+
+            if rel.__item_class__.__name__ != 'WorkflowApprovalTemplate':
+                if key in post_fields:
+                    continue
+
             is_relation = rel.__class__.__name__ in EXPORTABLE_RELATIONS
-            is_dependent = (_page.__item_class__.__name__, key) in DEPENDENT_EXPORT
+
+            # determine if the parent object and the related object that we are processing through are related
+            # if this tuple is in the DEPENDENT_EXPORT than we output the full object
+            # else we output the natural key
+            is_dependent = (_page.__item_class__.__name__, rel.__item_class__.__name__) in DEPENDENT_EXPORT
+
             is_blocked = (_page.__item_class__.__name__, key) in DEPENDENT_NONEXPORT
             if is_blocked or not (is_relation or is_dependent):
                 continue
 
-            rel_post_fields = utils.get_post_fields(rel_endpoint, self._cache)
+            # if the rel is of WorkflowApprovalTemplate type, get rel_post_fields from create_approval_template endpoint
+            rel_option_endpoint = rel_endpoint
+            export_key = key
+            if rel.__item_class__.__name__ == 'WorkflowApprovalTemplate':
+                export_key = 'create_approval_template'
+                rel_option_endpoint = _page.related.get('create_approval_template')
+
+            rel_post_fields = utils.get_post_fields(rel_option_endpoint, self._cache)
             if rel_post_fields is None:
                 log.debug("%s is a read-only endpoint.", rel_endpoint)
                 continue
@@ -147,24 +175,28 @@ class ApiV2(base.Base):
                 continue
 
             rel_page = self._cache.get_page(rel_endpoint)
+
             if rel_page is None:
                 continue
 
             if 'results' in rel_page:
                 results = (x.get_natural_key(self._cache) if by_natural_key else self._export(x, rel_post_fields) for x in rel_page.results)
-                related[key] = [x for x in results if x is not None]
+                related[export_key] = [x for x in results if x is not None]
+            elif rel.__item_class__.__name__ == 'WorkflowApprovalTemplate':
+                related[export_key] = self._export(rel_page, rel_post_fields)
             else:
-                related[key] = rel_page.json
+                related[export_key] = rel_page.json
 
         if related:
             fields['related'] = related
 
-        natural_key = _page.get_natural_key(self._cache)
+        if _page.__item_class__.__name__ != 'WorkflowApprovalTemplate':
-        if natural_key is None:
+            natural_key = _page.get_natural_key(self._cache)
-            log.error("Unable to construct a natural key for object %s.", _page.endpoint)
+            if natural_key is None:
-            self._has_error = True
+                log.error("Unable to construct a natural key for object %s.", _page.endpoint)
-            return None
+                self._has_error = True
-        fields['natural_key'] = natural_key
+                return None
+            fields['natural_key'] = natural_key
 
         return utils.remove_encrypted(fields)
 

awxkit/awxkit/api/pages/workflow_approval_templates.py

@@ -0,0 +1,25 @@
+from awxkit.api.pages.unified_job_templates import UnifiedJobTemplate
+from awxkit.api.resources import resources
+from . import page
+
+
+class WorkflowApprovalTemplate(UnifiedJobTemplate):
+
+    pass
+
+
+page.register_page(
+    [
+        resources.workflow_approval_template,
+        resources.workflow_job_template_node_create_approval_template,
+    ],
+    WorkflowApprovalTemplate,
+)
+
+
+class WorkflowApprovalTemplates(page.PageList, WorkflowApprovalTemplate):
+
+    pass
+
+
+page.register_page(resources.workflow_approval_templates, WorkflowApprovalTemplates)

awxkit/awxkit/api/resources.py

@@ -82,6 +82,9 @@ class Resources(object):
     _inventory_variable_data = r'inventories/\d+/variable_data/'
     _workflow_approval = r'workflow_approvals/\d+/'
     _workflow_approvals = 'workflow_approvals/'
+    _workflow_approval_template = r'workflow_approval_templates/\d+/'
+    _workflow_approval_templates = 'workflow_approval_templates/'
+    _workflow_job_template_node_create_approval_template = r'workflow_job_template_nodes/\d+/create_approval_template/'
     _job = r'jobs/\d+/'
     _job_cancel = r'jobs/\d+/cancel/'
     _job_create_schedule = r'jobs/\d+/create_schedule/'

docs/release_process.md

@@ -2,6 +2,7 @@
 
 The release process for AWX is completely automated as of version 19.5.0.
 
+If you need to revert a release, please refer to the [Revert a Release](#revert-a-release) section.
 ## Get latest release version and list of new work
 
 1. Open the main project page for [AWX](https://github.com/ansible/awx/releases) and [AWX Operator](https://github.com/ansible/awx-operator/releases).
@@ -135,3 +136,33 @@ We're happy to announce that [AWX version 21.1.0](https://github.com/ansible/awx
 We're happy to announce that [AWX Operator version 0.22.0](https://github.com/ansible/awx-operator/releases/tag/0.22.0) is now available!
 
 ## Send the same IRC message (less the @newsbot) to #awx:ansible.com
+
+## Revert a Release
+
+Decide whether or not you can just fall-forward with a new AWX Release to fix a bad release. If you need to remove published artifacts from publically facing repositories, follow the steps below.
+
+Here are the steps needed to revert an AWX and an AWX-Operator release. Depending on your use case, follow the steps for reverting just an AWX release, an Operator release or both.
+
+
+1. Navigate to the [AWX Release Page](https://github.com/ansible/awx/releases) and delete the AWX Release that needs to be removed.
+
+![Revert-1-Image](img/revert-1.png)
+
+2. Navigate to the [AWX Tags Page](https://github.com/ansible/awx/tags) and delete the AWX Tag that got created by the Github Actions Workflow from when you originally tried to release AWX. You need delete the release in step 1 before you can do this step. The tag must not be tied to a release if you want to delete a tag.
+
+![Tag-Revert-1-Image](img/tag-revert-1.png)
+[comment]: <> (Need an image here for actually deleting an orphaned tag, place here during next release)
+
+3. Navigate to the [AWX Operator Release Page]() and delete the AWX-Operator release that needss to tbe removed.
+
+![Revert-2-Image](img/revert-2.png)
+
+4. Navigate to [quay.io](https://quay.io/repository/ansible/awx?tag=latest&tab=tags) and delete the published AWX image(s) and tags.
+
+5. Navigate to [quay.io](https://github.com/ansible/awx-operator/releases) and delete the published AWX Operator image(s) and tags.
+
+6. Navigate to the [Ansible Galaxy Collections](https://galaxy.ansible.com/awx/awx) website and remove the published AWX collection with the bad tag.
+
+7. Navigate to the [PyPi](https://pypi.org/project/awxkit/#history) and delete the bad AWX tag and release that got published.
+
+8. [Restart the Release Process](#releasing-awx-and-awx-operator)

requirements/requirements.in

@@ -38,6 +38,7 @@ prometheus_client
 psycopg2
 psutil
 pygerduty
+pyjwt>=2.4.0 # https://github.com/ansible/awx/security/dependabot/58
 pyparsing
 python3-saml==1.13.0
 python-dsv-sdk

requirements/requirements.txt

@@ -258,8 +258,9 @@ pycparser==2.20
     # via cffi
 pygerduty==0.38.2
     # via -r /awx_devel/requirements/requirements.in
-pyjwt==2.3.0
+pyjwt==2.4.0
     # via
+    #   -r /awx_devel/requirements/requirements.in
     #   adal
     #   social-auth-core
     #   twilio