AAP-17690 Inventory variables sourced from git project are not getting deleted after being removed from source (#15928)

* Delete existing all-group vars on inventory sync (with overwrite-vars=True) instead of merging them.

* Implementation of inv var handling with file as db.

* Improve serialization to file of inv vars for src update

* Include inventory-level variable editing into inventory source update handling

* Add group vars to inventory source update handling

* Add support for overwrite_vars to new inventory source handling

* Persist inventory var history in the database instead of a file.

* Remove logging which was needed during development.

* Remove further debugging code and improve comments

* Move special handling for user edits of variables into serializers

* Relate the inventory variable history model to its inventory

* Allow for inventory variables to have the value 'None'

* Fix KeyError in new inventory variable handling

* Add unique-together constraint for new model InventoryGroupVariablesWithHistory

* Use only one special invsrc_id for initial update and manual updates

* Fix internal server error when creating a new inventory

* Print the empty string for a variable with value 'None'

* Fix comment which incorrectly states old behaviour

* Fix inventory_group_variables_update tests which did not take the new handling of None into account

* Allow any type for Ansible-core variable values

* Refactor misleading method names

* Fix internal server error when savig vars from group form

* Remove superfluous json conversion in front of JSONField

* Call variable update from create/update instead from validate

* Use group_id instead of group_name in model InventoryGroupVariablesWithHistory

* Disable new variable update handling for all regular (non-'all') groups

* Add live test to verify AAP-17690 (inv var deleted from source)

* Add functional tests to verify inventory variables update logic

* Fix migration which was corrupted by a rebase

* Add a more complex live test and resolve linter complaints

* Force overwrite_vars=False for updates from source on all-group

* Change behavior with respect to overwrite_vars

awx/main/management/commands/inventory_import.py

@@ -33,6 +33,7 @@ from awx.main.utils.safe_yaml import sanitize_jinja
 from awx.main.models.rbac import batch_role_ancestor_rebuilding
 from awx.main.utils import ignore_inventory_computed_fields, get_licenser
 from awx.main.utils.execution_environments import get_default_execution_environment
+from awx.main.utils.inventory_vars import update_group_variables
 from awx.main.signals import disable_activity_stream
 from awx.main.constants import STANDARD_INVENTORY_UPDATE_ENV
 
@@ -457,19 +458,19 @@ class Command(BaseCommand):
         """
         Update inventory variables from "all" group.
         """
-        # TODO: We disable variable overwrite here in case user-defined inventory variables get
-        # mangled. But we still need to figure out a better way of processing multiple inventory
-        # update variables mixing with each other.
-        # issue for this: https://github.com/ansible/awx/issues/11623
-
         if self.inventory.kind == 'constructed' and self.inventory_source.overwrite_vars:
             # NOTE: we had to add a exception case to not merge variables
             # to make constructed inventory coherent
             db_variables = self.all_group.variables
         else:
-            db_variables = self.inventory.variables_dict
-            db_variables.update(self.all_group.variables)
-
+            db_variables = update_group_variables(
+                group_id=None,  # `None` denotes the 'all' group (which doesn't have a pk).
+                newvars=self.all_group.variables,
+                dbvars=self.inventory.variables_dict,
+                invsrc_id=self.inventory_source.id,
+                inventory_id=self.inventory.id,
+                overwrite_vars=self.overwrite_vars,
+            )
         if db_variables != self.inventory.variables_dict:
             self.inventory.variables = json.dumps(db_variables)
             self.inventory.save(update_fields=['variables'])