Unpin channels-redis (#15329) (#6647)

* unpin channels-redis

The bug that initially caused the upgrade block has been resolved https://github.com/django/channels_redis/issues/332

* replace aioredis Exception with a redis Exception

Version 4.0.0 of channel-redis migrated the underlying Redis library from aioredis to redis-py. The Exception has been changed to an equivalent

* remove unused license

* remove UPGRADE BLOCKER in README

* remove hiredis

it was an indirect dependency from aioredis which was removed

* remove unused license

* add back hiredis

it's potentially providing a performance boost. install explicitly as a part of redis. upgrade to more recent version

* remove UPGRADE BLOCKER for hiredis

it was also addressed as a part of this PR

awx/main/wsrelay.py

@@ -8,7 +8,7 @@ import ipaddress
 
 import aiohttp
 from aiohttp import client_exceptions
-import aioredis
+import redis
 
 from channels.layers import get_channel_layer
 
@@ -199,7 +199,7 @@ class WebsocketRelayConnection:
                         return
 
                     continue
-                except aioredis.errors.ConnectionClosedError:
+                except redis.exceptions.ConnectionError:
                     logger.info(f"Producer {name} lost connection to Redis, shutting down.")
                     return
 

licenses/aioredis.txt

@@ -1,22 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2014-2017 Alexey Popravka
-Copyright (c) 2021 Sean Stewart
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

requirements/README.md

@@ -120,24 +120,6 @@ in the top-level Makefile.
 
 If modifying this library make sure testing with the offline build is performed to confirm it is functionally working.
 
-### channels-redis
-
-Due to an upstream bug (linked below), we see `RuntimeError: Event loop is closed` errors with newer versions of `channels-redis`.
-Upstream is aware of the bug and it is likely to be fixed in the next release according to the issue linked below.
-For now, we pin to the old version, 3.4.1
-
-* https://github.com/django/channels_redis/issues/332
-* https://github.com/ansible/awx/issues/13313
-
-### hiredis
-
-The hiredis 2.1.0 release doesn't provide source distribution on PyPI which prevents users to build that python package from the
-sources.
-Downgrading to 2.0.0 (which provides source distribution) until the channels-redis issue is fixed or a newer hiredis version is
-available on PyPi with source distribution.
-
-* https://github.com/redis/hiredis-py/issues/138
-
 ## Library Notes
 
 ### pexpect

requirements/requirements.in

@@ -7,7 +7,7 @@ azure-keyvault
 boto3
 botocore
 channels
-channels-redis==3.4.1  # see UPGRADE BLOCKERs
+channels-redis
 cryptography>=41.0.7  # CVE-2023-49083
 Cython<3 # due to https://github.com/yaml/pyyaml/pull/702
 daphne
@@ -29,7 +29,6 @@ djangorestframework-yaml
 filelock
 GitPython>=3.1.37  # CVE-2023-41040
 grpcio<1.63.0 # 1.63.0+ requires cython>=3
-hiredis==2.0.0  # see UPGRADE BLOCKERs
 irc
 jinja2>=3.1.3  # CVE-2024-22195
 JSON-log-formatter
@@ -60,7 +59,7 @@ receptorctl
 social-auth-core[openidconnect]==4.4.2  # see UPGRADE BLOCKERs
 social-auth-app-django==5.4.0  # see UPGRADE BLOCKERs
 sqlparse>=0.4.4   # Required by django https://github.com/ansible/awx/security/dependabot/96
-redis
+redis[hiredis]
 requests
 slack-sdk
 tacacs_plus==1.0  # UPGRADE BLOCKER: auth does not work with later versions

requirements/requirements.txt

@@ -7,8 +7,6 @@ aiohttp==3.9.5
     #   twilio
 aiohttp-retry==2.8.3
     # via twilio
-aioredis==1.3.1
-    # via channels-redis
 aiosignal==1.3.1
     # via aiohttp
 annotated-types==0.6.0
@@ -30,7 +28,6 @@ asn1==2.7.0
 async-timeout==4.0.3
     # via
     #   aiohttp
-    #   aioredis
     #   redis
 attrs==23.2.0
     # via
@@ -88,7 +85,7 @@ channels==3.0.5
     # via
     #   -r /awx_devel/requirements/requirements.in
     #   channels-redis
-channels-redis==3.4.1
+channels-redis==4.2.0
     # via -r /awx_devel/requirements/requirements.in
 charset-normalizer==3.3.2
     # via requests
@@ -201,10 +198,8 @@ grpcio==1.62.2
     # via
     #   -r /awx_devel/requirements/requirements.in
     #   opentelemetry-exporter-otlp-proto-grpc
-hiredis==2.0.0
+hiredis==3.0.0
-    # via
+    # via redis
-    #   -r /awx_devel/requirements/requirements.in
-    #   aioredis
 hyperlink==21.0.0
     # via
     #   autobahn
@@ -449,8 +444,10 @@ pyzstd==0.15.10
     # via -r /awx_devel/requirements/requirements.in
 receptorctl==1.4.4
     # via -r /awx_devel/requirements/requirements.in
-redis==5.0.1
+redis[hiredis]==5.0.1
-    # via -r /awx_devel/requirements/requirements.in
+    # via
+    #   -r /awx_devel/requirements/requirements.in
+    #   channels-redis
 referencing==0.33.0
     # via
     #   jsonschema