* There are 2 data-structures that django channels redis uses: (1) zset
and (2) list. (1) is used for group membership where the key is the
logic user group and the value(s) are websocket clients. The score of
the zset entry is used for group expiration. We can not rely on group
expiration for clean-up because there is no interface privided by redis
channels to refresh the expiration. Choosing a small value for
group_expiry could result on our websocket backplane group expiring,
which would result in job events not being delivered. Instead, we
increase the group expiration to 5 years and clean up on daphne service
start.
* The list (2) data-structure is used by django channels redis to queue
websocket events per-websocket-client as needed. The need arises to
queue per-websocket-client events when the consumer can not keep up with
the producer. The consumer here is daphne, the producer is AWX.
* When AWX is operating healthy group membership in Redis is reflective
of the real-world. When AWX is unhealthy i.e. daphne cycles, the zset
will contain stale websocket client entries. This can be observed by
running `zrange asgi::group:jobs-status_changed 0 -1`. If the entries
returned look like:
specific.fUkXXpYj!DKOIfwPICNgw
specific.fUkXXpYj!FQcdopZeiRdG
specific.lpTSAgnk!IOKldfzcfdDp
specific.lpTSAgnk!NbvRUZsDpIQx
The entries with `fUkXXpYj` are stale. Note that this changeset fixes
this by removing all `asgi:*` entries on daphne start.
* Also note that individual message themselves have an expiration that
is configurable and defaults to 60.
* Also note that zset's tracking group membership will be deleted by
django channels redis when they are empty.
* Sending health about websockets over websockets is not a great idea.
* I tried sending health data via prometheus and encountered problems
that will need PR's to prometheus_client library to solve. Circle back
to this later.
* Separates test messages from application messages
* Removes test runner and groups, processes, and streams from network_ui
* Adds network_ui_test
* Fixes routing for network_ui_test
* Removes coverage_report tool from network_ui
* Fixes network_ui_test test workflow
* Sets width and height of the page during tests
The ansible-network-ui prototype project builds a standalone Network UI
outside of Tower as its own Django application. The original prototype
code is located here:
https://github.com/benthomasson/ansible-network-ui.
The prototype provides a virtual canvas that supports placing
networking devices onto 2D plane and connecting those devices together
with connections called links. The point where the link connects
to the network device is called an interface. The devices, interfaces,
and links may all have their respective names. This models physical
networking devices is a simple fashion.
The prototype implements a pannable and zoomable 2D canvas in using SVG
elements and AngularJS directives. This is done by adding event
listeners for mouse and keyboard events to an SVG element that fills the
entire browser window.
Mouse and keyboard events are handled in a processing pipeline where
the processing units are implemented as finite state machines that
provide deterministic behavior to the UI.
The finite state machines are built in a visual way that makes
the states and transitions clearly evident. The visual tool for
building FSM is located here:
https://github.com/benthomasson/fsm-designer-svg. This tool
is a fork of this project where the canvas is the same. The elements
on the page are FSM states and the directional connections are called
transitions. The bootstrapping of the FSM designer tool and
network-ui happen in parallel. It was useful to try experiemental
code in FSM designer and then import it into network-ui.
The FSM designer tool provides a YAML description of the design
which can be used to generate skeleton code and check the implementation
against the design for discrepancies.
Events supported:
* Mouse click
* Mouse scroll-wheel
* Keyboard events
* Touch events
Interactions supported:
* Pan canvas by clicking-and-dragging on the background
* Zooming canvas by scrolling mousewheel
* Adding devices and links by using hotkeys
* Selecting devices, interaces, and links by clicking on their icon
* Editing labels on devices, interfaces, and links by double-clicking on
their icon
* Moving devices around the canvas by clicking-and-dragging on their
icon
Device types supported:
* router
* switch
* host
* racks
The database schema for the prototype is also developed with a visual
tool that makes the relationships in the snowflake schema for the models
quickly evident. This tool makes it very easy to build queries across
multiple tables using Django's query builder.
See: https://github.com/benthomasson/db-designer-svg
The client and the server communicate asynchronously over a websocket.
This allows the UI to be very responsive to user interaction since
the full request/response cycle is not needed for every user
interaction.
The server provides persistence of the UI state in the database
using event handlers for events generated in the UI. The UI
processes mouse and keyboard events, updates the UI, and
generates new types of events that are then sent to the server
to be persisted in the database.
UI elements are tracked by unique ids generated on the client
when an element is first created. This allows the elements to
be correctly tracked before they are stored in the database.
The history of the UI is stored in the TopologyHistory model
which is useful for tracking which client made which change
and is useful for implementing undo/redo.
Each message is given a unique id per client and has
a known message type. Message types are pre-populated
in the MessageType model using a database migration.
A History message containing all the change messages for a topology is
sent when the websocket is connected. This allows for undo/redo work
across sessions.
This prototype provides a server-side test runner for driving
tests in the user interface. Events are emitted on the server
to drive the UI. Test code coverage is measured using the
istanbul library which produces instrumented client code.
Code coverage for the server is is measured by the coverage library.
The test code coverage for the Python code is 100%.
