diff --git a/awx/network_ui/CONTRIBUTING.md b/awx/network_ui/CONTRIBUTING.md new file mode 100644 index 0000000000..cd028ef1e1 --- /dev/null +++ b/awx/network_ui/CONTRIBUTING.md @@ -0,0 +1,133 @@ +Network UI +========== + +See [awx/ui/client/src/network-ui/CONTRIBUTING.md](../ui/client/src/network-ui/CONTRIBUTING.md) for the introduction +to the Network UI client-side development. + +Server-Side Development +----------------------- + +This document covers the Network UI server-side development. + +The Network UI is a UX driven feature to provide a graphical user +experience that fits well into the network engineer's normal workflow. Their +normal workflow includes a diagram drawn in a graphical drawing program, a +spreadsheet, and the command line interface of their network gear. Network +architects design the network on the graphical diagram and then hand off the +architecture to network operators who implement the architecture on the network +using spreadsheets to manage their data and manually converting the data into +CLI commands using their networking expertise and expertise with their physical +gear. + +The server-side code supports the persistence needed to provide this graphical +user experience of architecting a network and using that information along with +additional information (stored in vars files) to configure the network devices +using the CLI or NETCONF using Ansible playbooks and roles. + +Network UI Data Schema +---------------------- + +For the 3.3 release the persistence needed includes the position information of +the devices on the virtual canvas and the type of the devices as well as +information about the interfaces on the devices and the links connecting those +interfaces. + +These requirements determine the database schema needed for the network UI which +requires these models: Topology, Device, Interface, Link, Client, and TopologyInventory. + +![Models](designs/models.png) + +This diagram shows the relationships between the models in the Network UI schema. + +The models are: + +* Device - a host, switch, router, or other networking device +* Interface - a connection point on a device for a link +* Link - a physical connection between two devices to their respective interfaces +* Topology - a collection of devices and links +* TopologyInventory - a mapping between topologies and Tower inventories +* Client - a UI client session + + +Network UI Websocket Protocol +----------------------------- + +Persistence for the network UI canvas state is implemented using an +asynchronous websocket protocol to send information from the client to the +server and vice-versa. This two-way communication was chosen to support future +features for streaming data to the canvas, broadcast messaging between clients, +and for interaction performance on the UI. + + +Messages +-------- + +JSON messages are passed over the `/network_ui/topology` websocket between the +test client and the test server. The protocol that is used for all messages is +in ABNF (RFC5234): + + + message_type = 'DeviceMove' / 'DeviceCreate' / 'DeviceDestroy' / 'DeviceLabelEdit' / 'DeviceSelected' / 'DeviceUnSelected' / 'InterfaceCreate' / 'InterfaceLabelEdit' / 'LinkLabelEdit' / 'LinkCreate' / 'LinkDestroy' / 'LinkSelected' / 'LinkUnSelected' / 'MultipleMessage' / 'Snapshot' + message_data = '{' 'msg_type' ': ' message_type ', ' key-value *( ', ' key-value ) '}' + message = '[ id , ' posint ']' / '[ topology_id , ' posint ']' / '[' message_type ', ' message_data ']' + +See https://github.com/AndyA/abnfgen/blob/master/andy/json.abnf for the rest of +the JSON ABNF. + +See [designs/messages.yml](designs/messages.yml) for the allowable keys and +values for each message type. + + +Initially when the websocket is first opened the server will send four messages +to the client. These are: + +* the client id using the `id` message type. +* the topology id using the `topology` message type. +* a Topology record containing data for the canvas itself. +* a Snapshot message containing all the data of the data on the canvas. + +As the user interacts with the canvas messages will be generated by the client +and the `network_ui.consumers.Persistence` class will update the models that +represent the canvas. + + + +Persistence +----------- + +The class `awx.network_uiconsumers.Persistence` provides persistence for the Network UI canvas. +It does so by providing message handlers that handle storage of the canvas change events +into the database. Each event has a message handle with name `onX` where `X` is the name of the message +type. The handlers use the `filter/values_list`, `filter/values`, `filter/update`, and `filter/delete` +patterns to update the data in the database quickly with a constant O(1) number of queries per event +often with only one query needed. With `filter/update` and `filter/delete` all the work is done +in the database and Python never needs to instaniate and garbage collect the model objects. + +Bulk operations (`filter/values`) in `send_snapshot` are used to produce a constant number of +queries produce a snapshot when the canvas is first loaded. This method avoids creating +the model objects since it only produces dicts that are JSON serializable which are bundled +together for the `Snapshot` message type. + +This method of persistence uses Django as a database query-compiler for transforms from +the event types to the database types. Using Django in this way is very performant since +Python does very little work processing the data and when possible the data never leaves +the database. + + +Client Tracking +--------------- + +Each user session to the network UI canvas is tracked with the `Client` model. Multiple +clients can view and interact with the network UI canvas at a time. They will see each other's +edits to the canvas in real time. This works by broadcasting the canvas change events to +all clients viewing the same topology. + +``` + # Send to all clients editing the topology + Group("topology-%s" % message.channel_session['topology_id']).send({"text": message['text']}) +``` + +API +--- + +There is no user accessible API for this feature in the 3.3 release. diff --git a/awx/network_ui/designs/README.md b/awx/network_ui/designs/README.md index 51772417dd..2c8ff94bda 100644 --- a/awx/network_ui/designs/README.md +++ b/awx/network_ui/designs/README.md @@ -4,6 +4,5 @@ The design files in this directory are used in the database schema designer tool * [models.png](models.png) - An image of the database schema design for network UI. * [models.yml](models.yml) - Provides the main schema design for the network UI project. -* [api.yml](api.yml) - Provides additional meta-data for the API. ![Models](models.png) diff --git a/awx/network_ui/designs/api.yml b/awx/network_ui/designs/api.yml deleted file mode 100644 index 590547908b..0000000000 --- a/awx/network_ui/designs/api.yml +++ /dev/null @@ -1,65 +0,0 @@ -models: -- name: Device - api: true - v2_end_point: /api/v2/canvas/device/ - topology_id_query: topology_id - v2_lookup_field: host_id - create_transform: - id: id - name: name - device_type: type - x: x - y: y - interface_id_seq: interface_id_seq - process_id_seq: process_id_seq - host_id: host_id - topology_id: topology_id -- name: Link - api: true - v2_end_point: /api/v2/canvas/link/ - topology_id_query: from_device__topology_id - create_transform: - id: id - name: name - from_device__id: from_device_id - from_interface__id: from_interface_id - to_device__id: to_device_id - to_interface__id: to_interface_id -- name: Interface - api: true - v2_end_point: /api/v2/canvas/interface/ - topology_id_query: device__topology_id - create_transform: - id: id - name: name - device__id: device_id -- name: Group - api: true - v2_end_point: /api/v2/canvas/group/ - topology_id_query: topology_id -- name: GroupDevice - api: true - v2_end_point: /api/v2/canvas/groupdevice/ - topology_id_query: group__topology_id -- name: Topology - api: true - v2_end_point: /api/v2/canvas/topology/ - topology_id_query: topology_id -- name: TopologyInventory - api: true - v2_end_point: /api/v2/canvas/topologyinventory/ - topology_id_query: topology_id -- name: Toolbox - api: true - v2_end_point: /api/v2/canvas/toolbox/ -- name: ToolboxItem - api: true - v2_end_point: /api/v2/canvas/toolboxitem/ -- name: Stream - api: true - v2_end_point: /api/v2/canvas/stream/ - topology_id_query: from_device__topology_id -- name: Process - api: true - v2_end_point: /api/v2/canvas/process/ - topology_id_query: device__topology_id diff --git a/awx/network_ui/designs/messages.yml b/awx/network_ui/designs/messages.yml index ef1914d6f7..06ee3b9b75 100644 --- a/awx/network_ui/designs/messages.yml +++ b/awx/network_ui/designs/messages.yml @@ -1,7 +1,5 @@ messages: - {msg_type: DeviceMove, fields: [msg_type, sender, id, x, y, previous_x, previous_y]} - - {msg_type: DeviceInventoryUpdate, fields: [msg_type, sender, id, host_id]} - - {msg_type: GroupInventoryUpdate, fields: [msg_type, sender, id, group_id]} - {msg_type: DeviceCreate, fields: [msg_type, sender, id, x, y, name, type, host_id]} - {msg_type: DeviceDestroy, fields: [msg_type, sender, id, previous_x, previous_y, previous_name, previous_type, previous_host_id]} - {msg_type: DeviceLabelEdit, fields: [msg_type, sender, id, name, previous_name]} @@ -14,40 +12,8 @@ messages: - {msg_type: LinkDestroy, fields: [msg_type, id, sender, name, from_device_id, to_device_id, from_interface_id, to_interface_id]} - {msg_type: LinkSelected, fields: [msg_type, sender, id]} - {msg_type: LinkUnSelected, fields: [msg_type, sender, id]} - - {msg_type: Undo, fields: [msg_type, sender, original_message]} - - {msg_type: Redo, fields: [msg_type, sender, original_message]} - - {msg_type: Deploy, fields: [msg_type, sender]} - - {msg_type: Destroy, fields: [msg_type, sender]} - - {msg_type: Discover, fields: [msg_type, sender]} - - {msg_type: Layout, fields: [msg_type, sender]} - {msg_type: MultipleMessage, fields: [msg_type, sender, messages]} - - {msg_type: MouseEvent, fields: [msg_type, sender, x, y, type, trace_id]} - - {msg_type: MouseWheelEvent, fields: [msg_type, sender, delta, deltaX, deltaY, type, originalEvent, trace_id]} - - {msg_type: KeyEvent, fields: [msg_type, sender, key, keyCode, type, altKey, shiftKey, ctrlKey, metaKey, trace_id]} - - {msg_type: StartRecording, fields: [msg_type, sender, trace_id]} - - {msg_type: StopRecording, fields: [msg_type, sender, trace_id]} - - {msg_type: ViewPort, fields: [msg_type, sender, scale, panX, panY, trace_id]} - - {msg_type: CopySite, fields: [msg_type, site]} - - {msg_type: GroupMove, fields: [msg_type, sender, id, x1, y1, x2, y2, previous_x1, previous_y1, previous_x2, previous_y2]} - - {msg_type: GroupCreate, fields: [msg_type, sender, id, x1, y1, x2, y2, name, type, group_id]} - - {msg_type: GroupDestroy, fields: [msg_type, sender, id, previous_x1, previous_y1, previous_x2, previous_y2, previous_name, previous_type, previous_group_id]} - - {msg_type: GroupLabelEdit, fields: [msg_type, sender, id, name, previous_name]} - - {msg_type: GroupSelected, fields: [msg_type, sender, id]} - - {msg_type: GroupUnSelected, fields: [msg_type, sender, id]} - - {msg_type: GroupMembership, fields: [msg_type, sender, id, members]} - - {msg_type: ProcessCreate, fields: [msg_type, id, name, type, device_id, x, y]} - - {msg_type: StreamCreate, fields: [msg_type, sender, id, from_id, to_id, label]} - - {msg_type: StreamDestroy, fields: [msg_type, sender, id, from_id, to_id, label]} - - {msg_type: StreamLabelEdit, fields: [msg_type, sender, id, label, previous_label]} - - {msg_type: StreamSelected, fields: [msg_type, sender, id]} - - {msg_type: StreamUnSelected, fields: [msg_type, sender, id]} - - {msg_type: FSMTrace, fields: [msg_type, order, sender, trace_id, fsm_name, from_state, to_state, recv_message_type]} - - {msg_type: ChannelTrace, fields: [msg_type, sender, trace_id, from_fsm, to_fsm, sent_message_type]} - - {msg_type: Snapshot, fields: [msg_type, sender, devices, links, groups, streams, order, trace_id]} - - {msg_type: EnableTest, fields: [msg_type]} - - {msg_type: DisableTest, fields: [msg_type]} - - {msg_type: StartTest, fields: [msg_type]} - - {msg_type: TestCompleted, fields: [msg_type]} - - {msg_type: TestResult, fields: [msg_type, sender, id, name, result, date, code_under_test]} - - {msg_type: Coverage, fields: [msg_type, sender, coverage, result_id]} - + - {msg_type: Snapshot, fields: [msg_type, sender, devices, links, order, trace_id]} + - {msg_type: id, type: int} + - {msg_type: topology_id, type: int} + - {msg_type: Topology, fields: [topology_id, name, panX, panY, scale, link_id_seq, device_id_seq]} diff --git a/awx/network_ui/designs/models.png b/awx/network_ui/designs/models.png index 012a5a74ca..c6b22910d8 100644 Binary files a/awx/network_ui/designs/models.png and b/awx/network_ui/designs/models.png differ diff --git a/awx/network_ui/designs/models.yml b/awx/network_ui/designs/models.yml index a8a32525a8..683dde4bfd 100644 --- a/awx/network_ui/designs/models.yml +++ b/awx/network_ui/designs/models.yml @@ -1,18 +1,7 @@ app: awx.network_ui external_models: [] models: -- api: true - create_transform: - device_type: type - host_id: host_id - id: id - interface_id_seq: interface_id_seq - name: name - process_id_seq: process_id_seq - topology_id: topology_id - x: x - y: y - display: name +- display: name fields: - name: device_id pk: true @@ -36,27 +25,13 @@ models: - default: 0 name: interface_id_seq type: IntegerField - - default: 0 - name: process_id_seq - type: IntegerField - default: 0 name: host_id type: IntegerField name: Device - topology_id_query: topology_id - v2_end_point: /api/v2/canvas/device/ - v2_lookup_field: host_id x: 348 y: 124 -- api: true - create_transform: - from_device__id: from_device_id - from_interface__id: from_interface_id - id: id - name: name - to_device__id: to_device_id - to_interface__id: to_interface_id - fields: +- fields: - name: link_id pk: true type: AutoField @@ -86,12 +61,9 @@ models: name: name type: CharField name: Link - topology_id_query: from_device__topology_id - v2_end_point: /api/v2/canvas/link/ - x: 837 - y: 10 -- api: true - display: name + x: 731 + y: -33 +- display: name fields: - name: topology_id pk: true @@ -111,15 +83,7 @@ models: - default: 0 name: link_id_seq type: IntegerField - - default: 0 - name: group_id_seq - type: IntegerField - - default: 0 - name: stream_id_seq - type: IntegerField name: Topology - topology_id_query: topology_id - v2_end_point: /api/v2/canvas/topology/ x: 111 y: 127 - fields: @@ -127,51 +91,9 @@ models: pk: true type: AutoField name: Client - x: -518 - y: 138 -- fields: - - name: topology_history_id - pk: true - type: AutoField - - name: topology - ref: Topology - ref_field: topology_id - type: ForeignKey - - name: client - ref: Client - ref_field: client_id - type: ForeignKey - - name: message_type - ref: MessageType - ref_field: message_type_id - type: ForeignKey - - name: message_id - type: IntegerField - - name: message_data - type: TextField - - default: false - name: undone - type: BooleanField - name: TopologyHistory - x: -205 + x: -162 y: 282 - display: name - fields: - - name: message_type_id - pk: true - type: AutoField - - len: 200 - name: name - type: CharField - name: MessageType - x: -501 - y: 428 -- api: true - create_transform: - device__id: device_id - id: id - name: name - display: name fields: - name: interface_id pk: true @@ -186,170 +108,9 @@ models: - name: id type: IntegerField name: Interface - topology_id_query: device__topology_id - v2_end_point: /api/v2/canvas/interface/ - x: 1157 - y: 337 -- api: true - fields: - - name: group_id - pk: true - type: AutoField - - name: id - type: IntegerField - - len: 200 - name: name - type: CharField - - name: x1 - type: IntegerField - - name: y1 - type: IntegerField - - name: x2 - type: IntegerField - - name: y2 - type: IntegerField - - name: topology - ref: Topology - ref_field: topology_id - type: ForeignKey - - len: 200 - name: group_type - type: CharField - - default: 0 - name: inventory_group_id - type: IntegerField - name: Group - topology_id_query: topology_id - v2_end_point: /api/v2/canvas/group/ - x: 407 - y: -379 -- api: true - fields: - - name: group_device_id - pk: true - type: AutoField - - name: group - ref: Group - ref_field: group_id - type: ForeignKey - - name: device - ref: Device - ref_field: device_id - type: ForeignKey - name: GroupDevice - topology_id_query: group__topology_id - v2_end_point: /api/v2/canvas/groupdevice/ - x: 739 - y: -234 -- api: true - fields: - - name: stream_id - pk: true - ref: Stream - ref_field: stream_id - type: AutoField - - name: from_device - ref: Device - ref_field: device_id - related_name: from_stream - type: ForeignKey - - name: to_device - ref: Device - ref_field: device_id - related_name: to_stream - type: ForeignKey - - len: 200 - name: label - type: CharField - - default: 0 - name: id - type: IntegerField - name: Stream - topology_id_query: from_device__topology_id - v2_end_point: /api/v2/canvas/stream/ - x: 709 - y: 527 -- api: true - fields: - - name: process_id - pk: true - type: AutoField - - name: device - ref: Device - ref_field: device_id - type: ForeignKey - - len: 200 - name: name - type: CharField - - len: 200 - name: process_type - type: CharField - - default: 0 - name: id - type: IntegerField - name: Process - topology_id_query: device__topology_id - v2_end_point: /api/v2/canvas/process/ - x: 654 - y: 778 -- api: true - fields: - - name: toolbox_id - pk: true - type: AutoField - - len: 200 - name: name - type: CharField - name: Toolbox - v2_end_point: /api/v2/canvas/toolbox/ - x: 179 - y: 644 -- api: true - fields: - - name: toolbox_item_id - pk: true - type: AutoField - - name: toolbox - ref: Toolbox - ref_field: toolbox_id - type: ForeignKey - - name: data - type: TextField - name: ToolboxItem - v2_end_point: /api/v2/canvas/toolboxitem/ - x: 391 - y: 645 + x: 977 + y: 312 - fields: - - name: fsm_trace_id - pk: true - type: AutoField - - len: 200 - name: fsm_name - type: CharField - - len: 200 - name: from_state - type: CharField - - len: 200 - name: to_state - type: CharField - - len: 200 - name: message_type - type: CharField - - name: client - ref: Client - ref_field: client_id - type: ForeignKey - - default: 0 - name: trace_session_id - type: IntegerField - - default: 0 - name: order - type: IntegerField - name: FSMTrace - x: -872 - y: 507 -- api: true - fields: - name: topology_inventory_id pk: true type: AutoField @@ -360,137 +121,10 @@ models: - name: inventory_id type: IntegerField name: TopologyInventory - topology_id_query: topology_id - v2_end_point: /api/v2/canvas/topologyinventory/ - x: -226 - y: -19 -- fields: - - name: event_trace_id - pk: true - type: AutoField - - name: client - ref: Client - ref_field: client_id - type: ForeignKey - - default: 0 - name: trace_session_id - type: IntegerField - - name: event_data - type: TextField - - name: message_id - type: IntegerField - name: EventTrace - x: -1087 - y: 202 -- fields: - - name: coverage_id - pk: true - type: AutoField - - name: coverage_data - type: TextField - - name: test_result - ref: TestResult - ref_field: test_result_id - type: ForeignKey - name: Coverage - x: -1068 - y: -4 -- fields: - - name: topology_snapshot_id - pk: true - type: AutoField - - name: client - ref: Client - ref_field: client_id - type: ForeignKey - - name: topology_id - type: IntegerField - - name: trace_session_id - type: IntegerField - - name: snapshot_data - ref: TopologySnapshot - ref_field: snapshot_data - type: TextField - - name: order - type: IntegerField - name: TopologySnapshot - x: -1123 - y: -277 -- fields: - - name: test_case_id - pk: true - type: AutoField - - len: 200 - name: name - ref: TestCase - ref_field: name - type: CharField - - name: test_case_data - type: TextField - name: TestCase - x: -1642 - y: -38 -- fields: - - name: result_id - pk: true - type: AutoField - - len: 20 - name: name - type: CharField - name: Result - x: -1610 - y: 120 -- fields: - - name: code_under_test_id - pk: true - ref: CodeUnderTest - ref_field: code_under_test_id - type: AutoField - - name: version_x - type: IntegerField - - name: version_y - type: IntegerField - - name: version_z - type: IntegerField - - name: commits_since - type: IntegerField - - len: 40 - name: commit_hash - type: CharField - name: CodeUnderTest - x: -1612 - y: 259 -- fields: - - name: test_result_id - pk: true - type: AutoField - - name: test_case - ref: TestCase - ref_field: test_case_id - type: ForeignKey - - name: result - ref: Result - ref_field: result_id - type: ForeignKey - - name: code_under_test - ref: CodeUnderTest - ref_field: code_under_test_id - type: ForeignKey - - name: time - type: DateTimeField - - default: 0 - name: id - type: IntegerField - - name: client - ref: Client - ref_field: client_id - type: ForeignKey - name: TestResult - x: -1336 - y: -49 + x: -204 + y: 12 modules: [] view: panX: 213.729555519212 panY: 189.446959094643 scaleXY: 0.69 - diff --git a/awx/network_ui_test/CONTRIBUTING.md b/awx/network_ui_test/CONTRIBUTING.md new file mode 100644 index 0000000000..eb82432402 --- /dev/null +++ b/awx/network_ui_test/CONTRIBUTING.md @@ -0,0 +1,221 @@ + +Network UI Test +=============== + +Network UI Test is an event driven test framework for testing the Network UI. +This tool works by setting up the UI with a pre-test run snapshot, replaying a +set of events to the Network UI, catching exceptions and then comparing the +state of the system after the events to a post test run snapshot. Test results +and test code coverage are stored for each test run through the system. This +allows us to determine which lines of test are run during each test and which +lines are run under any test. This can be very helpful during development to +determine that the code is executed as expected given a certain input, to find +code that needs additional tests, and to find code that is not be run under any +of the given inputs (a.k.a. dead code). + +Using this test framework it is fairly easy to achieve 90%+ code coverage under +test with a few days of work recording and crafting tests. + +Test Steps +---------- + +The tests suppported by this test framework perform the following steps: + +* Reset code coverage records +* Recreate a pre-test snapshot on the UI +* Replay a set of events to the UI +* Check for exceptions thrown by the UI +* Check the state of the system based on the post-test snapshot +* Report the pass/fail/error status of the test by test name, code version, and client id +* Report the code coverage per test result +* Repeat for next test + +Test Case Data Schema +--------------------- + +The tests are completely data driven and the data for the test snapshots and events are stored in a JSON +structure in the `TestCase` model under the `test_case_data` field that has the following structure: + + { + "event_trace": [], + "fsm_trace": [], + "snapshots": [ + { + "devices": [], + "inventory_toolbox": [], + "links": [], + "message_id": 4, + "msg_type": "Snapshot", + "order": 0, + "sender": 0, + "trace_id": 2 + }, + { + "devices": [], + "inventory_toolbox": [], + "links": [], + "message_id": 31, + "msg_type": "Snapshot", + "order": 1, + "sender": 0, + "trace_id": 2 + } + ] + } + + +The pre-test snapshot has order 0 and the post-test snapshot has order 1. + + + +Network UI Test Models +---------------------- + +![Models](designs/models.png) + +* FSMTrace - The record of an event that happend during an FSM trace. This is used in the recording phase of test case data. +* EventTrace - The record of an event that happened during an event trace. This is used in the recording phase of test case data. +* Coverage - Per-line test case coverage as returned by istanbul JS code coverage tool. +* TopologySnapshot - A snapshot of the state of the network UI before or after the test was run. +* TestCase - The definition of a test case with a given name using the test case data schema defined above. +* Result - One of passed, failed, errored, skipped, aborted, not run, or blocked. +* TestResult - The record of one test case being run at a certain code location by a client at a certain time. +* CodeUnderTest - The record of what exactly was the code being tested at the time the test was run. + + +Messages +-------- + +JSON messages are passed over the `/network_ui/test` websocket between the test client and the test server. +The protocol that is used for all messages is in ABNF (RFC5234): + + + message_type = 'MultipleMessage' / 'MouseEvent' / 'MouseWheelEvent' / 'KeyEvent' / 'StartRecording' / 'StopRecording' / 'ViewPort' / 'FSMTrace' / 'ChannelTrace' / 'Snapshot' / 'EnableTest' / 'DisableTest' / 'StartTest' / 'TestCompleted' / 'TestResult' / 'Coverage' + message_data = '{' 'msg_type' ': ' message_type ', ' key-value *( ', ' key-value ) '}' + message = '[ id , ' posint ']' / '[ topology_id , ' posint ']' / '[' message_type ', ' message_data ']' + +See https://github.com/AndyA/abnfgen/blob/master/andy/json.abnf for the rest of the JSON ABNF. + +See [designs/messages.yml](designs/messages.yml) for the allowable keys and values for each message type. + + + +Loading Tests +------------- + +Tests can be imported from exported data dumps from the `awx-manage datadump` command. From the Tower shell run +the following command: + + awx-manage loaddata /awx_devel/network_ui_test_cases_2018_03_12.json + +This will load the tests from the `network_ui_test_cases_2018_03_12.json` file. + +Exporting Tests +--------------- + +Use the standard Django dumpdata command to dump the test case data to a file: + + awx-manage dumpdata network_ui_test.TestCase > /awx_devel/network_ui_test_cases_YYYY_MM_DD.json + + +Writing Tests Manually or Generating Tests +------------------------------------------ + +Use the empty test case schema above and add messages to the event_trace list. Then upload the +JSON test data to the upload test URL below. + +Recording Tests +--------------- + +Tests can be reruns of previously recorded manual interactions with the network UI. To start a recording +open the JavaScript console and run this command: + + scope.onRecordButton() + +To stop the recording run this command: + + scope.onRecordButton() + +To download the recording use this command: + + scope.onDownloadRecordingButton() + + +After downloading the recording upload the data using the following URL to make a test of it. + +Uploading Tests +--------------- + +Go to the URL: https://SERVER:PORT/network_ui_test/upload_test and use the form to upload test data +and choose a test case name. + +Instrumenting a UI Build +------------------------ + +Code coverage is collected automatically if the UI is built with instrumented code. Add this section to the +rules in awx/ui/build/webpack.base.js to build instrumented code. + + { + test: /\.js$/, + use: { + loader: 'istanbul-instrumenter-loader', + options: { esModules: true } + }, + enforce: 'pre', + include: [ + /src\/network-ui\// + ] + }, + + +Then run: + + make ui-devel + +or: + + make ui-docker + + +To rebuild the code with istanbul instrumentation. + + +Running Tests +------------- + + +To kick off tests in the web browser navigate to the Network UI page under an inventory and open the JavaScript console. +Run this command in the JavaScript console: + + scope.onRunTestsButton(); + + +Building a Coverage Report +-------------------------- + +To build a coverage report for the last set of tests that were run as in above. Edit the `tools/Makefile` and change +`SERVER` to match your Tower server. Then run the following command from the `tools` directory: + + make coverage + +This will download all the coverage data from all the test results that were previously recorded and build a coverage +report with istantbul. You can then view the report at the address http://localhost:9000 or http://server:PORT +where you executed the command. + +http://localhost:9000/coverage will contain the coverage for all test cases merged together. +http://localhost:9000/Load/coverage will contain the coverage for just loading the page. +http://localhost:9000/TestX/coverage will contain the coverage for just the TestX test where +TestX is the name of one of the tests run previously. + + + +3.3 Hardening Road Map +---------------------- + +* Add post-test snapshot comparison +* Add FSM trace comparison +* Add an ability to run a single test +* Add an ability to run a subset of the tests +* Add a big red light when event recording is on. + + diff --git a/awx/network_ui_test/designs/messages.yml b/awx/network_ui_test/designs/messages.yml new file mode 100644 index 0000000000..ff0721b535 --- /dev/null +++ b/awx/network_ui_test/designs/messages.yml @@ -0,0 +1,18 @@ +messages: + - {msg_type: MultipleMessage, fields: [msg_type, sender, messages]} + - {msg_type: MouseEvent, fields: [msg_type, sender, x, y, type, trace_id]} + - {msg_type: MouseWheelEvent, fields: [msg_type, sender, delta, deltaX, deltaY, type, originalEvent, trace_id]} + - {msg_type: KeyEvent, fields: [msg_type, sender, key, keyCode, type, altKey, shiftKey, ctrlKey, metaKey, trace_id]} + - {msg_type: StartRecording, fields: [msg_type, sender, trace_id]} + - {msg_type: StopRecording, fields: [msg_type, sender, trace_id]} + - {msg_type: ViewPort, fields: [msg_type, sender, scale, panX, panY, trace_id]} + - {msg_type: FSMTrace, fields: [msg_type, order, sender, trace_id, fsm_name, from_state, to_state, recv_message_type]} + - {msg_type: ChannelTrace, fields: [msg_type, sender, trace_id, from_fsm, to_fsm, sent_message_type]} + - {msg_type: Snapshot, fields: [msg_type, sender, devices, links, groups, streams, order, trace_id]} + - {msg_type: EnableTest, fields: [msg_type]} + - {msg_type: DisableTest, fields: [msg_type]} + - {msg_type: StartTest, fields: [msg_type]} + - {msg_type: TestCompleted, fields: [msg_type]} + - {msg_type: TestResult, fields: [msg_type, sender, id, name, result, date, code_under_test]} + - {msg_type: Coverage, fields: [msg_type, sender, coverage, result_id]} + diff --git a/awx/network_ui_test/designs/models.png b/awx/network_ui_test/designs/models.png new file mode 100644 index 0000000000..93327b3c78 Binary files /dev/null and b/awx/network_ui_test/designs/models.png differ diff --git a/awx/network_ui_test/designs/models.yml b/awx/network_ui_test/designs/models.yml new file mode 100644 index 0000000000..120f183340 --- /dev/null +++ b/awx/network_ui_test/designs/models.yml @@ -0,0 +1,168 @@ +app: awx.network_ui_test +external_models: [] +models: +- fields: + - name: client_id + pk: true + type: AutoField + name: Client + x: -518 + y: 138 +- fields: + - name: fsm_trace_id + pk: true + type: AutoField + - len: 200 + name: fsm_name + type: CharField + - len: 200 + name: from_state + type: CharField + - len: 200 + name: to_state + type: CharField + - len: 200 + name: message_type + type: CharField + - name: client + ref: Client + ref_field: client_id + type: ForeignKey + - default: 0 + name: trace_session_id + type: IntegerField + - default: 0 + name: order + type: IntegerField + name: FSMTrace + x: -872 + y: 507 +- fields: + - name: event_trace_id + pk: true + type: AutoField + - name: client + ref: Client + ref_field: client_id + type: ForeignKey + - default: 0 + name: trace_session_id + type: IntegerField + - name: event_data + type: TextField + - name: message_id + type: IntegerField + name: EventTrace + x: -1087 + y: 202 +- fields: + - name: coverage_id + pk: true + type: AutoField + - name: coverage_data + type: TextField + - name: test_result + ref: TestResult + ref_field: test_result_id + type: ForeignKey + name: Coverage + x: -1068 + y: -4 +- fields: + - name: topology_snapshot_id + pk: true + type: AutoField + - name: client + ref: Client + ref_field: client_id + type: ForeignKey + - name: topology_id + type: IntegerField + - name: trace_session_id + type: IntegerField + - name: snapshot_data + ref: TopologySnapshot + ref_field: snapshot_data + type: TextField + - name: order + type: IntegerField + name: TopologySnapshot + x: -1123 + y: -277 +- fields: + - name: test_case_id + pk: true + type: AutoField + - len: 200 + name: name + ref: TestCase + ref_field: name + type: CharField + - name: test_case_data + type: TextField + name: TestCase + x: -1642 + y: -38 +- fields: + - name: result_id + pk: true + type: AutoField + - len: 20 + name: name + type: CharField + name: Result + x: -1610 + y: 120 +- fields: + - name: code_under_test_id + pk: true + ref: CodeUnderTest + ref_field: code_under_test_id + type: AutoField + - name: version_x + type: IntegerField + - name: version_y + type: IntegerField + - name: version_z + type: IntegerField + - name: commits_since + type: IntegerField + - len: 40 + name: commit_hash + type: CharField + name: CodeUnderTest + x: -1612 + y: 259 +- fields: + - name: test_result_id + pk: true + type: AutoField + - name: test_case + ref: TestCase + ref_field: test_case_id + type: ForeignKey + - name: result + ref: Result + ref_field: result_id + type: ForeignKey + - name: code_under_test + ref: CodeUnderTest + ref_field: code_under_test_id + type: ForeignKey + - name: time + type: DateTimeField + - default: 0 + name: id + type: IntegerField + - name: client + ref: Client + ref_field: client_id + type: ForeignKey + name: TestResult + x: -1336 + y: -49 +modules: [] +view: + panX: 213.729555519212 + panY: 189.446959094643 + scaleXY: 0.69 diff --git a/awx/ui/client/src/network-ui/CONTRIBUTING.md b/awx/ui/client/src/network-ui/CONTRIBUTING.md index f0e8b0d5d3..bf3c2b3a28 100644 --- a/awx/ui/client/src/network-ui/CONTRIBUTING.md +++ b/awx/ui/client/src/network-ui/CONTRIBUTING.md @@ -258,6 +258,24 @@ Key events are captured by the following code: $document.bind("keydown", $scope.onKeyDown); ``` +**Event Processing** + +This code works as an event processing pipeline where the source of the events +may be mouse clicks, keystrokes, or messages from the server over the +websocket. This allows the appropriate processor to handle each event in turn +or delegate the message to another processor. + +The following diagram documents the pipeline processors that handle the events. +Events are injected into to the pipeline at `Start` and travel through the +pipeline along the arrows. Events may be handled at a node in the pipeline, +passed along to the next node, discarded, or transformed into another message +and sent along the pipeline. For instance `hotkeys_fsm` generates new and +different type of events based on key presses that are injected at the +beginning of the pipeline. + +![Event Pipeline](designs/pipeline.png) + + **Describing Behavior with Finite State Machines** To implement complex UI interactions predictably and correctly is a tough @@ -556,35 +574,6 @@ The messages defined are [messages.js](messages.js): * StreamSelected - A stream was selected * StreamUnSelected - A stream was unselected -**Message Passing** - -Messages are passed along channels between FSMs and over the websocket to and -from the server. Messages from the server over the web socket and user input -events from the web browser are passed to the `first_channel` where they are -passed along the chain of FSMControllers until they reach the end with -`NullChannel` or they are handled and the models are updated. - -* See: [network.ui.controller.js](network.ui.controller.js#L115) - -The order (from first to last) of message handling is: - -* Mode FSM -* Site Toolbox FSM -* Rack Toolbox FSM -* Inventory Toolbox FSM -* App Toolbox FSM -* Time FSM -* Buttons FSM -* Site FSM -* Rack FSM -* Group FSM -* Stream FSM -* Link FSM -* Move FSM -* Device Detail FSM -* View FSM -* Null FSM - Widget Development ================== diff --git a/awx/ui/client/src/network-ui/designs/pipeline.png b/awx/ui/client/src/network-ui/designs/pipeline.png new file mode 100644 index 0000000000..c5fb2b12f7 Binary files /dev/null and b/awx/ui/client/src/network-ui/designs/pipeline.png differ