Add spec doc, update README (#3)

Import spec from google doc, along with some updates and reformatting

Author: jtbandes

README.md

@@ -1,19 +1,38 @@
-## Virtualenv usage
+# Foxglove Studio WebSocket protocol libraries
+
+This repository provides a protocol specification and reference implementations enabling [Foxglove Studio](https://github.com/foxglove/studio) to ingest arbitrary “live” streamed data.
+
+The protocol is encoding-agnostic, i.e. it can support Protobuf messages, ROS 1 or 2 messages, etc. (as long as the desired encoding is supported by both client and server).
+
+## Documentation
+
+- [Protocol spec](docs/spec.md)
+
+## Development
+
+### Virtualenv usage
 
 ```
 python3.8 -m venv venv
 . venv/bin/activate
-pip install -r requirements.txt -r dev-requirements.txt
+pip install -r python/requirements.txt -r python/dev-requirements.txt
 ```
 
-### h5py installation on M1 Macs
+#### h5py installation on M1 Macs
+
 ```
 brew install hdf5
 HDF5_DIR=/opt/homebrew/opt/hdf5 pip install -v --no-build-isolation h5py
 ```
 
-## Run example server
+### Run example server
+
+```
+python -m python.src [hdf5 file]
+```
+
+### Run example client
 
 ```
-python -m src /path/to/person.hdf5
+yarn workspace @foxglove/ws-protocol example [host] [topic]
 ```

docs/spec.md

@@ -0,0 +1,189 @@
+# Foxglove Studio WebSocket protocol v1
+
+## Protocol overview
+
+- An application wishing to provide data for streamed consumption by Foxglove Studio hosts a [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API) server.
+
+- The client (Foxglove Studio) will specify supported subprotocols (a standard part of the WebSocket handshake) when establishing the connection. The current version of this document corresponds to subprotocol `foxglove.websocket.v1`. The server must select a subprotocol with which it is compatible for the connection to continue.
+
+  - Example client code in JavaScript:
+    ```js
+    new WebSocket("ws://...", ["foxglove.websocket.v1"]);
+    ```
+
+- Both text and binary messages are used on the WebSocket connection.
+
+  - Each text message must be a JSON object having a field called `op` which identifies the type of message. The interpretation of the other fields depends on the opcode.
+
+  - Similarly, each binary message starts with a 1-byte opcode identifying the type of message. The interpretation of the remaining bytes depends on the opcode.
+
+- Upon establishing a connection, the server must send clients a Server Info message with a list of supported capabilities.
+
+## Summary of messages
+
+### Sent by server
+
+- [Server Info](#server-info) (json)
+- [Status](#status) (json)
+- [Advertise](#advertise) (json)
+- [Unadvertise](#unadvertise) (json)
+- [Message Data](#message-data) (binary)
+
+### Sent by client
+
+- [Subscribe](#subscribe) (json)
+- [Unsubscribe](#unsubscribe) (json)
+
+## JSON messages
+
+Each JSON message must be an object containing a field called `op` which identifies the type of message.
+
+### Server Info
+
+- This message is always sent to new clients upon connection.
+
+#### Fields
+
+- `op`: string `"serverInfo"`
+- `name`: free-form information about the server which the client may optionally display or use for debugging purposes
+- `capabilities`: array of strings (reserved for future expansions to the spec)
+
+#### Example
+
+```json
+{
+  "op": "serverInfo",
+  "name": "example server",
+  "capabilities": []
+}
+```
+
+### Status
+
+- The server may send this message at any time. Client developers may use it for debugging purposes, display it to the end user, or ignore it.
+
+#### Fields
+
+- `op`: string `"status"`
+- `level`: 0 (info), 1 (warning), 2 (error)
+- `message`: string
+
+#### Example
+
+```json
+{
+  "op": "status",
+  "level": 0,
+  "message": "Some info"
+}
+```
+
+### Advertise
+
+- Informs the client about newly available channels.
+- At least one Advertise message is always sent to new clients upon connection.
+
+#### Fields
+
+- `op`: string `"advertise"`
+- `channels`: array of:
+  - `id`: number. The server may reuse ids when channels disappear and reappear, but only if the channel keeps the exact same topic, encoding, schemaName, and schema. Clients will use this unique id to cache schema info and deserialization routines.
+  - `topic`: string
+  - `encoding`: string
+  - `schemaName`: string
+  - `schema`: string
+
+#### Example
+
+```json
+{
+  "op": "advertise",
+  "channels": [
+    {
+      "id": 1,
+      "topic": "foo",
+      "encoding": "protobuf",
+      "schemaName": "ExampleMsg",
+      "schema": "ZXhhbXBsZSBkYXRh"
+    }
+  ]
+}
+```
+
+### Unadvertise
+
+Informs the client that channels are no longer available.
+
+#### Fields
+
+- `op`: string `"unadvertise"`
+- `channelIds`: array of number, corresponding to previous Advertise
+
+#### Example
+
+```json
+{
+  "op": "unadvertise",
+  "channelIds": [1, 2]
+}
+```
+
+### Subscribe
+
+- Requests that the server start streaming messages on a given topic (or topics) to the client.
+- A client may only have one subscription for each channel at a time.
+
+#### Fields
+
+- `op`: string `"subscribe"`
+- `subscriptions`: array of:
+  - `id`: number chosen by the client. The client may not reuse ids across multiple active subscriptions. The server may ignore subscriptions that attempt to reuse an id (and send an error status message). After unsubscribing, the client may reuse the id.
+  - `channelId`: number, corresponding to previous Advertise message(s)
+
+#### Example
+
+```json
+{
+  "op": "subscribe",
+  "subscriptions": [
+    { "id": 0, "channelId": 3 },
+    { "id": 1, "channelId": 5 }
+  ]
+}
+```
+
+### Unsubscribe
+
+- Requests that the server stop streaming messages to which the client previously subscribed.
+
+#### Fields
+
+- `op`: string `"subscribe"`
+- `subscriptionIds`: array of number, corresponding to previous Subscribe message(s)
+
+#### Example
+
+```json
+{
+  "op": "unsubscribe",
+  "subscriptionIds": [0, 1]
+}
+```
+
+## Binary messages
+
+All binary messages must start with a 1-byte opcode identifying the type of message. The interpretation of the remaining bytes depends on the opcode.
+
+All integer types explicitly specified (uint32, uint64, etc.) in this section are encoded with **little-endian** byte order.
+
+### Message Data
+
+- Provides a raw message payload, encoded as specified in the Advertise corresponding to the channel.
+- Subscription id must correspond to a Subscribe that was previously sent.
+
+| Bytes           | Type    | Description                     |
+| --------------- | ------- | ------------------------------- |
+| 1               | opcode  | 0x01                            |
+| 4               | uint32  | subscription id                 |
+| 8               | uint64  | receive timestamp (nanoseconds) |
+| remaining bytes | uint8[] | message payload                 |

python/src/server.py

@@ -86,7 +86,7 @@ async def _run(self):
             self._handle_connection,
             self.host,
             self.port,
-            subprotocols=[Subprotocol("x-foxglove-1")],
+            subprotocols=[Subprotocol("foxglove.websocket.v1")],
         )
         for sock in server.sockets or []:
             self._logger.info("Server listening on %s", sock.getsockname())

typescript/src/FoxgloveClient.ts

@@ -51,7 +51,7 @@ interface IWebSocket {
 }
 
 export default class FoxgloveClient {
-  static SUPPORTED_SUBPROTOCOL = "x-foxglove-1";
+  static SUPPORTED_SUBPROTOCOL = "foxglove.websocket.v1";
 
   private emitter = new EventEmitter<EventTypes>();
   private ws!: IWebSocket;