PixelStreamingInfrastructure/SignallingWebServer/Docs/SignallingProtocol.md

Pixel Streaming Signalling Protocol

The following is a complete reference to the current signalling server messaging protocol. These messages are sent as stringified JSON packets. Some parameters are JSON strings themselves and require escape sequences to be contained in the string parameter.

Version

1.0.0 - Current

Major version number - breaking protocol change such as a required new message or field or deleting an existing message.
Minor version number - independent new message.
Hotfix version - a non-breaking new field in an existing message type.

Table of Contents

• Terms
• Example Streaming Message Sequence
• Player Sent Messages
  • answer
  • dataChannelRequest
  • iceCandidate
  • listStreamers
  • offer
  • peerDataChannelsReady
  • stats
  • subscribe
  • unsubscribe
• SFU Sent Messages
  • answer
  • offer
  • peerDataChannels
  • streamerDataChannels
• Signalling Server Sent Messages
  • config
  • identify
  • playerConnected
  • playerCount
  • playerDisconnected
  • pong
  • streamerDisconnected
  • streamerList
• Streamer Sent Message
  • answer
  • disconnectPlayer
  • endpointId
  • iceCandidate
  • layerPreference
  • offer
  • ping

Terms

Signalling Server

The server that is responsible for routing messageas and controlling new connections.

Streamer

The entity that is streaming the media.

Player

A consumer of the stream. Could be passive or active with the use of data streams.

SFU

Selective Forwarding Unit. This is a special type of Player that consumes the stream and then can forward the stream to new connected Players. This is useful when many users need to be consuming the stream and encoding resources on the Streamer are not adequate to keep up with the demand. Additionally allows the use of simulcast which allows the Streamer to stream a handful of different quality streams that the SFU can selectively forward to individual Players based on their connection quality.

SDP

Session Description Protocol. A message format that is used to negotiate a media connection between two endpoints. There are many resources for further details such as here or here

ICE Candidate

Interactive Connectivity Establishment. Describes protocols and routing needed for WebRTC to be able to communicate with a remote device. Further details can be read here

Example Streaming Message Sequence

sequenceDiagram
Streamer->>Signalling: Open Connection
Signalling->>Streamer: config
Signalling->>Streamer: identify
Streamer->>Signalling: endpointId
Player->>Signalling: Open Connection
Signalling->>Player: config
Signalling->>Player: playerCount
Player->>Signalling: listStreamers
Signalling->>Player: streamerList
Player->>Signalling: subscribe
Signalling->>Streamer: playerConnected
Streamer->>Signalling: offer
Signalling->>Player: offer
Player->>Signalling: answer
Signalling->>Streamer: answer
Streamer->Player: WebRTC negotiation
Note over Streamer, Player: Streaming Session
opt Player Disconnect
	Player->>Signalling: Close Connection
	Signalling->>Streamer: playerDisconnected
end
opt Streamer Disconnect
	Streamer->>Signalling: Close Connection
	Signalling->>Player: streamerDisconnected
end

Player Sent Messages

answer

Message is forwarded to the currently subscribed to Streamer. Sends the SDP answer back to the Streamer The Player must have previously used the subscribe message to successfully subscribe to a connected Streamer and received an offer from the Streamer.

Param Name	Type	Description
sdp	string	The WebRTC SDP package in string format

dataChannelRequest

Message is forwarded to a connected SFU. Tells the SFU that the Player requests data channels to the Streamer.

Param Name	Type	Description

iceCandidate

Message is forwarded to the currently subscribed to Streamer. Sends an ICE candidate to the Streamer. This is part of the WebRTC negotiation and should come after the answer

Param Name	Type	Description
candidate	Object	The JSON object describing the ICE candidate

listStreamers

Message is consumed by the Signalling Server. Requests the list of streamer ids currently connected to the Signalling Server. The server will reply with a streamerList

Param Name	Type	Description

offer

Message is forwarded to the currently subscribed to Streamer. Supplies an SDP offer to the Streamer to answer. The Player must have previously used the subscribe message to successfully subscribe to a connected Streamer.

Param Name	Type	Description
sdp	string	The WebRTC SDP package in string format

peerDataChannelsReady

Message is forwarded to a connected SFU. Tells the SFU that the Player is ready for data channels to be negotiated.

Param Name	Type	Description

stats

Message is consumed by the Signalling Server. Will print out the provided stats data on the console.

Param Name	Type	Description
data	string	The stats data to log.

subscribe

Message is consumed by the Signalling Server. Tells the Signalling server that the player requests to subscribe to the given stream.

Param Name	Type	Description
streamerId	string	The id of the stream to subscribe to.

unsubscribe

Message is consumed by the Signalling Server. Tells the Signalling Server that the Player wishes to unsubscribe from the current stream. The player must have previously used the subscribe message for this to have any effect.

Param Name	Type	Description

SFU Sent Messages

answer

Message is forwarded to the Streamer. Sends the SDP answer back to the Streamer. This should be in response to a previous offer from the Streamer.

Param Name	Type	Description
sdp	string	The WebRTC SDP package in string format

offer

Message is forwarded to a Player. Sends the SDP offer to the specified Player. This begins the WebRTC negotiation between the SFU and the Player.

Param Name	Type	Description
playerId	string	The id of the player to send the offer to.
sdp	string	The WebRTC SDP package in string format

peerDataChannels

Message is forwarded to a Player. Sends information to the Player about what data channels to use for sending/receiving with the Streamer.

Param Name	Type	Description
playerId	string	The player id of the player to send the message to.
sendStreamId	number	The datachannel id for sending data.
recvStreamId	number	The datachannel id for receiving data.

streamerDataChannels

Message is forwarded to the Streamer. Sends a request to the Streamer to open up data channels for a given Player.

Param Name	Type	Description
playerId	string	The id of the player the request is for.
sendStreamId	number	The datachannel id for sending data.
recvStreamId	number	The datachannel id for receiving data.

Signalling Server Sent Messages

config

Message is used to send the peer connection options such as stun and turn servers to a connecting Streamer

Param Name	Type	Description
peerConnectionOptions	Object	The object describing the peer connection options for this server.

identify

Message is used to request an identity from a connecting Streamer. The Streamer should reply with an endpointId message. See note in endpointId about legacy Streamers and temporary ids.

Param Name	Type	Description

playerConnected

Message is used to notify a Streamer that a new Player has subscribed to the stream.

Param Name	Type	Description
playerId	string	The id of the new connected player.
dataChannel	boolean	Indicates whether the player wants a datachannel or not.
sfu	boolean	Indicates if the player is an SFU or not.
sendOffer	boolean	Indicates if the new player want's an offer or not.

playerCount

Message is sent to Players to indicate how many currently connected players there are on this signalling server. (Note: This is mostly old behaviour and is not influenced by multi streamers or who is subscribed to what Streamer. It just reports the number of players it knows about.)

Param Name	Type	Description
count	number	The number of players currently connected to this signalling server.

playerDisconnected

Message is used to notify a Streamer that a Player has unsubscribed/disconnected from the stream.

Param Name	Type	Description
playerId	string	The id of the new connected player.

pong

Message is a reply to ping from a Streamer. Replies with the time from the ping message.

Param Name	Description
time	The timestamp of the ping message. Will be returned in the pong message

streamerDisconnected

Message is used to notify players when a Streamer disconnects from the Signalling Server.

Param Name	Type	Description

streamerList

Message is a reply to listStreamers from a Player. Replies with a list of currently active Streamers connected to this server.

Param Name	Type	Description
ids	Array<string>	A list of string ids that are currently active on the signalling server.

Streamer Sent Messages

answer

Message is forwarded to a Player. Sends the SDP answer back to the player. This should be in response to an offer from the specified player.

Param Name	Type	Description
playerId	string	The id of the player to send the answer to.
sdp	string	The WebRTC SDP package in string format

disconnectPlayer

Message is consumed by the Signalling Server. Requests that the Signalling Server disconnect a Player.

Param Name	Type	Description
playerId	string	The id of the player to disconnect
reason	string	The reason for the disconnect

endpointId

Message is consumed by the Signalling Server. Specifies an id for the Streamer. This is used to uniquely identify multiple streamers connected to the same Signalling Server. Note: to preserve backward compatibility when Streamer ids were optional, when a Streamer first connects it is assigned a temporary ID (__LEGACY__) which allows use of older Streamers if needed.

Param Name	Type	Description
id	string	The id of the Streamer

iceCandidate

Message is forwarded to a Player. Sends an ICE candidate to the specified player.

Param Name	Type	Description
playerId	string	The id of the player to send the ICE candidate to.
candidate	Object	The JSON string describing the ICE candidate

layerPreference

Message is forwarded to a connected SFU. Sends a preferred layer index to a connected SFU for a specified Player. Useful for switching between SFU quality layers to force a certain resolution/quality option either as part of UX or testing.

Param Name	Type	Description
playerId	string	The id of the player to give the preference to
spatialLayer	number	The index of the spatial layer to prefer
temporalLayer	number	The index of the temporal layer to prefer

offer

Message is forwarded to a Player. Sends the SDP offer to the specified player. Begins the WebRTC negotiation with a player.

Param Name	Type	Description
playerId	string	The id of the player to send the offer to.
sdp	string	The WebRTC SDP package in string format

ping

Message is consumed by the Signalling Server. A keepalive ping message that initiates a pong response.

Param Name	Type	Description
time	number	The timestamp of the ping message. Will be returned in the pong message