* Updated github actions to work on UE5.3 branch and reset release version to 0.0.1 for the UE5.3 branch.
* Update bug report template to include UE5.X prefix.
* Refactored frontend deps to move from 5.2 to 5.3
* Update container-images Github action to use 5.3 tag
* Bump 5.2 t 5.3 in various docs and log strings.
* add unit test coverage reports to gitignore
* add jest dependencies
* add jest config
* add Config unit tests
* add AFKController unit tests
* mock video codecs list
* test for PixelStreaming SettingsChangedEvent
* mock/unmock functions for RTCRtpReceiver
* mock WebSocket
* test for PixelStreaming.connect()
* unit test for disconnect
* unit tests for reconnect
* added eventemitter events in tests
* test that listStreamers is sent on WS connect
* mock RTCPeerConnection and RTCIceCandidate
* test webRtcConnected event
* test RTCPeerConnect close
* fixed variable name for consistency
* check for navigator.getGamepads before calling it
* mock addTransceiver, createAnswer, getTransceivers, setRemoteDescription
* test for receiving a connection offer
* add null checks for peerConnection since it might be null if connectiong closed
* mock addIceCandidate
* test for receiving ICE candidate message
* mock setLocalDescription, getStats
* test for statistics
* test webRtcDisconnected event
* mock RTCDataChannel and RTCDataChannelEvent
* test for dataChannelOpen event
* mock RTCTrackEvent
* mock MediaStream and MediaStreamTrack
* mock video element play()
* test playStream and playStreamRejected events
* mock video readyState
* mock WebRTC data channel send()
* test emitCommand
* test emitUIInteraction
* test emitConsoleCommand
* Jest mock for HTMLMediaElement.play()
* check boolean return value of all emit* commands
* test dataChannelClose event
* added TextEncoder and TextDecoder to Jest globals
* mock datachannel onmessage
* test UE -> browser data channel message
* clarified in test description that we are using a Response message
* extracted commonly used setup steps as util functions
* triggerSdpOffer -> triggerSdpOfferMessage for consistency
* run unit tests as a Github action
* Revert "break one of the tests to test the GH action"
This reverts commit 05f5742cf0.
* run tests only if files changed under Frontend/library
* added unit test run instructions to README.md
* move registerLockedMouseEvents and registerHoveringMouseEvents to MouseController
* add a utility that keeps track of registered event handlers for easy unregistering
* fix stream handler name MoveMouse -> MouseMove
* support unregistering gamepad events
* support unregistering keyboard events
* support unregistering mouse events
* support unregistering touch events
* unregister input event handlers before registering new ones
* configurable keyboard/mouse/touch/gamepad support
* start/stop game pad polling on config enable/disable
* UI: add user input config to Settings menu
* added config flag EnableXRControllerInput for enabling/disabling XR controller messages
* added the new config flags to documentation
* smaller version of the settings panel image to better match the old image resolution
* Simplify input flag naming. Separate input toggles into new section in the settings panel
This PR splits the Frontend/library code into two parts:
1) /library - that provides an API for establishing Pixel Streaming sessions
2) /ui-library - that contains all the UI components like settings overlay panel, connection strength indicator, buttons, etc.
The library code is intended to be used as a library through the Javascript/Typescript API, and it can be used programmatically without the default UI implemented in ui-library. This allows the developers to bring their own UI if they wish to customize the user experience, or even start a Pixel Streaming session without any overlay UI.
The wish is to keep the library API stable and try to not make breaking changes if possible. If new non-breaking features are introduced to the API, it would be great if it was reflected in the version numbering following semantic versioning. Breaking changes should increase the major version number, while non-breaking changes increase only the minor or patch version number.
* add jss dependencies to UI project
* move PixelStreamingApplicationStyles to the UI project
* remove jss dependencies from library project
* add pixel streaming lib as ui dependency
* renamed lib Application -> PixelStreaming
* expose AFK callbacks from API
* expose callbacks for overlays
* onDismissAfk function added
* fix possible NPE in statistics panel (Firefox only)
* build both library projects in setup.sh|.bat
* extracted overlays from PixelStreaming library to UI
* bind callbacks to have access to `this`
* emit onStatsReceived callback
* replaced on\* callbacks with a typed event emitter
* reuse types in on() and emit()
* fix web-xr on Firefox (no navigator.xr available)
* latency test API and callback events
* move stats panel to UI project
* move settings panel to UI project
* extract video quality indicator to UI project
* move buttons to UI project
* events for freezeFrame functionality
* import UI project in stresstest.ts
* move UI wrapper elements to UI project
* optional override for videoParentElement
* created uiless.html and uiless.ts, which together are a sample UI application with no overlay UI
* let -> const everywhere in sample applications
* used named imports everywhere, not import * as libfrontend
* document overrides
* added onOpen, onClose, onError handlers for data channels
* events for webRtc data channel open, close, error
* added missing javadocs for classes and functions
* renamed webRtcDisconnect -> webRtcDisconnected
* added disconnect function to the API
* use EventTarget to emit events
* emit streamerListMessage, move new streamer select overlay to UI side
* Emit config change events
* API for changing settings
* add getSettings() for symmetry
* set partial initial settings in Config constructor
* configure initial params in uiless.ts to auto start muted
* add missing type exports
* extracted config UI from the config components
* configurable parameter saving to URL
* save config params to URL only on demand
* removed console.log
* removed unused import
* Extract light/dark mode config out of Pixel Streaming library
* update labels for custom settings
* Build the new library in setup scripts
* added a fallback click-to-play handler into uiless.ts
* hide non-public attributes in Config
* make the public API more compact by hiding private attributes
* added documentation for the public functions
* underscored some methods in Config
* make useUrlParameters read-only
* underscore for Config event listeners: public API users use settingsChanged events
* underscore for Config event listeners
* fix imports: @epicgames-ps/lib-pixelstreamingfrontend-ue5.2, not -dev
* added a comment to request function API docs: expected to be changed later
* reorganized directory structure
* ui-library build now uses NPM dependency, build-all linked filesystem dependency
* added ui-library build to GH release
* added github workflow for ui-library publishing
* Link base library when building ui-library
* Bring dispatchEvent, addEventListener and removeEventListener to the top level PixelStreaming class, simplifying API usage
* Allow settings to take a default onChange listener. This listener has two params of the new value as well as the calling settings, allowing you to modify the underlying settings members in this callback
* Fix overlay to properly notify is none or multiple streamers are connected
* Make dark mode the default color scheme regardless of browser preference.
* Make numeric settings spinner width take us less space.
* Make SettingUIOption match styling width of numeric settings.
* renamed EpicGames build-all-dev -> build-dev-all to match other projects
* added Click to play overlay in uiless.html and show/hide it when needed
* revised documentation for Config/*.ts
* revised documentation in PixelStreaming.ts
* documented EventEmitter events and event parameters
* emit webRtcConnected event when connected
* Fix max bitrate not being set correctly.
* Update package-lock.json for ui-library
* Refactor sendEncoderSettings to instead be sendEncoderMinQP and sendEncoderMaxQP
* Bump version numbers of library NPM package to 0.1.0 and bump 5.2 release to 0.2.0 as all changes here a non-breaking as far as we have tested.
---------
Co-authored-by: William Belcher <william.belcher@xa.epicgames.com>
Co-authored-by: Luke Bermingham <1215582+lukehb@users.noreply.github.com>
* Add WebGL program. Refactor input to reduce arguments when calling handlers.
* Add XR gamepad handling. Make use of input API more consistent.
* Add support for rendering the video stream into the headset.
* Add handling for float data type when sending message to streamer.
* Run prettier pass on repo.
* Refactor javascript frontend into a typescript library and a reference implementation.
* fixed restart stream
* fixed bug where the afk was still active after a reguler dissconnection
* added overlay for when the stream was loading
* added dissconect overlay that allows reconnection
* added comments for disconnection overlay
* replaced all bind methods with arrow functions to make code cleaner
* seperated auto webrtc connection from auto play with a new config paramater
* added the new QP icon
* added fully functional qp indicator
* removed redundant code and changed image restart icon to svg
* added the ability to toggle fullscreen and added logic
* added the ability to toggle settings and stats but need to clean up css and make icon images show
* fixed qp stat bug
* fixed bug with the dissconect action overlay and autoplay
* added icons for tooltips
* changed images to be served as base 64
* configured svg button in the index for fast loading
* moddifed styles in the settings and stats menu
* fixed matchviewport resolution
* added favicons and fixed matchviewportresolution
* refactored VideoPlayerController to StreamController
* upgraded the logger to be more usefull
* remove hard-coded signalling server address from the frontend
* seperated playing audio from playing video in webrtcplayer controller and fixed logger bug for ff and ios
* reworked sound and video
* added a fix for ios making svg icons invisible when having filteres aplyed to them
* added qol change where the settings pannel closes when clicking restart stream
* turned off autoplay
* logged unknown dc messages
* logged unknown dc messages moved log before error
* add support for sending back a pong message to the signalling server on response to a ping message
* Bug fixed multitouch zooming the page
* ensure that the max width of the panel wrap can never exceed the max width of the viewport
* fix the force turn button so that it appears off to the right of the label
* Added support for handling websocket messages sent as binary.
* added a feature that looks for turn servers if turn is forced.
* reworked turn to dissconect if forced and there are no servers
* add support for overriding the disconnection message in the webrtcplayercontroller
* Improve grammar on the error message when there is no TURN server
* prevent the disconnect message from incorrectly appearing as undefined
* added message to closing the websocket
* consolidated commands and ui descriptors
* moved message methods to SendMessageController
* integrated input rework
* added datachannel sender
* refactored mouse/keyboard input
* refactored normalise and quanatize
* refactored pointerlock functionality
* refactored touch controller
* fixed resizing bugs
* fixed player resolution and mouse movment bugs
* Adding AFK setting to settings panel
* Add support for signalling server url passed as /?ss= as url parameter.
* Fix video fill display option
* Added programmatic way to create togglable settings
* Added programmatic surface to construct settings flags
* Added FillWindow as a new flagged setting in the new settings.
* Move control scheme setting into new settings config.
* Added buttons programmatically instead of in html
* Added numeric settings for min/max qp
* Added a new type for creating numeric settings: `SettingNumber`
* Added safe codepath if KeyboardEvent.keyCode is deprecated in the browser.
* Moved WebRTC settings to new programmable approach
* Created settings panel that can be constructed programmatically
* Moved all overlay construction into Overlays.ts
* Removed `showStats` boolean that was now unused.
* Added programmatic construction of video qp indicator.
* Fullscreen icon not constructed programmatically, including icons.
* Settings icon now programmatically constructed
* Stats Icon is now programmatically constructed.
* Moved all icons to a controls element that is programmatically constructed.
* Programmatically construct latency test element.
* Create stats panel programmatically
* Programmatically construct uiFeature element and playerUI root element of Pixel Streaming application
* Remove dependency on events lib
* Added italic headings to stat panel headings and changed tooltip on stats panel to say 'information'
* Allow browser to receive offers - UE5 compatibility
* Hooked up new signalling protocol that can be extended to support custom signalling messages
* Fix bug in signalling protocol handler calls. rename 'browser send answer' to browser send offer' to reflect what it actually does.
* Add SFU support. Also removed usages of console.xxx for logging and replaced with use of the Logger.
* Fix all linter errors and most warnings.
* Fix: Config control scheme wasn't being updated with the value parsed from the URL
* Remove control scheme from config. It is no handled directly from the settings panel and the URL
* Enable PS to embed properly
* Moving signalling server in setting text that can be accessed and set programmatically
* Moved construction of signalling server url into library
* Merge SFU work into signalling url changes by creating a construct signalling url function
* Create Epic Games reference implementation.
* Moved all config options from members to be settings which can set in the UI or URL
* Remove reference to any type and fix remaining linter errors.
* Refactor normalize and quantize to be called coordinate converter.
* Refactored AFK logic to be AFK Controller
* Removed UiController and moved logic into video player
* Fix security vulnerabilities in implementations. Remove unused packages.
* Fix spelling errors.
* Add ability to handle default mouse scheme from an offer. Also fixed some bugs in the settings that prevented onChange callbacks getting called.
* Add stresstester to implementation. This unveiled some points where we were using document.getElementByID(xxx) which isn't useful for when there are > 1 players
* Add login page
* Rename libfrontend to pixelstreamingfrontend
* Add copyright notice
* Fix double connections. Remove logging from websocket onerror as the error event doesn't contain any information
* Fix body filling more than the screen size on iOS
* Add toggle for light and dark mode. defaults to system preference
* Fix mic not working
* Fix light mode
* Fix backspace not working
* Adding jss support
* Fix webpack compilation
* Fixed css in js to handle camel case names for styles
* Remove tag level css rules that may leak into someone elses application
* Added light mode, fixed css, fixed to US spelling of color, fixed loading overlay.
* Changing stress test default values
* Remove esbuild
* Added fix for vh100 creating scroll of iOS devices
* Cleanup websocket disconnect message to contain reason only if one is valid
* Add webpack devserve to contain dist dir
* Fix backspace not working
* Fixing use mic option that was not usable locally
* Cleaning up package.json files
* Updating readme for release
* Move the new typescript frontend into a /Frontend directory.
* Remove old frontend files and update implementation to build into webserver directory
* Renamed index to player.html to match existing conventions in Cirrus
* Added script to install frontend if not built and also an option to force it to build with --build
* Cleaned up indentation and trailing whitespace
* Updated setup_frontend.bat to install npm packages too
* Updated bash setup scripts to build frontend if missing or directed
* Revised arg handling to match rest of the bash scripts
* Added a disclaimer if the frontend build is skipped
* Added --build description to --help display
* Removed uncessary includes from tsconfig
* Remove superfluous package-lock.json in root.
* Added prettier and performed whitespace pass on the library.
* Add missing typescript dependency.
Co-authored-by: hmuurine <hmuurine@users.noreply.github.com>
Signed-off-by: Luke Bermingham <1215582+lukehb@users.noreply.github.com>
* Update Frontend/library/package.json
* Add missing typescripe and webpack dependencies.
Co-authored-by: hmuurine <hmuurine@users.noreply.github.com>
Signed-off-by: Luke Bermingham <1215582+lukehb@users.noreply.github.com>
* Add ability to set preferred codec during negotiation
* Fix missing stats in stats panel
* Set update selector based on what was negotiated once video starts
* Fix sfu not working due to fmtp line reordering
* Update default codec for use with ios
* Fix signalling server not connecting if frontend is hosted on anything other than port 80
* Fix locked mouse 'MouseDown' events using the wrong coordinate location
* Fix: On Windows always use our bundled NodeJS to build the frontend.
* Adding SetEnv.exe tool that is used on Windows to temporarily modify path during frontend install.
* Linux: Add npm to the path temporarily when running `setup_frontend.sh`
* Remove preferred codec from text parameters as it's now an option parameter
---------
Signed-off-by: Luke Bermingham <1215582+lukehb@users.noreply.github.com>
Co-authored-by: Adrian Zahra <adrian@tensorworks.com.au>
Co-authored-by: Daniel Holden <daniel@tensorworks.com.au>
Co-authored-by: David MacPherson <david@tensorworks.com.au>
Co-authored-by: William Belcher <william.belcher@hotmail.com>
Co-authored-by: Michael Stopa <michael.stopa@xa.epicgames.com>
Co-authored-by: hmuurine <hmuurine@users.noreply.github.com>
Co-authored-by: William Belcher <william.belcher@xa.epicgames.com>
Matthew Cotton
dependabot[bot]
Luke Bermingham
hmuurine
William Belcher