ORCA – Object-RTC API Community Group
ORTC Walk Through
IETF 88
Robin Raymond, Hookflash
Bernard Aboba, Microsoft
Adalberto Foresti, Microsoft Open Technologies, Inc.
Meeting Purpose
• The purpose of this meeting is explain:
–
–
–
–
–
Why a community has come together to work on an alternative API
High level features of the ORTC API
How to use the API to implement a simple video chat demo
How you can get involved with developing the spec and code
Introduce open source effort / call to action
• There will be time for refreshments and Q&A the last 30 minutes
• Session is recorded and streamed to the web
(sponsored by Hookflash)
Introduction to ORTC
ORTC – Object Realtime Communications
– A next-generation JavaScript API for browser,
mobile, and cloud realtime audio / video scenarios
– ORTC is an alternative API to the W3C WebRTC 1.0
draft API
– It is being developed in the ORCA W3C
Community Group
The ORCA W3C Community Group
W3C Community Groups are forums open to all
without fee
– Simple IPR policy – royalty-free commitments to
one’s own contributions
– Not on standards track
– Mature CG specs can get broader IPR
commitments, transition to standards track
– Don’t have to be W3C member!
The ORCA W3C Community Group
Object RTC API Community Group,
– Formed in August 2013
Chair and editor: Robin Raymond
– Goal to develop WebRTC API that exposes an
object API rather than blobs of SDP
– Community group founded by JS developers
coming from both SIP/other protocol signaling
backgrounds, node.js and WebRTC enthusiasts.
This is not a community group
meeting
• Goal here is to explain mission and progress,
recruit new members
• To suggest API changes, join the Community
Group – we can’t accept suggestions from
those who have not signed the Contributor
License Agreement
Special Thanks to…
ORCA Community Group asked Microsoft Open
Technologies if they would implement ORTC as
demo / validation of concept.
Special thanks for Microsoft Open Technologies,
Inc. for implementing demo / validation of
concept.
(and the refreshments!)
WebRTC Needs
• Mobile: SDKs enabling development of mobile apps protocolcompatible with WebRTC.
• Analytics. Diagnostics and monitoring for carrier scale
operation.
• Cloud. Easy setup and management of WebRTC systems.
– Tools. Enabling connection of WebRTC clients to legacy
infrastructure (e.g. node.js)
– APIs. Enabling management of STUN/TURN credentials,
audio/video mixers, SIP proxies, XMPP connection
managers, etc.
• Extensibility: Enabling addition of functionality beyond the core
(e.g. mobile-friendly video codecs such as H.264/SVC)).
• A next generation JavaScript API compatible with the above.
What are ORTC potential uses?
• Browser code (simplified RTC control)
• Mobile development via “ortclib” SDK
under development https://github.com/openpeer/ortclib
• Server code via Node.js
new services or legacy inter-op solutions – call to contribute
under development https://github.com/openpeer/ortc-node
Open Source projects are under way!
Write ORTC apps using existing
WebRTC 1.0 Browsers
An WebRTC 1.0 “shim” will be made available to
maintain backwards compatibility
If you like the WebRTC 1.0 API, you can still use
the 1.0 API!
ORTC Migration Strategy
1.0 Strategy
2.0 Future ?
Web App
Web App
SDP / SIP (shim)
ORTC (shim)
WebRTC 1.0
(SDP)
Chrome /
Firefox / etc
Node.JS
Browsers
(see demo)
ORTC
Mobile
(ortclib)
Browsers
Node.JS
Mobile
Isn’t ORTC just CU-RTCWeb?
No
Both are object models but at different levels
While CU-RTCWeb inspired ORTC, they are are
different.
Issues with CU-RTCWeb:
- too powerful/difficult to use
- exposing too much socket level control (i.e. security risks)
- couldn’t shim CU-RTCWeb
Where will ORTC be supported by
vendors?
Support is via the shim over WebRTC 1.0 for now for
browsers.
Direct ORTC API source code is being developed.
Microsoft Open Tech developed a shim plugin for IE
(but not a product announcement).
openpeer/ortc
GitHub Specification Repo
https://github.com/openpeer/ortc
•Original home of the ORTC API discussion
•Spec discussion spun off to CG, OSS code for ORTC stays in
GitHub
•Why do we need the GitHub repo?
– Pull requests – discussed on public-orca mailing list
http://www.w3.org/community/orca/
openpeer/ortclib
GitHub Mobility Repo
https://github.com/openpeer/ortclib
•
•
•
•
ORTC mobile implementation
Maintained by Hookflash
Exposes C++ WebRTC library as an ORTC API
To be wrapped in iOS, Java (for Android) and JavaScript hybrid
openpeer/ortc-node
GitHub Node.JS Repo
https://github.com/openpeer/ortc-node
•ORTC Node.JS implementation
•Call to action – seeking contributors
– What should implementation look like?
– What are native code integration points?
Will contributing to ortc
break protocol compatibility with web browser?
No
Wire level compatibility is maintained
Maintaining wire compatibility is a tenet of ORTC
implementations with WebRTC enabled browsers*
We encourage expansion of features, not removal of
basic wire support.
* as much as they are compatible with each other
Motivation for
ORTC Design
Traditional Browser Rendering
Engine
Resource
JS
Description Format
(HTML/CSS)
Render Engine
Output
This is not how we program HTML on the web…
1
JS asks browser “Give
me webpage that
meets this criteria”
2
Browser generates
some HTML (gives to
JavaScript)
3
Modify browser
provided HTML
4
Set HTML
WebRTC 1.0 Capture / Render Engine
{ Cloud }
JS
JS
Description Format
(SDP)
Description Format
(SDP)
Resource
Resource
Capture/Render Engine
Capture/Render Engine
Input/Output
Input/Output
ORTC Capture / Render Engine
{ Cloud }
JS
JS
Capture/Render Engine
Capture/Render Engine
Input/Output
Input/Output
ORTC Simple Use Case
{ cloud }
Browser
Browser
Application
Application
RTCConnection
RTP / DATA / DTLS / ICE
RTCConnection
Questions?
ORTC API
ORTC Simple Use Case
{ cloud }
Browser
Browser
Application
Application
RTCConnection
RTP / DATA / DTLS / ICE
RTCConnection
ORTC - Object Relationships
remote
RTCIceCandidate
RTCIceCandidate
RTCConnectionOptions
RTCConnection
RTCSocket
local
RTCIceConnectionInfo
RTCIceConnectionInfo
RTCConnectionInfo
RTCConnectionInfo
RTCDtlsConnectionInfo
RTCDtlsConnectionInfo
RTCStream
RTCStream
MediaStream
MediaStream
RTCTrack
RTCTrack
MediaStreamTrack
MediaStreamTrack
RTCTrack
RTCTrack
MediaStreamTrack
MediaStreamTrack
RTCIceCandidate
RTCIceCandidate
RTCConnection
[Constructor(RTCConnectionOptions options)]
interface RTCConnection : EventTarget {
readonly attribute RTCSocket
readonly attribute RTCConnectionState
attribute RTCConnectionRole
readonly attribute RTCConnectionInfo
attribute RTCConnectionInfo
void
void
void
(RTCSTream? or RTCTrack?)
streamOrTrack);
(RTCStream? or RTCTrack?)
sequence<RTCStream>
sequence<RTCTrack>
sequence<RTCStream>
sequence<RTCTrack>
void
attribute
attribute
attribute
attribute
attribute
attribute
};
socket;
state;
role;
local;
remote;
addRemoteCandidate(RTCCandidateDescription candidate);
connect();
gather();
send((MediaStream or MediaStreamTrack or RTCTrack or RTCStream)
receive((RTCStream or RTCTrack) streamOrTrack);
sendStreams();
sendTracks();
receiveStreams();
receiveTracks();
close ();
EventHandler
EventHandler
EventHandler
EventHandler
EventHandler
EventHandler
oncandidate;
onendofcandidates;
onactivecandidate;
onstatechanged;
onnetworkchange;
onunknowntrack;
RTCConnection (p1)
- RTCSocket (attached to one network socket)
- States
-
“new” – connection not started
“searching” – searching for connectivity
“halted” – no connectivity found
“connected” – found connectivity (ready)
“closed” – connection is finished (one-time-use)
RTCConnection (p2)
- role (“controlling” vs “controlled”)
- Local and remote information
- ICE credentials (usernameFrag / password)
- DTLS fingerprint (generated or discovered)
- ICE Candidates – local and remote (for
connectivity checks)
RTCConnection (p3)
- Gathers local ICE candidates
- Performs ICE connectivity checks
- Causes sending / receiving of media tracks
RTP data over the “nominated” ICE
connection
RTCConnection (p4)
RTCStreams and RTCTracks (sending or
receiving)
- Contains mapping of on-the-wire
RTC/RTP/RTCP/etc information to media streams /
media stream tracks)
- Contains respective “MediaStream” or
“MediaStreamTrack” objects.
RTCConnection “connect”
Set remote “RTCIceConnectionInfo” ICE
“usernameFrag” and “password”
Call “connect” to cause candidate search
Will causes state changes from “new”,
“searching”, “haulted”, “connected”
RTCConnection “gather”
Find new candidates
- Restarts network topology ICE candidate gathering
Does not affect connection state by itself
Start gathering candidates before “connect”
RTCConnection “send”
Send “MediaStream”, “MediaStreamTrack”,
“RTCStream”, “RTCTrack”
Convenience to auto-map “MediaStream” and
“MediaStreamTrack” into RTP layer constructs
Manipulate RTP mapping by first creating
“RTPStream” / “RTCTrack” from “MediaStream”
and “MediaStreamTrack”
RTCConnection “receive”
Send “RTCStream”, “RTCTrack”
Setup mapping for “MediaStream” and
“MediaStreamTrack” for RTP layer constructs
Manipulate mapping by constructing
“RTPStream” / “RTCTrack” objects
RTCConnection “close”
Shuts down RTCConnection object
Object goes to “closed” state
RTCConnection events
oncandidate
Fires for local candidate (can be cancelled)
onendofcandidates
Fires when all local candidates are discovered
onactivecandidate
Fires when local:remote candidate connection changes
onstatechanged
Fires as connection state changes
onnetworkchange
Fires when network topology change has been detected
(auto “gather” called but can be cancelled)
onunknowntrack
Fires when track discovered which has no mapping
RTCSocket
interface RTCSocket
};
{
RTCSocket
- No exposed attributes, members or events
- Represents one physical local socket (as well
as virtualized paired firewall/relay socket)
- Re-use the same RTCSocket in another
RTCConnection to send over same physical
local socket
RTCStream
[Constructor,
Constructor(MediaStream)]
interface RTCTrack {
attribute MediaStream
attribute DOMString
sequence<RTCTrack>
void
void
void
};
source;
msid;
tracks();
start();
stop();
remove();
RTCStream
- References single MediaStream
- Collection of RTCTracks
- Change collection then call RTCConnection
“send”/”receive” again
- Simplifies MediaStream usage in RTCConnection
- Convenience “start” / “stop” methods
- “remove” method to remove from RTCConnection
RTCTrack
[Constructor,
Constructor(MediaStreamTrack)]
interface RTCTrack {
attribute MediaStreamTrack
attribute DOMString
attribute DOMString
attribute unsigned long
attribute sequence<RTCCodec>
attribute sequence<RTCMediaAttributes>
attribute Object
void
void
void
};
start();
stop();
remove();
source;
id;
kind;
ssrc;
codecs;
mediaAttributes;
rtpExtHeaders;
RTCTrack
- References single MediaStreamTrack
- Mapping of RTP/RTCP/etc wire information such as:
-
ssrc
codecs
media attributes (e.g. max width / height)
- Change properties then call RTCConnection “send” /
“receive” again
- Convenience “start” / “stop” methods
- “remove” method to remove from RTCConnection
RTCConnectionInfo
dictionary RTCConnectionInfo {
attribute RTCIceConnectionInfo
readonly
attribute RTCDtlsConnectionInfo
};
ice;
dtls;
RTCIceConnectionInfo
dictionary RTCIceConnectionInfo {
DOMString
usernameFrag;
DOMString
password;
};
RTCDtlsConnectionInfo / CertificateFingerprint
dictionary RTCDtlsConnectionInfo {
attribute CertificateFingerprint
};
fingerprint;
dictionary CertificateFingerprint {
getter ArrayBuffer (DOMString hashFunction);
};
RTCIceCandidateInfo
dictionary RTCIceCandidateInfo {
DOMString
int
DOMString
int
DOMString
int
DOMString
DOMString
int
};
foundation;
component;
transport;
priority;
connectionAddress;
connectionPort;
type;
relAddress;
relPort;
Object Realtime Communications API
(ORTC)
• Next generation WebRTC API, under development in the W3C
ORCA community group:
– http://www.w3.org/community/orca/
– Rationale: http://tools.ietf.org/html/draft-raymond-rtcwebwebrtc-js-obj-api-rationale
• Specification:
– http://openpeer.github.io/ortc/
• Github repository:
– ORTC API drafts and shim: https://github.com/openpeer/ortc
– ortclib: https://github.com/openpeer/ortclib
– Demos: https://github.com/openpeer/webrtc-demo
To Learn More
• ORCA Community Group:
http://www.w3.org/community/orca/
Demo and discussion tomorrow to follow…