This demo of the Microsoft Surface Hub 2 is pretty damn cool…
I don’t run a lot of Microsoft product anymore, switched to mac when the intel chip landed + Apple moved to a unix underpinning. That said, I have seen much better quality in products coming from Microsoft in the last few years, so maybe they deserve a second look.
Surface Hub 2 sort of reminds me of a product called Perch, built a by a local Vancouver team which was meant to serve as a portal into disparate global offices. Perch was way before it’s time. WebRTC was still in its infancy and personal device video conferencing had not really crossed the chasm, which is a shame considering where we are today.
Now there are many of video conferencing companies and products, and plenty of alternatives / platforms for developers to build on. It certainly seems plausible now that we could see the Microsoft Surface Hub 2 in boardrooms across the globe. Apparently it will be interoperable with WebRTC endpoints as well, which could make this a powerful work tool indeed. That would enable collaboration with peers over IP on various endpoints including laptops, tablets and mobile, regardless of the OS. Sharing product ideas, riffing on concepts and polishing final features on a product release using the Microsoft Surface Hub 2 as a tool, could be a refreshing new way to work.
It will be interesting to see what developments come about from the Microsoft press event in NYC in April, as reported by The Verge.
I haven’t blogged here in some time, so I figured that since the topic is relevant this would be good a good opportunity to dust off the old blog (webrtc.is / sipthat.com) and post something we have been working on at SignalWire. I am quite passionate about WebRTC and real-time communications so it’s great to be helping bring it to life at SignalWire!
We all know and love <cough> SIP, so we decided we would enable the use of SIP over WebSockets at SignalWire. This new offer also enables functionality like WebRTC with SIP over WebSockets.
This means our customers can now use off the shelf JS libraries, like JSSIP to create basic web experiences for their users, powered by SignalWire. It used to be a bit of a PITA, to create services that provided users with seamless online communications. Now it’s a breeze, and when using SignalWire it’s also very affordable.
For now, we are enabling basic calling and video capabilities, the advanced functionality (including video conferencing) will come in conjunction with a future release of a SignalWire RELAY JS library.
Personally, I can’t wait to see what creative minds will build using this technology with SignalWire on the backend.
For more than 6 years, we have been working on and looking forward to a simpler way to build RTC (Real Time Communications) applications on the web. In order for this technology to truly show its value, the major browser vendors needed to show up.
Now that Apple has joined the party in earnest, does the technology have the coverage required in order for developers to make good use of WebRTC on mobile devices? Let’s find out.
Until now, in order for WebRTC to work on iOS, we were relegated to wrapping WebRTC code in Objective-C and Swift, in our native iOS apps. Basically, we had to take the Chrome code and build an app that was sent to the app store for approval and wait in line, like all the other chumps (yours truly included). Conversely, on Android we could run much of that same code from our desktop Chrome apps, on the Android device as well, within reason of course.
Now that Safari and Chrome are shipping compatible WebRTC on mobile, we get to reuse the same code, right!? Well, mostly, they are different code bases, after all.A word about hardware acceleration.
If ubiquitous mobile video is to take off, the battery life of the device has to last more than the length of the 10 minute video call (ok, I am exaggerating a bit, but I think you get the point) and the performance needs to be at least adequate enough to distinguish facial features. My bar is set a little higher, baby steps for now.
Without h/w acceleration the CPU is likely working too hard to encode the local video and decode the inbound video + service the other processes required at the same time. That really means there needs to be hardware onboard the device dedicated to video coding. That in turn means H.264, since there are very few vendors that offer VP8 or VP9 h/w acceleration.
Question: Does this mean that mobile apps written with VP8 will not be able to deliver decent mobile video conferencing?
Answer: No, not at all, but they will likely not be as performant as those taking advantage of hardware acceleration.
Suffice to say that SVC (Scalable Video Coding) usage would be another reason why we need h/w acceleration, but that’s for another day.Who’s using what?
The majority of desktop and mobile WebRTC apps written today, are using VP8 for video.
Since Apple and Microsoft both use H.264 and Google uses VP8 and H.264 (recently shipped Open H.264 – on the desktop and mobile). Also, many of the Enterprise RTC developers are already on that H.264 bandwagon.
Question: If Apple and Microsoft devices ship with H.264, what is the case with Google Chrome on desktops and android, are they preferencing VP8?
Answer: Chrome for desktop and android now have H.264 native. Many of the Android devices that ship today all have H.264 hardware acceleration onboard. In order to understand which units have H.264 and hardware acceleration, you can run use the Android APIs to pull a list of available codecs, but in the case of WebRTC, you will only get H.264 in Android WebRTC if there is a h/w encoder on the device.Is H.264 the answer for WebRTC video?
Here is a recent test:
Host 1 – (before joining):
macOS Sierra, Macbook, Safari (Technology Preview 32)
Host 2 (after joining):
Android 7, Samsung 7, Chrome 55
Host 1 (after joining):
According to the Chrome Status page, Chrome for Android should have H.264. So why is the session barfing when trying to set up video? The logs do not lie…
Safari – offer:
Chrome on android – answer:
Err, huh? No H.264 in reply?
So, I updated to latest Chrome on android (58) and tried again…
Next topic, paying the man!
… et voilà!!
Shipping your product with H.264 enabled, means you may potentially need to deal with the MPEG-LA royalty police for H.264 royalties, but there are some grey areas.
In the case of Apple and Microsoft, where H.264 royalties are already being paid for by the parent vendor, the WebRTC developer is riding on the coattails of papa bear, at least in theory.
Cisco’s generous OpenH.264 offer means that those using this binary module, can do so at potentially no cost:
We will not pass on our MPEG-LA licensing costs for this module, and based on the current licensing environment, this will effectively make H.264 free for use on supported platforms.
Q: If I use the source code in my product, and then distribute that product on my own, will Cisco cover the MPEG LA licensing fees which I’d otherwise have to pay?
A: No. Cisco is only covering the licensing fees for its own binary module, and products or projects that utilize it must download it at the time the product or project is installed on the user’s computer or device. Cisco will not be liable for any licensing fees incurred by other parties.
That seems to mean (I am no lawyer) every developer shipping WebRTC apps supporting Open H.264 binary module, get a free ride. Those using some other binary, or shipping the above source code for that module, could be on the hook for those royalties. That said, since there are royalties being paid by parent vendors where devices are shipping H.264 anyways, developers may not get hassled regardless.Summary:
So what did we learn here?
- Apple has joined the party, now we have a full complement of browser vendors!
- If you want to leverage WebRTC video to deliver a ubiquitous mobile and desktop experience for your users, you should likely consider including both H.264 and VP8.
- VP8 is (still) free and powers most of the WebRTC video out there today.
- You can make use of the Open H.264 project and get a free H.264 ride, albeit baseline AVC.
- WebRTC on Android does not support software encoding of H.264, so unless there is local hardware acceleration, H.264 will not be in the offer.
- H.264 is not fully enabled (or buggy) in Chrome 55 (I was using it on Samsung S7 Edge (Android 7), but it does work with Chrome 58.
- WebRTC is not DOA!
- SDP still sucks and ORTC can’t come soon enough!!
As a side note, it would be interesting to see something like this open sourced; VP8 / H.264 conversion without transcoding, if only to service the existing desktop apps currently running VP8 <-> mobile H.264. It would likely overwhelm the mobile device, but it would be cool if it worked!
Disclaimer: The views expressed by me are mine alone and do not necessarily represent the views or opinions of my employer.
Looks like VP8 is not there after all, bummer. More political jostling afoot, which sucks for the development community.
This is a big deal, to have Apple / Safari onboard is really the final major obstacle in the adoption of this awesome standard.
More info (thanks Marc Abrams !!)…
Based on the beta for macOS High Sierra – that was made available yesterday…
– Test samples: webrtc.github.io/samples/ (It passed most of the tests)
– Video codec support is VP8 and H.264 (I have not seen a test that shows H.265 or HEVC but I know it’s there)
– Audio codec support is Opus, ISAC16, G.722 and PCMU
– Basic datachannel support is there but none of the tests seem to work
AWESOME!!! This took a bit longer that many of us were expecting, but hey better late than never!
Next week I will be joining friends old and new at PulverHWC to rediscover – How We Communicate.
Here is an email from Jeff Pulver inviting all of you to join us in Los Gatos for what is sure to be a landmark occasion.
Hope to see you there!
The Keys to the Communications Universe
Next week I return to doing the one thing that I love best – bringing together brilliant, interesting people.
Leaders, visionaries, dreamers and market makers from the worldwide communications industry have accepted my invitation to take part in the Pulver HWC Summit, May 18 – 19 at Testarossa Winery in Los Gatos, CA. I am grateful for both the people who are speaking and the tech legends who have signed up to join us for an intimate conversation. I believe understanding the message behind “How We Communicate” (“HWC”) is the next great area of growth in the communications space. Trillions of dollars of opportunity will be created and there are relationships to be forged, deals to be made, and knowledge to be shared.
There are a limited number of tickets still for sale. To join the conversation and to register, please click here. I would appreciate it if you could share this email with your friends and family involved in the communications industry.
Warm hugs, Jeff
- Added the gather() method, as noted in: Issue 165
- Removed “public” from RTCIceGatherPolicy, as noted in: Issue 224
- Removed the minQuality attribute, as noted in: Issue 351
- Made send() and receive() asynchronous, as noted in: Issue 399, Issue 463, Issue 468 and Issue 469
- Provided additional information on ICE candidate errors, as noted in: Issue 402
- Added state attribute to RTCSctpTransport, as noted in: Issue 403
- Provided an example of RTX/RED/FEC configuration, as noted in: Issue 404
- Clarified payloadType uniqueness, as noted in: Issue 405
- Updated the list of header extensions, as noted in: Issue 409
- Added “goog-remb” to the list of feedback mechanisms, as noted in: Issue 410
- Added kind argument to the RTCRtpReceiver constructor, as noted in: Issue 411
- Clarified send() restrictions on kind, as noted in: Issue 414
- Added getAlgorithm() method, as noted in: Issue 427
- Changed RTCDataChannel protocol and label to USVString, as noted in: Issue 429
- Clarified nullable attributes and methods returning empty lists, as noted in: Issue 433
- Clarified support for the “direction” parameter, as noted in: Issue 442
- Clarified the apt capability of the “red” codec, as noted in: Issue 444
- Clarified usage of RTCRtpEncodingParameters attributes, as noted in: Issue 445
- Clarified firing of onssrcconflict event, as noted in: Issue 448
- Clarified that CNAME is only set on an RTCRtpSender, as noted in: Issue 450
- Updated references, as noted in: Issue 457
- Described behavior of send() and receive() with unset RTCRtpEncodingParameters, as noted in: Issue 461
- Corrected dictionary initialization in the examples, noted in: Issue 464 and Issue 465
- Corrected use of enums in the examples, noted in: Issue 466
- Clarified handling of identity constraints, as noted in: Issue 467 and Issue 468
- Clarified use of RTCRtpEncodingParameters, as noted in: Issue 470
- Changed hostCandidate type, as noted in: Issue 474
- Renamed state change event handlers to onstatechange, as noted in: Issue 475
- Updated description of RTCIceGatherer closed state, as noted in: Issue 476
- Updated description of RTCIceTransport object, as noted in: Issue 477
- Updated description of relatedPort, as noted in: Issue 484
- Updated description of RTCIceParameters, as noted in: Issue 485
- Clarified exceptions in RTCDataChannel construction, as noted in: Issue 492
- Provided a reference to error.message, as noted in: Issue 495
- Clarified RTCRtpReceiver description, as noted in: Issue 496
- Clarified default for clockRate attribute, as noted in: Issue 500
- Removed use of “null if unset”, as noted in: Issue 503
- Updated RTCSctpTransport constructor, as noted in: Issue 504
- Clarified behavior of getCapabilities(), as noted in: Issue 509
- Addressed issues with RTCDataChannelParameters, as noted in: Issue 519
Last Thursday we had the first virtual w3c webrtc wg interim meeting. Once we sorted out a few technical details it went quite well!
Meeting Home Page:
ORTC, WebRTC, H.264, VP8, RID, RtpEncoding, Simulcast and much more. Google, Microsoft and Hookflash leading the discussion, join us!
We have an immediate WebRTC development contract opportunity that has just come up in the Seattle area. The contract requires 4-5 full-time developers onsite, remote will not fit the bill on this one.
For this contract we are looking for a team lead, 2 x Node.js, 2 x common JS developers
You have built commercial web applications using WebRTC libraries and are intimately familiar with the WebRTC and ORTC specs and respective libraries.
Start date: ASAP
If you are interested please forward your resume firstname.lastname@example.org
Our initial ORTC implementation includes the following components:
- ORTC API Support. Our primary focus right now is audio/video communications. We have implemented the following objects: IceGatherer, IceTransport, DtlsTransport, RtpSender, RtpReceiver, as well as the RTCStatsinterfaces that are not shown directly in the diagram.
- RTP/RTCP multiplexing is supported and is required for use with DtlsTransport. A/V multiplexing is also supported.
- STUN/TURN/ICE support. We support STUN (RFC 5389), TURN (RFC 5766) as well as ICE (RFC 5245). Within ICE, regular nomination is supported, with aggressive nomination partially supported (as a receiver). DTLS-SRTP (RFC 5764) is supported, based on DTLS 1.0 (RFC 4347).
- Codec support. For audio codecs, we support G.711, G.722, Opus and SILK. We also support Comfort Noise (CN) and DTMF according to the RTCWEB audio requirements. For video we currently support the H.264UC codec used by Skype services, supporting advanced features such as simulcast, scalable video coding and forward error correction. We’re working toward to enabling interoperable video with H.264.
W3C WebRTC working group chairs [Harald Alvestrand (Google), Stefan Håkansson (Ericsson), Erik Lagerway (Hookflash)], made a decision recently to add a new editor to the working group, as Peter St. Andre (&yet) has resigned as editor.
Bernard Aboba (Microsoft) has now been appointed as editor.
Bernard’s attention to detail and advocacy for transparency, fairness and community has been refreshing. It has been my pleasure (as chair of the W3C ORTC CG) to work with Bernard whom also is an author in the W3C ORTC CG alongside Justin Uberti and Robin Raymond (editor). I look forward to working more with him in the WG.
The new charter for the WebRTC Working Group has been approved. Current members will need to re-join, from the WebRTC WG mail list…
Great news, the new W3C WebRTC Working Group charter  has been officially approved by the W3C Director .
The revised charter adds a deliverable for the next version of WebRTC, has an updated list of deliverables based on the work started under the previous charter, clarifies its decision policy, and extends the group
until March 2018.
The charter of this Working Group includes a new deliverable that require W3C Patent Policy licensing commitments from all Participants.
Consequently, all Participants must join or re-join the group, which involves agreeing to participate under the terms of the revised charter and the W3C Patent Policy. Current Participants may continue to attend meetings (teleconferences and face-to-face meetings) for 45 days after this announcement, even if they have not yet re-joined the group. After 45 days (ie. September 10, 2015), ongoing participation (including meeting attendance and voting) is only permitted for those who have re-joined the group.
Use this form to (re)join:
Instructions to join the group are available at:
Vivien on behalf of the WebRTC WG Chairs and Staff contacts
As newly appointed co-chair in the W3C WebRTC WG, I just participated in my first Editor’s Call, and I’m impressed.
We had to address nearly dozens of Pull Requests and Issues on the associated github repos. We managed to knock down quite a few that ended up getting merged and a few that were closed today, despite not having 1 co-chair and 1 editor present.
There were some suggestions on how we could make the processes a bit more effective, allowing everyone to understand more what’s expected of them. It’s going to take a few meetings I suspect to get a real feel for how I can be adding the most value possible.
Overall, it feels like we are all trying our best to do what the new charter has set out, to get 1.0 done before getting on with the next chapter. I am excited to be part of it and look forward to continue helping!
If you have any thoughts on how the WebRTC Working Group could be doing things differently to be more effective and efficient, I would like to hear your thoughts.
Big thanks to everyone (especially Bernard) for putting in the extra work required here for our next CG meeting:Draft Community Group Report 22 June 2015
B.1 Changes since 7 May 2015
- Addressed Philipp Hancke’s review comments, as noted in: Issue 198
- Added the “failed” state to RTCIceTransportState, as noted in: Issue 199
- Added text relating to handling of incoming media packets prior to remote fingerprint verification, as noted in: Issue 200
- Added a complete attribute to the RTCIceCandidateComplete dictionary, as noted in:Issue 207
- Updated the description of RTCIceGatherer.close() and the “closed” state, as noted in: Issue 208
- Updated Statistics API error handling to reflect proposed changes to the WebRTC 1.0 API, as noted in: Issue 214
- Updated Section 10 (RTCDtmfSender) to reflect changes in the WebRTC 1.0 API, as noted in: Issue 215
- Clarified state transitions due to consent failure, as noted in: Issue 216
- Added a reference to [FEC], as noted in: Issue 217
With the forthcoming re-charter @W3C WebRTC Working Group, there were also a few managerial changes:
- Peter Saint Andre (@andyet fame), will be joining as co-editor
- Erik Lagerway, yours truly (co-founder @hookflash), will be joining as co-chair
- Vivien Lacourba, W3C staff, will be helping out Dominique Hazael-Massieux with increased W3C staff time in the WebRTC Working Group
I am personally flattered and over the moon excited to have been asked to co-chair the WebRTC Working Group and look forward to working with Harald and Stefan to help usher in the next era of WebRTC standards work.
We are holding our ninth CG meeting on the 24th of June…
Where: Online (TBD)
When: June 24, 2015 10am PDT
Review action items from last meeting:
– RTCIceCandidateComplete dictionary
– RTCIceGatherer.close affect on RTCIceTransport / RTCDtlsTransport
– Comments added to #200
Incoming media prior to Remote Fingerprint Verification
– Comments added to #170, Peter to send fuller proposal to list
Response to connectivity checks prior to calling iceTransport.start()?
– Original #188 – Priority Calculation, new bug #209
Trying to remove RTCIceTransport.createAssociatedTransport(component)
– Philipp Hancke’s Review Comments
Review open issues: https://github.com/openpeer/ortc/issues?q=is%3Aopen
Review current draft: http://ortc.org (upper right hand side)
Review implementation progress: ORTC Lib, MS Edge, Google ?
Review ORTC CG alignment with WebRTC WG and 1.0 spec.
Plan next meeting.
Fresh out of Google IO, Justin Uberti provides a WebRTC update via WebRTC Meetup in SFO at the Twilio HQ. Slides and demos are not visible, I am attempting to get a copy of the slides. UPDATE: Most of the slides were captured via photos.
Justin talking points:
– Renewed focus on mobile
– HD bitrates and bandwidth estimation
– Goal H.264 coming to Chrome 45 via Cisco’s OpenH264 (whoa!)
– VP9 & hardware support
– Demo on Nexus 6 using VP9 and hardware encoder
What’s coming next..
– Mobile performance
– Complete call setup should be 500ms
– Encryption (we don’t hold the keys)
– ECDSA coming soon!
– HW encode on android capable of 1080p
– New Echo Cancellation via DAEC (Delay Agnostic Echo Canceller)
– Mobile Networks
– Network Handoff
– Scaling Quality
– Better performance on lossy networks
New domain for “WebRTC and Web Audio resources”
Q What’s the story on spec deviation?
A We want to make sure we add promises to the spec.
Q Get Stats?
A Working on it
Q Unified plan support
A Organizationally challenged and taking back seat to encoding performance and other “on fire” must fix immediately
Q What is going to evolve in screen sharing in spec and Chrome?
A Things work “ok” for screen sharing but not great for some things like scrolling, people are also interested in using in tabs versus window. Screen refresh is not as fast as we would like but we think we have fixed that.
Q Changing framerate and resolution mid-call?
A RTPSender gives you some of these knobs (Note: Object from ORTC Spec!), which is on its way.
Q Battery life for hw encoded apps?
A 3 categories, voice only, video on sw, video on hw. Video demo was on hw at 1080p at 30% of CPU. HW video will compete with a baseband voice call on wifi.
Feross Aboukhadijeh & John Hiesey (creators of PeerCDN
– Using WebRTC DataChannel to stream content
– Demo: can’t see the screen
– Hosting websites in Browsers via WebTorrent
– NAT traversal via regular STUN / TURN
Q Justin asks, what will it take to have this work with existing bittorrent clients
A They need to add WebRTC, then it will work