News from Industry

Register to the two free webinars I am hosting this month in areas around supporting WebRTC with Talkdesk and Poly.

I am shifting gears this year. Looking back at last year, what I’ve noticed is that there’s been a shift in what clients are asking of me. Many of them are more interested in issues that are support related rather than architecture or development. While a lot of the work I do revolves around assisting with defining architectures and dealing with roadmaps of products, there’s been an ongoing increase in the questions related to supporting WebRTC.

This led to a few changes in the things that I have on offer:

1. At testRTC, where I am a co-founder and apparently also the CEO, we’ve launched a new product for network testing. This is focused on helping people who support clients with network related issues around WebRTC applications. We’re now working on another product specific to this domain
2. At BlogGeek.me, I am now offering a new course called Supporting WebRTC. With 40 people who were there in the prelaunch and the feedback I’ve been getting, this seems like the topic is really relevant to many

Somehow, I found myself scheduling two separate free webinars for this month with partners that are around WebRTC support.

Talkdesk and how to support WebRTC-based call centers

At testRTC, we’ve created a product in 2019 to assist support teams analyze network issues for their users. Our first client for this product were Talkdesk who were kind enough to share their experience with us in a nice testimonial.

On Tuesday next week, João Gaspar from Talkdesk will join me in a webinar titled How to analyze WebRTC network issues in minutes and not hours (or days). In this webinar, I’ll explain a bit about the challenges WebRTC poses when it comes to connectivity from a support perspective, and João will share with us what Talkdesk are doing today to assist their users.

I’ve learned a lot from working with João and his team last year, and I am sure this will be interesting to you as well.

How to analyze WebRTC network issues in minutes and not hours (or days)

Tuesday, January 21, 2020

14:00-14:45 EST; 11:00-11:45 PST

Register here Poly and picking the right headset to improve WebRTC session quality

In the last year I’ve had a lot of conversations with support engineers. The people who end up needing to troubleshoot, figure out and explain issues to their users. Many of these issues end up being related to network connectivity. This made me create the new Supporting WebRTC course (now open for all to enroll). One thing I wanted to add there but had no clue about is headsets.

Headsets are this thing that I have at home and use for most of my conference calls. But I never really gave them a second thought. The last pair I purchased at the local computer equipment store, not even making an informed decision about what I needed.

That lead me to reach out to Poly, to get a briefing about headsets and how they affect quality in WebRTC, which lead to me understanding that this boring topic known as headsets is quite fascinating. Obviously, I used what I learned in that briefing to create that lesson I needed in my course.

The great thing though, is that Richard Kenny from Poly (who briefed me), was kind enough to accept joining a webinar about this topic.

Picking the best headset for your next WebRTC session

Tuesday, January 28, 2020

14:00-14:45 EST; 11:00-11:45 PST

Register here How are you handling your support efforts with WebRTC?

The people who usually follow me here are developers or product managers. Seldom are they support-oriented. I know that based on the comments and conversations I have on and off this website.

My suggestion to you is to go check what your support team is challenged with. What is keeping them up at night. What is it they need assistance with. What knowledge are they missing.

And then once you do, see if these webinars might be useful to them so you can share this with them. Let’s make 2020 the year we start solving more of the connectivity issues for our customers.

The post Supporting WebRTC: Two webinars coming your way (with Talkdesk & Poly) appeared first on BlogGeek.me.

WebRTC isn’t like Node.js or TensorFlow. Its purpose isn’t adoption in general, but rather adoption in browsers. If you believe otherwise, then there’s a problem of expectations you need to deal with.

As we are starting 2020, with what is hopefully going to be an official spec for WebRTC 1.0, it is time for a bit of reflections. I started this off when writing about Google’s WebRTC roadmap and I’d like to continue it here about WebRTC goals and expectations.

When I explain what WebRTC is, I start off with the fact that it is two things at the same time:

1. A standard specification
2. An open source project

The open source project angle is interesting.

Is WebRTC an open source project?

The main codebase we have for WebRTC today is the one maintained by Google at webrtc.org. There are other open source projects that implement the spec, but none to this level of completeness and quality.

By the ecosystem and use of WebRTC, one may think that this is just another popular open source project, like Node.js or TensorFlow.

It isn’t.

If I had to depict Node.js, it would be something like this:

TensorFlow?

How would I draw a diagram of WebRTC? Probably something like this:

From an administrative point of view, WebRTC is part of Blink, Chromium’s rendering engine. Blink is part of Chromium, the open source part of Chrome. And Chromium is what Chrome uses as its browser engine.

WebRTC isn’t exactly an independent project, sitting on its own, living the life.

Need an example why? WebRTC’s version releases follow the version releases of Chrome in terms of numbering and release dates. But mobile doesn’t follow the exact same set of rules. Olivier wrote it quite eloquently just recently:

“For web developers, release notes are very good and detailed. But for IOS and Android developers… I expect the same level of information.”

There’s an expectation problem here…

WebRTC isn’t like other open source projects that stand on their own, independent from what is around them. WebRTC is a component inside Chrome. A single module.

The WebRTC team at Google are assisting developers using the codebase elsewhere. It took a few years, but we now have build scripts that can build WebRTC separately and independently from Chromium. We have official pre-compiled mobile libraries for WebRTC from Google, albeit not a 1:1 match to the official WebRTC/Chromium releases.

At the end of the day, the WebRTC team at Google are probably being measured internally at Google by how they contributed to Chrome, Google’s WebRTC-based services AND to the web as a whole. Less so to the ecosystem around their codebase. If and how WebRTC gets adopted and used in mobile first applications or inside devices and sensors is harder to count and measure – and probably interests Google management somewhat less.

Who contributes to WebRTC?

I took the liberty of checking the commit history of the WebRTC git project over the years, creating the graph below:

There were various different emails associated with the committers, but they fell into these broad categories:

1. People with a webrtc.org email address. These are Google employees working directly in the WebRTC project (at least I don’t know of a non-Googler with a webrtc.org email address)
2. People with a google.com email address
3. Commits done with a “chromium-webrtc-autoroll@” or similar “email” address in them. I’ve categorized these as bots
4. All the others

It is safe to say that the majority of committers throughout the years are Googlers, and that the ones who aren’t Googlers aren’t contributing all that much.

Is that because Google is protective about the codebase, as it goes right into Chrome which servers over a billion users? Or is it because people just don’t want to commit? Maybe the ecosystem around WebRTC is too small to support more contributors? Might there be other reasons?

One wonders how such a popular project has so little external contributors while there are many developers who enjoy it.

Is webrtc.org Google’s RTC or ours?

A few years back, Google introduced a new programming language – Go (or Golang). It is getting quite a following (and its own WebRTC implementation, though unrelated to this article).

In May 2019, quite a stir was raised due to a post published by Chris Siebenmann titled Go is Google’s language, not ours. Interestingly enough, if you replace the word “Go” with “WebRTC” in this article – it rings true in many ways.

Golang has over 2,000 lines in its CONTRIBUTORS file versus WebRTC’s 100+ AUTHORS. While Golang identify individual contributors, WebRTC uses wildcard “corporate” contributions (I wouldn’t count too many contributors in these corporates though). WebRTC is smaller, and I dare say more centralized.

The simple answer to those who complain is going to be the same – “this is an open source project, feel free to fork it”.

For WebRTC, I’d add to this that what goes into the API layer is what the W3C and IETF decide. So Google isn’t in direct control over the future of WebRTC – just of its main implementation, which needs to adhere to the specification.

Then there’s the Node.js community forks that took place over the years (latest one from 2017). These disputes, technical and political, always seem to get resolved and merged back into the main project. In hindsight, this just seems like attempts to influence the direction of the project.

Can this be done for WebRTC?

It already occurred with the introduction (and slow death) of ORTC. ORTC (Object-RTC) started and was actively pushed by Microsoft, ending with most of what they wanted to do wrapped up into WebRTC (and probably causing a lot of the delays we’ve had with reaching WebRTC 1.0).

What does that mean to you?

Should you complain about Google? Maybe, but it won’t help

For Google, it makes sense to push WebRTC into Chrome as that is its main objective. Google is improving in tooling and capabilities of using WebRTC outside of Chrome, but this objective will always be second to prioritization of Chrome’s needs and Google’s services.

As an open source project, you are free to use or not use it. You’re not paying for it, so what would you be complaining about?

Google have invested and is still investing heavily in WebRTC. It is their prerogative to do so, especially as they are the only ones doing it today.

You should make an educated decision, weighing your requirements, risks and challenges, when developing a service that makes use of WebRTC.

The post Google’s WebRTC goals – a problem of expectations appeared first on BlogGeek.me.

Google’s plans for WebRTC have either changed or finally got revealed. Where? In its internal WebRTC roadmap.

WebRTC is many things.

On one hand, it is a standard specification at the W3C (and is reaching 1.0 milestone).

On the other hand, it is an open source project. While there are a few such projects today, the most important one is Google’s webrtc.org. This is the code that gets into Chrome itself and the one being adopted by many (simply because it is already highly optimized for the main scenarios. And… it is free).

Google made it super simple for companies to adopt its WebRTC implementation – it uses a BSD open source license, making it quite permissive.

In the last 8 years, we’ve been treated like royalty, having access to a world-class media engine implementation for free.

The WebRTC roadmap we’ve seen so far from Google had 3 types of features in it:

1. Making sure the implementation fits the spec
2. Improve the architecture to perform better
3. Add features specific to Google’s needs in other projects (not necessarily abiding to the spec)

At all times, these were available to everyone.

Google’s intent in open sourcing WebRTC

When WebRTC was first introduced it was about who has the balls to take something that up until that point was considered a core competency and make it freely available. This was a piece of technology that video conferencing companies protected fiercely, battling about through their sales and marketing pitches, each claiming to have superior media quality. At the time, media quality wasn’t in the “good enough” position that it is today:

Google made the calculated risk at the time:

• Media quality was improving. So were bandwidth available and compute. It made sense that it would get to a point of “good enough” within a few years time
• A migration to the cloud for video conferencing wasn’t at most companies agenda yet, but as cloud migration started picking up everywhere, it made sense to occur here as well. These cloud migrations took place hand in hand with the use of browsers
• Dominance in browsers and lack of a real operating system footprint meant needing to have a media engine as part of the browser
• Google had no leading service in video conferencing. Google Hangouts was available, but wasn’t any real competition to the leading platforms at the time, so they didn’t have much to lose by the decision

Other vendors just following along in the ride, making minor contributions here and there. Today, the leading (and only) media engine out there for WebRTC is still the Google one. At least in any meaningful way. So much so that Google’s “competitors” are using Google’s WebRTC stack directly in their products.

Where has this lead Google?

WebRTC is a huge success. All modern browsers now support it. They interoperate (to a good extent). Today, in every industry and market where live or real time media is needed, WebRTC is playing an important role.

But what about Google and WebRTC? What success did Google exert from WebRTC?

Not a lot. Or at least not enough.

Google uses WebRTC in the following services it offers:

• Hangouts / Google Meet
• Duo
• Stadia
• Chrome Remote Desktop
• YouTube Live

Lets see how well did Google fare in each.

Hangouts / Google Meet

I use these two services almost on a daily basis. My calendar meetings default to them simply because they are so each to schedule with the Google Calendar. They offer what I need without any of the complexity.

But.

When you read or hear discussions about the video conferencing market, the vendors mentioned are usually Zoom and Cisco. Maybe Microsoft Teams or Skype for Business. Also Bluejeans and Pexip. A few others. Google isn’t one of the top vendors that come to mind here. Even though their service is rather good.

Did I mention that almost all their competitors are using WebRTC as well?

Duo

Duo. Google’s answer to Apple’s FaceTime.

It is a standalone video calling app available on Android and iOS. It isn’t installed by default on most smartphones and users need to actively find it, install it and make a decision on using it. Not an easy feat.

Why hasn’t Google nailed and bolted it smack into Android? Probably due to carriers and not wanting to hurt their feelings (and Google’s relationship with them). Otherwise, it makes no sense for Google to try and compete with the likes of FaceTime with one hand tied behind their backs.

Anyways… Duo is quite popular. Even on iPhone. It is ranked #7 in the social apps in the Apple App Store. This is higher than Houseparty (positioned somewhere at #17-20), which is rather interesting considering the high engagement Houseparty sees for its users.

Google doesn’t share any stats on usage of Duo. The only thing we know is downloads and the number of people who ranked it – two stat points that are useless for social networks. This is quite telling to the real usage numbers – not publishing them means they aren’t on par with the competition.

Curious myself, I’ve put out a quick poll on Twitter:

This is most definitely NOT the way to know or understand usage, but it is interesting.

My audience is probably tech savvy. Those answering the poll are highly likely to know about WebRTC. And still. We have over 50% who never tried it and 13% who use it. I’d consider 13% quite a lot and surprising. But it isn’t scratching the surface of where it should be given that Google owns and controls Android.

Stadia

Google Stadia is something totally different. It is cloud gaming. The game is being processed and rendered in “the cloud” and gets streamed in real time to your device using WebRTC. Google even made modifications to its WebRTC implementation to make it a better fit for gaming.

The concept is great. The technology is solid. The experience is said to be good (if you’re close enough to the data center and have a good network connection).

From the media, it seems like there are hurdles and challenges to the Stadia launch – this type of an article titled “Stadia’s biggest problem? Google” or this one titled “Google Duo is the best video calling service you’re not using” are rather common. Especially when put in comparison to the Apple Arcade launch.

Looking at Google Play store numbers for the Stadia app, things look rather disappointing: below 1M installs so far:

I have this feeling Google expected more.

Cloud gaming is still new and nascent. It will take time to happen and mature.

Take this into an adjacent industry, Netflix introduced streaming in 2007. It took them 3-4 years for the stock to take notice and the service to mature enough to make a dent in the industry. Whereas today, every other production studio is launching their own streaming service.

Will Google have the patience with Stadia to get there or will it end up shutting it down like many other “experiments” it has been running throughout the years? The thought itself is making it hard for Google to entice game developers to jump on its platform.

Chrome Remote Desktop

Google apparently has a remote desktop service. It makes use of WebRTC’s screen sharing capability and is called Chrome Remote Desktop.

While I haven’t used it myself, this does seem to have quite a following. 10M+ installs on Android, The Chrome extension shows ~4.8M users.

There is no apparent business model as the service is offered freely, and while the market has similar paid services, it doesn’t seem to be big enough to attract a company like Google. This isn’t interesting enough to value an investment in WebRTC itself by Google.

YouTube Live

YouTube has the ability to host live events. And it does that with the help of WebRTC.

That said, its use of WebRTC isn’t an impressive one – it is just a window into the service if you want to broadcast from your browser. It isn’t used for live streaming to the users themselves. There’s more on the technical side of it on webrcHacks, where they analyze what goes on the wire with YouTube Live.

Here’s the thing – just like Chrome Remote Desktop, this is Google exploiting a technology that is there. It isn’t about leading the industry or the market with it. And as with Chrome Remote Desktop, it isn’t of enough value to make it worth their while to invest in making WebRTC itself better.

–

WebRTC is now part of HTML5 and part of what browsers need to do, so Google needs to invest in having it in Chrome. How much to invest is the real challenge.

To WebRTC or not to WebRTC?

Meet, Duo and Stadia seem to be the leading factors in whatever Google is doing in WebRTC, other than dealing with complaints and feedback from the community.

Google Meet

Google Meet is using VP9. It is one of the only group calling services running in production at scale that have made that shift.

By harnessing WebRTC and owning its roadmap, Google is able to experiment and build their service faster than others can on WebRTC.

Two interesting examples we’ve had in the past year –

1. At Kranky Geek 2018, Google showed an experiment of using WebAssembly with WebRTC to improve video switching in a conference by distinguishing noise and speech:

Did it find its way into Google Meet? Maybe.

Then there’s the new captioning feature in Google Meet, which Gustavo nicely explains. It uses the data channel in WebRTC to send back the results. Assuming anything in WebRTC was needed to change to make this work better, Google could do that as it owns the WebRTC roadmap.

Google Meet, being predominantly a browser based experience, will need to rely on changes made directly into WebRTC or things that can be bolted on top using WebAssembly.

Google Duo

Google Duo is a mobile first service. It has browser support via Duo for Web, but for the most part, it is meant to be used on your smartphone.

Last month, Google announced some new features in Pixel phones, but also 3 machine learning based improvements for Duo:

Auto-framing:

“Auto-framing keeps your face centered during your Duo video calls, even as you move around, thanks to Pixel 4’s wide-angle lens. And if another person joins you in the shot, the camera automatically adjusts to keep both of you in the frame.”

We’ve seen Facebook do that in Portal and a few video conferencing vendors adding that to their room systems.

Packet loss concealment:

“When a bad connection leads to spotty audio, a machine learning model on your Pixel 4 predicts the likely next sound and helps you to keep the conversation going with minimum disruptions.”

Packet loss concealment using machine learning is something not many are doing (or publishing that their are doing).

Background blur:

“you can now apply a portrait filter as well. You’ll look sharper against the gentle blur of your background, while the busy office or messy bedroom behind you goes out of focus.”

Another nice feature, which is available in other services such as Zoom.

From the looks of it, auto-framing and background blur rely on hardware based capabilities of the Pixel devices. Packet loss concealment… a lot less so.

Could we see machine learning based packet loss concealment find its way into the WebRTC codebase? (where it makes the most sense to add it instead of as an external piece of software). Not soon…

Google Stadia

For Google Stadia, Google went with QUIC instead of SCTP for the controls. It decided to make use of WebRTC for live streaming itself.

But it wasn’t enough. It needed the low latency of WebRTC to be even lower. So it added a Chrome experiment to enable them to reduce the playout delay in WebRTC. A few of my clients have already adopted it and are happy with the results for their own use case.

Google also tweaked and improved the VP9 decoder to make it work with 4K 60fps streams.

In the case of Stadia, the changes need to be made inside the WebRTC codebase to apply well for its service anywhere.

What is changing with Google’s strategy about WebRTC in 2020?

WebRTC 1.0 is “out”. Almost.

The latest CR (Candidate Recommendation) is dated December 13. Hopefully the last one before we go to the next step. It is interesting to look at the original charter of WebRTC:

It took somewhat longer to get here than originally expected, but we’re almost there.

Google held its internal milestone of WebRTC 1.0 code complete two months back.

What now?

Besides housekeeping, bug fixes, and talking about WebRTC NV (the next version), I think a lot of it will change internally at Google to how can they make more of their investment in WebRTC and stay or become more competitive in the market. This being an open source project, means that some features will need to be kept out of the open source codebase. Like the new packet loss concealment mechanism in Google Duo.

How is that achievable?

The leading factor is going to be adding more flexibility and control to developers over what WebRTC is and how it operates. Ideally by using WebAssembly and in the future by using WebTransport and WebCodecs, two new initiatives that will unbundle a lot of what WebRTC is.

This gives the ability to take out improvements out of the baseline implementation and introduce them as proprietary features.

The demarcation line of what will go into the WebRTC codebase by Google and what will be kept out of it is going to be the use of machine learning and artificial intelligence. Whenever a feature makes use of learned machine learning models, Google will most probably try to keep that implementation out of WebRTC. Why? Because it has the greatest value and the highest investment today.

Should this worry you?

Maybe, but it is to be expected.

Google has invested heavily in WebRTC. Without this investment nothing that we see and use today in WebRTC and take for granted would have been possible.

It is even surprising that it lasted this long…

WebRTC closes the basic gaps and requirements of media engines. It is good enough. If you want to improve upon it, differentiate or be at the cutting edge of the WebRTC technology, you will need to invest in it yourself as well. Relying only on Google isn’t an option. And probably never really was.

Here’s to an interesting and eventful 2020 with WebRTC!

The post Google’s private WebRTC roadmap for 2020 = AI appeared first on BlogGeek.me.

Review of Chrome's migration to WebRTC's Unified Plan, how false metrics may have misguided this effort, and what that means moving forward.

Continue reading Is everyone switching to Unified Plan? at webrtcHacks.

Conference calls were always complex. WebRTC might have made joining them simpler, but it does come with its own set of headaches.

I’ve been in the industry for the last 20 years or so (a dinosaur by now). I had my share of conference calls that I joined or scheduled. As humans, we tend to remember the bad things that happened. The outliers. There are many of those with conferencing.

When I saw this Dilbert strip the other day, it resonated well with the “Supporting WebRTC” course I’ve been working on these past few months:

One of the things I am dabbling with now in the course are media quality issues. This was spot on. So of course I had to share it on Twitter, which immediately got a colleague to remind me of this great Avengers mock video conference:

The funny thing is that this still occurs today, even if people will let you believe networks are better and these problems no longer exist. They do. Unless you are Zoom – Zoom always works. At least until it doesn’t…

What can possibly go wrong?

This one was just published today, so couldn’t resist…

A modern WebRTC service today will have a few potential failure points:

1. The cloud vendor’s infrastructure
2. Your own infrastructure
3. The user’s network
4. The user’s browser
5. The user’s device

Let’s try to break these down a bit

1. The cloud verndor’s infrastructure

Here’s a secret. AWS breaks from time to time. So does Azure and Google and Digital Ocean and practically everyone else.

Some of these failures are large and public ones. A lot more are smaller and silent ones that aren’t even reported in the main status pages of these cloud vendors. We see that in testRTC – as I am writing these words, we are struggling with a network or resource issue with one of the cloud vendors that we are using, which affects one of our services (thankfully, we’re still running for most of our customers).

Your service might be unreachable or experiencing bad media quality because of the cloud vendor you are using. Fortunately, most cases, these are issues that don’t last long. Unfortunately, these issues are out of your control.

2. Your own infrastructure

This one is obvious but sometimes neglected. What you run in your backend and how the client devices are configured to use it has a profound effect on the quality of experience for your users.

I’ve seen anything from poor ICE servers configuration, through bad scaling decisions to machines that just need a reboot.

WebRTC has a lot of moving parts. You need to take good care and attention to them.

3. The user’s network

Now we head towards the things that you have no control over… and primarily that is the user’s network.

You. don’t. have. control. over. what. network. your. customer. Uses.

He might be over a poor 3G connection (yes, we still have those). Or just be too far from the closest WiFi hotspot he is connected through. Or any other set of stupid issues.

In enterprises, problems can easily include restrictive firewall configurations or use of an HTTP proxy or a VPN.

Then there’s the congestion on the user’s network based on what OTHER people are doing on it.

Here, what you’ll need to do is to be able to understand the issue and explain it to the user to help him in squeezing more out of the network he is using.

4. The user’s browser

Here’s another challenging one.

The first one is a bit obvious – modern browsers automatically upgrade. This means you will end up with a new browser running your app one day without Apple, Google, Microsoft or Mozilla calling you to ask if you agree to that. And yes – these upgrades may well change behavior for customers and affect media quality.

Then there’s the opposite one – in enterprise environments, IT administrators sometimes lock browser versions and don’t let them upgrade automatically.

The biggest challenge we’re now facing though are Google experiments, like the one conducted with mDNS in WebRTC. Google is conducting experiments in Chrome on live users sporadically. You have no control over these and no indication where and how they are conducted. The whole purpose of this is to surface issues. Problem is, you won’t know if it breaks things for you until someone complains (or unless you monitor your deployment closely).

5. The user’s device

The device the user uses affects quality. Obviously.

Tried recently to use an iPhone 4 with a WebRTC service?

The CPU, memory, software and other processes your user has on the device will affect quality. Add to that the fact that certain devices and peripherals behave differently and have their own known (or unknown) issues with WebRTC, and you get another minor headache to deal with.

The things we can control in our WebRTC conference calls

Here’s where we started – a modern WebRTC service today will have a few potential failure points:

1. The cloud vendor’s infrastructure
2. Your own infrastructure
3. The user’s network
4. The user’s browser
5. The user’s device

In WebRTC calls, you can control your own infrastructure. And you can build it to work around many cloud vendor’s infrastructure issues.

You can try to add logic that deals with the user’s device.

You can probably deal with many of the user’s browser issues by more testing and running their unstable and developer preview releases.

The things we can’t control in our WebRTC conference calls

The main thing you can’t control is the user’s network.

What you can do here is to provide better support, assisting your users in finding out the issues that plague their network and suggesting what they can do about it.

Two things you will be needing to get that done: tooling and knowledge

The tooling side I’ll probably touch in a future article. The knowledge part is something I have a solution for.

How can you better serve your customer?

In the last few months I’ve been working on the creation of a new “Supporting WebRTC” course. This course is geared towards support people who get complaints from users about their service and they need to understand how to help them out.

The course started through conversations with support teams in widely known providers of WebRTC services, which turned into a suggested agenda that later turned into a real course.

There are already close to 6 hours of content split into 33 lessons, with more to be added in the next month or so.

I’ve decided to open up registration to the course to everyone and not limit it to the limited pre-launch users I’ve shared it with. I feel it is the right time and that the content there is rock solid.

If you want to improve your knowledge or your support team’s knowledge of WebRTC, with a focus on getting them making your users happy and using your service, then check out my course.

Register to the Supporting WebRTC course

The post WebRTC conference calls. What could possibly go wrong? appeared first on BlogGeek.me.

WebRTC isn’t only about guest access or even interoperability. It is about the whole infrastructure and service.

My article last month about guest access, the use of WebRTC for it AND how it is now used for “interoperability” between Microsoft and Cisco had its nice share of feedback and comments. Both on the article and off of it in private conversations. I think there is another trend that needs to be explained, which in a way is a lot more important. This one is about video conferencing hardware being dominated by HTTP and WebRTC. This in turn, is affecting how modern video infrastructure is also shifting towards WebRTC.

Where video conferencing hardware meets WebRTC

Check out this recent session from Kranky Geek last month. Here, Nissar Mahamood from Lifesize explains how WebRTC got integrated into their latest meeting room systems (=hardware), getting it to 4K resolutions.

It is a good session for anyone who is looking at embedded platforms and systems or needs to customize WebRTC for his own needs, using it outside of a web browser.

There are two things in this video that surprised me, for two very different reasons:

1. Using GStreamer as the basis of the media engine
2. Selecting Node.js as the client application environment

Using GStreamer as the basis of the media engine

I started seeing more and more developers using GStreamer as part of the technology stack they use with WebRTC. On Linux, your best bet with processing media using open source is either ffmpeg or GStreamer. Due to the real time nature of WebRTC, GStreamer is often the more sought after approach. In the past year or so, it also added WebRTC transport, making it a more viable option.

In many cases, the use of GStreamer is for connecting non-WebRTC content to WebRTC or getting content from WebRTC to restream it elsewhere. Lifesize has done something slightly different with it:

As the illustration above from their Kranky Geek session shows, Lifesize replaced the media engine (voice and video engines) part of WebRTC with their own which is built on top of WebRTC. They don’t use the WebRTC parts of GStreamer, but rather the “original” parts of it and replacing what’s in WebRTC with their own.

It is surprising as many would use WebRTC specifically for its media engine implementations and throw its other components. Why did they take that route? Probably because their existing systems already used GStreamer that is heavily customized or at the very least fine tuned for their needs. It made more sense to keep that investment than to try and reintroduce it into something like WebRTC.

This approach, of taking the WebRTC source code and modifying it to fit a need isn’t an easy route, but it is one that many are taking. More on that later.

Selecting Node.js as the client application environment

We’ve been so focused on development with WebRTC on browsers and mobile, that embedded non-mobile platforms are usually neglected. These have their own set of frameworks when it comes to WebRTC.

The one selected by Lifesize was Node.js:

They created a Node.js wrapper that interfaces directly with the WebRTC native C++ “API” with an effort to create the same JS API they get in the browser for WebRTC.

Why? Their meeting room systems now use HTML for its visual rendering and the application logic is driven by JavaScript.

Why JavaScript?

Because of Atwood’s Law

any application that can be written in JavaScript, will eventually be written in JavaScript

Lifesize simply made their application to one that can be written in JavaScript.

This is doubly true when you factor in the need to support web browsers where you have WebRTC with a JS API on top anyways.

The hidden assertion of WebRTC cloud infrastructure

What I like the slide above is the cloud with the wording “Lifesize Cloud Service” in it. The fact that Lifesize is connecting to it via Node.js speaks volumes about where we are and where we’re headed versus where we’re coming from.

A few years ago, this cloud service would have been based on H.323 or SIP signaling.

H.323 is now a deadend (something that is hard for me to say or think – I’ve been “doing” H.323 for the better part of my 13 years at RADVISION). SIP is used everywhere, but somehow I don’t see a bright future for it outside of PSTN connectivity (aka SIP Trunking).

Lifesize may or may not be using SIP here (SIP over WebSocket in this case) – due to the nature of their service. What I like about this is how there is a transition from WebRTC at the edge of the network towards WebRTC as the network itself. Let me try and explain –

Video conferencing vendors started off looking at WebRTC as a way to get into browsers. Or as a piece of open source code to gut and reuse elsewhere. If one wanted to connect a room system or a software client to a guest (or a user) connecting via WebRTC on a web browser, this would be the approach taken:

(I made up that term transcoding gateway just for this article)

You would interconnect them via a gateway or a media server. Signaling would be translated from one end to the other. Media would be transcoded as well. This, of course, is a waste of processing and bandwidth. It is expensive and wasteful. It doesn’t scale.

With the growing popularity of WebRTC and the increasing use and demand for browser connectivity to video conferences, there was/is no other way than to rethink the infrastructure to make it fit for purpose – have it understand and work with WebRTC not only at the edge.

That’s when vendors start trying to fit WebRTC paradigms into their infrastructure:

(guess what? Translating gateway? Also made up just for this article)

Things they do at this stage?

• Add support for SRTP in their infrastructure
• Think of adding BUNDLE and rtcp-mux
• Implement trickle ICE, or at the very least ice-lite
• Try to align the video codecs properly

There are a lot of other minor nuances that need to be added and implemented at this stage. While some of these changes are nagging and painful, others are important. Adding SRTP simply means adding encryption and security – something that is downright mandatory in this day and age.

The illustration also shows where we focused on making the changes in this round – on the devices themselves. We’ve “upgraded” our legacy phone into a smartphone. In reality, the intent here is to make the devices we have in the network WebRTC-aware so they require a lot less translation in the gateway component.

Once a vendor is here, he still has that nagging box in the middle that doesn’t allow direct communication between the browser and the rest of his infrastructure. It is still a pain that needs to be maintained and dealt with. This becomes the last thing to throw out the window.

At this last stage, vendors go “all in” with WebRTC, modifying their equipment and infrastructure to simply communicate with WebRTC directly.

This migration takes place because of three main reasons:

1. The need to support web browsers with WebRTC
2. The cost of interworking across WebRTC and whatever the rest of the vendor’s infrastructure supports
3. The popularity of WebRTC and its vibrant ecosystem marks it as the leading technology moving forward

That third reason is why once a decision to upgrade the infrastructure of a vendor and modernize it takes place, there is a switch towards adopting WebRTC wholeheartedly.

This isn’t just Lifesize

Microsoft took the plunge when adding Skype for Web and went all in with Microsoft Teams.

With their hardware devices for Teams they simply support web technologies in the device, with WebRTC, which means theoretical ability to support any WebRTC infrastructure deployed out there and not only Teams.

Same as the above is what we see with Cisco recently.

BlueJeans and Highfive both live and breath web technologies.

Forgot to mention you? Put a comment below…

There were other good Kranky Geek sessions around this topic this year and last year. Here are a few of them:

• Discord (2018) – how they use WebRTC and the changes made to it to fit their needs. In their case, that was very large audio conferences
• Microsoft (2018) – an overview of WebRTC for UWP and Hololens
• PION (2019) – an alternative WebRTC stack written in Go, and using WASM while at it
• RingCentral (2019) – server apps with Node.js+WebRTC
• Phantom Auto (2019) – replacing the video encoder with an external hardware one running on an NVIDIA GPU

The winning video conferencing hardware software stack

Here’s what seems to be the winning software stack that gets shoved under the hood of video conferencing hardware these days. It comes in two shapes and sizes:

Linux

• Linux as the underlying operating systems
• HTML/JS as the visualization layer (Node.js, Chromium or WebKit as its baseline)
• WebRTC embedded in there as part of the HTML implementation

This gives a vendor a hardware platform where web development is enabled.

Android

• Android as the underlying operating system
• Android app used to implement the device UI and application logic
• WebRTC embedded natively as part of the app

This diverts from the web development approach a bit (while it does allow for it). That said, it opens up room for third party applications to be developed and delivered alongside the main interface.

Linux or Android, which one will it be? Depends on what your requirements are.

A word about Zoom in this context

Why isn’t Zoom using WebRTC properly?

I don’t know. But I can make an educated guess.

It all relates to my previous analysis of Zoom and WebRTC.

Zoom were stuck with the guest access paradigm, trying to take the first step was too expensive for them for some reason. Placing that interworking element to connect their infrastructure to web enabled Zoom clients didn’t scale well with pure WebRTC. It required video transcoding and probably a few more hurdles.

At their size, with their business model and with the amount of guest access use they see with the Zoom client on PCs, it just didn’t scale economically. So they took the WASM route that they are following today.

It got them on browsers, with limited quality, but workable. It got them an understanding on WASM and video processing in WASM that not many companies have today.

And it put them on an intersection in how they operate in the future.

Would they:

1. Switch towards WebRTC, as most of their competitors have; or
2. Continue with WASM, waiting for WebCodecs and WebTransport to progress to a point where they are clearly defined and available in web browsers

If I were the CTO of Zoom, I am not sure which of these routes I’d pick at this point in time. Not an easy decision to make, with a lot to gain and lose in each approach.

Need help figuring this out?

This whole domain is challenging. Getting WebRTC to work on devices, around devices, in new or existing infrastructure. Deciding how to define and build a hardware solution.

Contact me if you need help figuring this out.

The post The software inside video conferencing hardware is… WebRTC appeared first on BlogGeek.me.

Our best Kranky Geek event ever. Or is it just that I have a short memory?

Earlier this month marked the highlight of the year for me. It happens every year now since 2015. The Kranky Geek event takes place in San Francisco. The event started by mistake and had become an immensely taxing and enjoyable undertaking for me.

WebRTC is a niche of an industry that are here to change the world and challenge how we communicate online with each other in real time. Kranky Geek became a place where our WebRTC niche meets, mingles and discusses many aspects of what it is that we’re doing. A lot of it is technology – and learnings people had and the scars they have to show for it. Some of it is more future looking, where new requirements are shared and semi-pitches are made. It is also a place where we get to talk and interact with the people behind the browser implementations.

I decided to share this slide about how niche WebRTC is:

This shows Stack Overflow Trends for WebRTC, VoIP and SIP. It is the percentage of questions in each month that shows these technologies as tags. WebRTC is higher than either SIP or VoIP by a factor of 3, which is nice. But overall, we’re still talking 0.05% of the questions, which isn’t much. WebRTC is a niche, but an important one (at least to me).

What is Kranky Geek all about

Kranky Geek is about the current state and the immediate future of the WebRTC ecosystem. It is first and foremost an event for developers.

Here’s what I understood at a client meeting earlier in that same week. After the meeting, the client comes to me and tells me how he is using the videos from past Kranky Geek events. Whenever there is a technical detail or a topic he knows is covered by one of our past sessions, he just goes and searches the videos to find that 2-3 minutes he needs.

It got me thinking. It is quite similar to how I use it. I end up referring people to a specific Kranky Geek video at least once a month if not more.

In the end, we are into learning and expanding the knowledge available out there about WebRTC.

The obligatory thanks

The Kranky Geek event isn’t funded by the audience’ tickets. These are practically free. We have a low registration fee that is a kind of a seriousness fee, which makes it easier to estimate the actual attendance rates we will see. That fees ends up being donated to good causes. In the case of Kranky Geek, we’ve been giving that money to GDI.

The event is only possible due to its sponsors.

–

There are a few people and companies that I need to thank for the Kranky Geek 2019 event.

First, to my partners in crime – Chris and Chad. Our different opinions and dispositions make a good mix for running Kranky Geek.

To Google and the Chrome WebRTC team at Google.

Google have been there with us from the beginning. They assist us tremendously with the logistics, their attendance and their sessions throughout the years.

To our sponsors of the event:

Their contribution is an important part of us being able to do this every year. I am also very happy that without exception, they treat their speaking slot and our rigorous process and dry runs seriously.

We had a new type of sponsors this year. Vendors who wanted to be part of the event, but didn’t speak (they came after we had a full agenda already).

Voximplant is a CPaaS vendor with WebRTC technology – one you should follow closely if you aren’t already.

Jamm just came out of stealth, and wanted to do that as part of our event.

What you can find in this year’s Kranky Geek sessions

We started off planning the event with a lot of AI in mind. This is what we had last year, and the trend is obvious to follow this year as well. It will probably still be a trend 5 years from now.

When we actually looked at our agenda, we found a nice mix of WebRTC topics, covering things from WebRTC specifications and best practices, through customizing and modifying WebRTC in production to new use cases and AI.

It is good we did a dry run to all of our speakers, since I didn’t really have the time and attention to listen to them during the event itself. I learned a lot of new things about WebRTC from the dry runs we have and I am sure you will find some very interesting and useful sessions here as well.

All of the videos are already available on YouTube and I encourage you to both subscribe and watch our 2019 playlist:

See you next year?

Maybe.

We never really know if we will be having a next ever. This is part of the fact that we’re not professional event organizers. We do it because we enjoy it. We also rely on others to make this happen.

If you are interested in a Kranky Geek 2020, then do one of the following things (or all of them):

1. Subscribe to this blog – I’ll be sending out an email at some point with the event’s date if we decide to do another one
2. Subscribe to WebRTC Weekly – that’s who we email to about Kranky Geek events officially
3. Contact me directly – especially if you’re interested in speaking or sponsoring a future event

The post Kranky Geek SF 2019 – post event summary appeared first on BlogGeek.me.

There are different ways to deal with interoperability. With WebRTC, the one selected is relying on the browser and offer guest access. Interestingly, while the industry is headed in that direction, the elephants are also headed… elsewhere.

When I first started with this blog, over 7 years ago, I wasn’t really sure where I was headed with it. What I did know, is that I have to write something about WebRTC to get it off my chest. WebRTC was the reason I stopped working at RADVISION and moved on. You see, as a CTO of my business unit I was told there’s no budget to invest in researching what we can do with WebRTC. Somehow, the future wasn’t important enough, which got me to understand there’s no future for a CTO there either.

I ended up deciding to write three posts – what is WebRTC, why signaling is irrelevant, and what a future meeting room would look like.

That third article? Here it is, from March 2012: The Post-WebRTC Video Conferencing Room System

We’re still slowly crawling towards that goal.

A short history lesson: the early days

For many years video meetings were an in-company luxury. A dubious luxury at that.

All Most video conferencing systems were based on a signaling protocol called H.323 and were *supposed* to be “interoperable”. This didn’t work that well, and in the end, companies tended to purchase all of their hardware from a single vendor. Multi Vendor was possible, but always at a loss of features or capabilities – either because these were proprietary to begin with or because interoperability is such an elusive target.

What was a person to do when he needed to communicate with someone *outside* the company? Dial his phone number. If video was what was needed, then the IT department had to be involved – on both sides. Fooling around with dialing plans, checking that the video conferencing devices interoperate, and then hand holding the users throughout that session. This happened not only in regular companies but also when the companies in question were video conferencing vendors.

Most systems at this point were hardware based. You had to purchase “meeting rooms” and install them.

The system was totally broken.

Rise of the federation

At some point a new concept started cropping up. If I recall correctly, Microsoft came with it, in their Microsoft Lync service. The idea was create federations.

Microsoft Lync was a semi-standards based service. It was SIP based in nature, but different – connecting to it was harder than connecting to other SIP devices and services as a lot of the spec was proprietary. Being Microsoft, they had a largish software-based market share, but one that was left unconnected.

Each company installed, operated and managed its own Microsoft Lync service. You couldn’t just reach out to another user on another installation directly. What you could do is involve the IT people (on both ends – yes), and get them to configure both installations to be aware of one another. This was referred to as a federation.

Think about it.

Thousands and thousands of installations. Each an island of its own. Each time you wanted to reach out to someone from a new island, you had to ask permission and get it setup – to federate with that other island of install base.

And guess what? This never really worked either. Not in real life. And not even for the video conferencing vendors themselves.

The friction was just too high to make this useful for the workforce.

Introducing the software client

Until a couple of years ago, video conferencing was a thing for hardware devices.

20 years ago? These devices were mostly built around DSPs and weird embedded operating systems.

15-20 years ago? The vendors learned about Linux and were comfortable enough to use it (!) for an embedded application such as video conferencing. The main concern was usually the real time nature necessary in encoding and decoding video.

About 15 years ago, the notion of being able to use a software client on a Windows operating system to join a video conference (not conduct a meeting – just join one) started to crop up.

The idea was this:

• If your company had a video conferencing installation, it could theoretically get a PC to connect to the video conferencing system at lower media quality and still be able to communicate properly
• Towards that goal, one could install a video conferencing software client and use it

This brought with it the headaches of having to deal with unmanaged networks – having employees (mainly managers) connect from their home, coffee shops or the occasional crappy hotel network.

This new capability started changing the business model around video conferencing. How do you license the software in a world where what was sold was hardware through channels and VARs?

What it also did was change behavior patterns. People now didn’t go to meeting rooms to join a call – they joined from wherever they wanted. Once the video client was installed in their PC they were relatively free.

It had another use case to it: technically, you could get someone to connect as a guest to a meeting. All he needed to do was install the specific software client of the specific video conferencing vendor from the specific landing page of the specific enterprise who purchased the video conferencing system and connect.

If you conducted a meeting with a company who had an installation of a specific vendor, then meeting with another company using the same video vendor usually meant you didn’t have to install the client again – unless it needed an upgrade of sorts.

Since these were early days, there were many installation issues with these clients. When it worked it was great, but when it doesn’t…

Enter the cloud

At around the same point in time, cloud services started taking potshots at the video conferencing industry. They didn’t call it video conferencing but rather web conferencing. Why? Because the center of the service wasn’t an on premise hardware video system installation, but rather a software based cloud service.

It wasn’t as performant and the quality was lower, but it was easier to use. Sadly, video conferencing companies didn’t see it as an existential threat.

Anyway, these services assumed that all users download and install a software client to connect to these web conferences.

Since this was their bread and butter, the idea of having guests connect became more prevalent and acceptable.

At any given point in time, I had on my laptop at least 3 such software clients. Services like WebEx, GoToMeeting and AT&T Connect.

Two challenge these services faced:

1. The concept was around “a company licenses a communicate tool to use internally but allows guest to participate”. This still wasn’t about the guests
2. That software install is still friction, and something no one really wanted to experience (besides maybe the vendor behind that software client)

Zoom

Out of these two challenges, Zoom came and solved the first one. For the most part, the first experience of a user with Zoom is by being invited to a Zoom meeting. By someone. Not necessarily an employee in a company who licensed Zoom – just by someone.

The change in business model, as well as the focus on the first time experience (making it simple), got Zoom to where it is today.

The problem that remained though is the software installation piece. That’s friction, and the browser-based solution that Zoom is offering is still subpar to what can be done on a browser.

The WebRTC guest access

In the past 5 years, what we’ve seen is that every video conferencing vendor except for Zoom has made it towards WebRTC.

Vendors still offer software clients for ongoing use of their service and for providing an improved experience, but all of them also have WebRTC access as well.

Need to have someone join a session? Create a calendar invite and get a meeting link. That link will allow you to either install a software client or just use the browser with WebRTC.

This has become the norm to the point that in many cases, I get invited to meetings just by receiving a URL on one messaging service or another.

Just in the last year we;ve seen UCaaS vendors joining this game by offering their own video conferencing services, usually called Meetings:

• 8×8
• Vonage

The race towards having video bolted on top of voice meetings and web conferences now relies on WebRTC support and guest access as key features.

The nice thing about this? There’s no need to interoperate, federate or connect the islands of services. Need someone to join a meeting? Just send them a link. They won’t need to install anything, just click and be connected. Magic.

Today – almost all services offer simple to use guest access via the browser using WebRTC.

Room systems “interoperability” in 2020

This all leads to this interesting announcement by Microsoft and Cisco. In two carefully crafted posts/announcements, the two companies appear to be collaborating more than ever. The plan?

Offer direct guest access for a room system of one vendor to meetings of the other vendor.

What does that mean? If you are invited to a Microsoft Teams session as a guest, you should be able to join it from a Cisco WebEx Room device. And vice versa.

This is no federation here – just pure use of an existing room system to join “any” meeting.

From Cisco’s announcement:

Cisco and Microsoft are working together on a new approach that enables a direct guest join capability from one another’s video conferencing device to their respective meeting service web app (WebRTC based).

From Microsoft’s announcement:

Cisco and Microsoft are working together on a new approach that enables meeting room devices to connect to meeting services from other vendors via embedded web technologies. Microsoft and Cisco will be enabling a direct guest join capability from their respective video conferencing device to the web app for the video meeting service.

A few interesting initial thoughts:

• This is provided not by interoperability of signaling protocols (my SIP can talk to your SIP, how about we federate?) but rather by making use of “web technologies” in Microsoft’s announcements, or simply using “WebRTC based” web app on Cisco’s announcement
• It will not work on older meeting rooms and legacy devices. This isn’t about interoperability. It is about the web
• Both Microsoft and Cisco rely here on WebRTC and web technologies. Their new devices use WebRTC internally and browser tech. They can now expose it by connecting using guest URLs
• Both companies are taking the first initial step of supporting a single type of guest access of a single other vendor. This is a controlled environment for them, and probably just a first step towards opening up the devices to “any” WebRTC based guest access

It is about time we got there.

The post Video meetings guest access: the new frontier of interoperability appeared first on BlogGeek.me.

We are now almost 8 years into WebRTC, and it seems like the same mistakes developers made 8 years ago are still being made today. Here are some common WebRTC mistakes that I see on a daily basis.

Last week, I took a quick business trip to Beijing for Agora.io’s RTC Expo event. I was invited by Agora.io to present there about a WebRTC topic, and I decided on “Common WebRTC mistakes and how to avoid them”. Why? Because it fits nicely with the fact that I’ve been promoting my WebRTC course recently, but also because it is an issue that crops up on a weekly basis.

RTC Expo is an interesting event. To begin with, it is a local event in China. It runs in three separate tracks and it was well attended – the rooms were usually filled to the brim during sessions. The number of foreigners could be counted on the fingers of a single hand. Agora.io offered there live translation, automated using Google Translator. During every session, the spoken words were transcribed and then translated to either Chinese or English, showing both languages to the side of the big screen. The results were mixed, and at times funny. It allowed understanding the gist of what was said but required some grasp of the language spoken by the presenter.

For my own presentation, I decided to work out with a simple structure:

1. Give a short explanation of WebRTC
2. Continue with a shopping list of common WebRTC mistakes

This structure gave me the ability to fit the content to the length of the session quite nicely, while driving home the three main concerns:

• Developers are clueless about STUN and TURN configuration and meaning
• Picking a signaling project in github is a tricky/risky endeavor
• Lack of knowledge brings with it mistakes. Better learn WebRTC

There are a lot more mistakes, but these definitely make it to the top of the list.

If you are interested in learning more, then here is the deck I used:

Common WebRTC mistakesand how to avoid them (RTC Expo 2019) from Tsahi Levent-levi

When the video of the session will be published, I will add it here as well.And if you are interested in solving such issues and reducing the risks of your WebRTC project, then I can always suggest my WebRTC courses.

The post Common WebRTC mistakes and how to avoid them [Slidedeck] appeared first on BlogGeek.me.

I am in the process of launching a WebRTC support course, alongside my WebRTC training for developers. This is by part taking place because of the work we’ve been doing at testRTC lately.

Supporting a technology is different than developing it. This is something I learned only recently. It is something I should have known some 20 years ago already. You learn something new every day.

I was always on the software development track. Be it as a developer, project lead, product manager or CTO. It was all about defining, designing, implementing and maintaining communication software. On good days, I interacted with product managers and developers. On bad days, I had to deal with support people (not because they are bad people, but because it meant we had product issues and bugs to deal with). On really bad days, I had to talk to a client who was on an escalation path.

A lot of that work with clients and support teams is frustrating as hell for developers. Oftentimes, there are two disconnected conversations going on, where both sides try to talk to each other but somehow there’s a mismatch in the languages.

This was never a fun experience for me.

Learning the trade of technical support

Earlier this year, at testRTC, where I am a co-founder and the CEO, we’ve partnered with Talkdesk, developing a new product to suit their needs. For the first time, my customers weren’t other developers, devops or entrepreneurs but rather support teams. What we essentially built was a network testing tool for WebRTC, which enabled Talkdesk’s support team to more easily collect and analyze network statistics from their clients. The end result for Talkdesk? This greatly reduced their turnaround time on incidents. This product is now being trialed by a few other customers, which is great.

I learned a lot from this experience – working with support teams, understanding their challenges and getting feedback from them on our initial alpha release and from there to the product launch itself.

At roughly the same timeframe, I found myself consulting more to support teams through BlogGeek.me, which was a different experience. The main bulk of my consulting is either around architecture and troubleshooting development issues in communication technologies or they are revolving around roadmapping and strategizing communication products. The people you deal with are different in each case, and trying to assist support people instead of making them go away as a developer in my distant past is an interesting experience (something that I should have experienced years back, when I was still young and beautiful).

Where is all that leading to?

New upcoming Supporting WebRTC course

My next pet project at BlogGeek.me is a new course. This one geared towards support people.

It isn’t a subset of the developers WebRTC courses that are already available, but rather a brand new course, created and recorded from scratch.

Why?

Because support teams need something different.

They don’t really need to know the internals of SRTP, or a detailed explanation of the patent situation of video codecs, or a lot of other technicalities. What they need is a basic understanding of WebRTC and then a lot of information around how things fail (as opposed to how they work).

If you want a peak at the agenda for this course, then it is available here.

I am in the process of creating the materials for the course and will switch gears towards recording and putting this live in two or three weeks.

There are 3 options here:

1. If you want to put your weight on it, affecting what content is on the course, and learning while I record these lessons, then you can join the pre-launch now at half the launch price. I am expecting your feedback in such a case, and will be giving priority to creating lessons you want. All you need to do is contact me
2. You can wait until this is all available, probably end of December or so. And then enroll in the course and take it in a linear fashion
3. You can skip and move on

My WebRTC courses for developers

Today, I have 3 WebRTC courses for developers:

1. WebRTC basics – a free course open to anyone
2. Advanced WebRTC – the main course, which already saw 500+ students enrolled to it
3. WebRTC Tooling – a growing set of snippets and interviews to be used when needed for reference

If you want to learn more about them, you can check the course syllabus (PDF).

Are you an employee and not a decision maker?

I think this doesn’t happen enough:

The part not happening enough is employees asking to take classes. Asking to get trained in technologies they need to get their job done. Why do I think that? Because I used to be like that as a developer myself. I was passive, waiting for things to happen to me, rarely going and asking for the tools to assist me in my work.

More often than not, I see managers interested in enrolling their employees you my courses. From time to time, there will be a developer who thinks this is important enough to go and ask for permission to take the course – or even more – go suggest the company to send the whole team to enroll in the course.

Think you need this course but don’t think management will approve? Try asking them. You might be surprised by the reply you get.

• Check my WebRTC developer training
• Check my upcoming WebRTC support training

The post Are you supporting WebRTC or developing with WebRTC? appeared first on BlogGeek.me.

I find myself looking at streaming platforms somewhat more lately. A topic that crops up from time to time is access to “open data”. Many write about the merits of open data but a lot less is written about the challenges related to making such data accessible and available.

I’ve asked Tom Camp, technical author and developer at Ably Realtime, a data stream network and realtime API management platform, to give a few pointers around the challenges in accessing open data streams.

Why realtime open data is useful

A well-known example illustrating the benefits of realtime open data is Transport for London and the ‘Citymapper effect’. Deloitte estimates that the 13,000 developers who started using this data created 600+ apps (including Citymapper), contributing £130m to the city’s economy within just a few years of the scheme’s launch. So it’s surprising large-scale examples like this are so rare (if you know of any similar success stories/ good sources of realtime data please comment at the end of this article). The EU’s data commission has also noted a distinct lack of publicly available, value-generating data sources (think traffic data, weather information, realtime financial updates) due to the costs involved of realtime distribution. In the UK, the Office of National Statistics (the ONS) has noted a widespread lack of data sources in realtime. Headlines aside, ask most developers and you’ll get the same answer. 

By allowing developers to publish and consumer realtime open data feeds on Ably’s API Streamer (a realtime API Management Platform) Ably’s Open Data Streaming Program aims to make public realtime data easier to work with. Work setting this in motion has involved identifying the most useful, publicly-available realtime data, converting it to a single realtime feed, and inputting it to the Ably Hub, which then re-distributes it to users (for free) in whichever realtime protocol and data structure they need. The process brought us into contact with hundreds of ‘open’ realtime data sets, and we soon became veterans in identifying and solving common problems developers experience when trying to consume realtime data feeds. Recurring obstacles range from a lack of ‘real’ realtime information, to a lack of protocol support, to heterogeneous data structures. 

Below we isolate three key potential problems to bear in mind when accessing ‘realtime’ data sources, and share what we learnt about how to overcome them. 

1. Polling takes up time and resources

Despite the fact many online experiences (B2C, C2C and B2B) now take place in realtime, we still see a lack of push-based realtime APIs. Developers have to poll for data if they want updates in near realtime. The internet’s infrastructure is built on REST-APIs, which fall short in terms of providing event-driven online experiences. 

Let’s take transport systems as an example. Although transport systems are subject to change at any minute, even here we notice a lack of realtime APIs that would be better suited to reflect this. When we looked into this we found just 2/10 cities provided actual realtime APIs. As it happens, these were the two cities with some of the best journey-planning and transport sharing apps. 

How do realtime APIs help? Consider an application which is meant to keep end-users updated with train arrival times, subject to change (as the city dwellers amongst us know), at any moment. Using pull-based protocols, those wanting to receive the information will need to poll the provider’s endpoint every few seconds for current information, with obvious impacts on server load as well as usability.

Leave it too long and you risk missing information on a train arriving at a different platform, and have the end user miss the train:

Make it too short, and you’re using a lot of bandwidth making requests for unchanged information, with each message also having a fairly large overhead:

What can we do about it? We can recommend data be provided using push-based systems, to lighten the engineering load both for producers, who only need to provide the initial connection point, and for subscribers, who no longer need to worry about intermittently polling the provider’s endpoint. The result is instantaneous updates and far lower bandwidth costs. 

Unlike pull systems, push bandwidth costs remain sustainable even when thousands of developers start using the data. For developers wishing to add realtime to their apps, look out for push-based APIs, such as WebSockets and MQTT, that allow for persistent, bidirectional connections. But while we are persuading data producers of the benefits of providing these, we can –  up to an extent – stick with long-polling BUT optimize how we long-polling with maximal efficiency.

2. Data structures are fragmented 

Developers looking for realtime updates have to spend a lot of time familiarizing themselves with each provider’s chosen protocol, be that HTTP or something like STOMP, working out its implementation, and how to convert this data into a unified format suited to a particular app or service. More widely though, and again using transport as an example, there is also a fundamental lack of standardization in the way transport providers structure their data. Some companies provide extended information – carriage formation, up-to-the-minute ETAs, and seat availability, others scrape by with the bare minimum of time and transport mode ID. A lack of standards across sectors mean developers wanting to expand the reach of their app (ie all developers) eventually come up against a host of additional problems to solve. With each new data structure developers need to work out which data corresponds to what, how to correlate similar data, in addition to allow for varying degrees of accuracy. 

A good illustration of lack of cohesion is the variety of options for what has caused a disruption. GTFS Realtime includes twelve possible reasons for delays. NationalRail on Darwin however, has a whopping 496 options (I kid you not). If open data is to have a meaningful impact on different sectors, we recommend industry-wide agreements on what data to provide. For developers, in the meantime, it’s a matter of knowing how to sift through the sources.

3. Some data sets are more open than others 

Most pull-based systems I’ve encountered don’t seem to be designed to handle large numbers of requests, which inherently reduce the value in the data as it becomes less accessible. Many transport data providers impose heavy rate limits and restrictions on data usage. For example, UK  train operator NetworkRail has a limit of 500 people using their queues at any one time. TFL’s RESTful API is limited to 500 requests a minute. I think that public data providers need to impose generous limits. For developers, so as not to get caught out when your app scales, it’s a wise precaution to bear in mind that you will likely need higher loads than you are anticipating. Here and elsewhere, before you dive into building an app, it’s best to read the smallprint around your chosen data source, gauging how it fits in both with other data sources, and your use case.

–

Ably is a global cloud network for streaming data and managing the full lifecycle of realtime APIs. Read more about concepts, design patterns and protocols underpinning realtime engineering on the Ably Engineering blog. 

Finally, if you know of realtime data feeds that would benefit from being on the Ably Hub, get in touch – tom@ably.io 

The post Data APIs: How to make the most of ‘public’ realtime data sources appeared first on BlogGeek.me.

Looking at the future of CPaaS, the lines are blurring in the cloud communication API future. And this isn’t only about UCaaS and CCaaS.

I’ve been asked recently by multiple clients to analyze for them the future of specific technologies they are developing. The process was very interesting and provided a lot of insights – somethings things that haven’t been obvious to me to begin with.

It got me into thinking. What if I do the same around CPaaS? Looking at how the future of cloud communication APIs look like, what are vendors after, what they pitch and brief analysts about, and what their customers are looking for.

I decided to do exactly that, ending up writing this article and creating a new comparison sheet and eBook (this eBook/sheet combo can be found in my WebRTC Course paid-for ebooks section).

–

When looking at what the future holds in the CPaaS domain, there are many aspects to review. If this topic interests you, then you should probably also read these other 4 articles I’ve written previously:

1. 7 CPaaS Trends to Follow in 2018 – the perspective I had a year ago. Mostly still true, but I think we’re accelerating the pace of change and evolving this a lot further
2. What Comes Next in Communications? – a look at how CPaaS, UCaaS and CCaaS vendors are looking at the market and at the blurring lines between them
3. CPaaS differentiation in 2019 – because it shows how different vendors try to operate differently and rise above the noise
4. Twilio Signal 2019 and the future of the programmable enterprise – a summary of the recent Twilio Signal event. Important simply because Twilio is the market leader and innovator in this domain

Now that we’re on “the same page”, here’s where I see things heading for communication APIs.

Want to figure out exactly what each vendor is doing in each of these future trajectories? You can purchase my CPaaS Vendors Comparison.

Learn more

nocode

There’s this new trend of making software development all-encompassing. It boils down to a single non-word used for it known as #nocode

Here’s some of the things people like saying about this trend:

As creating things on the internet becomes more accessible, more people will become makers. It’s no longer limited to the <1% of engineers that can code resulting in an explosion of ideas from all kinds of people. #NoCode

— Shaheer Ahmed ✪ (@Boringcuriosity) September 13, 2019

The best code you could write is #nocode at all

— Denis Anisimov (@dbanisimov) September 14, 2019

Interestingly, the place where you see people talk the most about #nocode is in the third party API space. Now that we’ve made integrating with third parties simpler via APIs, it is time to make it even more so by requiring less development skills to do so.

This has been a long time coming to the communication API space as well.

We’ve had visual IVRs for quite some time, and we’ve seen in the past 2-3 years many of the CPaaS vendors adding visual drag and drop tools. Twilio calls their tool Twilio Studio, while the rest of the industry settled on the name Flow.

Who is doing it today with CPaaS?

• IMImobile IMIconnect
• Infobip Flow
• MessageBird Flow Builder
• Plivo PHLO
• Twilio Studio
• Voximplant SmartCalls

Others, like Nexmo, opted for releasing a Node-RED package, enabling developers more flexibility in the integration points the Flow tool has to offer them.

What I fail to understand is why so little activity is taking place in the serverless trend. It is as if CPaaS vendors knowingly decide NOT to offer these and instead jump directly towards the visual drag & drop flow tool.

Look at the diagram above. It shows why I believe it is a mistake to skip the serverless opportunity. We’ve started with APIs, to simplify the task of inhouse development, going towards cloud so we don’t need to install complex systems. We’ve seen a shift towards serverless (think AWS Lambda), where developers can focus on their use case and not think too much about the whole non-functional infrastructure stuff. Then came the visual drag and drop tools, which made life even simpler, as for many scenarios, there is no more need to code anything – just express your intents by connecting dots to boxes.

Developers end up using ALL of the tools given to them. They will use a visual drag & drop tool to speed up development when the flow is easier to express in that tool. They’ll write code when necessary. And they will use serverless functions to reduce the effort of scaling and maintenance if that is needed. So why not give them all of these tools?

CPaaS vendors are doing APIs and moving towards visual. The serverless part is an internal implementation which most don’t expose to their customers. Why? I am not sure.

What should you expect in the coming years?

Visual Flow tools will become an integral part of any CPaaS offering, with more widget types being added into these tools – supporting new features, adding new channels or integrating with external third parties.

Omnichannel

Omnichannel is the biggest thing in CPaaS at the moment.

There are two reasons for this:

1. SMS is crap. And it is getting worse
2. SMS (and voice) is being commoditized. Omnichannel means less churn for CPaaS vendors

Why is SMS crap? Because in the last week or so I’ve received so much spam on SMS related to the election here in Israel that it made that channel useless. I am sure I am not the only one and that this isn’t only in Israel.

SMS is being marketed to marketers as the channel that gets the highest attention rate from the spammed audience. What it gets is the highest deliverability – maybe. Definitely not the highest attention. This makes SMS great for transactional messages but I am not sure how good it is for sales or marketing promotions if done in the current stupid carpet-bombing tactics.

How does omnichannel change that? It doesn’t. But the social networks that act as channels treat their users better than carriers, which means they are guarding the entry to their garden from sales people and marketers, trying to bake the rules of permission marketing into the engagement. This is done by things like manually approving message templates, not letting businesses send unsolicited messages, forcing identity on the sender, allowing users to mark crap they receive as spam, etc.

It does one more thing – it brings the game into a new field which is murkier than SMS today. There are many channels already, with a promise of more channels to come in the future. Will you develop it on your own or rely on a third party CPaaS vendor for that? Most will choose the CPaaS vendor approach.

Timing is also good. Social networks are opening up their APIs, letting CPaaS vendors (and other vendors) access to their users, in an effort to enhance their usefulness to their users and to have more monetization options on their platform. They are doing that while trying really hard not to piss off their users, so spam levels are low and will be kept that way for years to come.

Omnichannel is the leading force of future CPaaS growth. This is where most invest their focus on, and where there’s an easy path for migrating SMS revenue/engagement from.

Email

Email was always shunned from. Akin to fax. A relic of a bad past.

But it isn’t.

Most of my business revolves around the ability to reach people via email. And it mostly works for me (don’t like my content? unsubscribe).

It isn’t a replacement for SMS messages. Not really. But it has many uses of its own. Especially if you factor omnichannel. Businesses need to communicate with their customers and prospects, and doing that only over SMS or WhatsApp is a limited worldview. There’s email as well.

Some CPaaS platforms already had email integrations and capabilities to some extent. Twilio has taken it to a whole new level with the acquisition of SendGrid. Did Twilio decide on this acquisition to increase their bottom line and appeal to Wall Street? Were they after an operation with less costs attached to it to increase their revenue per share? Was it a genuine strategic move towards email?

Doesn’t matter anymore. Email is part of the game of CPaaS. I don’t think many agree with me on that. The reason it is becoming part of CPaaS is because we need to look at communications holistically. As we head towards the enterprise with CPaaS, email is yet another channel of interaction – same as SMS, WhatsApp and others. Being better at email means answering more of the needs of an enterprise communications which means appealing more in a vendor selection process.

Email will take a bigger and more important position in CPaaS. The more omnichannel becomes the norm, the more customers will ask about Email support and capabilities.

Streaming media to third parties

We call it AI – Artificial Intelligences. If we’re not overly hyped, then ML – Machine Learning. And if we’re true to ourselves, then most of it is probably statistics, sometimes sprinkled with a bit of machine learning.

CPaaS is too generic and broad to be able to cover all possible algorithms and models. What do you want to do with that recorded voice call? Transcribe it? Translate to another language? Maybe do some emotion analysis? Find intents? Summarize? Look for action items?

Too many alternatives, with too much data to train from to get a good enough model. And then each scenario needs its own data to train for and get a specialized model to use.

The end result?

CPaaS vendors offer a few out of the box integration with popular features and frameworks. The known culprits are speech to text and text to speech. Or just connectivity to AWS or Google machine learning algorithms in this speech analytics domain.

Another approach which is gaining a lot of traction is to be able to stream the media itself to any third party – be it an on premise/proprietary machine learning model or a cloud based machine learning API. Usually over a WebSocket, but sometimes on top of other transport mechanisms.

The name of the game here? Simplicity and real time.

Enabling easy access to the media streams is key. The easier it is to access the media streams and integrate them with third parties that do machine learning the more attractive the CPaaS vendor will be moving forward.

Chatbots and voicebots

The digital transformation of enterprises is a transition that is taking now over a decade and will continue for many years to come. Part of that transition is figuring out how businesses communicate with users. Part of that communication needs to be relegated to bots.

Why?

1. Because as a business we want greater scale. The more we can automate the more we can accomplish for less price, less friction and less mistakes
2. Because users seem to prefer self service in many cases. “Empowering” users to do more by having a lot of their interactions taken care of with bots help that
3. Interaction interfaces are moving from button clicking towards voice interactions. And text is the main form of communications on social networks (I am ignoring emojis and gifs here)

I’ve written about this trend and its reasoning when reviewing the two recent acquisitions of Cisco and Vonage in this space.

There are startups focusing solely on the bots industry, which is great. But in many ways, this is part of what a CPaaS vendor can offer – enablement of communications at scale.

Some CPaaS vendors today integrate directly or indirectly with bot frameworks such as Dialogflow or have built their own bot infrastructure. Moving forward, expect to see this more.

Enabling easy creation and configuration of chatbots and voice bots will be an important feature in CPaaS. The better tooling a CPaaS vendor will have in this space, the easier it will be for him to maintain enterprise customers looking to better communicate with their users.

UCaaS and CPaaS

Acronyms might be confusing in this section and the next so follow closely (or skip altogether)

UCaaS vendors are looking at CPaaS as a potential growth opportunity.

Vonage has seen that first with the acquisition of Nexmo.

Since then we’ve had Cisco acquire Tropo (and botch that one), RingCentral introducing developer APIs and 8×8 acquiring Wavecell.

There are definite synergies at the infrastructure level of UCaaS and CPaaS, though it is a bit less obvious what synergies there are on the frontend/application/business side. They do exist, but just a bit harder to see.

UCaaS vendors are adding APIs and points of integrations to their service because it makes sense. Everyone’s doin’ it in one way or another. It isn’t CPaaS but in some minor cases it can replace the need for using CPaaS.

What you don’t see, is CPaaS vendors heading towards UCaaS. Yet.

And you don’t see any successful independent UCaaS vendor using a 3rd party CPaaS vendor to operate all of its communication infrastructure. Yet.

For UCaaS, CPaaS is a growth potential. For CPaaS, UCaaS is just another use case. The lines are blurring between these two domains but not enough to matter.

CCaaS and CPaaS

Cloud contact centers take the exact opposite powerplay than UCaaS.

Many of the cloud based contact centers are using CPaaS and not their own infrastructure.

Twilio decided to build a contact center solution – Twilio Flex. In a way, it competes with some of its own customers. As successful companies grow large, they go toward adjacencies and CPaaS is an adjacency.

Will Twilio succeed with Flex? Too early to know.

Will more CPaaS vendors introduce contact center solutions? Probably not, but they are being bunched up and consolidated as larger entities – just see what Vonage and 8×8 have been doing in their acquisitions.

Twilio Flex is a singular occurrence. The norm would be other larger communication players who have CCaaS, acquiring smaller CPaaS players. The end result? A blurring of the lines between the various communication vendors.

For Twilio, Flex might be just the beginning. If this bet succeeds, Twilio will find the appetite to look at other adjacent enterprise applications it could build or acquire and make its own.

M2M / IOT

This. isn’t. part. of. CPaaS.

Or is it?

I’ll start by splitting this one into two areas:

1. M2M (cellular stuff)
2. IOT (messaging between devices)

M2M – Wireless

Twilio has their Programmable Wireless offering, which at its core is a modern M2M solution (for me M2M and IOT are one and the same).

In this domain, communication is needed between devices. Less human intervention for the most part, so some of the requirements are different.

But this is still communications.

CPaaS will redefine M2M/IOT as one of the use cases it covers. I don’t see a reason why CPaaS vendors wouldn’t take that route in an effort to grow their product line horizontally.

IOT – serverless infrastructure for real-time messaging

I tried to find a name for this subdomain and settled on what vendors like PubNub, Pusher and Ably end up with (or something in-between). There’s a set of vendors offering a kind of general purpose managed messaging that developers can use when they build their apps.

These vendors are settling on something like serverless infrastructure for real-time messaging as a name.

Serverless because it sounds modern, advanced and cool (marketing asked for that).

Infrastructure because this is what they have.

Real-time messaging because this is what they do.

How is that related to CPaaS? It doesn’t directly. Because no CPaaS vendor offers a “serverless infrastructure for real-time messaging”.

Here’s a surprising thing.

All of the CPaaS vendors who support WebRTC have a global backend real-time messaging infrastructure already. It is used to drive signaling across the network.

It might be more centralized. It might be slightly slower. It might be simplistic.

But at the end of the day – it is a serverless infrastructure for real-time messaging.

These CPaaS vendors can slap an API on top of that infrastructure and offer that as yet another distinct service. And they will. Either by inhouse development or through acquisitions.

Serverless infrastructure for real-time messaging will be wrapped into CPaaS.

Cloud native, no hybrid

There were attempts in the past by CPaaS vendors to offer both cloud and on premise alternatives.

Some are probably doing it still.

The vendors that see more growth though are cloud native and offer no on premise alternative.

Things aren’t going to change here.

The future of CPaaS is cloud. Hybrid is a nice idea, but until cloud vendors themselves don’t offer an easy (and cost effective) path towards that goal, the hybrid model makes less sense – it becomes too expensive to develop and maintain.

Measurements and SLAs

Quality across vendors, carriers, networks, infrastructures, time of day, day of the week or any other parameter you wish to use is variable at best. CPaaS vendors are “supposed” to handle that. They track and optimize media quality and connectivity across their services. They strive to maintain high uptime and reliability. Some even use quality as reasons for opting for their service.

At some point, TokBox and Twilio started offering quality measurement tools. TokBox introduced Inspector, a way for its users to troubleshoot network issues of recent sessions. Twilio launched Voice Insights, offering its users a quality dashboard of the calls conducted through its service.

A similar aspect is the use of SLAs as part of the service – a binding of what type of service expectations the customer should expect and what happens when the expectation isn’t met. These apply mostly to enterprise plans of some of the CPaaS vendors.

Why am I mentioning it here? Because it see it happening. It is what got Talkdesk to pick testRTC for a network testing tool (I am a co-founder at testRTC). It is also an issue that causes a lot of challenges to customers – understanding the quality their own users experience.

Measurement and SLAs will take bigger roles in customer’s buying decision making. As the market evolves and matures, expect to see more of these capabilities crop up in CPaaS offerings. It will happen due to pressure from competitors but more likely due to pressure from enterprise customers.

Vying towards the Programmable Enterprise

We’re shifting from on premise to the cloud. From analog to digital. From siloed solutions towards highly integrated ones. This migration changes the requirements of the enterprise and the types of tools it would require.

I think we will end up with the Programmable Enterprise. One where the software used is highly integratable. Many of these early trends we now see in CPaaS will trickle and find their way across all enterprise software.

Want to figure out exactly what each vendor is doing in each of these future trajectories? You can purchase my CPaaS Vendors Comparison.

Learn more

The post Future of CPaaS; a look ahead appeared first on BlogGeek.me.

An analysis of the most popular open-source WebRTC repos on GitHub with a review of how WebRTC itself is doing there.

Continue reading and the WebRTC Open Source Popularity Contest Winner is… at webrtcHacks.

Some updates you might want to be aware of.

This is going to be mainly about updates of things that are going on that you may want to be aware of. Mainly:

• Kranky Geek 2019
• Available WebRTC related sponsorships
• Revamping my consulting pages

Kranky Geek 2019

Kranky Geek 2019 is coming up fast.

Date is set to Friday, November 15 2019

At our traditional location: Google’s office at 345 Spear St, San Francisco

We are going to continue this year in our look at WebRTC and machine learning in communications as our main theme.

Want to register for Kranky Geek?

Registration to the Kranky Geek event are now open.

We’ve got limited room, so you should register earlier rather than later.

There’s a token registration fee ($10) – it is how we make sure everyone has a place to sit during the event.

Register now

Want to speak at Kranky Geek?

If you’re into sharing your knowledge and experience with others, then how about speaking at Kranky Geek?

We’re working on the agenda at the moment, and are looking for speakers to join us. Each year we get one or two such requests that end up quite well. Need examples? Check out last year’s Facebook session on Portal or maybe Discord on their infrastructure.

Want to try this out? Contact us.

Want to sponsor Kranky Geek?

We get to do Kranky Geek on a yearly basis due to our great sponsors.

Our sponsors this year include:

This leaves room for one or two more sponsors. If you’d like to help us our, and show off your brand where it matters when it comes to WebRTC, then let us know.

Meet me in person

In the next couple of months I’ll be traveling. If you’d like to meet, ping me.

October 24-25, Beijing

I’ll be heading to Beijing for Agora.io’s RTC 2019 event.

My session at the event is “Common WebRTC mistakes and how to avoid them”. Still need to work out on my presentation.

If you’re in Beijing for the event, it would be great to see you in person.

November 11-16, San Francisco

Kranky Geek takes place November 15. I’ll be in San Francisco for the duration of that week.

My time in San Francisco is usually limited and hectic, but I am always happy to catch up and talk when I can find an open slot for it.

If you are interested in meeting up – just tell me.

Available WebRTC related sponsorships

There are sponsorship opportunities that are available if you want to highlight your products, services or even job listings. These are available not directly on BlogGeek.me, but rather in a few partner domains:

• webrtcHacks – that’s where most WebRTC developers end up when they need to learn a trick or two or a best practice around WebRTC
• WebRTC Weekly – a weekly newsletter that people interested in WebRTC are subscribed to
• WebRTC Index – a list of vendors offering services in the WebRTC ecosystem
• Kranky Geek – as stated above, we’re looking for a last sponsors or two for this event

There’s now an orderly media kit you can review for the webrtcHacks and WebRTC Weekly sponsorships. Check it out.

New testRTC product: Network Testing

At testRTC, we’ve launched a new product a few months back – Network Testing

While our other products are geared towards developers, testers and IT, this new product caters for support teams.

What it does is connects to your backend directly (there’s an onboarding/integration associated with this product), and then runs a battery of network tests from the machine you use our service. It ends up providing the information it gathers to both the person running the test as well as your support team.

This was developed with the help of Talkdesk, one of our first clients for this product. Check out the testimonial we did with Talkdesk using testRTC’s Network Testing.

Interested in learning more? Contact us @ testRTC

A new WebRTC course – for support teams

I have started working on a new course called “Supporting WebRTC”. The purpose of this course is to assist support teams that need to deal with issues related to WebRTC to better understand and handle them.

This comes as I celebrate my 500 course students in my developer focused Advanced WebRTC training.

Anyways, ping me if youre interested in learning more about the new Supporting WebRTC course – or even want to be there during the prelaunch, providing feedback as I create lessons.

Revamping my consulting pages

This how the menu bar on my website looked until yesterday:

And this is how it looks now:

I’ve replaced the “non-performing” and somewhat cluttered Workshops/Consulting combo with the more usual Products/Services alternative.

Why the change?

Because there are many of my services that were gone unnoticed. I found that out while speaking to clients and potential clients. So it made sense to change the structure. Another reason is the recent launch of my ebooks section – while these are part of the WebRTC Course website (along with the courses themselves), I wanted to be able to share everything on my main site – BlogGeek.me.

I’ve decided to make this change available now and not wait for it, but these pages will be updated soon. I have commissioned a few unique illustrations for these new pages and can’t wait to get them up.

Here’s a glimpse of one of the concept sketches I received (this one for the courses):

Doing something with communications? I am here to help.

The post Kranky Geek, WebRTC sponsorships and other updates around my services appeared first on BlogGeek.me.

There are different ways to use WebRTC. Zoom is using WebRTC, just not in the most common way possible today.

Zoom seems to be an interesting topic when it comes to WebRTC. I’ve written about them two times recently (and a bonus one from webrtcHacks:

1. When Jitsi played with Zoom vs Jitsi in bandwidth limiting
2. That in turn led to webrtcHacks looking at the browser implementation of Zoom
3. Just two months back we had the security vulnerability in Zoom

That in itself begged the question where WebRTC starts and where it ends, since Zoom uses getUserMedia to access the media to begin with.

What was found lately is even more interesting:

Looks like @zoom_us has switched it's web client from web sockets to #WebRTC data channels. Performance a lot better compared to their old web client. pic.twitter.com/SQhP9XhHXP

— Nils Ohlmeier (@nilsohlmeier) September 5, 2019

Nils (Mozilla) noticed that Zoom is using WebRTC’s data channel. Which led webrtcHacks to update that Zoom article.

Interesting times

Want to effectively connect WebRTC sessions with the success rate that Zoom is capable of? Check out my free mini course

Connect more WebRTC sessions

What does “use WebRTC” mean?

If you go by the specification components in W3C, then the split looks something like this:

From the W3C specifications standpoint, WebRTC is support for Peer Connection and the Data Channel. This encompasses in it other elements/components such as getStats, SDP negotiation, ICE negotiation, etc.

But at its core, WebRTC is about sending data in real time in peer-to-peer fashion across browsers. Be it voice, video or arbitrary data.

getUserMedia and getDisplayMedia have their own specification – Media Capture and Streams. This is what Zoom has been using out of WebRTC. It allows browsers access to cameras, microphones and the screen itself. These are used for things that have nothing to do with communications – like MailChimp or Whatsapp taking a snapshot for a long time now. Others are doing the same as well.

Then there’s the MediaRecorder component, which is defined in MediaStream Recording. Its use? To record media locally. Dubb and Loom use it for example.

Is MediaStream Recording WebRTC? Is Media Capture and Streams WebRTC?

I like taking an encompassing view here and consider them part of what WebRTC is in its essence when used in a browser.

Zoom’s route to WebRTC

Back to Zoom.

Zoom started by using only getUserMedia. This allowed them access to other browser technologies such as WebAssembly. They got their real time media processing somewhere else.

The next step is what Nils just bumped into – Zoom decided that streaming the media over a WebSocket is nice but not that efficient. Since it ends up over TCP, the performance and media quality is subpar once packet losses kick in. That’s because TCP starts retransmitting the media when it is already too late for a real time task like video calling to use it, ending up with even more congestion and more packet losses.

What is a company to do at such problem? Find a non-reliable connection to send their data on. There are two alternatives today to do that in web browsers:

1. WebRTC’s data channel (which uses SCTP today)
2. QUIC (HTTP/3), which is still a bit too new

Zoom decided on WebRTC’s data channel in its current SCTP implementation. They haven’t gone for the Google Chrome experiment of a QUIC data channel (which should be rather “safe” considering Google Stadia is said to be using it). And they haven’t decided to use HTTP/3, which I find as a bit odd.

The end result? Zoom is using WebRTC. Somewhat. With a data channel. To handle live video streams, with their previous WebSocket architecture as fallback. And not the peer connection itself. It is really cool, but… don’t try this at home.

Is this the end of the road for WebRTC in Zoom?

I don’t think so.

They still have the installation friction and now all them pesky security experts breathing down their necks looking for vulnerabilities. It won’t hurt their valuation or their revenue, but it will eat into management’s attention.

And frankly? Zoom on a data channel will still be subpar, since doing everything in WebAssembly isn’t optimized enough. At some point, Zoom will need to throw the towel and join the WebRTC game.

Why?

Because of either VP9 or AV1. Whichever ends up being the breaking point for Zoom.

What will be the next step for Zoom’s adoption of WebRTC?

Zoom has two main things working for it today, as far as I can see:

1. It just works
2. Quality is great

Both are user/market perception more than they are an objective reality (if there even is such a thing).

1. It just works

It just works is about simplicity. It is the reason Zoom started with using GetUserMedia and later the data channel. Without it, guest access to Zoom would mandate installing their app. At a time when all of their competitors require no installation, that’s a problem. The problem is that this small friction that is left means that “it just works” is no longer a Zoom advantage. It becomes their hindrance.

2. Quality is great

Zoom uses H.264, at least from the analysis done by webrtcHacks (based on packet header inspection).

Since WebRTC has H.264 support, my assumption is that Zoom’s H.264 implementation is proprietary or at the very least, not compliant with the WebRTC one. They might have their own H.264 implementation which they like, value and can’t live without – or at least can’t replace in a single day.

At some point, that implementation is going to lose its luster and its advantages, and that day is rather close now.

H.264 is computationally simpler than VP9 and AV1 – a good thing. But at the same time, VP9 and AV1 offer better quality than H.264 at the same bitrate.

When Zoom’s competitors migrate to using VP9 or AV1, what is Zoom to do?

It can probably adopt VP9 or go for HEVC. It might even decide to use AV1 when the time comes.

But what if it does that without supporting WebRTC? Would running an implementation of a video codec twice or three times as complex as H.264 in WebAssembly make sense? Will it be able to compete against hardware implementations or optimized software implementations that will be found at that point in web browsers?

Without relying on WebRTC, Zoom will be impacted severely in its web browser implementation, and will need to stick to installing an app. At some point, this will no longer be acceptable.

If I were Zoom, I’d start working on a migration plan towards WebRTC. One lasting at least 2-3 years. It is going to be long, complicated, painful and necessary.

Microsoft has taken that route with Skype. Cisco did the same with WebEx.

Both Microsoft and Cisco are probably mostly there but not there yet.

Zoom should start that route.

The end of proprietary communications

In a way, this marks the end of proprietary communications. At least for the coming 5-10 years.

It is funny how things flip.

The market used to look like this:

Companies standardized on signaling, placed acceptable standardized codecs. And then pushed proprietary non-standard improvements to their codecs.

And now it looks like this:

Companies standardize on codecs, using whatever WebRTC has available (and complaining about it), placing their own proprietary signaling and infrastructure to make things work well.

In that same challenge, you’ll find additional vendors:

Agora.io, who has their own proprietary codecs, claiming superior error resiliency. They just joined the AOMedia, becoming part of the companies behind the AV1 video codec.

Dolby, who has their own proprietary voice codec, offering a 3D spatial experience. It works great, but limited when it comes to the browser environment.

As WebRTC democratized communications it also killed a lot of what proprietary optimizations in the codec level can do to assist in gaining a competitive advantage.

It isn’t that better codecs don’t exist. It is that using them has an impossibly high limitation of not being able to be used inside browsers – and that’s where everyone is these days.

Want to effectively connect WebRTC sessions with the success rate that Zoom is capable of? Check out my free mini course

Connect more WebRTC sessions

The post When will Zoom use WebRTC? appeared first on BlogGeek.me.

What I like at how companies are tackling spam calls and robocalling is that the solutions they bring to the table are based a lot on their DNA.

There is more than one way to solve a problem. There is usually more than one way to solve a problem effectively. Which means that it isn’t that easy to pick the best solution – simply because there are a few good alternatives. This is the case with spam calls. These spam calls are also called robocalls, which we’ll get to later.

When I wanted to explain it through an analogy – it hit me!

It is akin to the many lightbulb jokes. At the end of the day, in each industry or persona type, a different approach is taken to change a lightbulb.

Let’s change the subject instead of the lightbulb though. We’re talking about calls. Here in Israel we get a few unsolicited spam calls. Not that many if you consider what’s going on in the US – and it is still not that fun.

My own spam calls experience(s) – or lack of

I used to live in a rather religious city a few years back. We are a secular family. The neighborhood and the city around us changed to become more religious over time through a kind of natural selection. At that time, I used to get a call or two a week, starting with a recording with a wording that can roughly be translated to a preacher saying “Precious Jews!” – that’s the point where I hung up automatically, so I have no clue how that “conversation” progressed.

This miraculously stopped as we moved to a city nearby. This time a secular one (almost too secular). It stopped not only in our landline but also in our mobile phones, which was interesting. This week though, I started receiving different calls, probably due to the upcoming election here. These calls start with something like “Save Liberalism” – which I again identified as my cue to hang up the call.

Here in Israel? This isn’t such a big deal. Probably due to the exorbitant cost of call automation or simply because the market is too small or too immature for it. In the US? It seems like there this plague is so common that many people don’t answer their phones for numbers they don’t have in their address book.

Here’s what Andy Abramson has to say about his spam calling experience:

Sure my regular dial up and mobile phone numbers rings throughout the day with calls from toll-free and from numbers that look like they’re from a neighbor, when they’re nothing but spam like calls. 

Most of his conversations take place over OTTs these days, which don’t carry spam.

Up until recently, this seemed like a necessary evil that no one is going to really handle. But something has changed this last year. So much so that this now looks to be the main issue in phone calling. Especially if what we’re looking for is maintaining a semblance of usefulness to using telecom carriers to handle our phone conversations.

How did we get to this point?

I get a feeling that it involves a mixture of reasons:

1. The low cost of calling (or sending an SMS) to people
2. The ability to programmatically automate that process and leave humans out of the equation for the spammer
3. We’ve gone through a digital transformation in telecom – from analog to digital communications – which made the interfaces towards telecommunication networks easier and more accessible via the internet. At the same time, the capacity of these networks to handle calls grew significantly
4. When carriers interconnected with each other, they didn’t really think that far into the future of the types of abuse and attack vectors available today

Remember that just until a few years ago, the concept of encrypting traffic other than financial transaction seemed an exaggeration (encryption and cryptographic authentication in communications was not part of an MVP, a version 1 or a version 2 of a product, and it almost never interoperated well out of the box). Today? We’re discussing end to end encryption as if that’s a human right and zero trust networks as if that’s the norm.

–

There is no doubt a problem. And this problem is getting bigger each year. How are companies tackling it? Each one with the tools its has available and the DNA it has.

Carriers: Lets standardize

One of the main concerns with spam calls is based on spoofing. The ability of the originator of the call to masquerade as any number he wants, including local numbers, close to that of the called number. This technique tries to add trust to the originating call, to pass the automated response of people (not answering calls that look somewhat fishy).

You’d think that by 2019 this wouldn’t be such a simple thing to do (zero trust anyone?), but it is. So much so that the standards suggested – SHAKEN/STIR – a cryptographic authentication of caller IDs. As explained by the FCC on combating spoofing:

This means that calls traveling through interconnected phone networks would have their caller ID “signed” as legitimate by originating carriers and validated by other carriers before reaching consumers. SHAKEN/STIR digitally validates the handoff of phone calls passing through the complex web of networks, allowing the phone company of the consumer receiving the call to verify that a call is from the person making it.

For the FCC and carriers such a solution makes a lot of sense:

1. You start by better defining the problem – it isn’t spam calls but rather caller identity spoofing
2. Then you continue by picking a solution – authentication of caller identity
3. And then you go spec it out as a standard – SHAKEN/STIR
4. Last but not least, you get all carriers (100’s of them) to implement the new standard

In the US, AT&T, T-Mobile and Comcast have started implementing SHAKEN/STIR (PDF). I didn’t find much information about other carriers around the globe.

Here are a few challenges with this approach:

1. SHAKEN/STIR doesn’t block calls. Just indicate if they are authenticated or not. Think of it as the green indicator of the past on your browser bar for websites served via HTTPS. Or even worse – the Extended Validation Certificates for HTTPS (now officially dead and useless). In other words, you will keep getting spam calls, but something on your display will allow you to better decide if you wish to answer or ignore
2. It requires software changes on mobile devices (and landline phones). Since it blocks no calls, the indication of unauthenticated caller ID needs to appear on your display when there’s an incoming call
3. It requires all carriers to be effective. Otherwise, a lot of them phone numbers will come unauthenticated adding too much noise
4. It requires OTTs, CPaaS vendors, UC vendors, contact centers, enterprises and anyone interconnecting with carriers for his voice traffic to authenticate his numbers using the same standard specification

These challenges means that until we see value in this initiative, we will be well into 2025 or something similar.

The “go it alone” carriers

There are instances where carriers are going it alone, trying to solve spam on their own.

The notable example here is Verizon, offering free and paid call filtering services, targeted at robocalling. They are now pre-enable it on Android phones.

Frankly? This approach is again within the realm of carriers-DNA. From Verizon’s website:

• With carriers, everything has a price. Caller ID – if that means authentication like SHAKEN/STIR (or SHAKEN/STIR itself), then why only under a paid plan? Aren’t carriers supposed to take care of spam similarly to how most email services do today?
• Automated call blocking by filtering them as spam is great, but what about false positives? How many important calls from businesses is this going to block? (and yes, I know I complained before about SHAKEN/STIR not blocking calls)

Verizon isn’t alone in this approach. Other carriers are offering similar solutions as well.

My challenge here? I’ve never seen a carrier app on mobile that works well. They always seem and feel half baked.

There’s an app for that

And a lot more than a single app.

Since our smartphones allow for apps, there are those who created apps that allow blocking incoming calls based on who the caller is. The intent is to be able to block robocalls/spam from coming in. Which is great.

The challenges are?

• Not all call blocking apps are created equal. Some offer an abysmal user experience while others integrate nicely with the operating system. It is left to the user to pick one that works for him
• These apps often build their database via crowdsourcing the spam indication from users. While great, this did block calls from my insurance company a few times when I really needed to receive these calls. This also means that different people have a different definition of what spam is and that will affect what gets blocked on your phone
• They are selling your data. Or at least that’s the current news. Robocall blocking apps collect more data than they should and use it for unknown reasons (it might just be developer log collection data, but some of these apps actually might sell data)
• You need to actively install these apps on your phone. Select one and register to it. So not frictionless

Operating systems: power to the user

Mobile operating systems allow some semblance of control that is/can be given to the user.

If you are using Google’s phone application on your Android device, then you can use Android’s caller ID & spam protection.

This relies on Google to decide if an incoming call is suspect of spam or not (more on that later), and be able to simply block it. Users also have the ability to mark calls as spam, which I am sure Google then uses as crowdsourced information as well.

Why this approach by Google? Google is a data-first company, so any challenge gets first solved using data.

Apple, on the other hand, decided to not look or rely on their users’ data. What they did was add a simple rule in iOS 13 to silence unknown callers. This will just not ring your phone if the caller ID isn’t found in your address book. While a nice feature, this doesn’t really scale and the result is too aggressive.

Why can Apple take this route? To get more businesses into its Apple Business Chat solution – effectively enticing businesses to communicate with iPhone users via Apple, and getting them into the user’s address book.

Google: AI

There’s one more thing Google is now doing for their new Pixel phones, called Call Screen.

Call Screen is a kind of a virtual assistant or a voicebot that “lives” in your phone. It can answer calls on your behalf, transcribing and checking on your behalf who is calling and why. You will then be continue interacting with the caller via menu buttons on the screen, instead of actually talking to him.

Why this approach?

It does what only Google can do. Run speech to text as well as text to speech on device, in real time, and do that with an accuracy that is good enough.

The funny thing is that it gets robocalls interacting with voicebots. I wonder if this is communications or can we start talking here about the M2M (machine to machine) market instead…

The problems? You still need to man-handle all these spam calls. Would be better if we could just make them go away to begin with. Oh… and it is available only on Pixel phones for now.

Twilio: Programmable Identification

In their recent Signal event, Twilio announced Verified by Twilio.

The idea here is to create a kind of a marketplace where Twilio customers add metadata to their outgoing calls to users – like who is calling and the reason for the call. And then that data gets picked up by caller id apps and shown to the users when that call rings on their smartphone.

This is a nice thing, but it does have its own set of challenges:

• It requires businesses to identify their intent via APIs. And they can do it only through Twilio today. This isn’t an open standard
• As a user you still need to install an app to make this work
• And it doesn’t block the calls – just gives you a bit more information before you answer it

That said, if Twilio can pull it off, it will secure its lead even further in the CPaaS market.

Is all robocalling spam?

No.

A lot of it is transactional.

I get a call every 6 months from the dentist. An automated reminder a day before a visit. If I don’t answer it and press “1”, it tries to hunt me down. Never checked what happens to my appointment if it fails to do so.

That term digital transformation is old by now, but the transition we are going through towards digitizing and automating interactions between businesses and users is a real one, and it is a growing trend. The purpose of it isn’t just to deflect incoming calls and communications so customers don’t bother “us businesses”. The purpose is to genuinely improve the customer experience and to do so at scale, while relying less on human agents (or at least not relying on them in the boring and the trivial).

Then how do we filter out these spam calls from the automated transactional ones that we really want to receive?

Today, it seems, there are two main solutions:

1. Block “spam”, which might catch real calls and block them as well. My guess is that false positives here are higher than what email spam shields are doing – think of it as being 10-20 years behind in the technology curve
2. Mark intent of the calls or “manage” incoming calls, which means users are still being bothered with it, just a bit less so

Not a good solution in sight yet.

Back to lightbulbs

I started with lightbulbs so better finish with that. Especially since there’s no aha-moment here for us, or a great lightbulb idea to work with when it comes to spam calls.

So… How many board meetings does it take to get a light bulb changed? (or to fight spam calls)

This topic was resumed from last week’s discussion, but is incomplete pending resolution of some action items. It will be continued next week. Meanwhile . . .

The post How different companies (and industries) are trying to fight spam calls appeared first on BlogGeek.me.

Resistance is futile when it comes to AI in communications. Are you going to get there on your own or dragged there?

I’ve worked with Chad Hart last year on a unique report that covers AI in RTC. Since then, we’ve seen the trends we’ve analyzed in the report strengthen and take shape. Why did we start on that route? Because of an understanding that the use of machine learning and artificial intelligence will be part and parcel of the communications industry – simply because it is now penetrating all industries. The difference here, though, is that with communications this isn’t just about finding churn or optimizing workflows – it is about looking at the data itself – the communications – in real time.

One of the things we looked at and debated at length was how communication vendors are going to integrate AI into their own products. Are they going to have that done in house? Will they outsource that part? If they outsource, would that be outsourcing everything or just certain parts? Will they be using cloud based AI APIs for that. And if they do, would they go for Amazon, Google, Microsoft and/or IBM. Or would they rather go for smaller, more specialized vendors?

The answers were all over the place, and the main challenge was deciding how the future would look like. Me? I thought and still think, that a lot of that technology and experience must come in-house for vendors who want to keep an edge and a competitive advantage.

You can’t use third parties for a technology like AI and assume you will gain differentiation.

Why? Because everyone else will be doing the same.

Need proof?

A year ago, almost everyone we spoke to said they were experimenting or rolling out call summaries on recorded calls. How were they implementing that? By using Voicera (turned Voicea and acquired by Cisco. More on that later).

When Google introduced their Contact Center AI, everyone and his uncle partnered with them for the announcement.

How does that assist in differentiation? I don’t know.

During August, two interesting acquisition announcements took place on the exact same day:

1. Cisco Announces Intent to Acquire Voicea
2. Vonage purchased Over.ai assets

Both acquisitions are in the communication space. Both acquisitions are around NLP/NLU capabilities (natural language processing and understanding).

Looking for better understanding of the AI space in communications? Check out the overview of our AI in RTC report

Download the preview

Cisco & Voicea

Voicea does two things:

1. Offers transcription
2. Captures decisions, notes and action items

The transcription part is usually referred to ASR (Automatic Speech Recognition) or NLP (Natural Language Processing). Dumbing it down a bit, this is speech to text capabilities.

The second part is about NLU (Natural Language Understanding). The “AI” or “assistant” is capable of reviewing what is said (usually on the textual level) and deducing from that what it means. In the case of Voicea and their EVA product, this is about the creation of meeting summaries and action items taking.

In many ways, this is similar to Dialpad’s acquisition of TalkIQ a year earlier.

Where do these two differ?

Cisco

From the press release, it seems that Cisco intends to wrap Voicea into WebEx, with a focus on collaboration. This means help facilitate meetings and conversations within an enterprise. As Amy Chang, SVP & GM, Cisco Collaboration states in the press release:

“Voicea’s true market leading technology will be a game changer for our Webex customers to experience more productive and actionable meetings”

Cisco’s approach stems from their focus on what they call “cognitive collaboration”. I am sure Voicea will find its way into the sales calls that Cisco enables for its customers, but first priority seems to be collaboration.

Dialpad

Dialpad focused on getting TalkIQ into its contact center offering, creating AI assistant to sales people and adding it to the unified communication offering that they have. In the acquisition press release, the initial capabilities mentioned are real-time call transcription, smart notes, real-time sentiment analysis for call center and real-time coaching.

–

Cisco decided to bring in-house the competency of NLP and NLU. It isn’t the first AI technology that Cisco is acquiring. It makes me wonder if they’ll keep the Voicea team independent, fold them under WebEx or wrap them into an AI team they already have (probably through a past acquisition).

Vonage & Over.ai

Vonage acquisition of Over.ai may seem somewhat similar to the Cisco acquisition of Voicea at first glance. Both acquired an NLP/NLU startup. Both plan on wrapping the tech into their communications offerings. Both acquired an AI team in a location close to one of their existing offices. But that’s where things start to look somewhat different.

Over.ai is focused more on voicebots that it is on just speech analytics. As such, it doesn’t seek to glean meaning, converting it into summaries and action items. It is geared towards understanding intents and “holding” a conversation between a human and a bot.

This requires knowledge and competencies in speech to text, text to speech, intent recognition and building something akin to Google’s Dialogflow (and the application logic on top of it).

What can Vonage do with such technology? Here are some initial thoughts and ideas:

• Offer its own transcription engine, across its product line (API for developers via Nexmo and TokBox, contact center via NewVoiceMedia and unified communications via Vonage Business Cloud
• Replace and/or augment IVRs across that same product offerings
• Create outbound calling bots, probably with a focus on NewVoiceMedia and maybe Nexmo
• Expand to other AI related challenges in the communication space

That last one is the most interesting.

The main challenge vendors have today with AI is finding experienced developers. Or more accurately, experienced employees. In this acquisition, Vonage got itself a complete team with expertise around communication related AI. this includes developers, testers, product managers, sales and marketing – the whole shebang. While we tend to focus on developers and their experience, AI has proven as a technology that needs all these added functions to be experienced in it as well.

This gives Vonage a nice head start in this, where others need to build such capabilities in house, or acquire them elsewhere (as Vonage did).

Communications + AI = Future

Dialpad has a really nice explanation of today’s state of AI in communications:

It is hard as hell and requires lots of customizations, so you can mostly use it at scale.

I believe that the interesting use cases are still ahead of us. And that many of them would require cracking the scaling issue – how to be able to deploy AI algorithms and models that can work well for small businesses and not only at the largest ones where customization and fine tuning is part of the process.

Looking for better understanding of the AI space in communications? Check out the overview of our AI in RTC report

Download the preview

The post AI in communications is inevitable appeared first on BlogGeek.me.

While this is a non-story, it does raise an interesting conversation about security, privacy and the tension between them.

This one’s going to be philosophical. Might be due to my birthday and old age. Feel free to skip or join me on this somewhat different journey of an article…

A month ago, an article on Forbes, started a storm in a teacup. The article discussed a Facebook plan to thwart encryption in WhatsApp by adding client side moderation of sorts:

“In Facebook’s vision, the actual end-to-end encryption client itself such as WhatsApp will include embedded content moderation and blacklist filtering algorithms. These algorithms will be continually updated from a central cloud service, but will run locally on the user’s device, scanning each cleartext message before it is sent and each encrypted message after it is decrypted.”

A few days later, Facebook disputed these as something they’d never do.

Problem solved.

As I was working at the time on security related product requirements for one of my clients, this story has stuck with me, especially bringing home the challenge and the difference between security and privacy – in most enterprise scenarios – you just can’t have them both.

I’d like to raise here a few of my thoughts on this subject, and also look at some of the differences between individuals, businesses and governments. This will also mix with the fact that I am a father of two children at ages 9 and 12 who both use WhatsApp on their smartphones regularly (that’s the norm here in Israel – if I could, I’d wait with smartphones for them a bit longer).

Trying to understand security in WebRTC? Here’s a developer’s checklist you should follow:

Download the WebRTC security checklist

Protecting our privacy

There’s an expectation here that whatever we do online will stay private, similarly to how things work in the real world.

It sounds great, but what exactly is privacy anyway? And how do we translate it to our daily lives without technology?

Without technology, conversations were transient, they never stored in any way, so people who talked to friends never really had the recording of that conversation either. They had no transcript either. And if we’re talking about technology, do we include the written word as part of a technology advancement or is that a pre-tech thing?

Today though, I can search and find a conversation with my daughter’s teacher from 5 years ago on WhatsApp. Is that a breach of the privacy of the teacher?

I don’t know the answers, and I am not advocating against privacy.

At the very least, I believe in encryption everywhere and in the concepts behind zero trust (ie – not trusting machines in your own network). Is that privacy? Or security?

The challenge with privacy – the idea that the things we do in private stay private – is when you try and mix it with security.

Securing our society

I live in a country which seems to be at constant war with its neighboring countries and with the many who want to harm it and its citizens.

When going to a shopping mall, I am used to having my bags scanned and my privacy breached. Why? Because in the context of where I live – it saves lives. In order to maintain security, some privileges around privacy cannot be maintained.

The challenge here is when do we breach privacy too much in the name of security. How far is too far?

Taking all this online makes things even more challenging. Can governments rely on the ability to “spy” on its own citizens in the name of security? Are there things that are better be spied on to make sure people don’t die? How far is too far?

Our society today values the lives of people so much. Is the life of a single person saved worth being able to spy on everyone?

Then there’s the bigger issue of corporations being multinational and social networks being global – who is securing society here? The corporations or the countries? Should corporations and social networks secure people against their governments or vice versa?

Securing our children

This one is where things get really tricky.

I’ve got kids. They’re small. I am in charge of them.

I’d like that whatever they do online will be private and ephemeral in its nature. Stupid stuff they do today shouldn’t haunt them as grownups. Good luck with that request…

On the other hand, how much privacy should they be allowed on social networks and on the Internet?

Should I be spying on them? Should I be able to filter content? Should I be alerted about questionable content that they get exposed to or are exposing themselves?

If anything, do my kids have the same privacy we so much value for ourselves against me being able to educate them on what’s out there lurking in the shadows of the internet?

There are different apps to help parents with that. Most of them are quite invasive. I decided to go with something rather lightweight here but I can’t say it lets me sleep well at night. Nothing really does when you have kids.

Securing our business

If you are a business owner, you somehow need to do what your employees do on your behalf. This affects how customers look and value your brand, so the privacy of your employees… well…

If a customer complains about a transaction, you’d like to go back and figure out the history of the interactions with that customer. If you’re in an industry that has strict rules and regulations, you might be forced by law to make a record of your employees’ interactions anyways.

How does that compare to the requirement for privacy? How does that fit with the march towards end-to-end encryption where the service provider himself (=you) can’t look at the interactions?

On one hand, you want and need encryption and security, on the other hand, this might not go hand in hand with securing the privacy of an individual employee. What works for consumers may not work in enterprise scenarios.

Our age of automation

Then there’s automation and machine learning and artificial intelligence.

As businesses, we want to automate as much of what we do in order to scale faster, better and at a lower price point.

As consumers, we want easier lives with less “steps” to make and remember. We’ve shifted from physical buttons on TVs to remote controls to voice control and content recommendations. At some point, these steps involve smarts and optimization that can only be obtained by looking at large collections of data across users.

In other words, we’re at a point in time that much of the next level of automation can only be introduced by collecting data, which in turn means breaching privacy.

Here are a few recent examples for all the great voice interfaces that are cropping around:

• Apple contractors were allegedly listening to 1,000 Siri recordings a day — each
• Google workers can listen to what people say to its AI home devices
• Your Xbox Is Listening to You, and So Are Microsoft’s Contractors
• Amazon reportedly employs thousands of people to listen to your Alexa conversations
• Facebook admits contractors listened to users’ recordings without their knowledge

Who do we TRUST?

Here’s the funny bit – it doesn’t really seem like there’s anyone we can trust, while we need to trust everyone.

As an employee, I need to trust my employer. At least to some extent.

As a citizen, I need to trust my government. Especially in democracies, where I choose that government along with my fellow citizens. At least to some extent.

As a user of “apps”, I need to trust the apps I use. At least to some extent.

And yet, none of these organizations have shown that they should be trusted too much.

So in Blockchain we trust?

I beg to differ, at least today, with all the data and security breaches, along with other scandals around it. I can’t see this as a trusting environment.

Can we have both privacy and security?

Companies are looking for ways to bridge between the two alternatives.

It is interesting to see how Apple and Google each takes a side. Apple vying towards privacy more than Google while Google trying to use security and math to be able to offer some extent of privacy while maintaining its machine learning advantage and ability to serve ads.

Then there are cloud-based end-to-end encryption solutions for enterprises where the privacy is maintained by letting the enterprise hold the keys to its messaging kingdom and not letting the cloud provider have a peak. Cisco Webex for example does a good job here, going as far as giving granular controls also on where end-to-end encryption works on an individual or a group level.

Today though, we still don’t have a good solution that can offer both privacy and security and work well at all the levels we expect it to. I am not sure if we ever will have.

Why Facebook’s idea isn’t farfetched?

While Facebook said this isn’t even planned, the solution makes sense on many levels.

What are the challenges Facebook has with messaging?

• The need to offer end-to-end encryption to its users. This is table stakes in social messaging these days
• The need to play nice with governments around the globe. Each government has its own rules and unances. Europe has GDPR and the right to be forgotten. The US is somewhat less restrictive in its privacy policies. China has all encryption keys to its kingdom. Different countries offer different privacy profiles to its citizens
• Facebook has everyone looking over its shoulder, waiting for it to fail with user’s privacy. And it fails almost on a weekly basis
• It has competitors to contend with. These competitors are working on bots and automation to improve the user experience. Facebook needs to do (and is doing) the same. Which requires access to user interaction data at some level

To do all this, Facebook needs to be able to access the messages, read them, decide what to do about them, but not do it on its own servers so they aren’t “exposed” to the actual content and data of the user. Which is exactly what this no-news item was all about. Lets go back to the original quote from the Forbes piece:

“In Facebook’s vision, the actual end-to-end encryption client itself such as WhatsApp will include embedded content moderation and blacklist filtering algorithms. These algorithms will be continually updated from a central cloud service, but will run locally on the user’s device, scanning each cleartext message before it is sent and each encrypted message after it is decrypted.”

will include embedded content moderation and blacklist filtering algorithms – there will be a piece of code running on the mobile device of the user reading the messages. Here’s the news for you – there already is. That piece of code is the one sending the messages or receiving them. It is now going to be somewhat smarter though and look at the content itself, for various purposes – “content moderation” and “blacklist filtering”. → I definitely don’t want Facebook to do this for my content, but I really do want them to do it for my kids’ content and report back to me

these algorithms will be continually updated from a central cloud service – they already are. We call this software updates. Each release of an app gives us more features (or bug fixes). With machine learning, which these algorithms are doing, there’s a need to tweak and tune the model continually to keep it relevant. Makes perfect sense that this is needed.

will run locally on the user’s device – the content itself isn’t going to be stored in the cloud by Facebook. At least not in a way they can read and share directly. Which is what end-to-end encryption is all about.

This immediately reminded me of someone else who is doing that and offering it as an API. Google Smart Reply:

The Smart Reply model generates reply suggestions based on the full context of a conversation, and not just a single message, resulting in suggestions that are more helpful to your users. It detects the language of the conversation and only attempts to provide responses when the language is determined to be English. The model also compares the messages against a list of sensitive topics and won’t provide suggestions when it detects a sensitive topic.

It runs on device, reads all messages in cleartext. Applies algorithms and determines what to do next offering it as reply suggestions. All running on device, without interacting with the cloud.

To get to this level though, Google had to look and read a lot of messages and build a model for the algorithms it uses.

–

Figuring security and/or privacy in modern applications and services isn’t easy. It comes with a lot of tradeoffs that need to be taken throughout the whole process – from requirements to deployment.

Trying to understand security in WebRTC? Here’s a developer’s checklist you should follow:

Download the WebRTC security checklist

The post Facebook eavesdropping Whatsapp? The everlasting tension between security and privacy appeared first on BlogGeek.me.

Twilio Signal 2019 was all about contact centers and legendary customer engagement. And then some.

This one is going to be long. There’s just a lot of ground to cover…

Twilio Signal is the biggest event of the programmable communications industry. I had the opportunity to join the first one and have been covering these events ever since – mostly remotely but also in person. This time, I had to do it remotely

I took the time last week to sit down for the two hours needed to watch the first day’s keynote and a few more hours thinking and reading about the event everywhere I could.

If you haven’t seen the event already, but CPaaS and programmable communications is of interest to you, then I warmly suggest you take the time to sit down and view the keynote:

From here on, I’ll try to explain my thoughts about the keynote and the new announcements in it.

The main theme?

160,000 customers.

Seriously. Jeff Lawson, CEO and co-founder of Twilio, and other Twilions brought on stage made certain they mention that number 160,000.

I’ll have more on numbers later, but this is something that Jeff and Twilio really wanted you to remember. That they are the biggest and baddest vendor offering programmable communications.

The real theme though was Legendary Customer Engagement:

It came up some 35 minutes into the event and set the pace for many of the scope, context and announcements that Twilio wanted to make at Signal.

“Legendary Customer Engagement” translates into a focus is on contact center use cases in 2019. Or is it the other way around?

Anyways, here’s what Twilio had in store for us in the keynote:

If you ask me, Flex dictated that focus. SendGrid gave it the marketing tint it was missing, but seemed it was there mostly as lip service. More on that later.

Topics covered

In the two hours of the keynote, Twilio tried to cover a lot of territory. A lot is going on for Twilio in a year to be able to cover it all. My coverage will be based on these announcements more than on the actual flow and topics dictated by the keynote. I’ll keep most of the coverage in the order that it was presented at the keynote.

Twilio by the numbers

This year, Twilio took the Moscone Center in San Francisco, the largest conferences building in town, to be able to fit all the visitors it wanted to cater for. I am assuming that meant doubling the number of visitors than the previous year.

Two main numbers were important to Twilio: Developers and Customers

That said, I’ll start in the reverse order of what Jeff shared… putting it in the order of interest to me.

Other Numbers

Here’s what got shared later on during the metrics section of the keynote:

• 750B human interactions / year
• 32,500 calls / minute
• 13,000 peak SMS / second (double than last year)
• 2.8B unique phone numbers / year (dialed in or out)
• 3B email addresses / quarter (in and out)

Twilio. Is. Big

But big where? And in what way?

Twilio were interested in showing peak traffic and not traffic totals.

What does 750B human interactions mean exactly?

We got the calls per minute at peak or maximum – a really impressive stat, but what’s the average? How about calls per day or month or year? And while we’re at it, how many of these calls are short or long? What’s the average call length on the Twilio platform?

Peak SMS is nice, but again, where are we with average? With total? The number shows the ability to scale more than actual scale.

Phone numbers and email addresses show touch points, but less amount of traffic. That part  was definitely missing from the numbers. I wonder why.

160,000 Customers

That’s the number of companies that use Twilio. How many of them actively use it and how many “forgot” a number they acquired and happen to pay a dollar a month out of credits on Twilio? I don’t know. But that’s a really big number.

I wonder how many of these are above $100/month customers. How many are really large customers. These would be the relevant/interesting ones.

6 Million Developers

Jeff started strong, revealing the number of developers using Twilio are 6 million. More accurately, though, this would be registered accounts. I have 2-3 accounts on the Twilio platform myself.

As big as this number is, it isn’t interesting. Not really.

When you look at mature platforms, they stop talking about registrations and start talking about engagement. That’s why most messaging platforms switched to revealing their Monthly Active Users (MAU), some going as far as daily active users. The idea being that people don’t care how many people are on the platform but rather how many are actively using it.

Twilio can definitely do the same today, and change the way CPaaS vendors flaunt their growth – I’d be surprised if anyone comes near to their size today.

Why? 160,000 customers

Enterprise Compliance

As Twilio is moving toward the enterprise for a few years now, they have been beefing up security, adding an enterprise plan, user roles, etc.

What comes next is compliance.

The part that Jeff Lawson mentioned and wanted to share was support for HIPAA compliance across Twilio’s product lines. This is coming in 2020, and will probably start with one or two products and spread from there to the rest, based on their own internal roadmap.

For those who aren’t aware of it, HIPAA is the regulation put in place in the US for patient privacy. Every and all vendors who wish to offer communication in this space must be HIPAA compliant, and to do so, if they are using a third party like Twilio or other CPaaS vendors, they need them to sign something known as BAA (Business Associate Agreement). Up until now, it wasn’t easy to get a BAA from Twilio, now it will be possible.

This announcement was kind of an aside in the keynote, done to get it out and into the open publicly on stage. It is a really important topic just not as glamorous as the rest. It wasn’t picked up by many who were covering the event. Searching Google for Twilio HIPAA gives results from 2017 but not anything from the keynote…

Twilio CLI

Twilio CLI is a command line interface to the Twilio API. Much like the Nexmo CLI or even WordPress CLI, it offers easy access for scripting scenarios without the need to “really code”. Here’s the Twilio CLI announcement.

This is a long overdue addition to Twilio. What sweetens the wait though is what this implementation includes out of the box:

• Coverage of a lot of the Twilio APIs already (maybe all of it, I am not sure)
• Plugins, which Twilio and third parties can build for the CLI and use
• Proxy capability to handle webhooks locally during development
• Ready-made templates in their documentation for the CLI
• Ability to pipe the tail of the log straight through the CLI

In order to show off the power of the CLI, they even created a kind of an online competition to make a point about it. You can watch it at the 24:30 mark in the keynote.

All in all, this is the only announcement around the Twilio developer tools, but definitely an important addition.

SendGrid

Up until that point at the keynote, Jeff was setting the stage. He got a few things out of the way – explained that Twilio is big by sharing numbers, shared Twilio’s diversity goals and progress, and announced Twilio CLI (because it didn’t fit in the overall theme of the event, but was hugely important to share on stage).

While getting to the communication part of it, and Twilio’s key points for 2019 and on, they had to say something about SendGrid. This was a big acquisition that was announced last year, and the intent was to show progress and cohesion.

Here’s how things went for this 20 minutes session:

Sameer Dholakia, the CEO of Twilio SendGrid came on stage. He spent most of the time explaining SendGrid, why email is important, and why this needs to be relegated to SaaS vendors.

Two things about SendGrid:

1. They’re also big
2. They’re APIs

That means a perfect match for Twilio.

The explanation given wasn’t really necessary for those used to using email APIs (which may seem like an obvious requirement to many developers). It was meant for the people in the room – Twilio customers – communications people who understand SMS and voice but are somewhat less conversant in the need and importance of email. Remember – most CPaaS vendors are used to explaining why carrier relations is complicated, but probably feel like sending emails should be way simpler.

Why spend all that time about SendGrid and emails? To increase the use of SendGrid by existing Twilio customers.

Sameer shared 3 new features/capabilities that SendGrid rolled out, none of them in too many words:

1. Email Validation API; built around the same concepts as Twilio Verify for phone numbers. The intent here is to be able to know if an email is valid or not and with what confidence. Sending emails to invalid addresses may ruin your standing and hurt the deliverability of emails you really want to send to real people, so this can come in really handy
2. Marketing campaigns; support for email sequences and the ability to send them as a campaign, which is nice. I use this in my own business by through a marketing automation vendor. This means SendGrid is starting to go up the stack and food chain as Twilio did with Studio and Flex
3. Ads; ability to integrate personalized ads in emails. Another usable feature (which was in beta since last year)

All these are targeted towards marketing activities and less towards communications.

The time spent on sharing these new features? 3 minutes. Not nearly enough to grok.

10 minutes into SendGrid, the discussion came back to Twilo. And SendGrid.

The intent was to show how Twilio+SendGrid = more. This makes perfect sense, especially when shifting towards a world of multiple channels and when that world is what Twilio is pitching at Signal. It wasn’t explained in such a way, or integrated as much at this point. A kind of a missed opportunity.

It was done by sharing live code of how you can add email sending via SendGrid within an application that uses Twilio for SMS communications. It was nice. I hope Twilio will have better integration and surprises for 2020 on the Twilio+SendGrid storyline.

Twilio Flex

Dave Michels said it quite well: “Flex was a major theme of Signal 2019, and I anticipate it will be an even bigger theme at Signal 2020.”

Flex had 3 parts to it in the keynote:

1. Why is Flex so great
2. What have we done new in Flex since last time?
3. Here’s how flexible Flex is

While the product is great from a concept point of view (and I am sure from an implementation point of view as well – never really used it myself), I think the delivery during the keynote could have been improved a bit.

1. Why is Flex so great

Here’s what Jeff said during the keynote about Flex:

“A year ago, if you’ve had told me that the legacy vendors in the space would have even heard of Flex I would have thought you were crazy. Let alone buying up all the ads around Moscone for Signal this year. I mean, isn’t that crazy? But you know what? I think they are afraid. I think they are afraid of what happens when we unleash the creative ability of millions of software developers to innovate in the contact center.”

Next, it went into positioning of Flex within the industry:

First, came the legacy vendors, who were positioned as:

• Hard to support
• Hard to maintain
• Expensive
• Hard to change

All true by the way.

Then came the turn of cloud contact centers in SaaS models, said to be 15% of the market now. These are “neither scalable nor flexible in the way we need […] not built for continuous improvement”.

Ouch.

True or not, that’s Twilio’s stance now, and it seems like they’re competing head to head with their cloud contact center customers – more verbally than last year.

For me, Flex is a great product – it shows that future enterprise products must be programmable products, ushering a new era of programmable enterprise. But that’s for some future article.

Two numbers were given for Flex:

1. %500 growth in interactions
2. 250 ecosystem partners

Both are somewhat meaningless without context, and there isn’t none as this product is just too new in the market.

2. What have we done new in Flex since last time?

There’s more work and focus on customer acquisition in Flex than in other products of Twilio.

We will probably see more of that in 2020 and 2021 – the same way that Amazon Connect is now winning a lot more deals and going into a lot more deployments than we’ve seen when it launched. It takes time for these products to really get adopted and integrated.

What did get into the announcements here, probably to give it some tint of progress, were the following:

1. Native Zendesk CTI integration. Mentioned in a full sentence throughout the whole keynote
2. Autopilot is now GA. The thing is that Autopilot is a distinct Twilio product in its own right, and it lives as a feature in Flex. The tie-in here to Flex is to show progress of Flex more than to say anything about Autopilot. More on the Autopilot announcement here
3. Announced Media Streams API in beta, a brand new product for Twilio. I think this should have gotten a section of its own rather than just a mention and a minute or two wrapped inside Flex

I think Flex is now in the grueling part of getting from an MVP towards a version 1.0 that Twilio can be proud of. Once there, Twilio will definitely open the back burners and start showing a lot more product specific progress in Flex itself.

3. Here’s how flexible Flex is

That was given to software engineer in the Flex team (I couldn’t catch her name and didn’t find her as a speaker in other sessions at Signal, so sorry about that).

She showed a few ways in which Flex can easily be extended, adding some custom features to it. It was an eye opener to the speed at which you can modify a deployed contact center and improve it, but I am not sure how well that was conveyed to the audience.

I think Twilio has to experiment here a bit further on how to make that into a wow moment.

Media Streams API

Media Streams API is the generic approach CPaaS vendors can offer (and already offer) ML (Machine Learning) to their customers.

With AI and ML you never truly know what a customer would like to use or how:

• There are many cloud based SaaS offerings today around ML that can bring value. Which ones would your customer want to use?
• Each customer is unique, as he might want to train a model using his own unique data
• As a generic player (CPaaS), it is hard to decide what solutions to offer in the ML space
• It is still early days

The approach here is to be able to interfere within an ongoing session, collecting the media out of the session and sending it towards a machine learning algorithm for classification. That can be used to handle things like speech to text, sentiment analysis, recognition, etc.

The easiest way to integrate such a thing today? By shipping that media over a websocket towards a cloud service. Other CPaaS vendors are doing this already. Twilio has added that capability now.

Out of the box? Amazon Lex and Google speech to text engines are provided via websocket interface. This means that there’s a bit of integration work on the developer’s end to get them to work, but it also means it is a kind of an open interface that can be fit (with some effort) to virtually anything.

Gridspace was added as an exclusive launch partner with a direct integration, making it simpler to use than others. Twilio promises this will change with more partners added into the roster, which stands to reason.

I am assuming this will come to Programmable Video at some point in the future (to analyze the voice channel only but also to analyze video).

Here’s the launch announcement for Media Streams API.

This got a really small portion of the keynote this year.

The most revealing slides about messaging

I found the following two slides the most interesting in all the keynote at Signal 2019.

In the above, Jeff indicates the reply rate Twilio customers see with the Twilio API for WhatsApp.

To me, this means the following:

1. With social messaging, enterprises are actually making an effort to make it a two-way interaction and not only one-way notifications
2. People are more inclined to reply over social networks (WhatsApp in this case) than they are to do the same over SMS. Why? Because we’re trained to assume companies won’t listen to us over SMS anyway
3. For companies seeking true conversations with their customers, social channels are better today than carrier based channels (SMS)

The second slide was revealing about how Twilio sees things:

This was given as context to unveiling Twilio Conversations.

Here, Jeff tried to explain what is driving these programmable conversations. He did it left-to-right.

Messaging Apps

The ones mentioned were Apple Business Chat, WhatsApp and Facebook Messenger.

Somehow, Telegram, Instagram, Twitter, Viber, WeChat and all the others were ignored. Why? Because the top 3 are probably over a billion users each. They are the biggest ones. And also the ones easiest for Twilio to reach (being predominantly a US West coast company).

One I found even more interesting here is the reference to Apple Business Chat – they haven’t allowed any of the CPaaS vendors to date access to their API. Up until now, Apple’s focus with it was on chat widgets and contact center type applications. Does this give us any indication about a possible announcement in the future of Twilio as the first CPaaS vendor with Apple Business Chat? Or maybe of Apple opening up to more CPaaS vendors and Twilio being just one of them.

The other thing that caught my attention came later, when Twilio Conversations was unveiled. It doesn’t yet include support for Facebook Messenger. For social, there’s only WhatsApp. For now.

One alternative here is that Twilio is banking on Facebook merging Facebook Messenger and WhatsApp infrastructure to a point where support for WhatsApp only would win. Or simply because they saw more traction to their Twilio API for WhatsApp than they did for their support of Facebook Messenger so this got prioritized for Twilio Conversations.

In any case, the coming year or two is going to be limited in the number of social channels that Twilio will be focusing on.

Carriers

For some reason, the slide includes RCS and 5G. I am not sure why 5G is there, besides trying to make the slide more appealing to the eye. Jeff might even agree with me, as he made no mention of 5G – on this slide or on any other part of the keynote.

RCS? That’s the carriers’ replacement for SMS. It has been promised in the last 20 years and with Google’s own adoption of it, it is getting more attention. Will it be deployed? I don’t know. Right now, the prospects aren’t that good.

Twilio isn’t alone here. Many CPaaS vendors made and are making announcements around RCS support.

Going back to my thoughts around WhatsApp and social, time will tell of enterprises will use RCS to “spam” customers with notifications and marketing or if they will truly try to use it for conversations. If the former happens, then conversations are better off done via social messaging platforms, leaving RCS to be the enterprise “notifications center” that SMS is today.

Data

For Data, Jeff had AI (Artificial Intelligence). Twilio decided to go for the marketing term as opposed to the technical term (ML for Machine Learning). Makes sense. For non-technical people, they are one and the same and AI just sounds way better.

ML is happening in communications. Got a whole report around AI in RTC on that written with Chad Hart if you’re interested.

This in the context of conversations is what will allow us to optimize and scale.

Twilio Conversations

Twilio Conversations was revealed as the last part of the keynote at Twilio Signal.

It is a shame that at this point, the main stage screen started fidgeting. It probably couldn’t wait to see what Twilio Conversations is about

After a long setup and explanation of how the world is headed towards multiple channels, how customers expect to be treated better, how enterprises who make an effort to strike conversations win, and how legendary customer experiences is what we should strive for, came Twilio Conversations.

This is a messaging component that handles conversations across multiple channels. Dumbing things down a bit, this is Twilio Programmable Chat does across channels and with history storage.

What does that mean?

It means that on the business side, users can use a single interface on their mobile device or in a browser to interact with customers. And that customers can use different channels to interact with the business. And all this is done as a conversation.

The result? A party.

The demo that Twilio put in place explains that conversations are messy and ever changing, and something that can assist there is Twilio Conversations.

Here’s what you get with Twilio Conversations:

• Support for multiple channels: Chat (IP messaging), SMS, MMS and WhatsApp
• Group chat, mixed with different channels
• Text, images and videos
• History storage of the conversations

Here’s where it can be improved (probably in future releases if this gets adopted):

• Multiple channels are nice. Somehow, Facebook Messenger which Twilio has support for, at least in beta
• Voice (and video) channels, as part of the conversation
• Email. They have SendGrid and decided not to use/include it in the first release of Twilio Conversations

And here’s the announcement on the Twilio blog.

Interestingly, Twilio decided not to cover pricing for Twilio Conversations during the keynote. They usually mention that for new products, at least to some extent. Checking on their website, it seems like Twilio went for a mixture of MAU+Storage approach.

You pay per Monthly Active User (count the number of users who did “something” in the last month, multiply them with the price per user). And then you pay for the storage you have inside conversations per GB. I am assuming storage is there for images and videos but might be also for text. Not written there, but also a reasonable assumption, you pay for each channel interactions separately based on their price list.

An interesting (and reasonable) choice.

Is Conversations necessary? It probably is.

As we’re moving forward with the use of CPaaS and communication APIs, there’s a need to provide ever higher abstractions, giving developers more of the tooling they need and having them write less of that tooling.

With omnichannel being hammered as the next big thing for about a decade, such tooling makes sense.

Twilio isn’t the only vendor headed in this direction either. Nexmo has their own variant, called Nexmo Conversation, somewhat hidden within their developer documentation.

As more conversation revolves around “Twilio is competing with their customers” and “Twilio is expensive”, having new constructs like Twilio Conversations makes perfect sense. It is a usable tool for customers, not competing with their own products (at least for most customers) AND it is a lot more than just SMS and voice.

If Twilio Conversations becomes a popular product for Twilio, this places many of the other CPaaS vendors yet again on the hamster wheel, playing catch-up.

Other announcements

There were other interesting announcements made by Twilio that weren’t introduced during the keynote itself.

Verified by Twilio

This is a different product.

It wasn’t part of the first day keynote, and all I could find is the post about it on Twilio’s blog, the press release and rehash of these two elsewhere. Not much on the product page besides an invitation to the beta program.

Here’s the problem: spam in calls. Lots of it. Most calls probably. Which is why people stopped answering their phone.

Everyone’s trying to solve that problem, each with his own concept – which I find really interesting. How the solutions proposed fit well into the DNA and thought processes of each company. This though, is for another, future article.

Twilio Verify connects between businesses who want their calls to pass through the noise of spam calls, customers who do not want to be bothered with spam and call identification apps. The deal is this one:

• A customer installs a call identification app (unrelated to Twilio)
• The call identification app integrates with Twilio (via an API obviously)
• Whenever a call comes in, the app checks with Twilio who the caller is and if the intent of the call is known. If it isn’t, it does whatever it did up until today. If Twilio has that information, then that information is shown to the customer
• The business identifies itself and its brand to Twilio, connecting that information to phone numbers
• The business can also indicate per call what the intent of that call is (or on the phone number level – that information is somewhat murky at this point)

Tada!

Problem solved.

It just needs most users to install these apps on their phones and businesses to use this new API.

Is that going to be opened to Twilio’s competitors? To the carriers themselves? Would this be adopted by Apple and Google?

Lots of questions. Great initiative.

More on the whole spam solutions space in September.

Missing at Twilio Signal 2019 keynote

A two hour keynote session is long. Twilio had to pick and choose what to talk about and what to leave aside. A few things that were left aside reveal what interests Twilio, or at least what they want to interest us with.

Here are 4 things that didn’t make it to the keynote that I felt were missing.

1. “Architecture” diagram

Every year, Jeff has shared the architecture view of the Twilio products portfolio. It was a fascinating view as to how Twilio sees its mix of products and how it thinks they fit into a customer’s thought processes.

I don’t recall if there was such a bird’s eye view for Signal 2018, but there definitely wasn’t for 2019.

Where is Twilio Conversations located compared to Authy and Notify? What about the new Media Streams APIs? How does Twilio rationalize SendGrid or simply email into the mix and within its ever-growing set of products?

If you head to the Twilio products page, you get this categorization:

• Solutions
  • Flex
  • Marketing Campaigns
• Services
  • Identity
    • Lookup
    • Verify
    • Authy
  • Intelligence
    • Autopilot
    • TaskRouter
  • Conversations
    • Twilio Conversations
    • Proxy
• Channels APIs
  • Programmable SMS
  • Programmable Voice
  • Twilio SendGrid
  • Programmable Chat
  • Programmable Video
  • Programmable Fax
  • Twilio APIs for WhatsApp
• Super Network
  • Phone Numbers
  • Programmable Wireless
  • Short Codes
  • Super SIM
  • Elastic SIP Trunking
  • Narrowband
  • Interconnect
• Tools
  • Runtime

Nothing on that page is visual. Just a long list of products. It is workable if you know Twilio, but feels somewhat hard for newcomers.

A few interesting things here:

• Conversations as a category under Services and Twilio Conversations as a product inside it. I am assuming marketing has debated this one a bit, trying to resolve the confusion it could bring and decided just to use the same word
• The word APIs appears only on the Channels. Services are considered higher level constructs here, even if (obviously) they are treated as APIs
• Tools includes only “Runtime”, wrapping into them things like Functions and Studio, and ignoring other free of charge tools and capabilities such as the new CLI
• Twilio Pay, Media Streams API and probably a slew of other capabilities get hidden inside the layers of products and not mentioned here at all

I think not speaking about these layers and how Twilio sees the world of communication APIs in a layered structure in their yearly keynote is a missed opportunity.

2. Video

No video. Nothing.

It wasn’t mentioned at all.

Was it because it isn’t a money maker?

Was it because there was nothing new to say about it?

Was it because the progress made wasn’t important enough to fit in the keynote?

3. Voice

This one’s interesting.

Thinking back at the keynote, there was nothing substantial in it about voice calls. Sure, it was somehow covered as part of the Twilio CLI and Flex, but nothing more.

I am guessing that support for voice is a done deal, so no real need to say anything new about it during the keynote itself.

Closest thing to coverage at Twilio Signal 2019 around voice was the new Media Streams API and even that wasn’t covered much.

4. IOT

The Twilio SIM card and its M2M play wasn’t part of Signal.

The focus was Flex and human interactions. Not bots-to-bots or machines-to-machines.

I am wondering what progress and announcements could have been made.

Where is CPaaS headed?

CPaaS is complicated these days.

SMS and voice? Sure.

But is SMS and voice mandatory? Some think it is, and place the whole focus on that part.

Is video part of CPaaS? What about only video?

WhatsApp support? Other channels?

Email anyone?

What makes a CPaaS platform well… CPaaS?

For me, it is being generic, appealing to developers and handling real time interactions (be it text, voice, video or whatever). And it also needs to somehow deal with communications between humans.

We’re at a point where CPaaS is appealing. Both for enterprises who need to improve their communications and to new entrants to the CPaaS market – either as vendors, enablers, partners, or whatnot. There’s an ecosystem building in the CPaaS space for a few years now.

Twilio is somehow managing to stay ahead of most vendors in both breadth and depth of their offering which is not an easy task.

Trying to figure out CPaaS?

• Who are the players in this market?
• How do they differentiate?
• Where is the industry headed?
• Is CPaaS being commoditized?
• How do you compare to other vendors in this space?

Ping me if you want answers to any of these questions for a private consultation.

The post Twilio Signal 2019 and the future of the programmable enterprise appeared first on BlogGeek.me.

WebRTC can be hacked-away with great results. Often though, this leads to sub-par experiences.

WebRTC as a VoIP technology is the best thing ever. It “democratizes” this whole domain, taking it from the hands of experts into the hands of the masses of developers out there. Slapping a bit of code and seeing real time video is magical. And we’re now starting to add it to more and more businesses using web technology.

While this all seems easy now (and it is a lot easier than it used to be before WebRTC), there are a few mistakes that many beginners make in WebRTC. And to be honest, these mistakes are not only made by beginners. That is why I am sharing a couple of common (beginner) mistakes in WebRTC that I’ve seen for a couple of years now.

1. Using an outdated signaling server (from github)

This happens all too often. You start by wanting to build something, you search github, you pick a project, and with WebRTC – it just doesn’t work. It might for the simple scenario but it won’t handle edge cases, or scale nicely, or accomodate for the more complex thing you’re thinking about.

The truth is, that today, there’s no single, goodly, off the shelf, out of the box, readymade, pure goodness, open source, signaling server for WebRTC that you can use. Sorry.

There might be a few contenders, but I haven’t seen any specific project that everyone’s using (unlike TURN for example, where coturn definitely rulz). The sadder truth? SFUs offer better signaling than signaling servers with WebRTC (and almost always I’d suggest against using their signaling directly in front of your WebRTC client).

2. Mis-configuring NAT traversal

This should have been trivial by now, but apparently it isn’t.

A few rules of thumb:

1. Don’t. Rely. On. Google. Public. STUN
2. Don’t use free github STUN and TURN server lists
3. Don’t decide not to deploy TURN because your server has a public IP address
4. And then a few

This is such a basic and common mistake that I even created a free video course for it: Effectively Connecting WebRTC Sessions.

3. Testing locally

This one’s also basic.

Locally things tend to work well. Due to different network configuration, but also due to fairy dust that I am sure you sprinkle over your local router (I know I do every morning).

Once you go to the real world with real networks, things tend to break.

Test in the real world and not on your machine using 2 tabs, or being professional, from a Chrome tab to a Firefox tab.

The real world is messy and messy isn’t healthy for naive deployments.

Need help with automating that? Look at testRTC, but don’t neglect real world testing.

4. Not using adapter.js

WebRTC is a great specification but it is rather new.

This means that:

• Different browsers are going to behave a bit differently
• Browser implementations are somewhat buggy
• Different versions of the same browser act differently

And I haven’t even started about getting WebRTC browser implementations to be spec compliant with 1.0.

This all boils down to you having to work out a strategy in your code where all that if/then directives to deal with these differences takes place.

The alternatives?

1. Whenever you find such an issue, add that if/then statement in the code directly (the most common approach, albeit not really smart in the long term)
2. Create a shim/polyfill/whatever you want to call it, where you do all these if/then thingies (great, but not easy to maintain)
3. Just use adapter.js

Guess which one I prefer?

5. Forgetting to take care of security

Two reasons for you to forget about security:

1. WebRTC is secure, so why should you do anything more about it?
2. Because your service doesn’t deal with payments or sensitive data so why bother?

Both reasons are won’t lead you to a good place. In 2019, security is coming to the front, especially with communications. You can ask Zoom about it, or go check what Google’s Project Zero did recently.

Want a good starting point? I’ve got a WebRTC security checklist for you.

6. Assuming you can outsource it all

You can’t. Not really.

Need a design for a whitepaper? An article written? A WordPress website created? Find someone on Upwork, Fiverr or the slew of other websites out there and be done with it.

With WebRTC? Don’t even think about it.

WebRTC is ever-changing, which means that whatever you deploy today, you will need to maintain later. If you are outsourcing the work – some of it or all of it – assume this is going to be a long term relationship, and that for the most part, you may be able to outsource the development work but not the responsibility.

Going this route? Here are 6 things to ask yourself before hiring an outsourcing WebRTC vendor.

7. Diving into the code before grokking WebRTC

1. Go to github.
2. Pick a project.
3. “Install” it.
4. Run it.
5. Fix a few lines of code.
6. Assume you’re done.

No. WebRTC is much more complicated than that scenario above. There are a few different servers you’ll need to deploy and use, there’s geography sensitivity to consider, and lots of other things.

You need to understand WebRTC if you want to really use it properly. Even if all you’re doing is using a 3rd party.

Don’t make these mistakes!

Be sure to review these to see if there’s anything you’re doin’ wrong:

1. Using an outdated signaling server (from github)
2. Mis-configuring NAT traversal
3. Testing locally
4. Not using adapter.js
5. Forgetting to take care of security
6. Assuming you can outsource it all
7. Diving into the code before grokking WebRTC

Check out my free WebRTC Basics course, or the bigger Advanced WebRTC one.

The post Common (beginner) mistakes in WebRTC appeared first on BlogGeek.me.