Now that it is getting relatively easy to setup video calls (most of the time), we can move on to doing fun things with the video stream. With new advancements in Machine Learning (ML) and a growing number of API’s and libraries out there, computer vision is also getting easier to do. Google’s ML Kit is […]
The post Smile, You’re on WebRTC – Using ML Kit for Smile Detection appeared first on webrtcHacks.
ML in RTC can fit anywhere – from low level optimization to the higher application layers.TL;DR – I am working with Chad Hart on a new ML in RTC report. If you are interested in it, scroll down to the end of this article.
Machine Learning (ML), Artificial Intelligence (AI), Big Data Analytics. Call it what you will. You’ll be finding it everywhere. Autonomous cars, ecommerce websites, healthcare – the list goes on. In recent years we’ve seen a flourish in this domain due to the increase in memory and processing power, but also due to some interesting breakthrough in machine learning algorithms – breakthroughs that have rapidly increased the accuracy of what a machine can now do.My ML Origin Story
I’ve been looking and dealing with machine learning for many years now. Never directly calling it that, but always in the vicinity of the communications industry.
It probably started in university. I decided to do an M.Sc because I was somewhat bored at work. I took a course in computational linguistics which then ended with me doing research in backward transliteration, looking at phonemic similarities between English and Spanish (#truestory). That was in 2005, and we used a variant of dynamic programming and the viterbi algorithm. That and other topics such as hidden markov model were my part and parcel at the time.
Later on, I researched the domain of Big Data and Analytics at Amdocs. I was part of a larger group trying to understand what these mean in telecommunications. Since then, that effort grew into a full business group within Amdocs (as well as the acquisition of Pontis, well after I left Amdocs for independent consulting).
Which is why when I talked to Chad Hart about what we can do together, we came to an agreement that something around ML and AI made a lot of sense for both of us, and taking it through the prism of RTC (real time communications), placed it in the comfort zone of both of us.
During that period, we thought a lot about what domains we wish to cover and what ML in RTC really means.Categorizing ML in RTC
Communications is a broad enough topic, even when limited to the type that involves humans. So we limited even further to real time communications – RTC. And while at it, threw text out the window (or at the very least decided that it must include voice and video).
Why do that? So we don’t have to deal with the chatbots craze. That’s too broad of a topic on its own, and we figured there should be quite a few reports there already – and a few oil snake sellers as well. Not our cup of tea.
This still left the interesting question – what exactly can you do with AI and ML in RTC?
We set out to look at the various vendors out there and understand what are they doing when it comes to ML in RTC.
Our decision was to model it around 4 domains: Speech Analytics, Computer Vision, Voice Bots / Assistants and RTC quality / cost optimization.1. Speech Analytics
Speech Analytics deals a lot with Natural Language Processing (NLP) and Natural Language Understanding (NLU).
Each has a ton of different use cases and algorithms to it.
Think of a contact center and what you can do there with speech analytics:
You will find a lot of speech analytics related RTC ML taking place in contact centers. A bit less of it in unified communications, though that might be changing if you factor in Dialpad’s acquisition of TalkIQ.2. Computer Vision
Computer Vision deals a lot with object classification and face detection, with all the derivative use cases you can bring to bear from it.
“Simple” things like face recognition or emotion recognition can be used in real time communications for a multitude of communication applications. Object detection and classification can be used in augmented reality scenarios, where you want to mark and emphasize certain elements in the scene.
Compared to speech analytics, computer vision is still nascent, though moving rapidly forward. You’ll find a growing number of startups in this domain as well as the cloud platform giants.3. Voice Bots & Assistants
To me, voice bots and assistants is the tier that comes right above speech analytics.
If speech analytics gets you to NLP and NLU, the ability to convert speech to text and from there moving to intent. Voice bots are about conversations – moving from a single request to a fluid interaction. The best example? Probably the Google Duplex demo – the future of what conversational AI may feel like.
Voice bots and assistants are rather new to the scene and they bring with them another challenge – do you build them as a closed application or do you latch on to the new voice bot ecosystems that have been rapidly making headway? How do you factor in the likes of Amazon Alexa, Google Home, Google Assistant, Siri and Cortana into your planning? Are they going to be the interaction points of your customers? Does building your own independent voice bot even makes sense?
Whatever the answers are, I am pretty sure there’s a voice bot in the future of your communications application. Maybe not in 2018, but down the road this is something you’ll need to plan for.4. RTC Quality & Cost Optimizations
While the previous 3 machine learning domain areas revolve around new use cases, scenarios and applications through enabling technologies, this one is all about optimization.
There are many areas in real time communication that are built around heuristics or simple rule engines. To give an example, when we compress and decompress media we do so using a codec. The encoding process (=compression) is lossy in nature. We don’t keep all the data from the original media, but rather throw away stuff we assume won’t be noticed anyway (sounds outside the human hearing range, small changes in color tones, etc) and then we compress the data.
The codecs we use for that purpose are defined by the decoder – by what you do if you receive a compressed bitstream. No one is defining when an encoder needs to look like or behave. That is left to developers to decide, and ecoders differ in many ways. They can’t brute-force their way to the best possible media quality, especially not in real-time – there’s not enough time to do that. So they end up being built around guesswork and heuristics.
Can we improve this with machine learning? Definitely.
Can we improve network routing, bandwidth estimation, echo cancellation and the myriad of other algorithms necessary in real time communications using machine learning? Sure we can.
The result is that you get better media quality and user experience by optimizing under the hood. Not many do it, as the work isn’t as high profile as the other domains. That said, it is necessary.Interested in ML in RTC?
Here are a few things you can do:Fill out our survey
This will get factored into the quantitative part of our report. If you fill it out, you will also receive a complimentary e-book we’re writing titled Intro to AI in RTC.
Interested in the report itself? Thinking of purchasing it? Great! We have a special launch discount.
You can find more information about the report itself in our research page.
Doing something interesting in this space? Share your thoughts with us.
Contact us via email@example.com to participate in our study.
The post Where does Machine Learning fit in Real Time Communication (ML in RTC)? appeared first on BlogGeek.me.
What should you be doing about the upcoming WebRTC 1.0 release?
That comic strip above? I think it embodies nicely what comes next.
We’ve started with WebRTC somewhere in 2011 or 2012. Depends who’s counting. So we’re 6 or 7 years in now.
I’ve been promised WebRTC 1.0 in 2015 I think.
Then again in 2016.
In 2017, I was told that WebRTC 1.0 is just around the corner. Definitely going to happen before year end.
Guess what? We’re now almost halfway through 2018. And no WebRTC 1.0. Yet.
But it is coming.
To give you the gist, Google will be ripping out some code, adding new code. Removing APIs. Modifying others. The timeline stated for all this in that posting?
Change is in the air…
That change is going to affect developers and testers everywhere, and the end result is going to be uncertainties and surprises in the coming months. How many months? Many months
There’s not much you can do about it besides allocating resources to the problem in the short and mid term future. These resources should be in development and testing.
I touched development in a previous webinar I did with Philipp Hancke when I launched the next round of my WebRTC training.
Now I want to talk about preparation aspects in the testing domain – what exactly you should be expecting moving forward.
To that end, I am a visitor of the upcoming WebRTC Standards webinar series. The webinar takes place later today –
Building an interactive application? There’s more than one WebRTC programming language that can fit your needs.
Last time I’ve written about WebRTC programming languages it was some two years ago. My focus then was how programming languages fit to different WebRTC components. That article is still relevant, so I suggest you read it as well.
This time, I want to focus on something slightly different. In recent months I’ve had the pleasure of watching as well as consulting teams of developers who are using programming languages that don’t always make sense to me when it comes to WebRTC. And yet, once they explained their reasoning, the decision and path they took would be one I couldn’t just dismiss if I were in their shoes.
There are two main places where you see JaveScript in WebRTC apps today: client side as well as signaling server.Clients
Client side is simple enough. It is due to the fact that this is how you use WebRTC in browsers, but it can also be seen in cross platform mobile development (though not that much) or when using Electron for WebRTC.Signaling
Signaling servers is about Node.js. And yes, the guest article from 2013 (!) is as relevant now as it was then. Twelephone (mentioned in that article) is long dead, and Chris has moved on to IOT and later decentralized networks. But the use of Node.js as a signaling server for WebRTC is going strong. If you take that route, just make sure to pick a popular framework.Media Servers
I’ve seen Node.js being used in media servers as well:
I’ll be using C and C++ together here, as I don’t see a huge distinction between the two (and in most cases, those that develop code in C create C++ abstractions, and those developing in C++ end up writing C code anyways).
C/C++ is a kind of a lowest common denominator. My guess is that a lot of the languages here have their compilers/interpreters written in C/C++ anyways. It is also a language that is available everywhere, though not always accessible directly.Clients
Go to webrtc.org and you’ll find out that the code WebRTC that Google open sourced is written in C++. The easiest thing to do if you want to support WebRTC on an embedded device is probably to start by taking that code and hammering it until it fits your device.
It isn’t the only way to get WebRTC into a device but it sure is a popular one.Signaling
Signaling with C++ isn’t common. Where you will find it is when SIP meets WebRTC.
Interestingly, all main open source SIP servers are written in C/C++: Asterisk, OpenSIPS and FreeSWITCH.
I am assuming this is because they are older than other WebRTC signaling implementations that tend to use higher languages.STUN/TURN
NAT traversal servers are written today in C/C++.
Not all of them. But the most popular one is (coturn).Media Servers
Media servers need to be highly performant, which is why most of them also end up being written in C/C++ – at least the parts that matter for performance.
Java is the most popular language based on the TIOBE index.
I don’t like Java that much, probably due to its verbosity and all the pain that its garbage collection causes to real time apps. But I don’t get to decide what others use
Java is probably one of the most popular programming languages also in WebRTC backend development, but not only there.Clients
Android requires Java, which means Android native development with WebRTC also requires Java coding.
Besides this obvious option, there’s also the part of writing WebRTC clients in Java from scratch.
While you won’t find any open source Java client for WebRTC, I know of two separate WebRTC client implementations that use Java.Signaling
There are many signaling servers out there that end up using Java when it comes to WebRTC. You’ll find it in enterprise software but not only.
A few of them are also open sourced, though there isn’t a specific one that is widely popular or highly recommended as far as I can tell.Media Servers
Several of the media servers out there use Java. Either for everything or for the higher level abstractions.
Here are two of them:
If Java is needed for development of WebRTC in Android, then Swift is what you need for iOS.
Unless you’re fine with using Objective-C and not adopt Swift.
Other than iOS, I haven’t seen Swift used anywhere else when it comes to WebRTC implementations (or otherwise for that matter).Python
As a higher level language, Python gets high marks. I’ve been introduced to it about a decade ago and loved it since, though I can’t say I used it too much myself.
When it comes to WebRTC, Python has its role in signaling servers, as many other higher level languages.
The most notable Python project related to WebRTC is [matrix]. Its open source Synapse server implementation is written in Python.
There are others using Python for their signaling. My guess is that this is just because of familiarity with the language.Ruby
You can’t mention Python without mentioning Ruby.
I am guessing that Ruby can fit in signaling servers just as well as Python. Only difference is that I know of no one that is doing that.C#
If someone is making use of a Windows based development stack, he is more likely to use C# than anything else.
In such cases, you will see the use of C# for all aspects of WebRTC – from native WebRTC client implementations, through signaling, to NAT traversal and media servers.
I am not a big fan of backend development on Windows, but if you are one of those who need it, then know you are not alone.PHP
Having such huge market share means more haters, especially when the language itself isn’t the most modern one out there.
What surprised me (it shouldn’t, but it did), was that there are companies who use PHP for their signaling server when it comes to WebRTC. I would never have thought that would be the case, but it is.
If you want to use PHP for your signaling server, then go for it. Just make sure you understand the limitations and implications of it.Erlang
We’re getting into the more exotic alternatives.
Erlang is such a programming language to me. Created by Ericsson and open sourced ages ago, Erlang offers some interesting capabilities (go read Wikipedia).
There are a few projects out there that make use of Erlang for signaling and one for NAT traversal. I am not aware of anyone using Erlang in a production service, though I am sure there is.Elixir
Elixir is built on top of Erlang (or at least on its virtual machine).
Only one I know who makes use of it for WebRTC is Slack.
In last year’s Kranky Geek event, Slack shared their plans of migrating from their Janus implementation to an in-house developed Elixir media server. You can watch that here:Go
Go is a programming language created by Google. It is somewhere between C and C++ as much as I can tell (never been an expert in Go).
This one came to my attention with the recent implementation of STUN/TURN in Go – Pion TURN.
Not sure how popular is Go with WebRTC developers.Where do these languages fit?
I tried taking the information above and placing it in an easy to use table, to give a quick summary of the state of WebRTC programming languages.
I am sure I missed a language or two. And I am also sure some people are using WebRTC programing languages differently than I’ve described here. Feel free to share in the comments for this article or by emailing me about it – I’d love to learn more.
What programming language should you use?
That’s a different question. I’d say it depends on several factors:
Google in 2018 is all about AI. But not only…
In November 2015, Google released TensorFlow, an open source machine learning framework. While we’ve had machine learning before that – at Google and elsewhere, this probably marks the date when machine learning and as an extension AI got its current spurt of growth.
Some time, between that day and the recent Google I/O event, Sundar Pichai, CEO of Google, probably brought his management team, knocked on the table and told them: “We are now an AI company. I don’t care what it is that you are doing, come back next week and make sure you show me a roadmap of your product that has AI in it.”
I don’t know if that meeting happened in such a form or another, but I’d bet that’s what have been going at Google for over a year now, culminating at Google I/O 2018.
After the obligatory icebreaker about the burger emoji crisis, Pichai immediately went to the heart of the keynote – AI.
Google announced AI at last year’s Google I/O event, and it was time to show what came out of it a year later. Throughout the 106 minutes keynote, AI was mentioned time and time again.
That said, there was more to that Google I/O 2018 keynote than just AI.
Google touched at its keynote 3 main themes:
I’d like to expand on each of these, as well as discuss parts of Smart Displays, Android P and Google Maps pieces of the keynote.
I’ll try in each section to highlight my own understanding and insights.Before we begin
Many of the features announced are not released yet. Most of them will be available only closer to the end of the year.
Google’s goal was to show its AI power versus its competition more than anything else they wanted to share in this I/O event.
This is telling in a few ways:
When it comes to AI, Google is most probably the undisputed king today. Runners up include Amazon, Microsoft, IBM, Apple and Facebook (probably at that order, though I am not sure about that part).
If I try to put into a diagram the shift that is happening in the industry, it is probably this one:
Not many companies can claim AI. I’ll be using ML (Machine Learning) and AI (Artificial Intelligence) interchangeably throughout the rest of this article. I leave it to you to decide which of the two I mean
AI was featured in 5 different ways during the keynote:
In each and every single thing that Google does today, there’s an attention to how AI can improve that thing that needs doing. During the keynote, AI related features in GMail, Google Photos and Android were announced.
It started off with four warm-up feel-good type use cases that weren’t exactly product announcements, but were setting the stage on how positive this AI theme is:
From here on, most sections of the keynote had an AI theme to them.
Moving forward, product managers should think hard and long about what AI related capabilities and requirements do they need to add to the features of their products.What are you adding to your product that is making it SMARTER?Google Assistant (=voice and speech)
Google Assistant took center stage at I/O 2018. This is how Google shines and differentiates itself from its main 3 competitors: Apple, Amazon and Facebook.
In March, Forbes broke some interesting news: at the time, Amazon was hiring more developers for Alexa than Google was hiring altogether. Alexa is Amazon’s successful voice assistant. And while Google hasn’t talked about Google Home, its main competitor at all, it did emphasize its technology differentiation. This emphasis at I/O was important not only for Google’s customers but also for its potential future workforce. AI developers are super hard to come by these days. Expertise is scarce and competition between companies on talent is fierce. Google needs to make itself attractive for such developers, and showing it is ahead of competition helps greatly here.
Google Assistant got some major upgrades this time around:
Besides these additions, where each can be seen as a huge step forward on its own right, Google came out with a demo of Google Duplex, something that is best explained with an audio recording straight from the keynote:
If you haven’t watched anything from the keynote, be sure to watch this short 4 minutes video clip.
There are a few things here that are interesting:
You’d like to read what Chad Hart has to write about Duplex as well.
For me, Duplex and Assistant are paving the way to where we are headed with voice assistants, chatbots and AI. Siri, Cortana and Lex seem like laggards here. It will interesting to see how they respond to these advancements.
Current advancements in speech recognition and understanding make it easier than ever to adopt these capabilities into your own products.If you plan on doing anything conversational in nature, look first at the cloud vendors and what they offer. As this topic is wide, no single vendor covers all use cases and capabilities.
While at it, make sure you have access to a data set to be able to train your models when the time comes.Google Lens (=vision)
Where Google Assistant is all (or mostly) about voice, Google Lens is all about vision.
Google Lens is progressing in its classification capabilities. Google announced the following:
That last one is interesting, and it is where Google has taken the same approach as Amazon did with DeepLens, one that should be rather obvious based on the requirements here:
It took it a step further, offering it also programmatically through ML Kit – Google’s answer to Apple’s Core ML and Amazon’s SageMaker.
Here’s a table summarizing the differences between these three offerings:Google Apple Amazon ML Framework TensorFlow Core ML + converters MXNet & TensorFlow Cloud component Google Firebase none AWS SageMaker Edge component ML Kit Core ML AWS DeepLens Edge device types Android & iOS iOS DeepLens Base use cases
Apple Core ML is a machine learning SDK available and optimized for iOS devices by Apple. You feed it with your trained model to it, and it runs on the device.
AWS DeepLens is the first ML enabled Amazon device. It is built on top of Amazon’s Rekognition and SageMaker cloud offerings.
Google ML Kit is Google’s machine learning solution for mobile devices, and has now launched in beta.
This started as Google Lens and escalated to an ML Kit explanation.Need to run ML? You need to think where training the model occurs and where classification takes place. These seem to be split these days between cloud and devices. In many cases, developers are pushing the classification algorithms towards the devices at the edge to gain speed and reduce costs and load on the backend. HWaaS
With everything moving towards the cloud, so does hardware in some sense. While the cloud started from hardware hosting of virtualized Linux machines, we’ve been seeing a migration towards different types of hardware recently:
We’re shifting from general purpose computing done by CPUs towards specialized hardware that fits specific workloads in the form of FPAG.
The FPGA in the illustration above is Google’s TPU. TPU stands for TensorFlow Processing Unit. These are FPGAs that have been designed and optimized to handle the TensorFlow mathematical functions.
TensorFlow is said to be slow on CPUs and GPUs compared to other alternatives, and somehow Google is using it to its advantage:
Google’s TPUs got their fair share of time at the keynote in the beginning and were stitched throughout the keynote at strategic points:
Pichai even spent time boasting large terms like liquid cooling…
It is a miracle that these TPUs aren’t plastered all over the ML Kit landing page.Going with TensorFlow? You’ll need to decide on the cloud platform you are going to use, especially when it comes to dataset processing and training. Google is working hard to differentiate itself there. Wellbeing
I am assuming you are just as addicted to your smartphone as I am. There are so many jokes, memes, articles and complaints about it that we can no longer ignore it. There are talks about responsibility and its place in large corporations.
Apple and Google are being placed on the spotlight on this one in 2018, and Google took the first step towards a solution. They are doing it in a long term project/theme named “Wellbeing”.
Wellbeing is similar to the AI initiative at Google in my mind. Someone came to the managers and told them one day something like this: “Our products are highly addictive. Apple are getting skewered in the news due to it and we’re next in line. Let’s do something about it to show some leadership and a differentiation versus Apple. Bring me ideas of how we can help our Android users with their addiction. We will take the good ideas and start implementing them”.
Here are a few things that came under Wellbeing, and one that didn’t but should have been:
In a way, this is the beginning of a long road that I am sure will improve over time. It shows the maturity of mobile platforms.Not sure how responsibility, accountability and wellbeing like aspects lend themselves to other products. If you are aiming at machine learning, think of the biases in your models – these are getting attention recently as well. Fake News
Under responsibility there’s the whole Fake News of recent years.
While Wellbeing targets mainly Apple, The Google News treatment in the keynote was all about addressing Facebook’s weakness. I am not talking about the recent debacle with Cambridge Analitica – this one and anything else related to user’s data privacy was carefully kept away from the keynote. What was addressed is Fake News, where Google gets way more favorable attention than Facebook (just search Google for “google fake news” and “facebook fake news” and look at the titles of the articles that bubble up – check it also on Bing out of curiosity).
What Google did here is create a new Google New experience. And what is interesting is that it tried to bring something to market that skims nicely between objectivity and personalization – things that don’t often correlate when it comes to opinion and politics. It comes with a new layer of visualization that is more inviting, but most of what it does is rooted in AI (as anything else in this I/O keynote).
Here’s what I took out of it:
One more thing Google did? Emphasized that they are working with publishers on subscriptions, being publisher-friendly, where Facebook is… er… not. Will this hold water and help publishers enough? Time will tell.AI and Machine Learning lends themselves well to this approach. It ends up being a mixture of personalization, trending and other capabilities that are surfaced when it comes to news. Can you see similar approaches suitable for your product offering? Smart Displays
Smart displays are a rather new category. Besides Android as an operating system for smartphones and the Waymo AI piece, there was no other device featured in the keynote.
Google Home wasn’t mentioned, but Smart Displays actually got their fair share of minutes in the keynote. The only reason I see for it is that it is coupled nicely with the Google Assistant.
The two features mentioned that are relevant?
I am unsure why Google gave smart displays the prominence it did at Google I/O. I really have no good explanation for it, besides being a new device category where Apple isn’t operating at all yet – and where Amazon Alexa poses a threat to Google Home.Android P
10 years in, and Android P was introduced.
There were two types of changes mentioned here: smarts and polish.
Smarts was all about AI (but you knew that already). It included:
There was really not much to say about Android P. At least not after counting all the AI work that Google has been doing everywhere anyway.App Actions and Slices are important if you develop Android Apps. ML Kit is where the true value is and it works on both Android and iOS – explore it first. Google Maps
Google Maps was given the stage at the keynote. It is an important application and getting more so as time goes by.
Google Maps is probably our 4th search destination:
This is where people look for information these days.
In Search Google has been second to none for years. It wasn’t even part of the keynote.
Google Assistant was front and center in this keynote, most probably superior to its competitors (Siri, Cortana and Lex).
YouTube is THE destination for videos, with Facebook there, but with other worries at this point in time. It is also safe to say that younger generations and more visual audiences search YouTube more often than they do anything else.
Maps is where people search to get from one place to another, and probably searching even more these days – more abstract searches.
In a recent trip to the US, I made quite a few searches that were open ended on Google Maps and was quite impressed with the results. Google is taking this a step further, adding four important pillars to it:
Smarts comes from its ML work. Things like estimating arrival times, more commune alternatives (they’ve added motorcycle routes and estimates for example), etc.
Personalization was added by the introduction of a recommendation engine to Maps. Mostly around restaurants and points of interest. Google Maps can now actively recommend places you are more likely to like based on your past preferences.
On the collaboration front, Google is taking its first steps by adding the ability to share locations with friends so you can reach out a decision on a place to go to together.
AR was about improving walking directions and “fixing” the small gripes with maps around orienting yourself with that blue arrow shown on the map when you start navigating.Where are we headed?
That’s the big question I guess.
More machine learning and AI. Expect Google I/O 2019 to be on the same theme.
If you don’t have it in your roadmap, time to see how to fit it in.
If you are contemplating build versus buy for your live video platform, or just undecided on which one to pick, check out this 10-part video series.
My consulting projects these days tend to be in one of 3 domains:
I like doing all of these types of projects simply because it keeps me interested. Especially since there’s no specific one that I like more than the others here. It does sometimes confuse potential customers, and probably doesn’t help me with “niching” or “focusing”, but it does give me a very wide view of the communications market.
I want to focus on the 3rd project type, the one where developers want assistance in making sure they pick the right technology, architecture the solution and get it to market with as little risk as possible, this is where things get interesting.
The first thing I do in such projects? Check for NIH.
NIH stands for Not Invented Here, and it is a syndrome of all developers. I know, because I suffer from it as well. Developers are builders and tinkerers. They like to make things work – not get them readymade, which is why when they have the opportunity of building something – they’ll go ahead and do it. The problem though, is that economies of scale as well as time to market aren’t in their favor. In many of the cases, it would be easier to just pick a CPaaS vendor and build your live video product on top of his platform instead of building it all from scratch.
There are many reasons why people go build their own video platform:
I spend some time uncovering and better understanding the reasons for the decision. Sometimes I feel they make sense, while other times less so.
Which is why when I sat down with Vidyo to think about an interesting project to do together some months back, the decision was made to put out a series of short videos explaining different aspects of live video platforms. I tried to cover as much ground as possible. From network impairments, through video coding technologies, through scale, devices and lots of other topics as well.
The purpose was to get developers and entrepreneurs acquainted with what is necessary when you go build your own infrastructure, and if you decide on buying a platform, to know what to look for.
The series is packed full with content. And I’d love to get your candid opinion of it. Check it out here:
The post Choosing a Live Video Platform – a new video series appeared first on BlogGeek.me.
One of the great things about WebRTC is that it is built right into the web platform. The web platform is generally great for WebRTC, but occasionally it can cause huge headaches when specific WebRTC needs do not exactly align with more general browser usage requirements. The latest example of this is has to do […]
There are opposite forces at play when it comes to the next wave of communication technologies.
There are a lot of changes going on at the moment, being introduced into the world of communications. If I had to make a shopping list of these technologies, I’d probably end up with something like this:
Each item is worthy of technobabble marketing in its own rite, but the thing is, they do affect communications. The only question is in what ways.
I have been looking at it lately a lot, trying to figure out where things are headed, building different models to explain things. And looking at a few suggested models by other industry experts.Communication domains – simplified
Ignoring outliers, there are 3 main distinct communication domains within enterprises:
Usually, we will be using the obligatory “aaS” to them: UCaaS, CCaaS and CPaaS
I’ll give my own simplified view on each of these acronyms before we proceed.UCaaS
Unified Communications looks inwardly inside the company.
A company has employees. They need ways and means to communicate with each other. They also need to communicate with external entities such as suppliers, partners and customers. But predominantly, this is about internal communications. The external communications usually takes a second-class citizen position, with limited capabilities and accessibility; oftentimes, external communications will be limited to email, phone calls and SMS.
What will interest us here will be collaboration and communication.CCaaS
Contact Centers are about customers. Or leads, which are potential customers.
We’ve got agents in the contact center, be it sales or customer care (=support), and they need to talk to customers.
Things we care about in contact centers? Handling time, customer satisfaction, …CPaaS
Communication Platform as a Service is different.
It is a recent entry to the communications space, even if some would argue it has always been there.
CPaaS is a set of building blocks that enable us to use communications wherever we may need them. Both CCaaS and UCaaS can be built on top of CPaaS. But CPaaS is much more flexible than that. It can fit itself to almost any use case and scenario where communications is needed.Communications in Consolidation
There’s a consolidation occuring in communications. One where vendors in different part of communications are growing their offering into the adjacent domains.
We are in a migration from analog to digital when it comes to communications. And from pure telecom/telephony towards browser based, internet communications. Part of it is the introduction of WebRTC technology (couldn’t hold myself back from mentioning WebRTC).
This migration opens up a lot of opportunities and even contemplation on how should we define these communication domains and are they even separate at all.
There have been some interesting moves lately in this space. Here are a few examples of where these lines get blurred and redefined:
These are just examples. There are other vendors in the communication space who are going after adjacent domains.
The idea here is communication vendors looking into the communications venn diagram and reaching out to an adjacency, with the end result being a consolidation throughout the whole communications space.External disruption to communications
This is where things get really interesting. The forces at play are pushing communications outwards:
UCaaS, CCaaS, CPaaS. It was almost always about real time. Communications happening between people in real time. When the moment is over, the content of that communications is lost – or more accurately – it becomes another person’s problem. Like a contact center recording calls for governance or quality reasons only, or having the calls transcribed to be pushed towards a CRM database.
Anything that isn’t real time and transient isn’t important with communications. Up until now.
We are now connecting the real time with the asynchronous communications. Adding messaging and textual conversations. We are thinking about context, which isn’t just the here and now, but also the history of it all.
Here’s what’s changing though:UC and Teams
Unified Communications is ever changing. We’ve added collaboration to it, calling it UC&C. Then we’ve pushed it to the cloud and got UCaaS. Now we’re adding messaging to it. Well… we’re mostly adding UC to messaging (it goes the other way around). So we’re calling it Teams. Or Team Collaboration. Or Workstream Collaboration (WSC). Or Workstream Communication and Collaboration (WCC). I usually call it Enterprise Messaging.
The end result is simple. We focus on collaboration between teams in an organization, and we do that via group chat (=messaging) as our prime modal for communications.
Let’s give it a generic name that everyone understands: Slack
The question now is this: will UC gobble up Team communication vendors such as Slack (and now Workplace by Facebook; as well as many other “project management” and messaging type tools) OR will Slack and the likes of it gobble up UC?
I don’t really know the answer.CC and CRMs
What about contact centers? These live in the world of CRM. The most important customer data resides in CRMs. And now, with the introduction of WebRTC, and to an extent CPaaS vendors, a CRM vendor can decide to add contact center capabilities as part of his offering. Not through partnerships, but through direct implementation.
Can contact centers do the same? Can they expand towards the CRM domain, starting to handle the customer data itself?
If salesforce starts offering a solid contact center solution in the cloud as part of its offering, that is highly integrated with the Salesforce experience, adding to it a layer of sophistication that contact center vendors will find hard to implement – what will customers do? NOT use it in favor of another contact center vendor or source it all from Salesforce? Just a thought.
There’s an additional trend taking place. That’s one of context and analytics. We’re adding context and analytics into “customer journeys”, sales funnels and marketing campaigns. These buzzwords happen to be part of what contact centers are, what modern CRMs can offer, and what dedicated tools do.
For example, most chat widget applications for websites today offer a backend CRM-like dashboard that also acts like a messaging contact center, and at the same time, these same tools act similarly to Google Analytics by following users as they visit your website trying to derive insights from their journey so the contact center agent can use it throughout the conversation. Altocloud did something similar and got acquired recently by Genesys, a large contact center vendor.CP and PaaS
CPaaS is different a bit. We’re dealing with communication APIs here.
CPaaS market is evolving and changing. There are many reasons for it:
That last one is key. We’ve seen the large cloud vendors enhancing their platforms. Moving from pure CPU and storage services up the food chain. Amazon AWS has so many services today that it is hard to keep up. The question here is when will we reach an inflection point where AWS, GCE and Azure start adding serious CPaaS capabilities to their cloud platforms and compete directly with the CPaaS vendors?
Where is CPaaS headed anyway?
The future can unfold in three different ways when it comes to communications:
How do you think the future will unfold?
A Linux device, such as PC Engines APU, can be equipped with an LTE modem, but sometimes it’s desirable to use the mobile connection only if the wired connection is unavailable.
The following scenario is for Debian 9 on an APU box, but it’s also applicable to any other Linux device.
The DHCP client is tweaked to ignore the DNS server addresses that are coming with DCHP offer. Otherwise, the LTE provider may provide DNS addresses that are not usable via the ethernet WAN link.
The “ifmetric” package allows setting metrics in interface definitions in Debian. This way we can have two default routes with a preferred metric over LAN interface. The default route with lower metric is chosen for outbound traffic.
The watchdog process checks availability of a well-known public IP address over each of the uplinks, and shuts down and brings up again the corresponding interface. It only protects from next-hop failures. If you want to protect from failures in the whole WAN service, you need to increase the Ethernet port metric if it fails, and then start checking the connectivity, and lower the metric when it’s stable again.
Also the second NIC on the box is configured to provide DHCP address and to NAT all outbound traffic.
Detailed installation instructions are presented here: https://gist.github.com/ssinyagin/1afad07f8c2f58d9d5cc58b2ddbba0a7
Ubiquiti EdgeRouter X is a tiny and cheap (around $50) router with a decent amount of memory: 256MB RAM and 256MB flash. The router offers 5 GigE copper ports, and there’s also a model with an additional SFP port. The device is produced since 2014, and it’s still up to date and a good value for money.
On hardware level, the device consists of a Gigabit Ethernet switch, with one GigE port attached to the MIPS CPU and used as a 802.1q trunk. Also inside the enclosure, serial console port is available for easy debugging or manipulating the boot loader.
The router comes with stock Ubuquiti software which is based on Debian Wheezy, so many files are from 2013-2014. OpenVPN package is pre-installed, but only version 2.3 is available. The software offers a nice GUI and SSH access.
OpenWRT provides excellent support for this hardware. The router is able to perform IP routing at more than 400Mbps (I haven’t tested it with back-to-back connection, so I don’t know the limit).
Also with OpenVPN 2.4 that is available in up-to-date OpenWRT packages, the box performs at up to 20Mbps with 256-bit AES encryption, and at about 55Mbps with encryption and authentication disabled.
In default OpenWRT configuration, the switch port 0 is dedicated to WAN link, and ports 1-4 are used as a LAN bridge. The WAN link acts as a DHCP client, and LAN is configured with DHCP service in 192.168.1.0/24 range. The command-line configuration utilities are quite straightforward, and there’s a Web UI as well.
Here’s a new repository for OpenVPN deployment scenarios and example configurations:
At the moment it lists two scenarios with configuration generation scripts:
WebRTC developers are really hard to come by. I want to improve my ability to help companies in search of such skill.
If there’s something that occurs time and again, it is entrepreneurs and vendors who ask me if I know of anyone who can build their application. Some are looking to outsource the project as a whole or part of it, and then they are looking for agencies to work with. Others are looking for a single expert to work with on a specific task, or someone they could hire for long stretches of time who has WebRTC skills.You a WebRTC Developer?
I’d like to know more about you IF you are looking for projects or for a new employer.
Here are a few things first:
Fill out this form for me please (or via this link):
I won’t be reaching out to you immediately (or at all). I’ll be using this list when others ask for talent that fits your profile.You looking for WebRTC Developers?
Got a need for developers that have WebRTC skills?
I am not sure exactly how to find them and where, but I am trying to get there.
Two ways to get there:
Chat won’t bring carriers to their SMS-glory days.
The Verge came out with an exclusive last week that everyone out there is regurgitating. This is my attempt at doing the same
We’re talking about Google unveiling its plans for the consumer chat experience. To put things in quick bulleted points:
I’d like to share my viewpoints and where things are going to get interesting.SMS is dead
I liked Mashable’s title for their take on this:
While an apt title, my guess is that beyond carriers and reports written to them, we all know that already.
SMS has long been dead. The A2P (Application 2 Person) SMS messages are all that’s left out of it. Businesses texting us either their PIN codes and passwords for 2FA (2 Factor Authentication) and OTP (One Time Passwords) or just sending us marketing junk for us to ignore.
I asked a few friends of mine on a group chat yesterday (over Whatsapp, of course) when and how do they use SMS and why. Here are the replies I got (I translated them to English):
These are 40 year olds in Israel. Most working out of the IT domain. The answers will probably vary elsewhere, but here in Israel, most will give you similar answers. Whatsapp has become the go-to app for communications. So much so, that we were forced to give our daughter her first smartphone at the age of 8 only so she can communicate with her friends via Whatsapp and won’t stay behind. Everyone uses it here in Israel.
You should also know that plans upwards of 2Gb of monthly data including unlimited voice and SMS in Israel cost less than $15 a month in Israel, so this has nothing to do with price pressure anymore. It has to do with network effects and simple user experience.
SMS is no longer ubiquitous across the globe. I can’t attest to other countries, but I guess Israel isn’t alone in this. SMS is just the last alternative to use when all else has failed.
Why is SMS interesting in this context?
Because a lot of what’s at stake here for Google relates to the benefits and characteristics of SMS.RCS is (still) dead
RCS is the successor of SMS for getting carriers into the 21st century. It has been discussed for many years now, and it will most definitely, utterly, completely, unquestionably get people back from their Messenger, WhatsApp and WeChat back to the clutches of the carriers. NOT.
RCS is a design-by-committee solution, envisioned by people my age and older, targeting a younger audience across the globe in an attempt to kill fast moving social network with a standardized, ubiquitous, agreed upon specification that then needs to be implemented by multiple vendors, handset manufacturers and carriers globally to make any sense.
Not going to happen.
Google’s take on this was to acquire an RCS vendor – Jibe – two years ago for this purpose. The idea was probably to provide a combination of an infrastructure and a mobile client to speed up RCS deployments around the globe and make them interoperable faster than the carriers will ever achieve on their own.
Two years passed, and we’ve got nothing but a slide (and the article on The Verge) to show for this effort:
An impressive list of operators, OEMs and OS providers that are behind this RCS initiative. Is that due to Google? To some part, probably so.
In a way, this reminds me also of Google’s other industry initiative – the Alliance of Open Media, where it is one of 7 original founding members that just recently came out with AV1, a royalty free video codec. It is a different undertaking:
Why is Google going into bed with the carriers on this one?Google had no choice
The Verge had an exclusive interview with Anil Sabharwal, the Google VP leading this effort. This led to the long article about this initiative. The numbers that Anil shared were eye opening as to the abysmal state of Google’s messaging efforts thus far.
I went ahead and placed these numbers next to other announced messaging services for comparison:
A few things to note here:
Google failed to entice its billion+ Android users to install or even use its messaging applications.
Without the numbers, it couldn’t really come up with a strategy similar to Apple iMessage, where it essentially hijacks the messaging traffic from carriers, onboarding the users to its own social messaging experience.
Trying to do that would alienate Google with the carriers, which Google relies on for Android device sales. Some would argue that Google has the klout and size to do that, but that is not the case.
Android is open, so handset manufacturers and carriers could use it without Google’s direct approval, throwing away the default messaging app. Handset manufacturers and carriers would do that in an effort to gain more control over Android, which would kill the user experience, as most such apps by handset manufacturers and carriers do. The end result? More users purchasing iPhones, as carriers try to punish Google for the move.
What could Google do?
Two years ago, Google decided to go for alternatives (1) and (3). Allo was their own social messaging app. Had it succeeded, my guess is that Google would have gone towards approach (2). In parallel, Google acquired Jibe in an effort to take route (3), which is now the strategy the company is behind for its consumer messaging.
The big risk here is that the plan itself relies on carriers and their decisions. We don’t even know when will this get launched. Reading between the lines of The Verge’s article, Google already completed the development and got the mobile client ready and deployed. It just isn’t enabled unless the carrier being used approves. Estimates indicate 6-12 months until that happens, but for which of the carriers? And will they use the stock Android app for that or their own ambitious better-than-whatsapp app?E2EE can kill this initiative and hurt Google
The biggest risk to Google is the lack of E2EE (end to end encryption).
In each and every regurgitated post of The Verge article and in The Verge itself this is emphasized. Walt Mossberg’s tweet was mentioned multiple times as well:
Bottom line: Google builds an insecure messaging system controlled by carriers who are in bed with governments everywhere at exactly the time when world publics are more worried about data collection and theft than ever.
— Walt Mossberg (@waltmossberg) April 20, 2018
Bottom line: Google builds an insecure messaging system controlled by carriers who are in bed with governments everywhere at exactly the time when world publics are more worried about data collection and theft than ever.
The problem for Google is that the news outlets are noticing and giving it a lot of publicity. And it couldn’t come at a less convenient time, where Facebook is being scrutinized for its malpractice of how it uses and protects user data in the Cambridge Analytica scandal. Google for the most part, has come unscathed out of it, but will this move put more of the spotlight on Google?
The other problem is that all the other messaging apps already have E2EE supported in one way or another. The apps usually mentioned here are Apple iMessage, Signal and Telegram. Whatsapp switched to E2EE by default two years ago. And Facebook Messenger has it as an option (though you do need to enable it manually per conversation).
Will customers accept using “Chat” (=RCS) when they know it isn’t encrypted end to end?
On the other hand, Russia is attempting to close Telegram by blocking millions of IP addresses in the country, and taking down with it other large services. If this succeeds, then Russia will do the same to all other popular messaging applications. And then other countries will follow. The end result will be the need to use the carrier (and Google’s) alternative instead. Thankfully, Russia is unsuccessful. For the time being.Who owns the data?
With RCS, the carriers are the ones that are intercepting, processing and forwarding the messages. In a way, it alludes to the fact that Google isn’t going to be the one reading these messages, at least not from the server.
This means that either Google decided there’s not enough value in these messages and in monetizing them – or – that they have other means to gain access to these messages.
Here are a few alternatives Google can use to accessing these messages:
Google might have figured it has other means to get to the messages besides owning and controlling the whole experience – similar to how Google Photos is one of the top camera apps in Apple iTunes.
By offering a better experience than other RCS client competitors, it might elicit users to download its stock Chat app on devices who don’t have it by default. Who knows? It might even be able to get people to download and use it on an iPhone one day.
The success of Google here will translate into RCS being a vehicle for Google to get back to messaging more than the means for carriers to gain relevance again.Ubiquity is here already, but not via SMS or RCS
I’ll put the graph here again – to make a point.
1.5 billion people is ubiquitous enough for me. Especially when the penetration rates in Israel are 100% in my network of connections.
People tend to talk about the ubiquity of SMS and how RCS will inherit that ubiquity.
They fail to take into account the following:
Think about it.
You can now send out an RCS message from your device. To anyone. If the other party has no RCS installed, the message gets converted to SMS. Sweet.
But what happens when the person you are sending that RCS message is located abroad? Are you seriously happy with getting a payment request from your carrier on a stupid international SMS message, or a full conversation of such for a thing you could have easily used Whatsapp for instead? And for free.
Ubiquity isn’t the word that comes to my mind when thinking about RCS.The holy grail is business messaging
Consumer messaging is free these days. There is no direct monetary value to be gained by offering this service to consumers. Carriers won’t be able to put that jinni back into its bottle and start collecting money from users. Their only approach here might be to zero-rate RCS traffic, but that also isn’t very interesting to most consumers – at least not here in Israel.
The GSMA already suggested where the money is – in business messaging. They see this as a $74bn opportunity by 2021. The problem is that rolling RCS 6-12 months from now, by only some of the carriers, isn’t going to cut it. Apple Business Chat was just released, vertically integrated, with a lot of thought put into businesses, their discovery process and free of charge.
Then there’s the rest of the social networks opening their APIs towards the businesses, and contact center solutions driving the concept of omnichannel experiences for customers.
Carriers are getting into this game late and unprepared. On top of that, they will try to get money out of this market similar to how they do with SMS. But the price points they are used to make no sense anymore. Something will need to change for the carriers to be successful here.
Will carriers be able to succeed with RCS? I doubt it.
Will google be able to succeed with Chat? Maybe. But it is up to the carriers to allow that to happen.
The post RCS now Google Messages. What’s Next in Consumer Messaging? appeared first on BlogGeek.me.