Skip navigation
All Places > Ignite Realtime Blog > Author: Dele Olajide

For anyone following my projects, you would be aware of me using  jVoiceBridge as my audio conferencing and VOIP engine with Openfire. I stopping using Asterisk a while back and all my telephony is done with jVoiceBridge and the SIP plugin for Openfire. The combination of Openfire, jVoiceBridge and WebRTC provides a very compact unfied communications solution.

 

One of the many cool features of jVoiceBridge I was never able to exploit up until now was the spatial audio feature that creates a 3D audio immersion experience from a stereo mix.

 

What's so cool about this?

 

As a home worker, I have been dreaming of when I would get to hear the separation of conference participant voices across a stereo field. By adding matching visual sitting postions, speech detection and improved voice quality, we might just get audio-conference calls back in fashion.

 

What makes this now possible is the use of the OPUS codec in WebRTC. However for a while, it looked like it was going to remain a dream unless I could figure out a way to access WebRTC audio streams directly from an Openfire plugin.

 

Introducing the Relay Plugin for Openfire.

 

I started out with the source code to the JingleNodes plugin for Openfire and got the WebRTC UDP packets wired into jVoiceBridge conferences. At first, I was limited to 8K mono audio from the PCMU (ulaw) codec, but thanks to the Jitsi project, I was able to add their OPUS decoder/encoder and finally got 48K stereo samples streaming into jVoiceBridge.

 

Image2.jpg

 

How does it work?

 

I have put together a demo web page of the plugin running on my server. To test, you have to access the same page from at least two PCs in order to test the 3D audio-conferencing effect. It only works with Google Chrome.

 

It will let you do the following things

 

  1. Join the demo conference choosing a codec and position on the stereo field (-90 to 90)
  2. Add a telephone call (SIP address) into the demo conference. OPUS re-sampling does not work with all SIP phones, use PCMU if you get no audio.
  3. Send DTMF to the telephone call.
  4. Watch the speech detection events
  5. Mute your audio from the conference

 

Where do we go from here?

 

The plugin will be an official Openfire plugin and the project will be hosted and supported from Igniterealtime. I am planning on exposing all the features of the plugin through the new Rayo XMPP Extension for remote call control. I hope to develop a Strophe plugin for Rayo to enable easy access from XMPP based web applications.

 

If you are interested in joining up with me to make this all happen sooner than later, send me an email or private message.

Now that Firefox Beta has WebRTC enabled by default, it was time to upgrade the WebRTC audio/video demo to support Firefox. Even though changes were minimal and I could do Firefox<->Firefox as well as Chrome<->Firefox, there is still an outstanding issue that is a show stopper for me.

 

Image1.jpg

 

Firefox does not yet support trickle ICE candidates and consequently does not work with Jingle Relay Nodes. This means we cannot use Openfire to act as a media bridge and relay media when both WebRTC peers cannot connect directly with each other as we can do with Chrome.

 

I am hoping this will be fixed before formal release in June. In the meantime, you can test it at http://redfire.4ng.net/webrtc. Just make sure you have Firefox beta or latest Chrome browsers. As usual, source code is attached for those curious to see or reuse in their own applications.

A new version of the Jappix plugin for Openfire is available with support for WebRTC restored back. It only supports Chrome version 26+ and is compatible with the WebRTC plugin for Spark. It uses The Openfire JingleNodes plugin for media relaying via Openfire server when WebRTC peers can connect directly with each other.

 

Video Chat

Image51.jpg

Group Video Chat

Image61.jpg

How to use

Click on the webcam icon to share video in an active chat/groupchat.

 

With chats, the other party receives a prompt to share their audio/video.

Image41.jpg

Groupchat does not prompt and auto shares audio and video with all participants.

For those following the use of Openfire for audio and video communication, The Jitsi project released very recently, the VideoBridge plugin for Openfire. If you use Jitsi with Openfire, you find it very useful for creating both audio and video conferences with multiple people.  I am hoping someone will try and make a Spark plugin for it

 

In the meantime, I did a quick experiment to see if I could convert the Redfire plugin for Spark to use WebRTC instead of RTMP/RTMFP

 

Image4.jpg

Guess what!!. It worked

 

It works the same way as the Redfire plugin for Spark. Click on the webrtc button to share audio and video with the person you are chatting with. On the other side, that person will receive a prompt to accept or reject the offer.

 

Image1.jpg

So here it is attached, if you want to play with it. It does not depend on any Openfire plugin, but will use the JingleNodes plugin if available to relay WebRTC audio/video media when a direct connection cannot be made between two Spark clients..

 

It needs Google Chrome version 26+ running on the client desktop as the default browser. It supports Google Chrome frame for Internet Explorer. The source code is in there as well. When Firexfox WebRTC support becomes stable, I will probably add support for that too.

 

*** UPDATE ***

I finally got multi-user video conferencing working in group chat like Redfire. Please note that there is no prompt. Each participant MUST click on the webrtc button to join the video conference and the plugin ensures a webrtc peer-connection is made between each and every participant. This should be done after everyone joins the groupchat.

Image1.jpg

If a new person enters the room and wishes to join the video-conference, each participant should close their browser windows and click on the webrtc button again.

Chrome version 26 stable and Firefox (Nightly build) now support full desktop screen sharing. In Chrome, you have to enable "Enable screen capture support in getUserMedia()" with the chrome://flags URL.

 

Image1.jpg

 

As the desktop screen is treated as a video device, you only have to make a change to the getUserMedia method for video and your existing application will work with no further changes. As an example, I only made the following change to my webrtc video example and now it allows two users to share their desktop screens

 

navigator.webkitGetUserMedia(
    {video:{mandatory: {chromeMediaSource: 'screen'}}},
    this.onUserMediaSuccess.bind(this),
    this.onUserMediaError.bind(this)
);

 

To see it in action, just click on this URL https://webrtc.free-solutions.org:8443/webrtc/screen.html from two different PCs. Register with two different user names (no spaces or special characters).

 

It is hosted by the very good kind and generous folks at free-solutions.org

Bla, bla bla...

WebRTC is on the march

 

With a stable implementation in Chrome since ver 23, applications are appearing as fast as flowers blooming in Spring. The nightly and developer builds (ver 25+) have more features including full desktop screen capture and ability to send it as a video stream over a peer connection. By adding DTLS encryption to Chrome nightly build, Google have achieved interoperability with Firefox nightly build.

 

Back in 2007 at ignite realtime, we built the first open source Flash to SIP gateway for SparkWeb and followed that up a few months later with Red5phone, the first open source web soft phone. Red5phone has now evolved into Apache OpenMeetings Red5Sip.

 

Fast-forward to today and we now have the iNum initiative from Voxbone which enables us to have a new geographical independent phone number for free phone calls on the Internet. iNums are global, supported by most telephone carriers including my former employer BT in the UK. Combine WebRTC with iNum and you can send and recieve free phone calls anywhere in the world provided you have a web browser and an iNum.

 

That was all the incentive I needed to get started on webrtc-phone.

http://webrtc-phone.googlecode.com/files/webrtc-phone.jpg

Ok, so what does it do?

 

The prime purpose is to make and recieve free phone calls with iNums. It can also make SIP calls with domains that accept them.

 

For more details go to the project page.

 

How did I make it. What did I use?

 

  • Customized version of Phono SDK for webrtc only that connects to the Voxeo SIP cloud and handles the Jingle signalling and WebRTC media. Flash support has completely been removed. It only works with Chrome ver 23+ for now. Firefox support will come later
  • Openfire Server with websockets and redfire plugins to lookup iNums and map to Phono SIP addresses using XMPP Presence for anonymous users.
  • Candybar and DialPad? GUI components from AT&T Foundry and &Yet
  • AddThis social sign on service for Facebook, Google and Twitter

 

Free calls! Whats the catch?

 

Calls from iNum to/from the public PSTN still costs money. If you divert your iNum to a mobile or landline, your voice service provider will charge you for diverting the call. You will be paying for all calls from your iNum to your mobile or landline phone.

 

Calls from the public PSTN to your iNum cannot be intercepted and are therfore are not supported by webrtc-phone. If your iNum is dialed from a regular phone, the call will end up on whatever device you have forwarded it to. It is only when the number is dialed from a webrtc-phone elsewhere that it will be re-mapped to your webrtc-phone session.

Can I play?....

 

Live demo is hosted at https://webrtc.free-solutions.org:8443/webrtc-phone  by the very kind folks at free-solutions.ch who are also members of the ignite realtime community.

 

Project source code is hosted on a Git repository at http://code.google.com/p/webrtc-phone/source/checkout, so go ahead and clone

WebRTCis definitely a game changing technology and desktop phones may soon be on their way to join VHS players in yesterdayland

 

A while back, I developed an Openfire plugin for Candy (Chats Are Not Yet Dead) which enabled multi-user voice chatting. It included a voice bridge to mix the audio streams on the server side and a Flash-based component to capture and play the audio streams in the web browser. It kinda worked, but has too many issues to be used in a production context.

 

I have replicated that application with WebRTC just using only HTML5 and JavaSccript. No server-side component for audio mixing and no browser component to handle streams. All audio mixing, capturing and playing is handled by the web browser.

 

As usual, there are caveats. A star topology with a server-side audio bridge will scale into hundreds of conference participants while client side peer to peer audio conferencing will struggle to get up to 10 users per conference. A ten person audio conference will consume 90 (10x9) peer connections.

 

Image1.jpg

 

However, getting away from conventional ringtone and telephone signaling using SIP or even Jingle allows to explore alternative approaches to voice communication and experience distant digital talking as naturally as possible without the restrictions of a telephone or software replacement. This is definitely the way to go. Sisko calling O'Brien

 

Click on a tab brings the group chat room into focus. Click again will turn voice on and broadcast everything you say to everyone in the group chat room. Click one more time and your voice is turned off. It works as a toggle. Click on another group chat room tab and your voice be turned off when focus moves away. On the other side, you can turn off the voice of any participant by using Candy's ignore feature. Just right click on the contact and select ignore. Select unignore to restore back the voice of the participant.

 

It is that simple. No ringtones, just use your voice with text and say hello

 

Don't take my word for it, you can try it it out here and the source code is attached to this blog for those would like to a look at the implementation. Even though it does not use SIP or Jingle signalling, It uses Jingle Nodes to act a a media relay when users are sitting behind restrictive routers or firewalls.

 

You need Chrome version 23+ or Chrome Frame with Internet Explorer. Candy has also been updated to work with the Openfire Websockets plugin as well.

The folks at Globility have been very kind to release back to the Ignite Realtime community their improvements to the Openfire monitoring plugin.

 

They have merged the monitoring plugin with the open archive plugin by Stefan Reuter and have sucessfully combined chat and group chat archiving (from monitoring) as well as XEP-0136 Message Archiving support (from open archive) removing all redundant and duplicated code. The monitoring graphing and statistics pages work just fine as before.

 

A change to the database schema of the Monitoring plugin was made to support the XEP-0136 spec and auto-upgrade DB scripts have been provided to handle the database upgrade for all the supported DBs as best as possible.

 

Source:

http://svn.igniterealtime.org/svn/repos/openfire/trunk/src/plugins/monitoring/

 

Latest build (1.3.0) of the plugin is attached. Any bugs please file in JIRA against leonroy

 

I have tested it against the latest version of OfChat (Chrome extension) web client as a drop-in replacement to the open archive plugin for chat history and it works ok.

 

For more details, see http://community.igniterealtime.org/message/225654

Now that WebRTC is now live in Chrome version 23, which is now rolling out to the public, expect some new exciting web applications with audio and video that will change the way we communicate beyond what we have seen so far with Skype and Flash.

 

The SIP community have already forged ahead with SIP over WebSockets and a number of SIP signaling libraries have started to appear. I am not convinced that employing heavy bloated SIP messages for WebRTC signaling is the best approach, but I am not ready to re-open an ancient debate.

 

Meanwhile at Ignite-realtime here, If you want to see what you can do with Openfire using the WebSockets and JingleNodes plugins, then take a look at my demo based on the Mobiecents SIP demo  I replaced their SIP JavaScript library with a simpler XMPP Jingle library. I also added support for Jingle Relay Nodes to bridge the media when both peers cannot communicate directly with each other.

 

 

Image1.jpg

 

To try it out, you would need two PCs with webcams both running latest Chrome version 23+. Use any two unique user names (eg user1 and user2) with no spaces and special characters. From user1, enter contact JID to call as user2@redfire.4ng.net/user2 or vice-versa.

 

For those interested in the source code, it is attached to this blog.

A few years ago, Matt and Gaston wrote a good introduction to XMPP Publish and Subscribe. You can read it here.  Back then, Pubsub was regarded the killer feature of XMPP looking for an application. Since then, Twitter has now shown us how to use Pubsub and many XMPP clients like Jappix now have realtime micro-blogging implementations using XMPP Pubsub.

 

I have been developing realtime business web applications that use Pubsub for a while now and my normal tools for such work would be Flex/ActionScript with XIFF to develop the Flash client and Java for the Openfire plugin. With Pubsub, I am not doing the classic client request and server response of multiple HTTP pages from a web server. It is usually a single page dynamically updated on each client responding to pushed subscribed events as they are published among clients. The server side code is just another publisher. Read this for more infromation about this approach.

 

Thanks  to Apple's IOS and Google's V8 JavaScript engine, Flex/ActionScript is now past tense and it is all about HTML5/JavaScript. JSON over WebSockets with NodeJs even makes a lighter and even sometimes, a better replacement for XMPP and Openfire plugins developed in Java.

 

Despite all the buzz about JavaScript as the single development language for client, server and network messages, the corporate market in which I do most of my work still has its user directories locked up in Microsoft Active Directories, all its mission critical data stored in Oracle and MS SQLServer databases and its enterprise applications living in Java bytecode. This means for some to come, Openfire, XMPP and all the application features (clustering, security, permissions, provisioning, user/groups, etc) will remain the choice application platform for my line for work.

 

However, I have to use HTML5 for developing my realtime web applications and the case for JavaScript everywhere is compelling. This lead me to investigate a JavaScript library or framework that I could use as my application platform and would be compatible with OpenfireJS on the server.

 

To cut a long story short, I have ended up with Backbone.Marionette. I seriously considered Derby and Meteor, but the heavy dependency on NodeJs was a show stopper.

 

Backbone has an XMPP Pubsub implementation in Strophe which I modified to work with Openfire.

 

In evaluating the myriad of JS frameworks/libraries, I used the excellent TodoMVC.

 

Image2.jpg

 

In the spirit of the TodoMVC project, here is a TodoMVC implementation of Backbone.Marionette with XMPP Pubsub all working in Openfire with no server side code. Pubsub manages the storing and distribution of the JSON data. Open as many application web pages. They are all showing the same data always in sync in realtime. If I wrote a server script in OpenfireJS, it would not be to respond to client requests, but to publish new Todos from an independent source.

By Tom Evans

 

A few of you more intrepid Openfire fans may have noticed a bit of recent activity in one of the branches of the Openfire SVN repository. Well, some of your fellow developers have been working behind the scenes to provide clustering support for PubSub, perhaps one of the lesser-known modules of our beloved real-time collaboration server. PubSub is an implementation of the XEP-0060 specification which extends the XMPP standard to add publish-subscribe functionality to the XMPP Core. However, if you have ever tried to use this module in Openfire, you may have been disappointed to discover that it was not designed to work in a clustered deployment. In fact, PubSub was forcibly disabled when deployed in a cluster! The main focus of the development effort was to address OF-205 and implement clustering support for the PubSub module. This work is now complete and the PubSub module is cluster-enabled and ready for action.

 

My Kingdom for  a Cluster!

 

However, during the course of this development effort, the team also took a critical look at the current clustering implementation itself (the "clustering" plugin). This solution is currently the only way to run Openfire in a clustered configuration (where multiple servers share the load). Unfortunately this plugin is inextricably tied to Oracle Coherence, an enterprise class (and enterprise priced) middleware component. A recent quote from Oracle put the price of Coherence (EE) at well over $300K for a smallish deployment ... clearly an untenable solution and incompatible partnership with the Openfire project.

 

We looked around for clustering alternatives that would have better affinity with Openfire, and landed with Hazelcast (Community Edition). Hazelcast is an open source clustering and highly scalable data distribution platform for Java. It enjoys a large deployment base and is licensed under the community-friendly Apache 2.0 license. There are also commercial licensing options available for deployments where professional support and enterprise security (among other features) are must-haves. This looked like a perfect fit for our needs, and likely for the Openfire community as well.

 

Where Two or Three are Gathered...

 

We are pleased to annouce the immediate availabilty of a new Hazelcast-based clustering plugin for Openfire. Starting today in the trunk of the Openfire SVN repository you will find the new plugin (/src/plugins/hazelcast/). Note that you will need to also setup the latest version of the Openfire core (currently 3.7.2-Beta) to use the new plugin.

 

We are looking for a few brave Openfire afficionados who can take the latest build and give it a whirl with your various deployment scenarios:

  • How many users and/or cluster member nodes do you have?
  • Which modules/components of Openfire are you using?
  • What is your typical JVM configuration? Preferred OS? Network topology (load balancer, LAN/WAN, etc.)?

Your feedback is very important and will help ensure that this new clustering solution will be a robust and stable component in the next Openfire release.

 

Those who have wrestled with the existing clustering plugin will hopefully find the new solution to be much simpler to configure and deploy ... and certainly much lower cost! There is a README file included with the new Hazelcast plugin that documents the basic steps for setting up an Openfire cluster, including links to the supporting Hazelcast documentation (if needed).

 

Testing ... Testing ... Is this thing on?

 

Please take the new build for a spin and report your feedback here. We will be posting an article to the main community page before long, but would love to have some initial feedback from the core developers before engaging a wider audience. No doubt there will be some bugs and configuration glitches ... can you help us find and fix them?

 

Thanks in advance for your consideration and assistance.

 

Cross posted from http://community.igniterealtime.org/message/224947#224947

 

--UPDATE--

 

I have added a slighlty modified version of hazelcast that is backwards compatible with Openfire 3.6.4 and Openfire 3.7.1. Unzip and copy the clustering.jar file to your plugins folder.

 

Please test and post your results at http://community.igniterealtime.org/message/224947#224947

Dele Olajide

Redfire-Phono

Posted by Dele Olajide Champion Jul 28, 2012

I use the Phono SDK from Voxeo to enable phone calls from a web page on a couple of projects which include the Inspired-Social (WordPress/BuddyPress) plugin for Openfire. I like using Phono SDK, but it is dependent on external cloud servers in the Internet that cannot be accessed in some situations.

 

Recently on request, I had to modify it to work directly with an in-house Openfire server to provide Jingle voice calls over the RTMP and RTMFP transport as well as SIP calls to PBX phones. The result of that work is now available as Redfire-Phono which replaces the Red5Phone softphone in the Redfire plugin for Openfire.

 

For more information about Redfire-Phono, see the latest release note for Redfire

Just posted version 0.4 of the Jappix plugin for Openfire  at http://code.google.com/p/openfire-jappix/

 

It includes:

 

  • Jappix Nemesis Alpha 2 [0.9.2~dev]
  • Quercus 4.0.25 PHP Java Engine

 

It also includes experimental support for:

 

  • WebSockets (needs Openfire 3.7.2)
  • WebRTC audio only with Jingle (needs Chrome Canary Version 22.0.1210.0)

 

It is compatible with Jitsi-Jingle  version 0.4 and Candy plugin for Openfire ver 0.4. You can make the following audio calls

 

  • Personal P2P Jingle voice chats between Jappix and Spark users

 

Image4.jpg

 

  • Group voice chats between, Jappix, Candy, Spark and SIP phones using an MUC

 

Image2.jpg

 

This is very experimental stuff and a few outstanding issues to be fixed to make WebRTC work properly with Jitsi-Jingle. use at your own risk

I recently started a project called jitsi-jingle primarily to add Jingle support to an audio-conferencing engine I was developing. It was also an opportunity to provide a replacement for the outdated smack jingle extension library (smackx-jingle) using the Jingle implementation of the active Jitsi project. This however turned out to be more challenging than it sounded as the OSGI dependecies in Jitsi and the internal abstraction model to support multiple protocols like SIP and XMPP made the code extraction difficult. There is a soon to be released libjitsi which should make the task easier, but I am running out of valuable time (I do this stuff in my spare time), so I had to choose another option for my immediate first release.

 

http://openfire-candy.googlecode.com/files/Image7.jpg

 

Fortunately, there are alternatives. I had used phono from labs.voxeo.com on the inspired-social project. It has a Java applet which first started life as the IAX applet and is now the backbone to the phono java library. It has an excellent media engine and even supports Jingle internally, but from Javascript. I on the other hand, needed a simple Java Jingle library.

 

Enter mini-jingle from Thiago Rocha, the original developer of smackx-jingle (while he was at Jivesoftware) and the architect of Jingle Nodes. Project mini-jingle is a simple, but working Jingle implementation minus the media engine which makes it a very good fit with phono-java-audio. By adding support for basic raw RTP UDP packets, the standard PCMU (ulaw) voice codec and the Openfire jingle-nodes plugin, I now have a working Jingle library for Smack that should work most of the time either directly peer to peer or via a Jingle nodes media relay plugin for Openfire. There will always be the exception cases of where UDP packets are completely blocked by a firewall.

 

In order to test the library and put into practical use, I created a Spark plugin for it as a replacement for the old outdated Jingle plugin for simple voice calls between Spark users. When it is used with the Candy plugin for Openfire, it offers Spark users, group and private audio-conferencing from MUCs with Candy web clients as well. Just click on the telephone icon in the chat room toolbar.

 

If you intend to use it, pick it up from here and remember to delete the old jingle plugin in both the Spark folder and User data folders. Both won't work at the same time

The ignite realtime community is happy to announce a plugin for the popular Candy (Chats Are Not Dead Yet) group-chat web client. Candy was developed by Michael Weibel (@weibelm) and Patrick Stadler (@pstadler) on behalf of their employer Amiado Group. It provides group and private chat for collaborative working between teams.

 

The Candy plugin for Openfire adds audio-conferencing and a telephone voicebridge service to Candy. In practice, it means a user can join a group-chat with voice using their PC mic/speaker or their SIP Phone. If the optional  PSTN/PBX gateway is configured, then an external mobile phone or home telephone can be used. The voice bridge can call the destination or accept an incoming call. The audio bridge is based on the jVoiceBridge project and can handle hundreds of concurrent users across multiple conferences,

 

The Candy web client has been extended by a voicebridge plugin which activates/deactivates either RTMP Flash or SIP audio-conferencing when user clicks on the  group-chat room/conference tab. Both group-chat and private chats are supported. It uses the XMPP Openlink XEP to handle the telephone messages

 

For more details read the rest of the overview document at http://community.igniterealtime.org/docs/DOC-2287 and visit the project web site at http://code.google.com/p/openfire-candy/

Filter Blog

By date: By tag: