Skip navigation
All Places > Ignite Realtime Blog > 2006 > December

Ignite Realtime Blog

December 2006 Previous month Next month
Matt Tucker

Roadmap Published

Posted by Matt Tucker Champion Dec 20, 2006

We just published a roadmap for the next releases of Wildfire and Spark. One thing you'll notice on the page is a list of the top 10 issues by votes. We make a big effort to prioritize new feature development based on user and customer feedback, so make your voice heard by casting votes!



The parody movie This is Spinal Tap includes a hilarious scene where the members  of the "loudest band in the world" point out that their amplifiers can actually be turned up to eleven. I was reminded of the scene as we've been doing scalability testing on the next version of Wildfire. It turns out we had little idea of how far we could push the limits -- we keep cranking the "scaling knob" louder and Wildfire just keeps scaling. 


So far we've hit 33k concurrent users with a single connection manager, running on a (old) Sun 280R server. CPU usage in the connection manager and core Wildfire server both hovered around 7% each. Those numbers are a pretty huge improvement over the previous version of Wildfire, which was barely able to hit 7500 concurrent users with maxed out CPU and memory usage. We're also only part way through the optimization process. The goal for the 3.2 release is to demonstrate 100k concurrent users on a single domain.


How did we get here? In Wildfire 3.2 we decided to replace our networking layer with Apache MINA, giving us support for asynchronous I/O and a foundation for better scaling. For testing, we created a Wildfire plugin that generates users, populates rosters and creates vCards. The rosters are populated with 20 to 30 contacts each. We've been using the load testing tool Tsung. Tsung is a master-slave tool, and for our tests we are using four slaves.


As we've tested ever higher scalability numbers, we've made lots of core code improvements along the way. For full details, see my forum post. The goal for this week is to reach 50K concurrent users. But based on our experiences over the past couple of weeks, that might not be much of a challenge at all.


Matt Tucker

Better Security With XMPP

Posted by Matt Tucker Champion Dec 17, 2006

Peter St. Andre recently blogged about the Jabber Software Foundation becoming an intermediate certificate authority. In his words:

"What: The ICA enables us to easily and cheaply issue real, RFC3920-aware digital certificates to administrators of Jabber servers http://....
Why: Easily obtainable digital certificates will result in more widespread use of channel encryption among servers and between users and servers, which will make the Jabber network even more safe and secure than it already is."

What's a standards organization doing handing out digital certificates? I don't think Peter is taking enough credit for the vision behind this move. The JSF is attempting something revolutionary: bringing strong security to an open internet-wide protocol. Let's look at the sad state of email security to see why this is important. The global email network is riddled with SPAM -- nine out of ten messages are junk. Email security and encryption technologies like PGP and S/MIME have never taken off, which means that for the vast majority of email, there's no way to know for sure who sent it or to encrypt it. Every user knows to only enter their credit card info on web pages with "https://" in the URL, so how come we accept virtually no security in email? The public IM networks like AOL, MSN and Yahoo are just bad given their own lack encryption support.


Providing free certificates is another layer of the XMPP security story and helps make security both easy and ubiquitous. It also shows a commitment by the JSF to not just create protocols, but to see them become widely adopted. The Jabber/XMPP community has always done things a bit differently than other standards organizations. We create standards using an open rather than closed process and use Open Source code to help foster interoperability. The JSF becoming an ICA is another example of doing things in a different and better way.


Of course, we're supporting the ICA efforts in Wildfire by doing everything we can to make certificate management as easy as possible. See Gato's latest blog entry for more.


Every time I've had to deal with SSL/TLS certificate handling in Java it's been hugely frustrating and time consuming. There's a lack of good information and examples published, and getting anything done requires using multiple libraries (the built-in JCE in some cases, Bouncy Castle in others, etc). That's resulted in a functional but not very easy to use certificate handling feature in Wildfire. The numerous threads in the Wildfire forum posted by users trying to create and import certificates proved to me that we had more work to do on this feature.


A guiding philosophy for Wildfire is ease of use and and certificate management should follow that same spirit. The current certificate management tools in Wildfire are anything but easy to use. The command line keytool application is used to manage the Wildfire certificate store to create certificates, generate signing requests and import signed certificates. Just to make things a little more difficult, the XMPP specification states that a particular extension should be included in certificates to specify the domain of the XMPP server. It's not possible to configure and set that extension using keytool, so the previous workaround was to use OpenSSL to create certificates. That's a major pain given that OpenSSL and keytool use different formats, which requires an extra format conversation step. No wonder users are having so many problems! Wildfire 3.2 does away with all this complexity by turning the painful process into a few clicks in a web-based interface.


The Jabber Software Foundation (JSF) has also been working to make the certificate process easier by becoming an Intermediate Certification Authority for the XMPP network. It will soon be easier (and cheaper) to obtain a signed certificate that provides much better security than a self-signed certificate. I'm happy to announce that Wildfire 3.2 will fully support certificates created by the JSF ICA.


I was hoping to show some screenshots of the new certificate handling pages in Wildfire, but they haven't gotten the Vanderzanden touch yet to make them pretty. However, the 3.2 release gets closer every week, so look for the feature soon.


We've been doing a lot of work integrating VoIP into Spark using Jingle and SIP (see Thiago's blog post), so of course we wanted to gather ideas from some existing user interfaces. The result? "Ahhh, my eyes hurt!" In my not so humble opinion, we found more people getting softphone UI's wrong rather than right. For your entertainment, I gathered together a visual tour of some of the clients we found.


One UI paradigm came up again and again, which is to make the softphone look like a physical object. I can imagine the discussions as this decision was made over and over again by various implementors -- "people know how to use their real phones, so we should make our softphone look like one." Check out the following screenshots:


[blog post| ne_middle.jpg|officeserv_softphone_middle.jpg][blog post| png|scrshot_softphone.png]


How a designer ended up with the examples above is beyond me. They're pretty hideous looking, waste a lot of screen real estate, and worst of all assume that users are so dumb that they need something that looks exactly like a real phone in order to figure out how to use it.


That brings us to a related category of UI designs. In these, the designers give up the idea of exactly duplicating a real telephone, but still preserve a tenuous connection by creating softphones that look like devices you could pick up and hold in your hands:


[blog post||soft phone.png] [blog post| g|zebra-softphone.png][blog post| d-advance.jpg|softphone-unlimited-advance.jpg][blog post||s oftphone_gr.jpg][blog post||s j-softphone.jpg][blog post|| dialer_v1_big.png][blog post||sof tphone3.jpg][blog post||sof tphone4.png][blog post||sof tphone5.jpg][blog post|| xtensoftphone.png]


These designs are a bad idea as well. When you're making a physical device, you create buttons for people to push with their fingers, keep the screen small to hold down costs, etc. Applying these same constraints to software on a computer (with a mouse and large screen) just doesn't make any sense. Why not show a list of people that the user recently called? A text area to type in is a lot faster than clicking buttons on a dialpad, so why not emphasize that? I could go on and on with more examples.


Not every VoIP client we found was bad. Skype does an ok job, and we like a lot of what we see in Gizmo. In a few weeks, we'll have the first version of our own softphone in Spark polished enough to start showing screenshots. At that point, we'll welcome feedback on whether we're as off-base with our UI as so many other sofphone creators seem to be.


Matt Tucker

Site Slowwwwness

Posted by Matt Tucker Champion Dec 12, 2006

Just wanted to write a quick note that we're aware of the the site slowness issues and the caching problems in the forums. We're working on the caching issues, and a new much more powerful server is on order. Ah, growing pains are fun.


Matt Tucker

Java 6 Released

Posted by Matt Tucker Champion Dec 11, 2006

Java 6 has been released (both Wildfire and Spark are built using Java). Normally, the fact that we use Java isn't relevant unless you're building plugins or making customizations to the code. However, the Java 6 release merits a mention for several reasons:

  • This is the first Open Source release of Java. That means that in the near future, you'll see Java 6 ship with many Linux distros. That will make it even easier to get Spark and Wildfire running on Linux.

  • Java 6 support for epoll on Linux fits in nicely with the scalability work we're doing for the next release of Wildfire. Linux users will be able to scale Wildfire to many tens of thousands of connections.

  • There are big improvements to Swing (the user interface library) in Java 6. That means Spark will be faster and prettier with things like font anti-aliasing, much faster re-draws, and quicker startup time.

You can install Java 6 now, or get it automatically bundled with the next releases of Wildfire and Spark.


A few months ago I took on a brand new JavaScript and HTML/CSS based project here at Jive. My experiences up to that point with JavaScript had been extremely limited. The only work I had done was through my work on Jive Forums and a little bit of time while working on Wildfire. In fact, when I started at Jive all of my experience up to that point had been in using Java, both server and client side; I had nothing as far as web experience was concerned.


The first thing I did to get myself started down the path where I could work effectively on the project was to get the book,  (also known as the "Rhino Book"). Everyone I talked to said that it was the book as far as learning JavaScript went and of course, it was featured in the Weird Al video, White & Nerdy. The book gave me the basics to get started, the syntax, and how to write basic things like functions.



Through my previous experience I had come across a Firefox extension called Firebug. Since my very first days learning Java I had learned how invaluable both a good IDE and a good debugger is to the development process. For developing JavaScript I wanted something comparable, to ease my development process and prevent me from pulling out my hair over silly bugs that an experienced JavaScript developer would find immediately. Firebug provided these debugging capabilities for me. As an aside, does anyone notice the pattern of awesomeness among products with fire-related names?

This is the Beginning of a Beautiful Friendship


The JavaScript library I am writing is somewhat complicated, the small scripts I wrote before were nothing in comparison. The value Firebug provides for me is allowing me to step directly through my program, see exactly what it is going on inside of the browser, and easily troubleshoot issues. It also provides a console, so, at any point during the running of the program you can print to the console using the following:


console.debug("This is a message.");


Any errors that occur, missing variables, missing functions, anything along those lines are logged to the console making troubleshooting extremely painless. What could've taken minutes or hours of dropping painful alerts in the code to see what was going on simply did not occur.


The newest beta of Firebug will also print out any and all HTTP requests happening inside of the browser. These print out in the console, listing all headers sent to the server, received from the server, the body content of the request and response and also the amount of round-trip time for the request.


I don't know how I would've survived without the capabilities that Firebug have given. It's proven itself to be an invaluable tool in my budding JavaScript arsenal -- so much so that we (Jive) just donated $100 to the project.


What am I doing with all of this crazy JavaScript? More on that soon...


Matt Tucker

Wildfire Product Review

Posted by Matt Tucker Champion Dec 7, 2006

Unix Review just posted a great review of Wildfire and Wildfire Enterprise. From the article:

"Wildfire really impressed me and I find it really hard to come up with a negative when it comes to this great IM server -- perhaps a Debian package would be nice for non-RPM distributions. Installation was a breeze as was setup. Working with the browser-based administrative interface is a joy. In fact, Wildfire is a beautiful, easy to use, configurable, customizable, extensible, and powerful instant messaging server."

The author also does a good job summarizing why companies should adopt XMPP (Jabber) as their IM protocol of choice:

"From a business standpoint, Jabber should be your clear IM choice. Because Jabber is an open protocol, it doesn't belong to anyone in particular, so there is no single company driving its destiny. Your business won't get locked down by proprietary formats. Jabber also uses a decentralized approach so the system is more robust. Best of all, any company can run its own private, secure, standards compliant, Jabber instant messaging server for little or no cost for the software."


Open Standards and Open Source

Posted by greg Dec 4, 2006

As we worked on Ignite we wanted to come up with a powerful core message for the community. We knew it was going to have something to do with both open standards and Open Source, but the real excitement came when we realized that it was the relationship of both of these forms of openness that we really wanted to focus on.


Open standards are an important part of any technology. From standard outlet sizes for electricity in homes to standard protocols on the Internet, open standards allow technology to reach more people and add more economic value than closed or proprietary standards. Open Source and open standards complement each other by increasing an open standard's relevance and by fostering interoperability between software projects. The XMPP community is a great example of this relationship and the benefits are visible in the wide range of areas that XMPP is used. Another favorite example is email -- once open standards took hold, email usage in businesses skyrocketed. Will the same happen with IM?!!


Open Source is a primary driver of open standards. Wide adoption is fostered by free and no cost implementations and APIs for open standards. Take the Smack API for example. It provides a simple API for Java developers to add XMPP functionality to their applications. Smack is used for everything from low-level application integration, event notifications from hardware devices, and desktop RTC clients like Spark. And because it is based on an open standard, integration with other platforms becomes easier since XMPP APIs are available in many programming languages.


Besides fostering wide adoption of open standards, Open Source also feeds back into the standards specification process. Open Source standards implementers contribute back to the standards process by recommending improvements, discovering ambiguities, and providing an implementer's perspective on the standard. The standard becomes stronger and easier to implement and everyone wins.


With all of this positive feedback streaming between Open Source and open standards it was only natural that the core message of Ignite Realtime be the promotion of the relationship between the two. Of course, we're a commercial company and not just about Open Source. We sell Wildfire Enterprise as a commercial extension to Wildfire and count on the fact that a good percentage of the community will find the Enterprise feature set compelling enough to make a purchase. We strongly believe that commercial software fits well into the Open Source/open standards relationship -- we think of it as the crank that keeps the wheel going (see Dave's blog entries for more on this). How do we visualize all of this? You can see our attempt as the image in this blog post, which can also be found on the Ignite Realtime home page.


Matt Tucker

Wildfire Integration Day

Posted by Matt Tucker Champion Dec 4, 2006

Two weeks ago (Friday) we had the first monthly Integration Day. Integration Day is when the entire Jive Software engineering team gathers to work on projects to integrate Wildfire and Spark with our other products (in this case, the upcoming product Clearspace). Bagels, pizza, and beer were enjoyed as well -- it was all very much in the spirit of some of our other dev team jamming days, Bug Day, and Sparkplug Day.


We considered two different architectures for the base integration:

  1. Wildfire connects to Clearspace using SOAP web services.

  2. Clearspace connects to Wildfire as an external component.

After much discussion, we settled on making Clearspace connect as a component. That means that Clearspace connects over XMPP to Wildfire and then exists as "" (if the XMPP server was ""). Just being connected as an external component doesn't buy a whole lot, so the next step was to build out some real integration code. The first step was to share presence data. We did so using two new Wildfire ad-hoc commands. The first returns a list of all current online presence information for users in the server. The second registers the external component to get a copy of all further presence changes. We're working on documenting these commands for the next release of Wildfire. With both in place -- voila! -- Clearspace knows the real-time presence status for every user in the system. I have a feeling this type of presence integration will be very useful to a lot of applications and not just ours. It's much more elegant than using the presence plugin, for example. The second integration component was to expose all Clearspace SOAP web services through XMPP ad-hoc commands. One of the Clearspace engineers (AJ) was able to do this with only a small amount of code using some clever XFire hacking.


These base integrations unlocked a lot of other feature developments, which we made good progress on. However, the next Integration Day is later this month so I'll wait until then to provide some more details, perhaps with some screenshots.


The sad reality is that the latest official release of XIFF (2.0 beta 2) is over a year old.  However, the project is far from dead. Regular forum readers know that Nick Velloff from Lymabean has bee doing active XIFF development and a new release should be coming soon.


Simplicity Accelerates Adoption

Posted by greg Dec 4, 2006

The fact that simplicity accelerates adoption may seem obvious, but it is worth pointing out. All of the projects on IgniteRealtime have simplicity and ease-of-use as one of their core goals. Igniting real-time collaboration is all about spreading the word about the productivity improvements and collaboration opportunities presented by real-time modes of communication. This principle doesn't just apply to the end-user experience either. Simplicity in administration and API usage allows administrators and developers to focus on solving their technical and business problems without spending unnecessary time on technical issues or awkward interfaces.


Why is accelerating adoption of real-time an important goal? I am looking forward to the day when RTC is as open and ubiquitous as email. This doesn't just mean having a federated set of IM addresses that everyone recognizes, though. That would be email parity. RTC ubiquity extends to seeing a colleague or partner's presence and being able to initiate the ideal method of contact, or combined methods of contact, simply and immediately. And making it easy to capture the output of the collaboration at the end of the interaction is just as important as starting the conversation in the first place.


I Just Like The Way It Feels

Posted by ddman Dec 4, 2006


One thing I've taken to heart during my years developing software is that one of the most important things you can do while developing a product is to think about how the end user ???feels??? when using it.  It's funny to think about, especially as an engineer, since most development is focused on implementing the low-level bits rather than UI or end-user experience. How are they going to feel when they initially open the app? How are they going to feel when using the app day-in and day-out, like we do with our IM clients? The real challenge for me was to finally accept the fact that I can't figure this all out on my own. Getting input from the community, both good and bad, is invaluable.Since Spark launched, there have literally been thousands of posts with feature requests. Some are as simple as adding extra emoticon packs, while others are a little more complicated (e.g. the ability to tie into any CRM application). When I initially saw some of these posts, I took the hard-nosed stance that adding these features was ridiculous (who needs a different way to smile?). But as I continued development on Spark and the feedback and feature requests kept rolling in, I realized people were taking time out of their days to give feedback because they actually cared about the product. Now when I say ???cared???, I mean that some emotion was evoked in them (good and bad), that motivated them to take the time to write up a request and send it off. That just blew my mind. Before Jive, I worked at Oracle and never had a chance to get this close to users.So you're probably now asking yourself, how am I going to benefit from Derek's realization? Well.. . one of the more obvious things to do is just make Spark feel better/cooler, or as Sam (who heads up marketing at Jive) might put it -- make Spark more lickable. There are two areas where we're pushing Spark in this respect. The first is the overall speed of the client itself. In the past we've been limited by the speed of the Java engine. With the Spark 3.0 release and the new and improved Java 6, known as Mustang (responsiveness of the application.



The second improvement area is in the UI. I really wanted to go down the road of having Spark feel light and sexy.  For the ???light??? part, I updated the parent frame that Spark runs in to be thinner and sleeker.  Take a look at the two screen shots below, the first one is the current released version of Spark and the one below it is just Spark with a thinner frame (although we're still playing with the final look).  Oh la la.






[| /2006/12/old_spark.png|Old Spark] [| /2006/12/new_spark.png|New Spark]



For the sexy part, I've updated Spark to fully support themes within the chat window. By following  a standard HTML template, organizations can now change the UI of their chats or select from a set of defined themes. The same thing applies to the new Adium for paving the way for us. We're actively working with them to define a cross-platform theme engine, but more on that in another blog post. An example of the new features are below.



[| /2006/12/chat_window.png|Chat Window] [| /2006/12/preferences.png|Preferences]









As a side note, I would like to say that this is my first blog entry ever.  I look forward to talking smack as time goes on.









As I'm writing this entry, we're in the last days before launching igniterealtime and I'm waxing nostalgic about the project that started all of our real-time efforts, Smack. It was four years ago that I looked around for a good Java XMPP client library and came up short. There were some other libraries out there, but most of them assumed that you already knew the in's and out's of the protocol and I was a full-on XMPP noob at the time. The goal for Smack simple: create an API easy enough to use that even someone  like me could figure it out.


We've stuck with that focus in each subsequent Smack release, even as the functionality has grown richer and richer. In between torrid work on the next Wildfire and Spark releases, we've slipped in work on Smack 3.0 (partially with the generous help of community members Francisco and Gabriel). Some of the major changes that are coming:

  • Uses JDK 1.5 for cleaner syntax and simplified code.

  • Support for re-using an XMPPConnection when the TCP/IP connection fails, including an auto-reconnect feature.

  • Rounded out support of the XMPP RFC's through better error packet handling and privacy lists support.

  • Ability to listen for new chat requests, which I felt was one of the major remaining design flaws in the library (thanks Alex!).

  • Tons of bug fixes.

There's no planned release date yet, but we've been using the latest code in Spark for quite some time.


So, a taste for where it all started and where it's going. But the really interesting part for me is where Smack is being used. Have an interesting application of Smack? Please post a comment to tell us about it!


Filter Blog

By date: By tag: