Instant messaging challenges

State of The Art

Instant messaging applications are traditionaly designed to require continuous access to a centralized server. The level of dependence on that central connection varies, with some protocols (e.g. AOL's Oscar and Yahoo's YMSG protocol) allowing clients to establish peer to peer connections for message and file exchange, although an established connection with the centralized server is still a requirement in most of those scenarios. And even in protocols that managed to seperate presence establishment from a continuous TCP connection (e.g. SIP/SIMPLE), access to the centralized server is still required to authenticate users, store and retrieve buddy lists, subscribe to other users' presence, and provide message routing between the different connected clients, without which none of the basic IM fuctionality is usable.

This approach was very beneficial for the development of instant messaging applications and protocols. It allowed instant messaging providers to provide a relatively high quality of service (you always expect your messages to go through, and get angry when your buddy appears online and doesn't answer those messages), while at the same time offering a large set of features that proved very useful in making instant messaging much more usable, and allow for the transition from the home PCs of teenagers to the busy desktops of Wall street brokers. Some of those features made possible through centralization is reliable immediate presence, offline messaging (Yahoo, ICQ, Sonork), roaming buddy lists (a problem email clients and servers are still struggling to solve) to name few.

Instant Messaging And The $100 Laptop

The challenges become very clear when you start comparing the networking environments in which the $100 laptops will be operating to the average reliability of an internet connection in a community in Maine or France. The $100 laptops will typically be used in environments where internet connectivity is very scarce if available at all. Laptops will also often be connected to a set of other laptops in a neighborhood through a mesh network, but with no access to a central school server that could help broker instant messaging communication.

To make the situation even more challenging, the expected quality of service from an instant messaging application is typically higher than other disconnected means of communication (e.g. email) due its synchronous and conversational nature. Presence also starts losing its usefulness fairly quickly as it starts getting inaccurate. The whole value of presence, is providing a user with the ability to optimize communication based on pushed presence information. You wouldn't try to send someone a message asking them a quick question, asking them to pass by, or quickly execute a transaction if they appear as away or offline.

The list of challenges that will need to be solved to allow instant messaging to operate in such an environment is simply the list of basic features that are offered by centralized servers today, and to which peer to peer or distributed alternatives will need to be developed.

One important thing to note before listing those features, is that any solutions developed are not supposed to replace a centralized instant messaging server. A centralized server is still required, at a minimum, to provide centralized-by-design services such as provisioning and trust establishment. It is also important to note that an end solution would probably work best if it utilizes a centralized server when one is available, and utilizes new techniques for connectivity and presence propagation when one is not available.

Also, an end solution does not necessarily require protocol modifications, or the development of a new set of tools. Solutions could be found by rearranging the tools that exist today in terms of function or the way they are deployed. This approach, given the need for reliability and immediate scalability (given the timelines), is deemed preferable. Although the end solution will most likely consist of a mix of those different approaches. I will not delve into what the solution would look like in this write up, leaving it to another write up and to further discussion as similar problems will need to be solved for other types of communication applications. This write up is intended to stimulate the discussion and provide a generic framework for thinking about the problems and their solutions.

Challenges

• Authentication: This one of the most fundamental features a centralized instant messaging server provides. Even in the most distributed of peer to peer application servers, authentication is almost always done through a centralized server (think Skype). The reason this is so, is because authentication is very tied to the provisioning functionality for which the server is a natural and secure host. This doesn't mean though, that end user authentication cannot be temporarily delegated to end points through distributed trust mechanisms (think public keys and kerberos as approximate models) and cached for in between connectivity to the centralized server.
  Another simpler, but at least one level less secure, is the association of user identities with hardware or network uniquely identifiable attributes. This options becomes much simpler if the solution is to be built on top of an IPv6 infrastructure. Security as noted though is a big risk. Given the state of email today, such a system will not necessarily be much less secure than email, but it will be a step backwards in terms of challenges that IM solved much more effectively than email, namely spam.

• Buddy List Management and Storage: Roaming buddy lists are one of the features that almost all instant messaging protocols implement today. They allow a user to have the same view regardless of which machine or client they log in from. In addition to storing a list of buddies whose presence the user is interested in monitoring, buddy lists store more and more information related to user's capabilities, preferences and privacy controls. They could be overloaded to store additional information that would help solve some of the other challenges, such as public keys.
  One additional challenge that needs to be addressed is offline buddy list management. What happens if you try to add a new user when you are in disconnected mode? One possible and totally justifiable approach would be to not allow offline editing of buddy lists, especially that no authentication facilities exist while the user is disconnected from the server. If security is less of a concern though, then it would be possible to allow for disconnected addition of newly discovered users, whose credentials could be verified through the central server the next time the user is connected. Some format of versioning would need to be used or re-used if offline editing is to be enabled.

• Message and Presence Routing: What used to be a presence and message routing challenge when implemented by the central server, turns into a discovery problem when in peer to peer disconnected environment. The client will now be responsible for polling (discovering) the presence of other users on the buddy list, and thus have enough information to establish peer to peer communication for presence updates and message delivery.
  Multicast-DNS, presence broadcasting, and network id association (targetted polling) are some of the possible approaches that could be taken to address this challenge. Which one is to be used will depend to a great degree on the capabilities made available through the chosen networking infrastructure. Network id association might be the best approach when the ids are unique and easily discoverable in an IPv6 network, while multicast-DNS might be the best approach in a NATed IPv4 network.

Bridging to traditional IM nets

For the case where the network has an Internet connection and the members wish to connect with "outside" IM networks, using the hosted IRC/IM gateway through bitlbee works well. You maintain an IRC connection to im.bitlbee.org (some IRC clients and servers support tunneling over SSL for security), which is much lower bandwidth and reasonably rugged against latency than most IM connections. The gateway server (MIT could host one, it's F/LOSS based on the GAIM code, so it keeps up to date with protocol changes), after an initial walkthrough, saves your buddy list and some preferences (using a password) for the major IM networks and acts as a gateway. All your IM buddies appear as users in a private-to-you chatroom. File transfers work, though voice obviously does not.

This was invaluable as the fragmentation and non-interoperability of the IM networks often requires you to have multiple clients, or (if you use Trillian/Miranda/GAIM) still multiple bandwidth-hogging connections.

There's also meebo.com, but I expect that to be laden with ads and slow on many computers.

There's also a GAIM plugin for Apple's iChat using the Bonjour protocol located in GAIM's CVS. The Bonjour protocol uses multicast DNS to automatically discover other Bonjour users on your local network without using a centralized server. See the discussion to find out more.

PSI is much smaller than GAIM. GAIM supports ICQ, MSN,... but the transports (including IRC) can be done server side. There is also a PSI Version in development that supports Jingle (JEP-0166) for VoIP.

Proposed approach

• Serverless use must be possible
• XMPP / Jabber is the protocol of choice
• XMPP's serverless mode ( http://www.jabber.org/jeps/jep-0174.html ) must be used as an option.
• Psi or any other jabber client will work fine

Thread in xmpp mailing list: http://thread.gmane.org/gmane.network.jabber.standards-jig/11274/focus=11274

Discussion

All of this is being discussed further (and in part even questioned) on Talk:Discussion of Instant Messaging Challenges.