Talk:Instant messaging challenges

Keep in mind before posting:

• Sign your posts. It makes figuring out whos who much easier.

See also SugarPresenceRefactor as it is related.

Mesh network part integrated into an IM system?

March 27-06: SoreThumb: "I may admit to be a bit ignorant of the situation even after reading the article, but sometimes outside and somewhat-ignorant thoughts offer new insights: Why not have the mesh network part integrated into an IM system specifically for the OLPCs?
"So to speak, each computer would have a unique serial #, and each computer would store the custom username.. If you can turn it on and log in, you technically would have the rights to using the IM name linked to the Serial.
"Also, it may be CPU or network-heavy, but when each computer signs onto the network, it obviously has to poll the local area for available OLPCs, right? Why not have this as signing-onto the IM system: all OLPCs recieving the network connection would send out notification of presence like a ripple from a pond.
"In any case, this network could technically not be that secure. If I'm not mistaken, you'd be sending your IM across other computers until it reaches its destination, with possible party-interest. Perhaps a global chatroom?
"I hope I adequately discussed the possibilities of a possible system." - NBST 2006

I think Sonork [1] would be excelent for the system. It has not only offline messaging, but offline File Transfer!

I think that what is needed is something that will work well over a peer to peer system. Probably what you want to have happen is that a person wants to orignally use a system they would have to connect up a centralized server. When they connect and get a userid, it would then send data back to the client of a signed digital certificate. Aka, when I sign up I would get a signed digital certificate saying that I am profdeadmeat signed by the server that we all know. Probably this should use the Jabber protocol. When we are using a regular non supported network of peer2peers then what would happen is that machines would query machines around them for instant messanging.
   It looks like the Zero Conf Group and the mDNS/DNS-SD groups may have figured parts of this out. A product that does some of what you are talking about is Bonjour ichat which is Apple's name for ZeroConf does do peer2peer chatting.

 Note: some of the above has moved to the main page if you need to speak to me contact me online
  via AOL or email me via gmail to my account.

Somebody commented: Linking a user account to a physical machine is very restrictive and is the wrong representation of reality. What you would be saying is that the identity of the machine and the identity of the user are one, which could be temporarily passable, but is a bad long term strategy. Think what will happen when the LCD breaks and the kid gets a new laptop.

A teacher should be able to register a hostname for a laptop, the kid may want to have its old laptop hostname back, like for instance superlaptop.laptop.org. So the hostname may very well be flexible enough to carry identities. In the long run XMPP needs an extension to provide for permanent redirect to a new identity, so should the laptop really die without replacement, the kid, by then a woman maybe, would make her old laptop hostname point to a redirection service which finally sends traffic to her new identity, which could be something like superwoman@fao.org. The advantages of having fully federated hosts on each laptop clearly outweighs the shortcoming of having an identity temporarily tied to a hostname. --lynX

It is reasonable though to use a machine's address and assume that it is "temporarily" associated with a user's id. IPv6 makes this much easier because of the partial mapping between of mac addresses to IPv6 IPs. This would be similar to the way in which a user's public key is cached to allow for p2p authentication. This will simplify the discovery process and could (more importantly) reduce network traffic.

Another point to make is that the scale and reach of discovery needed for mesh networking is very different than that for an IM network overlap only partially. Which means that the minimum discovery area needed to cover both will be larger than both, which leads to the undesired effect of more network traffic. In simpler words, when you start a mesh network you only need to find your immediate neighbors, when you start an IM network, you need to find a specific set of users in your body list, regardless of whether they are your neighbors or 3000kms/10 hops aways.

Problems of Serverless Mode (superceded, see below)

5/16/2006 - khassounah - thoughts on proposed approaches: XMPP is most likely the best option out there. It is widely supported, mature, and is the most extensible option. The challenge though is that the current proposed approach for serverless XMPP has limitations that make it not suitable for HDLT's purposes (yet) and introduces some serious security issues.

It doesn't provide anyway for users to authenticate each other for instance. This means that the proposed protocol is introducing a huge security flaw that email suffered from for ages (think smtp). Suddenly, any user with a client can impersonate any other user and send message to people connected to them. This is not a huge issue in a physical link-local network (it still is, but not as critical), but is more of an issue in a logical link-local mesh network where any wifi-enabled computer across the neighborhood can impersonate any other user, and where a child can receive a message that appears as if it's coming from their friend when it is actually coming from a pervert.

The severity of this issue becomes greater when we realize that these children will have addressable ipv6 addresses. Unless you want an application to worry about layer 3 addressing and differentiate between local addresses and global ones. Which means access to those children is not limited to neighbors.

The other issue is that it assumes that access to the server and the peer to peer mode of operation are mutually exlusive (at least by design not statement). Which is valid when thining about link-local networks, especially that link-local addresses (by specification) are not to be used in parallel with normal network addresses (except in a limited way while in transition). The implications of that are that smooth transition between the two modes when you gain and loose connectivity to the server is harder to achieve, and that you can't use peer to peer connectivity while you have access to the server (which might be a considerably more efficient in some cases)

These are not necessarily flaws with the proposed standard, since the standard seems to have been created to provide a simple presence-enabled paging system that makes use of link-local connectivity. What is needed though is more of a peer to peer alternative to server connectivity, and which could still function as flawlessly and as "securly" as a server-based connection.

5/24/2006 - philbo - why not email?: I can't see how usually-disconnected IM is any better than email, and might actually result in a worse user experience because most of the assumptions behind IM are violated in a disconnected scenario. How is severless XMPP any better than an email client with a buddy list style address book and nice threaded message display.

6/2/2006 - khassounah - email v.s. server-less IM:Disconnected here meant disconnected from the server not from each other. A better name would have been peer-to-peer. We are talking about a situation where the kids are in reach of each other, but only of each other. They should still be able to instantly communicate

Messaging Servers On Each Laptop

Totally distributed. Completely independent from the Internet.

• centralistic? no no no. distributed. you can even call it federated. yes please!
• if every laptop has a unique and reliable IPv6 address and DNS name, it can run its own messaging server (be it ejabberd or psyced or whatever) and host all of its local users.. mums dads sisters brothers grannies grandpas uncles and aunts.
• given this architecture, all client apps can reliably talk to their home server no matter what the network situation is, because it is running as a daemon on the same machine.
• only the server needs to figure out when it is in reach of other servers, or when it isn't. this may be the whole big internet, but it may just as well be just the next ten laptops within radio reach.

to this purpose we need a special hook that will pass the information of available stations from the radio drivers of the kernel to the messaging daemon, so it can get in touch with them.

• yes philbo, why not also email. email has a decades long tradition of dealing with temporary link-ups and offline messaging. think UUCP POP3 etc. considering that a network of laptops will want to exchange email even if the internet is unavailable for a week, good ole UUCP technology is the way to go when it comes to email. but then again... SPAM!

--lynX 10:00, 25 May 2006 (EDT)

6/2/2006 - khassounah - messaging servers: I don't see why not, but this is really an implementation detail. Whether you call it a server or a client accepting connections, at the end it is whether one machine can connect directly to the other or not. If you are talking about registration and user management though, then that's a hairy question, and needs to be thought out carefully. The reason is that you want a system to work both locally and globally, and there likely will need to be more than one solution combined.

XMPP clients cannot operate p2p, they need their servers. And it is good to have a daemon seperate from a client, because there are hundreds of clients, and you don't want to teach them all how to do p2p. You need to have a daemon on those laptops to handle the p2p nature of networking we're facing. Registration is no issue, because with a fully distributed IM system, each user has the name of its laptop as part of its identification (xmpp:ronaldo@ronaldinho.laptop.org, psyc://ronaldinho.laptop.org/~ronaldo, whatever). You don't need a single nickspace for OLPC. every laptop has its own nickspace. --lynX 03:20, 23 June 2006 (EDT)

Just to give you a taste of the design challenge: if I have a server on my machine, then why do I ever need to login to the server on your machine? is it that not all machiens have servers? in that case how will anyone know where I'm logged in at anytime, especially that it will have to be dynamic since your machine might not be always on or in sight (from a networking perspective), so I should be able to login to another machien available. the challenge then is where does user x find me? if they want to subscribe to my presence and be notified when I come online, which server do they login to?

You never log into my machine, because it is mine, not yours. But my WLAN card tells my server, that you are in reach, so it can show me your name on my buddy list and vice versa. If I want to receive your presence, then I make friends with khassounah@khassounahs.laptop.org. Whenever your laptop comes into radio distance with mine, our daemons will start talking to each other and we see each other in our clients, no matter which brand. I don't need the Internet to be able to IM with you. All I need for you to be in my radio neighborhood. If you're going to do buddy lists through a central server in the Internet, then nothing will work when the Internet is down. Why should we do this? There is no need to. Unix laptops can run their own p2p Internet over WLAN. --lynX

You are trying to solve two problems at once: discoverability and robustness. Any solution needs to solve both problems. Skype is very robust, but has serious discoverability issues. Most others (MSN, AIM, Yahoo, Jabber) are discoverable, but extremely rigid (no internet, no IM).

Trying? I don't see how the plan should fail. And using some company's technology won't help in a situation where the Internet isn't easily available and even if, it may be a luxury we might not be able to afford. The least we depend on it, the better.

2006.06.03 - regek - server discoverability: Why not just use something like OSPF to let the servers discover one another and assemble their own peer-to-peer network? Companies such as Cisco and Juniper have spent vast amounts of time and money developing efficient methods of discovering peer nodes and connecting them to one another. The easiest system I see would be to use a link-state routing protocol to enable the laptops to discover each other and route messages. Assign each username its own IP address. You wouldn't ever need to log into servers on other machines.

Each individual machine gets its own IPv6 /64 (or whatever size you want) address space and is responsible for all of the addresses inside that. Each time a user sees another, the username/address mapping is stored for X amount of time. If the two users never talk, the mapping is flushed to make room for newer ones. If they do talk, then the mapping gets stored somewhere more permanent.

Basically, I'm trying to say that to solve the issue of "no internet, no IM", you could make each of the devices into a router. Sure, there would be some overhead, but it would be nice and decentralized without too much new work being required.

Thanks regek, that sounds like an option. If I'm not mistaken however the WLAN routing drivers of our cards already implement neighborhood discovery, so all we need to do is pass the event of a machine plugging in or out to all the applications that need it, which in this case is our IM daemon. It's probably a few lines of perlscript (or whatever) that do the glue. Maybe somebody has already implemented this in fact. --lynX

2007.11.28 - elcman at gmail: I'm just randomly poking around the site to see how IM is being implemented with the XO. I can imagine that I'm quite late to this discussion, but seeing the issues around serverless IM is more than a little daunting. Here are some disjointed thoughts I had while reading through this discussion:

• Security model based on proximity
  • local (classroom, "known legitimate"), green or selectable
  • neighborhood/villages (small communities, less secure), yellow
  • "global" (worldwide communities), red
• Using a "RAID 5"-like striping of user data
  • A stripe of all local clients and their public keys
  • determining correct identity of each client based on a passive check of local, neighborhood to potential global repository (as available)
  • would require an eventual use of a public key/client identifier repository (local Teacher XO, global repository, GPG key server?)
• Treating IM like a p2p message delivery system, as has been discussed
  • Meshed XOs could passively save messages to be delivered according to historical activity and proximity.
  • Message could expire with notification when message cannot be delivered the message would be delivered to originator after a set amount of time (days, weeks)

Distribution: The idea would be that, like a p2p network, it would passively find how many clients could validate a client and its public key. Ideally validating the identity of the client users with those who know the client's user (as a local classroom collaborator). It would be a simple, and hopefully effective way for peers to know whom they were dealing with their public key should be created for each user, not just each client since the laptop will likely be shared.

Authentication: You would always have to acknowledge that you might not be talking to whom you expect unless you met the person and could validate that they are who they are using a simplified tool.

The server-less server: The initial "striping" of all local clients. There would be some laptops that are most consistently exposed to many laptops at once. Historical data, which would likely be an XML file consisting of client, user, user's public key, and shorthand for user's activity with expiration details, which will show that a teacher's laptop consistently has access to all laptops on a regular basis making it a pseudo-keyserver. Using this information, the teacher's laptop would be best to deliver delayed messages since both XOs regularly get in contact with the teacher's XO. It isn't fool-proof, but it would give the most consistent results if direct connect is not available.

SPAM, E-Mail, Messaging

With young people in the world e-mail is slowly going out of fashion - it is considered too formal and at the same time too SPAM-prone. They turn to massive instant messaging and web-based communication platforms like myspace and friendster. Both these technologies are typically proprietary. We can't and shouldn't keep anyone from joining in on this, but we can take our new users by the hand and offer them a friendly non-proprietary SPAM-unlikely technology to head out with, first. The obvious candidate would be Jabber/XMPP, this raises questions though:

• Is the protocol too verbose for Real World Internet bandwidths?
• Can it be extended to serve as a spam-free E-Mail replacement?
• Does it have enough profiling and social networking fun aspects?
• Is it architected well enough to handle the load?

Myself I am working on a protocol which does address these issues, and is gateway-compatible to Jabber and E-Mail, but I wouldn't dare to say we're ready to step into these shoes. From an insider's point of view I know that using XMPP to these purposes will be tricky at best.

--lynX 08:57, 25 May 2006 (EDT)

GSM SMS a solution!? (superceded)

Where is the creative thinking???

Quite frankly, this whole IM discussion is silly. You are all thinking like Americans who sit behind desktop computers in the office all day. Have you ever thought that this problem might have already been solved somewhere else where people do not live in the same technology environment as you do?

I already have an OLPC IM prototype that has been functioning for the past 7 years for me. It has existed longer than that, but I've only used it for 7 years. When I write an IM, it goes into my "To be sent" box. I select the destination from my "Buddy List" or I type in the raw destination address. On my OLPC IM device I also have a Python interpreter and I use a Python app to search a Russian-English dictionary that I have installed on my 512 megabytes of memory.

The device is called a Nokia N-70 and it is a typical European GSM mobile phone. The IM service is called SMS and it functions without any persistent connection to a server khim:only because mobile service providers spend millions of dollars every single day to support persistent connected central servers. The "Buddy List" is called the "Phone Book".

The CPU power and storage capacity of my Nokia N-70 are quite similar to that of the OLPC. My mobile phone is not always "Online". Sometimes I am in an underground train or I might even turn it off. But when I connect again, the SMS messages pour into my Inbox and I can deal with them. khim:This all is possible only because mobile service provider keep everything on central servers when phone is disconnected or turned off.

Not true Khim. All the messages that I have sent and received are stored in my mobile phone. The central SMS system only transports messages. It does not keep them after they are delivered to the recipient.

The Motoman service in Cambodia is probably even a better model. http://dailywireless.org/modules.php?name=News&file=article&sid=2003

Note that this blurs the lines between e-mail and IM. Since the OLPC is going into an environment where western-style Internet connectivity will not be available for a long time, I think that we need to accept the fact that IM on the OLPC will be more like Pony Express email than like traditional American AOL/Yahoo/Google/Microsoft IM.

lynX, who forgot his password and isn't getting e-mail from the wiki, would like to comment:

Alternate technologies need the adequate infrastructure. I doubt GSM is available in all involved countries at affordable prices. You just ignored the point that SMS cost money, and in our constellation even GPRS or UMTS are too expensive to allow for laptop.org chat and file exchange, unless you can convince providers in those countries to give away GPRS for free to everybody (How could you limit it to OLPC laptops?). That might work in Estonia, but not where these laptops are going. Also I dare to suppose that the biggest part of the social network of these children is in close geography, so chat and file exchange based on wireless LAN technologies that need no outside support sound like a quite good idea. And yes, I am European too.

Memracom replied:

lynx, I'm not saying that we should use SMS mobiles instead of IM on the OLPCs. I'm trying to point out that there is a widespread working model of IM that does not use longlasting connections to servers and it works. khim:It works real well in my imagination without any central server or persistent connection, too bad it does not work this way in real life. IM on the OLPC does not need all the clutter of the American centralized IM model. In fact, IM on the OLPC needs to move even further than the European SMS model and function with no servers at all.

For example, a kid should be able to send an IM direct to another kid's device. The "buddy list" should automatically populate with any devices within wi-fi connectivity. But, at the same time, the kid should be able to address non-connected machines to be delivered later or to be relayed via relay-points. I think I will write some Use Cases on the main page.

<lynX> yes yes that's what i've been saying, and the technology is there, we just need a little glue. khim, why do you think it doesn't work in real life? have you tried linking wlan discovery to xmpp or psyc servers?

5/16/2006 - khassounah - connected or not: It's not as silly as you think. Kids will each have their own laptops, and will be connected often. If your friend is not online, you send her an email or an offline message as lynx discussed.

<lynX> Maybe in the first years some nice company will sponsor some Internet dialup, but consider we are going into regions where there are no ISPs, where doing Internet means going through satellite or tremendously old flaky and expensive phone links. I cannot believe it is a good idea to make communication between the laptops dependent from the existence of the Internet, especially if it only takes such little effort to avoid it.

<Old Joe> Assume we can get the cost of each laptop down to $100. 40 kids in a class, 12 classes in a school so about 500 laptops per school, worth $50,000. This is a minimum, many schools will have 2 or 3 time that. If we can find $50,000 then we can find enough for an internet connection as well - especially since for maybe half the schools (in urban areas) this will be a simple cellphone modem. The problem will be bandwidth and reliability but there will, I'm sure be something.

<lynX> It is good there will be occasional Internet link-up, even if at expensive rates and low bandwidth. Still, since we have a realistic chance of making a wireless messaging backbone with or without uplink, why should we go for anything more dependent on expensive infrastructure?

That is exactly how SMS works.

<lynX> Don't say that. Thanks for the theoritical model of SMS. It is nice that SMS may appear like magically p2p, but each SMS is in fact an ISDN phone call with dialing and ringing and everything before those 160 bytes can be pushed to the other side. So from a technical point of view, SMS should never have existed in the first place. It's just a hack because the bellheads didn't want a packet switched network to run digital telephony on. Nowadays we have packet switched networks (GPRS, UMTS) so SMS is definitely ready to be seen as a piece of history. In the western world the big telecoms can continue to keep the end user under the oath of SMS, as it is a fabulous cash cow for them, but in the 3rd world we have no cows to cash, so let's start using state of the art technology!

SMS is peer to peer?

6/16/2006 - khassounah:

I copied this from the main page:

This summary betrays a strongly American-centric viewpoint. In Europe, instant messaging has been widespread since the late 1990's without continuous access to a central server for either sender or recipient. It is called SMS and is a component of the GSM cellular service used by hundreds of millions of people outside the USA. The OLPC is more like a mobile phone in Europe, than it is like a desktop computer in an office in the USA. Currently, my mobile phone has Internet access when I want it, and a Python interpreter. My main Python app is a Russian-English dictionary that allows me to type Cyrillic on my phone keypad (T9) even though the phone does not support Cyrillic directly. It is the Nokia N-70.

SMS (at least protocol wise) is not peer to peer. The message goes over an established wireless connection to a server that then routes it to the right cell (or network before that) before it is delivered to the destination phone. In that sense, it works exactly the same way as any centralized server-based public IM service.

If it was truely peer to peer, then you will be able to send someone an sms message if both of you are sitting across from each other, but neither of you has a tower signal on yours phones.

Bridging to traditional IM nets

For the case where the network has an Internet connection and the members wish to connect with "outside" IM networks, using the hosted IRC/IM gateway through bitlbee works well. You maintain an IRC connection to im.bitlbee.org (some IRC clients and servers support tunneling over SSL for security), which is much lower bandwidth and reasonably rugged against latency than most IM connections. The gateway server (MIT could host one, it's F/LOSS based on the GAIM code, so it keeps up to date with protocol changes), after an initial walkthrough, saves your buddy list and some preferences (using a password) for the major IM networks and acts as a gateway. All your IM buddies appear as users in a private-to-you chatroom. File transfers work, though voice obviously does not.

This was invaluable as the fragmentation and non-interoperability of the IM networks often requires you to have multiple clients, or (if you use Trillian/Miranda/GAIM) still multiple bandwidth-hogging connections.

There's also meebo.com, but I expect that to be laden with ads and slow on many computers.

There's also a GAIM plugin for Apple's iChat using the Bonjour protocol located in GAIM's CVS. The Bonjour protocol uses multicast DNS to automatically discover other Bonjour users on your local network without using a centralized server. See the discussion to find out more.

PSI is much smaller than GAIM. GAIM supports ICQ, MSN,... but the transports (including IRC) can be done server side. There is also a PSI Version in development that supports Jingle (JEP-0166) for VoIP.

<lynX> Nice to see good old IRC never dies, but I think we should not confront kids to learn to use both IRC and XMPP clients, and BitlBee requires an IRC client as its end user interface. Funny for me to advocate a Jabber solution (Read my PSYC wiki and you understand how much criticism I have for Jabber/XMPP, yet it seems the most viable way for this problem), but wouldn't it be possible to install so-called transports on the user's laptop capable of logging themselves into any proprietary messaging service once the Internet comes available? That would neatly integrate into the Jabber clients. A different kind of solution would be that kids who want to use proprietary IM would have to install a multi-protocol client like gaim to be able to do both proprietary IM and distributed Internet-independent IM with the same user interface.

Serverless Mode (XEP 0174)

On the main page uls and Memracom wrote:

No I didn't --Memracom 16:02, 4 July 2006 (EDT)

In fact, if stuff is wrong or irrelevant then just delete it. No point in trying to guess who said what. The main pages should be a blend of many people's work.

Even on the talk pages, junk can be deleted and important dangling points can be extracted and restructured to make discussion more organized.

• XMPP's serverless mode ( http://xmpp.org/extensions/xep-0174.html ) must be used as an option.
• Existing Jabber clients that support serverless mode include iChat and Gaim. Support is being added to Psi. Other clients will probably add it in the future.

Thread in xmpp mailing list: http://thread.gmane.org/gmane.network.jabber.standards-jig/11274/focus=11274

I have removed these limitations from the main page as they are not necessary and incorrect. Serverless mode requires DNS servers to be available for SRV resolution, so you might just as well use a centralized IM server then. Instead wireless infrastructure will provide us with node discovery which needs to be passed to localhost IM servers. Each laptop needs to run its own IM daemon (see above), so it can get in touch with every other laptop no matter if Internet is available as a whole, and is capable of providing the full set of protocol features a server provides to a client. This allows for any client to work.

--lynX 06:48, 23 June 2006 (EDT)

DNS servers are not required -- read the zero-configuration network specifications for details.

--Stpeter 19:44, 31 July 2006 (EDT)

Updated JEP-0174 to XEP-0174 above. Note that https://guardianproject.info/apps/gibber implements this on Android.

--devrandom 23:43, 19 June 2012

Using distributed hash table technology

CSpace

I'm not sure if you guys are aware of this, but CSpace is a decentralized secure open-source chat platform that incorporates many of the ideas that have been discussed here.

Perhaps it would be easier to build off of this?

-Shrimpster -E-Mail =>cyberguy34000@gmail.com

<lynX> I'm looking through this. Interesting piece of work. Having user resolution by pubkey instead of URLs is interesting, but I don't see how this can work when you only have a couple more laptops in reach on your wireless LAN and cannot consult the big DHT network. How will the technology figure out, those keys (people) are already in reach? Also, how is the event of a laptop entering the wireless reach integrated into a chat function based on this technology?

-<Chris> I'm guessing it would have a local repository of known keys (people), but I'm not sure. You should really contact the people who created it and talk to them about it. Wouldn't hurt to give 'em a call

<lynX> Sorry, Chris, but CSpace requires to access the worldwide DHT to be able to work. A local repository lets you recognize people when they talk to you, but it doesn't let you find them. Also CSpace uses a centralistic kickstart key server additional to the DHT, which isn't decentralized at all. Whenever you haven't talked to a person before, some preconfigured fixed IP address is consulted. Nice possibility also to compromise the system - simply insert fake pubkeys. Serverless.. haha what a joke. CSpace requires more servers than any IM system I have seen so far. The only sense of serverlessness it gives you is (a) you have no control over the servers you depend on (b) they do so little for you, they feel almost as invisible as DNS. Sorry for the tone, I'm just disturbed by aggressive marketing without facts to back it. Why don't you market things that are worth it? Hmm, okay, better not.

What about gnutella like net?

An anonymous user asked the question What about gnutella like net? - This sounds just as technologically aware of the implications as asking What about BitTorrent? or Why don't we just use the Web 2.0? - We are talking about how to make laptops independent from the Internet, decentralized applications are related, but they may very well not be suited. If you can come up with a distributed WLAN-capable Internet-independent chat/IM solution based on gnutella, please bring forward your ideas. Otherwise discuss why the use of existing XMPP or PSYC technology enhanced by a little WLAN discovery gadget should fail. --lynX 04:11, 23 June 2006 (EDT)