Scripting News - Latest Comments in FAQ: Is decentralized Twitter just IRC? (Scripting News)

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Randy — Tue, 23 Jun 2009 06:14:58 -0000

It is like IRC, but with every channel +m.

They did hire atleast one IRC guru, you know.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

rahul — Mon, 21 Jan 2008 23:13:13 -0000

That sounds interesting...would be willing to install it. I have 1.0 on my server, but 2.0rc1 does seem to support PEP

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Matt Terenzio — Mon, 21 Jan 2008 19:56:05 -0000

re: ejabberds can be clustered. That sounds pretty interesting. Ialso noticed that the latest release supports PEP which was mentioned above. OpenFire does as well.

I'd be willing to install it if someone wants to install a peer and experiment with some API features. Never used Erlang but I don't think that matters.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Matt Terenzio — Mon, 21 Jan 2008 19:47:13 -0000

I noticed you were sent a weather query. Thanks for trying it Rahul. So far the Gtalk user a programmmable TwitterBot has been working. Only a few users though. We'll see.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

rahul — Mon, 21 Jan 2008 18:52:47 -0000

sulu.tv is sweet. Presumably you have a bot at the other end of your gtalk?
This could be a nice indirect way to use google's scalability instead of running ones own jabber servers.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

rahul — Mon, 21 Jan 2008 18:22:57 -0000

Seems like they already support federation to some degree through s2s, as Blaine has replied to my question above...
So if we were to set up a federating server, and configure a subset of twitter users to register there, it seems we could take load of twitter.

I think outages are partially a case of capacity tuning for average and not maximum usage. Maximum usage would have two many fallow servers. Its perhaps possible to set up an Amazon ec2 image with ejabberd which can be turned on when requited and pay for peak usage/bandwidth as one goes. The ejabberd's can be clustered, so scaling would be a matter of adding another image...modulo how exactly amazon allocates virtual machines and bandwidth.

Twitter dosent have, sensibly( for spam reasons), account creation API's. So it wouldnt be possible to create accounts on a separate server and mirrir to twitter. If Twitter would provide a way to hook into the registration process as a redirect from another site with the site's info filled in federation would be easy. The other server could intercept twitter traffic between members on it so as to not load twitter but send non-members messages over to twitter.

It would remain to implement the twitter API on the other server which would be a big task in itself but not insurmountable if we reduce the scope of the transports we support and some features.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Matt Terenzio — Mon, 21 Jan 2008 12:26:25 -0000

One important XMPP fact I forgot to mention. GTalk is XMPP. I already use a GTalk account to access Twitter via notifications, and I've built real-time systems ontop of that Gtalk user . See http://sulu.tv for a simple example.

Gtalk is very stable, so we could build apps on top of it. In other words, you don't need to run out and get a jabber server up and running to get in on the fun.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Matt Terenzio — Mon, 21 Jan 2008 12:22:08 -0000

Taking Blaine's posts into consideration, can anyone suggest why we wouldn't go ahead and try to use XMPP as the basis for a decentralized microblogging platform?

Here are a couple thoughts:

1. If Twitter were ever to support such a federation, it sounds as if XMPP might help the movement gain some traction with them.

2. Trying to work out an RSS based solution does have the advantage of a ton of developers familiar with it (more than XMPP, I'd think), but any RSS-based solution would have to be more complex than
current RSS development, anyway.

3. Can XMPP and RSS co-exist using RSS cloud element? I can imagine an "RSS Server" that gets notified via XML-RPC from the XMPP Federation, allowing for RSS to poll it for the latest results.
Such a system wouldn't alienate the RSS-only developers.

To me it sound good, even if I'll have to learn more XMPP, but I'm interested in objections out there.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Nathan Snell — Mon, 21 Jan 2008 01:18:47 -0000

I agree, I don't much see Twitter like IRC. I always saw it more like an online forum, except there is only one thread and everyone posts in it. Or stated differently, it's instant messenger designed with eaves droppers in mind. Neither are bad, just new, and new is good :)

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Blaine Cook — Sun, 20 Jan 2008 13:12:34 -0000

Yes; we just use XMPP's s2s protocol. If you have your own Jabber server, you can configure your account to receive all Twitter updates there right now. The nice thing about Jabber is that its overall architecture treats all servers as equals, and builds in ways for servers to scale at different rates.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Chuck Shotton — Sat, 19 Jan 2008 17:31:10 -0000

Need to elaborate on the RSS payload meme. The shortcoming of RSS as a method of syncing up posts in a distributed Twitter is the lack of explicit support for unique message IDs that have been a part of the ubiquitous MIME header (in NNTP in particular) for decades. If 2 twitter servers have the same message, how does an RSS-based implementation disambiguate or identify duplicates?

If all my server ever does is give you locally originated messages, we have a N-squared network traffic problem because I will be forced to talk to EVERY subscriber to my feed. So the ability to forward new messages beyond your immediate peers is required. Which means the concept of unique IDs is mandatory. It can be retrofitted by convention into any one of several RSS entities, but that's not optimal.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

rahul — Sat, 19 Jan 2008 13:27:40 -0000

So Blaine,
Is there an architecture for s2s built into twitter? (s2s is server to server communication amongst xmpp servers). This would seem to be a good thing to have for corporate twitters and for the capacity issue.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Chuck Shotton — Sat, 19 Jan 2008 11:11:04 -0000

Yes, as it can with any two protocols swapping formatted messages. But there are existing protocols already designed to normalize and sync messages/transactions between two nodes. Using RSS as the payload is probably not optimal for semi-interactive content. There are better ways.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

andrewbaron — Fri, 18 Jan 2008 21:27:56 -0000

This point about being bi-directional can happen with RSS when two clients speak p2p, I think. Hers is my post on this, also directed towards Dave:
http://dembot.com/post/2411...

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

dave — Fri, 18 Jan 2008 15:23:27 -0000

We have to try to get a Twitter running outside your company if only to prove it can be done. I also think it's got to be decentralized.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Blaine Cook — Fri, 18 Jan 2008 15:12:30 -0000

Agreed (see my other post). Joe and I discussed this extensively at the Jabber DevCon at OSCon last year. Expect more of this.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Blaine Cook — Fri, 18 Jan 2008 15:10:42 -0000

Sure, you can try to use trackbacks, but we don't have an authentication method that can prevent spam. RSS polling is fundamentally broken for low-latency and social applications.

Twitter as implemented scales just fine --- we're running up capacity constraints, which we're working hard to address. It's probably worth noting that during the macworld outage, which affected our databases and web front-ends, our messaging infrastructure ran basically uninterrupted. For what it's worth, the majority of our traffic is inefficient polling-based API traffic (which uses our JSON, XML, Atom, and RSS endpoints).

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

jon — Fri, 18 Jan 2008 14:13:39 -0000

With RSS that is what blogging does. You host your content and then consume content hosted by others (RSS reader). You only consume the content that you want, just like with Twitter. It's not a perfect comparison, but it's pretty close.

The main difference I see is that Twitter makes it dead obvious who's following who. Not so with RSS.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Chuck Shotton — Fri, 18 Jan 2008 14:04:48 -0000

Your assumption. FWIW, Mine is that a decentralized Twitter is a server for me that is a client to multiple other servers, all containing content I am interested in. And vice versa. For me, Twitter has nothing to do with blogging and blogging in the wild has no explicit support or expectations of bi-directionality. It is inherently a broadcast-only tool (excluding bolted-on comment systems, etc.).

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

dshaw — Fri, 18 Jan 2008 13:11:15 -0000

Thanks for the insight Blain.
[ @dshaw Runs off to try to figure out what XMPP PubSub is. :) ]

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

dshaw — Fri, 18 Jan 2008 13:06:25 -0000

I just ran into a new (to me, at least) service called FriendFeed which appears to be similar to what you are describing (or at least what I think you are describing). http://friendfeed.com/

I don't anything like think this is going to replace Twitter (or Google Reader shared feeds for that matter) any time soon.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

jon — Fri, 18 Jan 2008 13:05:19 -0000

A decentralized Twitter is blogging.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Eric link — Fri, 18 Jan 2008 13:03:59 -0000

The semantics are exactly as you describe. Adding a pub/sub notion on top of jabber/chat protocols (that we know scale) seems to make a lot of sense to me. Also the twitter guys (see previous comment) seem to agree as they have that in beta.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

dave — Fri, 18 Jan 2008 13:00:36 -0000

Be careful, not everyone's RSS aggregator works the way you describe Blaine.

And it's pretty clear that Twitter, as currently implemented, doesn't scale either. :-)

Something needs to change.

Re: FAQ: Is decentralized Twitter just IRC? (Scripting News)

Joe Hildebrand — Fri, 18 Jan 2008 12:57:22 -0000

Twitter is just an application of PEP: http://urltea.com/2jga