-
Top Commenters
-
Community
-
Popular Threads
-
Recent Comments
- Got it, thanks for the clarification. One feature I would like is an option to get a feed of the full posts as opposed to just title and snippet. It would look something like this... http://pipes.yahoo.com/pipes/pipe.run?_id=b18aa9fb0d95e467f0ab2cc288bdf34f&_render=rss That's ...
- Dave, on my blog (http://gravitationalpull.net/wp/), about half of the posts are potentially of interest for techjunk, IMHO. Is there a mechanism to ping or message techjunk for posts potentially worth ...
- I love it! I love the original newsjunk also. Great work, Mr. Winer....
- good stuff, Dave. ;) Is there a search for the site too? That would be good. Also, do we get to know 'who' is choosing/ 'favouring' these stories? (As I see you mentioned that below) If you want ...
- I am not able to figure it out what special is techjunk offering ? I would prefer techmeme.com or smashingfeeds.com
FAQ: Is decentralized Twitter just IRC? (Scripting News)
32 comments
-
Blaine Cook 5 months ago with 3 points
Twitter isn't like an RSS aggregator, because (1) it's real-time, (2) it's a push technology, and (3) there is no concept of "read" or "unread"; it's a stream that you can dip into or out of. Twitter is essentially the same as an XMPP PubSub service with a web front end that provides a social layer and access to archives.
RSS doesn't scale to real-time data exchange. Even Google measures their RSS aggregation in the minutes or hours. We (at Twitter) measure our delivery times in milliseconds. Even with the concepts that have been discussed for federation of Twitter-like infrastructures, the delivery time is between 10 to 100 milliseconds. RSS also lacks real concepts of privacy - it's all or nothing; no amount of social overlays will fix that, really. OAuth might help, but it's a hard problem.
XMPP really does do all of these things, but essentially no-one has implemented them. To my knowledge, Twitter currently has the most built-out XMPP infrastructure of any social web service, and we're really aggressively moving towards implementing and launching more XMPP features (e.g., PubSub).
You're absolutely correct about the *use* of Twitter, though --- it's entirely up to the user to decide what it is. Some people use it for jokes, for poetry, for recipes, all sorts of writing. Others use it for diaries, questions, answers, all sorts of community endeavors. Some people use it for system-level logging! I'm not sure that we're blind, just that a simple answer (or question) isn't going to pin it down. -
Be careful, not everyone's RSS aggregator works the way you describe Blaine.
And it's pretty clear that Twitter, as currently implemented, doesn't scale either. :-)
Something needs to change. -
Sure, you can try to use trackbacks, but we don't have an authentication method that can prevent spam. RSS polling is fundamentally broken for low-latency and social applications.
Twitter as implemented scales just fine --- we're running up capacity constraints, which we're working hard to address. It's probably worth noting that during the macworld outage, which affected our databases and web front-ends, our messaging infrastructure ran basically uninterrupted. For what it's worth, the majority of our traffic is inefficient polling-based API traffic (which uses our JSON, XML, Atom, and RSS endpoints). -
Thanks for the insight Blain.
[ @dshaw Runs off to try to figure out what XMPP PubSub is. :) ] -
It's definitely not IRC or any other chat system, since I can control the noise. That's huge. (Though I wish I could filter out @guykawasaki 's Truemor's feed posts.)
I still don't see how the decentralized RSS approach would encourage community like Twitter does, though. I get some discovery out of shared feeds in Google Reader, but not community. I like your idea, but I think we'd see the greatest benefit from some rearchitecting of the Twitter message queue system. -
I completely agree. IRC is definitely a synchronous experience where Twitter is very asynchronous in that you choose who you listen to and others choose to listen to you. Send and Receive are separate from each other, but intertwine.
I follow you, but you don't follow me. I think that one way to approach this is in a p2p manner instead of a centralized server. Each person would own their node, chapel@mysite.com which then I could send updates and people could opt to follow me. By opting to follow me they are subscribing basically to an RSS feed of what I say. I can do the same for them or others. I think to really make it work best though would be in the use of xml-rpc for pinging everyone. Each persons node would track who is subscribed and ping them when there is an update, then their node will know to update. So each node is responsible for those that follow, but not those that they are following. The only problem I see is for a public list of updates. The only way that would work is if there was a centralized server that each node pinged with updates. Somewhat like technorati does.
I am interested in where this is going, because I like sites like Twitter and Pownce and discussion is key. -
The asymmetry of Twitter is without a doubt it's biggest difference from any other medium. The closest previous technology analogy is probably UseNet or ListServs. The difference is Twitter attempts to be more timely than those examples with message delivery, but far less granular in management of information sources. In no case are any of these like the cocktail party, symmetric broadcast/receive behavior of IRC.
Where Twitter falls down and a distributed version could succeed is in allowing multiple overlapping private communities of interest to form. Twitter is more or less public broadcasting on one channel. It entices people to communicate bidirectionally but by its very nature makes it quite hard to do. Twitter with multiple channels (e.g., my public broadcast channel, my discussion of politics with friends channel, my software product support channel, etc.) would be far more useful to me than just replicating Twitter's functionality in a distributed architecture.
Son of Twitter needs communities the same way we have seen them emerge in other social nets. -
I'd say the elephant isn't related to anything on the web -- Twittr has its roots in mobile technology. It's public, social SMS.
-
s/web/internet/
-
It would be interesting to know at this point what percentage of Twitter users care about the SMS component. My view of the elephant places virtually no value on the SMS interface (I have it disabled) and a very high priority on the AIM interface (use it exclusively). On my phone, I still use the web and IM interfaces before I touch the SMS option. While its roots may be in supporting SMS broadcasts to friends, I suspect it's no longer the key value proposition for most users. But that might just be my elephant talking.
-
Yeah, that's fair. I do use the SMS component for tracking selected keywords, but I meant more where its roots are from. My intuition os that Ev, et al. have just tapped the same thing that drives the popularity of SMS (outside the U.S.) and packaged it nicely for the web.
-
Joe Hildebrand
Twitter is just an application of PEP: http://urltea.com/2jga
-
Agreed (see my other post). Joe and I discussed this extensively at the Jabber DevCon at OSCon last year. Expect more of this.
-
We have to try to get a Twitter running outside your company if only to prove it can be done. I also think it's got to be decentralized.
-
Eric link
The semantics are exactly as you describe. Adding a pub/sub notion on top of jabber/chat protocols (that we know scale) seems to make a lot of sense to me. Also the twitter guys (see previous comment) seem to agree as they have that in beta.
-
jonknee
A decentralized Twitter is blogging.
-
Your assumption. FWIW, Mine is that a decentralized Twitter is a server for me that is a client to multiple other servers, all containing content I am interested in. And vice versa. For me, Twitter has nothing to do with blogging and blogging in the wild has no explicit support or expectations of bi-directionality. It is inherently a broadcast-only tool (excluding bolted-on comment systems, etc.).
-
jonknee
With RSS that is what blogging does. You host your content and then consume content hosted by others (RSS reader). You only consume the content that you want, just like with Twitter. It's not a perfect comparison, but it's pretty close.
The main difference I see is that Twitter makes it dead obvious who's following who. Not so with RSS. -
This point about being bi-directional can happen with RSS when two clients speak p2p, I think. Hers is my post on this, also directed towards Dave:
http://dembot.com/post/24117475 -
Yes, as it can with any two protocols swapping formatted messages. But there are existing protocols already designed to normalize and sync messages/transactions between two nodes. Using RSS as the payload is probably not optimal for semi-interactive content. There are better ways.
-
I just ran into a new (to me, at least) service called FriendFeed which appears to be similar to what you are describing (or at least what I think you are describing). http://friendfeed.com/
I don't anything like think this is going to replace Twitter (or Google Reader shared feeds for that matter) any time soon. -
So Blaine,
Is there an architecture for s2s built into twitter? (s2s is server to server communication amongst xmpp servers). This would seem to be a good thing to have for corporate twitters and for the capacity issue. -
Yes; we just use XMPP's s2s protocol. If you have your own Jabber server, you can configure your account to receive all Twitter updates there right now. The nice thing about Jabber is that its overall architecture treats all servers as equals, and builds in ways for servers to scale at different rates.
-
Need to elaborate on the RSS payload meme. The shortcoming of RSS as a method of syncing up posts in a distributed Twitter is the lack of explicit support for unique message IDs that have been a part of the ubiquitous MIME header (in NNTP in particular) for decades. If 2 twitter servers have the same message, how does an RSS-based implementation disambiguate or identify duplicates?
If all my server ever does is give you locally originated messages, we have a N-squared network traffic problem because I will be forced to talk to EVERY subscriber to my feed. So the ability to forward new messages beyond your immediate peers is required. Which means the concept of unique IDs is mandatory. It can be retrofitted by convention into any one of several RSS entities, but that's not optimal. -
I agree, I don't much see Twitter like IRC. I always saw it more like an online forum, except there is only one thread and everyone posts in it. Or stated differently, it's instant messenger designed with eaves droppers in mind. Neither are bad, just new, and new is good :)
-
Taking Blaine's posts into consideration, can anyone suggest why we wouldn't go ahead and try to use XMPP as the basis for a decentralized microblogging platform?
Here are a couple thoughts:
1. If Twitter were ever to support such a federation, it sounds as if XMPP might help the movement gain some traction with them.
2. Trying to work out an RSS based solution does have the advantage of a ton of developers familiar with it (more than XMPP, I'd think), but any RSS-based solution would have to be more complex than
current RSS development, anyway.
3. Can XMPP and RSS co-exist using RSS cloud element? I can imagine an "RSS Server" that gets notified via XML-RPC from the XMPP Federation, allowing for RSS to poll it for the latest results.
Such a system wouldn't alienate the RSS-only developers.
To me it sound good, even if I'll have to learn more XMPP, but I'm interested in objections out there. -
Seems like they already support federation to some degree through s2s, as Blaine has replied to my question above...
So if we were to set up a federating server, and configure a subset of twitter users to register there, it seems we could take load of twitter.
I think outages are partially a case of capacity tuning for average and not maximum usage. Maximum usage would have two many fallow servers. Its perhaps possible to set up an Amazon ec2 image with ejabberd which can be turned on when requited and pay for peak usage/bandwidth as one goes. The ejabberd's can be clustered, so scaling would be a matter of adding another image...modulo how exactly amazon allocates virtual machines and bandwidth.
Twitter dosent have, sensibly( for spam reasons), account creation API's. So it wouldnt be possible to create accounts on a separate server and mirrir to twitter. If Twitter would provide a way to hook into the registration process as a redirect from another site with the site's info filled in federation would be easy. The other server could intercept twitter traffic between members on it so as to not load twitter but send non-members messages over to twitter.
It would remain to implement the twitter API on the other server which would be a big task in itself but not insurmountable if we reduce the scope of the transports we support and some features. -
re: ejabberds can be clustered. That sounds pretty interesting. Ialso noticed that the latest release supports PEP which was mentioned above. OpenFire does as well.
I'd be willing to install it if someone wants to install a peer and experiment with some API features. Never used Erlang but I don't think that matters. -
That sounds interesting...would be willing to install it. I have 1.0 on my server, but 2.0rc1 does seem to support PEP
-
One important XMPP fact I forgot to mention. GTalk is XMPP. I already use a GTalk account to access Twitter via notifications, and I've built real-time systems ontop of that Gtalk user . See http://sulu.tv for a simple example.
Gtalk is very stable, so we could build apps on top of it. In other words, you don't need to run out and get a jabber server up and running to get in on the fun. -
sulu.tv is sweet. Presumably you have a bot at the other end of your gtalk?
This could be a nice indirect way to use google's scalability instead of running ones own jabber servers. -
I noticed you were sent a weather query. Thanks for trying it Rahul. So far the Gtalk user a programmmable TwitterBot has been working. Only a few users though. We'll see.
