Scripting News - Latest Comments in How to fix URL-shorteners, part II (Scripting News)

Re: How to fix URL-shorteners, part II (Scripting News)

Dean Michael Berris — Sat, 29 Aug 2009 00:15:33 -0000

Nice concept. I know that CNAME's and hosting on third-party storage providers would work -- heck you can even host it on your own servers if you really want to. But then you run into the problem (again) of your distributed storage of shortened links to be the single point of failure for all your links.

Thinking about it a little more, if we all went distributed and each one had a separate storage for each of our shortened URL links, then the likelihood of failure of an individual node (in aggregate) is higher than if you just had a company that has the economies of scale to do it for you -- like Amazon, Google, Yahoo, Microsoft, [insert favorite internet megacompany here]. For instance, what's stopping any distributed link from acting bad and doing the wrong thing and redirecting to spam or porn? Don't you think we'll be in the same conundrum?

Re: How to fix URL-shorteners, part II (Scripting News)

dave — Fri, 28 Aug 2009 23:58:20 -0000

It's a good question. Before implementing this solution with Adjix, yes I would have preferred to use something from Amazon or Google. Now, because of the safeguards we've implemented, I'd prefer to use Adjix because Joe Moreno is a good guy, and he returns my calls, and when I ask him for a feature he usually says yes. I like Amazon and I trust Google, but neither of them consider me that important a customer or user of theirs.

Re: How to fix URL-shorteners, part II (Scripting News)

sull — Wed, 26 Aug 2009 23:22:46 -0000

fair enough. it's a good system that lets you leverage any service. i get it :)
thanks.

Re: How to fix URL-shorteners, part II (Scripting News)

dave — Wed, 26 Aug 2009 23:09:33 -0000

Also, I may not be on the SUL or be a verified user, but I am on the
whitelist and am allowed an unlimited number of API calls.

Re: How to fix URL-shorteners, part II (Scripting News)

dave — Wed, 26 Aug 2009 23:09:30 -0000

Poor guy has no clue how the API works.

Re: How to fix URL-shorteners, part II (Scripting News)

dave — Wed, 26 Aug 2009 23:06:53 -0000

Me too. I've written a number of URL-shorteners. it's not the wriitng of
them that matters, it's as I said -- the keeping them running. It's a pain
in the ass, and I'd rather let someone else do it.

Re: How to fix URL-shorteners, part II (Scripting News)

Jesse Stay — Wed, 26 Aug 2009 23:05:31 -0000

Thanks Dave. Thinking back at it, that makes total sense and I was about to
conclude that myself. That brings some great ideas though for the option
I'm considering.

Re: How to fix URL-shorteners, part II (Scripting News)

dave — Wed, 26 Aug 2009 21:53:20 -0000

If you run your own software it's yet another web app you have to run
forever.

Re: How to fix URL-shorteners, part II (Scripting News)

Ravi Pinjala — Wed, 26 Aug 2009 20:48:38 -0000

Serious question: If a company as large and established as Amazon or Google created a URL shortener, would you use it? It seems that it would be exactly as stable as a S3-based solution, minus the hassle of redirecting through another layer of DNS.

Re: How to fix URL-shorteners, part II (Scripting News)

sull — Wed, 26 Aug 2009 13:48:37 -0000

that was the original reasoning adjix had for doing this... a request from clients to utilize analytics code on the url.

however, would this not be a better solution?

How do I tag (and track) my links?

http://www.google.com/suppo...

Re: How to fix URL-shorteners, part II (Scripting News)

Robert Lowe — Wed, 26 Aug 2009 13:24:31 -0000

I assume it works that way to ensure the Google Analytics script, which is embedded in the page, runs.

Re: How to fix URL-shorteners, part II (Scripting News)

sull — Wed, 26 Aug 2009 12:45:46 -0000

good to know. i just did not see that tracking data when i logged in. i only saw the # of clicks.
great regarding CSV export.

Re: How to fix URL-shorteners, part II (Scripting News)

sull — Wed, 26 Aug 2009 12:39:05 -0000

good point, joe.

however, for a public facing url redirection/tracking service (a startup), the cost is inconsequential.
for ordinary folk, you can run a private service for your own needs or even let others use it too and you could get away with using a shared hosting service for $5-$20 per month... running side by side with your website/blog.

as for option 4 - domain redirect.... oof. a redirect to a redirect is one redirect too many for me. but i suppose it is a workaround that wouldnt bother some. i have to imagine that search engines would not look upon this whole setup very well :/ thoughts?

Re: How to fix URL-shorteners, part II (Scripting News)

Joe Moreno — Wed, 26 Aug 2009 12:13:19 -0000

Sull, Adjix tracks every single click, by IP address and referrer. Additionally, this is archived into a CSV about once/month. I'm not aware of any other service that does this.

Re: How to fix URL-shorteners, part II (Scripting News)

Joe Moreno — Wed, 26 Aug 2009 12:10:23 -0000

Cost is the big reason. Running an EC2 server will cost at least $70/month. Using this S3 option will probably only cost about $0.25/month.

It's possible to use a root-domain. See option 4 under How to do it:
http://blog.adjix.com/2009/...

Re: How to fix URL-shorteners, part II (Scripting News)

Joe Moreno — Wed, 26 Aug 2009 12:07:28 -0000

You can do it with a root-domain. See option 4 under How to do it:
http://blog.adjix.com/2009/...

Re: How to fix URL-shorteners, part II (Scripting News)

abrahamvegh — Wed, 26 Aug 2009 11:09:24 -0000

Allow me to reiterate: I am completely cognizant of the way all Twitter clients work. All I'm saying is, if everyone backs off just a little from hitting the API then it's better for everyone, and doing what Dave is doing, he can afford to back off without losing functionality.

Re: How to fix URL-shorteners, part II (Scripting News)

myquealer — Wed, 26 Aug 2009 00:10:47 -0000

Abraham, what do you think every twitter client (Tweetie, Tweetdeck, Nambe, Seesmic, etc.) do? They are hitting the twitter API every minute or so to show you the latest tweets (whether you are actively reading them or not. And they, on an individual basis, are less important than Dave archiving his tweets. There are millions of those clients running and you're worried about Dave's one archiving script?

Twitter allows everyone 150 API requests per hour. If you ask twitter nicely, chances are they will whitelist your account and allow you to make 20,000 API requests per hour. I think they can handle Dave's 20 or so per hour just fine.

Re: How to fix URL-shorteners, part II (Scripting News)

Mason Lee — Tue, 25 Aug 2009 23:59:12 -0000

An improvement for use in Twitter for sure. You're no longer dependent on a third-party, and for the rest of us, your redirects are simple files that can be mirrored by archivists for when c.oy.ly goes the way of the dodo. (Does the bucket have a full public index?)

If I may ask: You've suggested that Twitter.com should include full links in their feeds, rather than pack everything into the 140 chars. If they did that, would you stop using c.oy.ly, or will you find too much value in this new click stats architecture?

I like the idea of public click stats. I'm glad you make your Top-40 report public.

Re: How to fix URL-shorteners, part II (Scripting News)

sull — Tue, 25 Aug 2009 22:54:09 -0000

i have cloned the core bit.ly (including tracking) without even using a database (uses flat static files).
and it's done in about 100 lines of code.

and click tracking is click tracking. date/time, ip, referrer url. standard stuff. i only spent a few moments on adjix.com but i did not even see these basic click stats. but maybe i missed where more detailed stars are or maybe their is a partner/api dashboard that is more thorough.

anyway, glad you are happy with a solution. at the same time, it's good that your readers/commenters are engaging the conversation with other ideas.

cheers!

Re: How to fix URL-shorteners, part II (Scripting News)

dai_vernon — Tue, 25 Aug 2009 22:41:13 -0000

I would argue that running a URL shortening service is a pretty non-trivial function so long as twitter is limited to 140 chars

Re: How to fix URL-shorteners, part II (Scripting News)

Carter Rabasa — Tue, 25 Aug 2009 22:31:48 -0000

But... isn't that something that a halfway decent web analytics tool could provide for retrieval of static files?

I don't mean to be a trouble-maker, but I just wonder if the "middleware" of a URL shortener is really adding that much value, given the trouble.

Re: How to fix URL-shorteners, part II (Scripting News)

dave — Tue, 25 Aug 2009 22:16:45 -0000

It's tracking again (it seems to be the answer to all the questions people are asking here).

Think about it this way. If I pointed the sub-domain to their server, they would know how which long URL to redirect to, but they wouldn't know who to give the credit to. The end result of all this linking is the Top 40 list, it's kind of a link-blog, fascinating to see what my 24K followers on Twitter and the retweeters and FriendFeed folk find interesting.

http://dave.40twits.com/

Re: How to fix URL-shorteners, part II (Scripting News)

dave — Tue, 25 Aug 2009 22:14:35 -0000

Happy to report the cold is in remission. Otherwise I never would have gotten this much done. :-)

Re: How to fix URL-shorteners, part II (Scripting News)

dave — Tue, 25 Aug 2009 22:14:02 -0000

Correct.