I read Robert Scoble’s post on “noise in the net” with morbid curiousity today. It’s a pretty interesting post, really. Seriously.
Oh, the glorious noise! Everyone loves beating me up for causing the noise. No, I am not the cause. I pass it along. You should see my inbound streams. Every second or two a new Twitter is aimed at me. Every few seconds, a new blog post comes into Google Reader. Every few seconds, a new thing on FriendFeed.
Scoble then goes on to give a few ideas on how to get past the noise and make some sense out of it all. The suggestions are OK, but not really great. They’re really more thoughtlets than anything useful.
Irony on ‘Roids
Anyhow, do you want to know something really ironic? I was just about to remove him from my twitter “follow” list, precisely because the amount of noise and gunk that he’s been generating has been enormous, and the value has been … well, not enormous.
Sort of like the opposite of Dolby Noise Reduction … but maybe worse. He not only passes along the noise, he adds more to it and injects back into the same stream, making it go from meta-stable to insanity inducing.
Sort of a human positive feedback loop. As in this excerpt (from the linked article)
A system in equilibrium in which there is positive feedback to any change in its current state is said to be in an unstable equilibrium … the end result of a positive feedback is often amplifying and “explosive”, i.e. a small perturbation results in big changes.
I know we’re trying to figure out how this can all work, be helpful and all that, BUT … is that really a good model for social communication? Then out of all that gunk he comes up with something interesting.
Maybe I’ll wait until tomorrow to break the loop…
Yesterday we talked about whether Twitter really ever need to be reliable or not … some said yes, others contend that it’s not necessary.
It’s been bugging me for awhile that something this popular … and Twitter is so … just keels over as often as it does.
Anyhow, the whole argument turned into a bona-fide debacle this morning when GroupTweet (a relatively new feature that seems to have been confusing) was at the heart of disclosing private messages (DMs in tweet-speak) to tons of folks.
So now it looks like Blaine Cook is out as chief architect, and Michael Arrington is calling it the end of amateur hour. That’s probably a bit harsh, because my (limited) interactions with Cook have been pretty decent.
Btw the comment thread on that last post is going crazy. My favorite so far is a short video comment from a Loren Feldman (warning … his language is a bit over the top, but you do know where he stands!) Btw, check here if the first link to the video doesn’t work.
Having said that, we just have to build apps that act like real, grown up (and you can call that boring if you want) apps … taking care of the data entrusted to them, working as expected, and working when we need them to work.
So I’m thinking that the answer to yesterday’s question is … YES. Twitter does need to figure out how to be reliable … and secure, scalable, and all the rest.
This is exactly the point that I’ve been making for awhile … why build to POC quality when it’s now possible to ensure reliability, scalability, and so forth from the beginning?
People are Still People
I don’t care if this is Web 2.0, Enterprise 2.0, or Web 10,000,000,000.0 … consumer or enterprise … people are still people. They still care about their privacy, the reliability of stuff that they come to rely on, basic stuff like that. No free pass.
Even consumer oriented web 2.0 apps need to ensure this, from the beginning.
Pretending that innovation in communication, biz, or technology somehow exempts us from the basics of social interaction is just … well, it’s just wrong.
This lesson is for our whole industry. Those who learn it will prosper, those who don’t …
In my earlier post I commented on my own little experiment about Web 2.0 infrastructure’s ability to handle even modest-interest events.
Well my initial verdict was that most players had fallen flat on their face … badly.
Of course, the Techcrunch post somehow overlooks the equivalent failures over at Crunchgear – imagine that!
On the negative side these failures show just how immature these architectures still are … of course, on the positive side it absolutely demonstrates the widespread need for something better … something like our EAF!
I was experimenting around a bit with different ways to track the macworld keynote from my cozy office. I figured this would be good nano-metric on how far we’ve matured web 2.0 scaling techniques, particularly when focusing on delivering an event experience. This is a perfect example of a specialized community – bigger than some, smaller than many.
Sorry to say that every venue that was directly trying to cover the keynote has to get a mad kitty. I was hoping for better.
Starting with a sort of “irritated kitty” were engadget and gizmodo … both of their live posts timed out quite a bit at first, then settled down and responded ok, albeit slowly.
Tried following the twitter feed of TUAW, but twitter’s website started timing out a few minutes before the start of the keynote. Kept timing out for another five or ten minutes, so I gave up. Btw, don’t know about their SMS distribution since I choose to turn it off momentarily (I do have biz to do!).
[update] Twitter died hard during the keynote.
[update] Crunchgear was apparently pretty hosed during the event.
Ran into “stevenotelive.com” and tried that … had it up for a half hour before the start of the event, during which they ran a continuos loop boasting on their “ridiculous bandwidth”. They had a little counter that was over a thousand 15 minutes before the start of the event, and then …
They were right, it was pretty ridiculous. So ridiculous that when the event started it just blew it’s guts all over the floor. I was never able to get a connection, much less sustain any streaming whatsoever.
Apple didn’t even try to broadcast the event, and the promised iphone updates were not available when Jobs said that they were.
But probably the biggest mad kitty of all goes to Randy Newman, who apparently compared the US and President Bush to Hitler and Stalin. The US has zillions of flaws, but that’s not even rational. I think this dude’s been on the road (or something else) WAY too long. Time to retire, Randy.
For a little speech that is big by tech-world standards, not much worked very well. As far as I’m aware there weren’t any live streaming systems that worked at all. The text-based wide-distribution stuff (I focused on twitter) didn’t make it. The blog-based stuff worked intermittently, and the iphone update server reported errors for awhile.
Of course, this is not a comprehensive view … just a little core-sample from our own industry’s back yard. I think the right answer is that our industry still has a long way to go to handle modest special-interest scale, without even beginning to deal with truly society-wide scale.
First things first, I suppose – let’s starting using more app fabrics to make the basics work better!
[update] Apple software update does appear to be creaking along now, a half hour after the keynote ended.
[update 2] The Macbook Air looks pretty cool though, I must admit!
[update 3] Engadget sort of owns up to their outages … nobody else does so far.
[update 4] Crunchgear posts their own mea culpa.
[update 5] In a big bit of irony, Techcrunch bags on Twitter, but gives Crunchgear a pass.
Ok, well the broken stuff for today (so far) includes Digg and Yahoo Small Business.
Digg appears to be a partial failure, which is probably the most common in the SaaS and enterprise worlds (as I discussed yesterday). While much of the mainstream functionality seems to continue working, much of the stuff that differentiates Digg (adds its personality, so to speak) is missing in action.
Interestingly enough, as I talked about a few months ago these are precisely the sort of functions that Digg’s dbas are proud of working hard to suppress. I understand that these can be difficult to scale (at least in old-school db-centric implementations that is!), but the business needs them to add value.
And now these value-added functions are gone.
The good news? It could be worse – at least Digg is doing this to themselves. Yahoo, on the other hand …
Yahoo Acts Like a …
On Monday at 6:00AM PT, the systems that power our merchant stores experienced outages, and shoppers of those stores were met with either error messages or they were unable to complete the checkout process …
These issues lasted until about 1:00PM PT
Other than that things were fine.
The Good News
One bit of good news is that both companies have talked a bit about having the problems. Perhaps it’s because the problems themselves affect such a high percentage of their customer bases – they’re just too prevalent to ignore.
The Bad News
Are you kidding me?!??!?!? Both of these SaaS offerings are broken. While Digg is sort-of-working (in a limp-home sort of way), the poor unfortunate merchants who believed and relied upon the promises of Yahoo Small Business have just suffered major losses.
Not even an SLA would do much good here. Check out some of the comments to Riley’s fessin’ up:
This outage cost us big time in terms of money, our time and customer goodwill … Yahoo! should immediately come up with a plan to compensate merchants for this disruption of service on the most highly publicized day of online shopping.
Just telling us the time line of what happened isn’t very useful. We already know that as we watched it happen and suffered the lost business because of it.
Please also give me a good reason(s) why I shouldn’t switch to a different shopping cart provider at my earliest opportunity.
this was catastrophic…
Why is there no redundancy? I have lost faith.
it will take a class action suit for it to be addressed unfortunately
The justifiable outrage goes on and on and on. After all, what’s a merchant going to get back … part of a day’s hosting fee? As if that would compensate for half a day’s lost sales during the make-or-break time of the year for most merchants!
This is Getting Ridiculous
As an industry we just have to do better … customers have a right to expect better, and we must deliver. Talking about reliability and SLAs is simply not enough … we need to get it done.
Btw, this business of broken SaaS offerings is becoming such a common occurrence that I’ve added a new category to this blog, for your convenience. All posts will be marked with the “mad kitty” (I’ll explain later), & I think I’ll probably go ahead and add a tab as well, because I’m pretty sure this topic is simply not going away for the foreseeable future (unfortunately).