Archive

Archive for the ‘Editorial’ Category

A Blunt Instrument

January 28, 2011 Leave a comment
Agence France-Presse/Getty Images

Egyptian protestors tearing down a portrait of then-President Mubarak in Alexandria (Agence France-Presse/Getty Images)

Overnight there were many, widespread reports that Egypt has severed most of it’s ties to the Internet (WSJ, CNN, Telegraph) and has also disabled SMS (text) services. What is not clear at this time is how stable the internal communications are at this time – mobile, landline, or net.

While isolating populations and breaking internal communications is probably step 4 in the “Totalitarian Handbook: How to Crush the Opposition”, this has not been tried at this scale before. My guess is that it will have precisely the opposite effect as intended, mostly because the action itself is far more disruptive than anyone realizes at this time – disruptive to the economy, to the normal interactions of the residents, for that matter, even disruptive to the actions of the government itself.

I’ll leave the considered analysis of the sociological, political, economic, and yes religious dynamics mostly to others who are undoubtedly far more qualified to think this one through … but I do want to begin to consider what impact cutting off an entire country from the net might have on that country.

A Precision Scalpel?

In graduate school I remember attending a seminar in which the speaker (whose name eludes me right now), who was working for the Federal Reserve Bank in San Francisco at the time, told us how the US had actually implemented the freeze of Iranian funds during the then-recent hostage crisis.

This is the front console of a PDP-11/70

The decision to freeze the funds was made in late 1979, so it turns out that after receiving a middle-of-the-night call to freeze all Iranian funds (about $8B dollars), he simply drove down to the SF Fed, walked over to the physical server that was used as a gateway for all large funds transfers (a PDP-11/70, if I remember correctly), and used a bit of tape to stick a handwritten note on the console saying something like “don’t release any Iranian funds”.

Apparently he used pretty decent tape, because that money stayed most-definitely out of Iranian hands.

While this arguably aggravated daily life in Iran, it generally functioned as intended – it caused pressure on the target country as a whole, yet did not directly impact most people within that country, at least not in a clearly discernible manner. This was one nation to another, and when considered in that context was actually a fairly targeted, precise instrument.

An Act of Desperation, A Very Blunt Instrument

By all accounts Egypt is in real turmoil right now, to say the least. While it may have seemed to make sense to the present government to cut off the Internet and the ability of people to text each other, I think that the impact of the unavoidable other stuff that will happen as a result of this action will end up making the situation far worse.

Bad enough, I think, to perhaps be the exact action that pushes the population over the edge and topples this government.

Here are a few of the unintended / at least unavoidable reasons why (in no particular order):

  • External Economic Disruption. Everything from the banks to the stock markets to all sorts of companies and individuals need to reliably communicate with their counterparts around the world to function. Breaking those will most likely hurt the Egyptian economy in entirely hard to anticipate, though likely significant, ways.
  • Internal Economic Disruption. It’s probably pretty safe to assume that much of the internal Egyptian economy will grind to a halt. Yes part of that is because of the riots themselves, but a much larger part of that will be because businesses can’t communicate with their customers or each other. Pretty simple, really.
  • Partial Isolation Externally. Cutting off much of the external net access will certainly create a sense of isolation; however, there will be some flow that continues. That will spread old-school, imperfectly and rather slow by comparison, yet it will spread.
  • Government Disruption. This is pretty mundane but probably non-trivial – the actions of the government across a large country will probably be impeded by the lack of the very tools that are targeted here. While there’s probably official communications channels that remain functioning, much of the day to day, informal actions probably are done the way people conduct most of their lives – and that is broken now.

Too Late, We Know How to Talk With Each Other
Perhaps if Egyptians had never become accustomed to facebook, youtube, twitter, email and every other aspect of the relatively open, functional net culture / economy (or at least one that is perceived to be fairly functional – China is a net that is probably far more controlled than we all think, though done in a much more precise, subtle manner), then these actions would have much less impact.

In other words, I doubt if North Korea cutting off international internet access (such as it is) would have much of an impact on that peculiar, sad country.

But Egyptians have, like most of the world, come to expect and rely upon pervasive, fairly reliable, net services for many aspects of daily life.

Take them away and people will notice – the economy suffers, society decays, people’s lives diminish. While technology is not causing the problems that Egypt is struggling with, disrupting that technology will certainly exacerbate those problems.

Let us keep the people of Egypt in our thoughts and prayers as they move through this time of extreme uncertainty and sorrow.

Categories: Editorial Tags: ,

Kindle DX, Newspapers, & Clouds

May 8, 2009 1 comment

i have been heads down on a writing project that we hope to be able to announce soon, but every once in awhile I’ll pop my head up (sort of Punxsutawney Phil style, but that may be insulting the groundhog) and look around …

Kindle DX

I’ll admit to not being much of a Kindle fan, in either it’s first or second iterations. Probably because I just like books. There – I’ll admit it. I like books. Physical books. Paper, stuff like that. Yes, an inordinate amount of my house is devoted to books … books on shelves, books in piles, books strewn here and there … (I’m going to mercifully spare you the Dr. Suess allusion!).

Having said that, my biggest heartburn with the first two Kindles has been the ratio of useful (viewable) screen to other stuff – be it the sort of Soviet-style industrial “design” in the first edition, or the as yet sizable, though clearly better looking case in the second.

So maybe I shouldn’t be surprised, but I’m warming up with the Kindle DX. Yes it’s pricey, I’m still concerned about the seizure-inducing etch-o-sketch erase flashes, and I still like physical books. Still, I am intrigued, somewhat for the device itself but probably more for the implications for electronic distribution of even more content.

Check out this short video that covers some of both of these points:

This video is from of an interesting startup called Newsy.com, which produces short video reports combining multiple perspectives into a single burst. It’s one of the new forms of journalism that I like, but more on that later.

clark kent

Clark Kent, Reporter

What Happened to Newspapers?
Most of us in the geek world long ago shifted to getting most of our news via the net in one for or another. That trend has clearly spread out to a much broader percentage of the population, enough that newspapers are really struggling.

Not struggling as in trim here, trim there, invest a bit and everything will be ok, but struggling as in mortally wounded, say your goodbyes, and if you work there start thinking about what’s next.

This happened for a fairly simple reason – advances in computing technology made it possible.

Some of it was the basic stuff (servers, decent storage, and all that), some of it was ubiquitous net-based distribution, some of it was more usable devices (I do love doing the web on my well-used, not so much better for the wear iphone), and some of it was what people did with it (blogging, aggregation sites, and the like).

Yes a few of the newspapers will make a transition to digital in a form that’ll be sort of recognizable, but not too many. The reasons are really complex and outside the bounds of this post (or my expertise, really), but in essence too many of them for too long have thought that they were in a technology-enabled distribution business (typewriters, printing presses, newsprint, kids on bicycles), while in reality they were in the content business (think Clark Kent).

Clouds Enable the Next Big Bang
The funny thing is all of that shifting has been mostly before cloud computing. Most of what has happened to date in the newspaper business has been because the distribution technology shifted, and the new distribution model enabled a bunch of new competitors. It also opened up a bunch of new customers, but that’s been harder to realize.

But with the basics of distribution in place over the net, and with devices like the iPhone, Kindle DX & competitors, netbooks and more that make the physical presentation of content better and more ubiquitous, the big race will be on to slice and dice the content that’s available in better and better ways.

All of this sifting and sorting and searching is going to take far more computing, storage, and network capacity than anything our industry has done to date.

And I know a perfect platform for all that cool stuff.

By happy circumstance I had an opportunity to speak on a panel this week at the Reynolds Institute of the University of Missouri School of Journalism. The panel was focused on entrepreneurial opportunities, particularly those revolving around content. These guys get that the rules of their business are changing, and are definitely doing some cool stuff.

Categories: Editorial

CloudIQ – The Evolution of a Platform, Part 1

March 25, 2009 Leave a comment

This is Part 1 of a planned three part series that traces the evolution of the CloudIQ Platform from first idea to what it is today, then considers what it is likely to become.

180px-View_from_the_La_Luz_Trail.jpg

Right after graduate school we spent a few years living in the foothills of a desert mountain range. I remember the first time that I hiked a narrow trail to the top of the nearest peak – standing at the bottom was rather intimidating; the circuitous ascent itself was such a tangled mixture of switchbacks, short ascents and descents, under cover and open trail that it could only be described as non-obvious (at best). However, it was only upon achieving the original goal that we gained any perspective on how, in fact, we had gotten there.

In this post I will reflect on the evolution of CloudIQ – the truly exciting (if I must say so myself!) cloud application platform that we announced a couple of weeks ago.

Roots
As some pondered the impending Y2K “crisis” and others looked for the best millennium parties, most of our founding team was deeply enmeshed in building, selling, and supporting an enterprise-grade (scalable, reliable, etc.) payment server.

Upon leaving that company I had time to reflect on my enduring frustrations -

why was it so hard to build software that we (and our customers) could rely upon?

It seemed that we were spending 60 to 70% of our engineering efforts not on core functionality, but in our best attempt to ensure that the resulting application could be relied upon.

Later on an early supporter coined the term “reliability tax” to refer to this overhead.

As I asked friends at other companies and enterprise shops most recognized the same problem – a few argued the overhead was actually higher, most thought that the cruel irony was that it was very, very difficult to ensure true reliability for enterprise apps – but all agreed that this just didn’t seem right, not nearly 50 years after Gracie Hopper did her most famous work.

With this question just really bugging me I had an opportunity to build the beginning of a digital recording studio. Using completely commodity gear – no name, cheap- I was genuinely shocked at the results. Serious performance, cheap.

So that led to the second question -

why weren’t we using commodity stuff like this for problems that we really cared about?

The answer to this seemed easy enough – who could trust this cheap stuff? What if it broke? (and it would break).

In pondering the first question it seemed to us that the core problem for software development was one of complexity – mainstream application architectures were simply too complex, and becoming inexorably more so.

The First Idea
Then it became fairly clear – we could solve both of these problems at the same time by enabling groups of commodity boxes to work together to ensure a stable platform for applications.

But what exactly did that mean? Or more to the point, could we build it? Ever MORE to the point, once we built it, how could people use it, and for what applications would this new thing be useful?

Over the course of a few months the founding team hammered out the first answers to those questions. Throughout this process we were driven by use cases – we wanted the resulting platform to be equally adept at running anything from fine grained, transactional applications to more computationally-intense enterprise applications.

This led to much refinement of the basic idea, which evolved to a self-organizing group of commodity machines that could act like one thing, reliably execute all sorts of applications, grow (and shrink) as needed without affecting the execution of any running applications, and be very simple to both write applications and operate.

The Hive is Alive
We decided to call this hive computing, and on June 12, 2002 we had the first successful demonstration of a running hive. We assembled a few commodity boxes on a re-purposed kitchen rack, loaded the prototype hive software, and … it worked!

We were able to (carefully) pull a few plugs and the application kept running without missing a beat – in fact, without even losing a bit of data.

Within two years we had our first paying customer (Sprint), a couple of patents filed, and a demonstration system on which we ran an eye-opening benchmark – a wall of 100 commodity computers that could legitimately double Visa’s then-current peak transaction load, for a total bill that was well under 10% of the conventional alternative.

The best part? It was arguably far more reliable as well. We were constantly amazed by the resilience and ease of use of this new type of application platform … though truth be told, we were not yet ready to use the “P” word.

Roadblocks
In our pursuit of the possible we (the founding team) sometimes thought that economics would do all the persuading for us. Well that turned out to be sometimes yes, but mostly no.

In fact, sometimes economics actually worked against us – the combination of 90% lower costs, simpler development, easier operations, and simultaneously increased reliability and scalability simply seemed too good to be true for many people.

The fact that we required some modification of the application also made adoption more complicated. While we supported several languages and multiple operating systems (and could easily support more of each), the plain, simple truth was that you did need to modify – albeit lightly – many components in each and every application.

borg_cube.jpg

This raised the adoption barrier a bit higher.

Then there was a little matter of language. Without a native category to call our own, sometimes we were put into all sorts of categories – everything from grid computing to autonomic computing, with several others between.

Early on I even told some folks that we were basically building the “Borg for applications”. While hard-core geeks loved that (and usually laughed), it didn’t exactly help us build trust with the typically non-technical executives responsible for making the final purchase decisions.

Yet, It Worked … Well
Despite these go-to-market difficulties the product itself worked well – really, really well. In fact, by mid 2002 several of us became firmly convinced that beyond a shadow of a doubt there would be a time in the future – say 10 or 15 years – where most mainstream computing would be done this way.

The economics and functional advantages were simply too compelling for any other outcome.

The only real question in our minds was when and who – when this transition would begin to occur and who would help make that transition happen.

So as 2004 came to a close we pondered solutions to these issues and continued to press rapidly forward.

In Part 2 we will talk about why this worked so well, and the transition to the application fabric.

Categories: Editorial Tags: , ,

Clouds for the Military – A Linchpin Technology

February 4, 2009 1 comment

linch-pin.jpg

Two applications areas for which cloud computing holds the most promise are in the related areas of intelligence and military applications.

Even if you are not already intimately familiar with the types of computing problems that dominate these application areas, it’s easy enough to see how cloud computing – and of course I mean all sorts of clouds, with a particular emphasis on private clouds – can help.

After all, the very attributes of clouds that are so attractive to startups and enterprise alike – easy sense of scale, flexibility, low cost, and more – have tremendous appeal for intelligence and military applications as well.

A Military Perspective
I was recently interviewed for a story that’s appearing In the current issue of Military Information Technology. entitled “COMPUTING IN THE CLOUDS”. The story covers a number of cloud initiatives, with a focus on some things that are working and challenges that are looming.

Here is a cool quote from the story:

Appistry offers a linchpin technology for cloud computing, called the Enterprise Application Fabric, a cloud application platform for developing and managing large-scale, selfhealing cloud applications rapidly on commodity hardware.

Why Is Appistry a “Linchpin Technology”?
In this quote the story captures precisely one of the concerns of both those pioneering and those contemplating cloud applications in military and intelligence – sure the inherent scale and flexibility are great, but what about the complexity?

Speaking from the IC side of the house, streaming full-motion video from a Predator UAV or a satellite image are huge files to deal with in terms of storage, processing and transport to a soldier in motion…

However, a disadvantage is the added complexity of virtualization, which is inherent in cloud architecture (em. added). “When we virtualize in a cloud, it is more difficult to unwind the problem should it arise. As virtualization increases, logical complexity grows,” Pierce pointed out.

- Ken Pierce, DIA-DS/C4ISR

He went on to say that his organization is already well-positioned to handle the added complexity – but what else can he really say?

The Real Value of a Cloud Application Platform
It is precisely in aggressively taking out complexity – both operational and development – while maintaining all of the goodness of clouds that this emerging thing the industry has begun calling a cloud application platform. delivers the goods.

As you might expect, Appistry EAF as it exists today makes an excellent cloud application platform, and stuff that we’re hard at work on – even as we speak – will expand that lead.

And that is why Appistry is becoming a “linchpin technology”.

Categories: Editorial Tags: ,

(Unintentional) Startup Humor – or – It Seemed Like a Great Idea

January 28, 2009 1 comment

From a great (free) daily feature produced by the WSJ called The Best of the Web by James Taranto comes this awesome bit of irony …

Think Locally, Act Globally
From the New York Times:

The local food movement has been all about buying seasonal food from nearby farmers. Now, thanks to the Web, it is expanding to include far-away farmers too.

A new start-up, Foodzie, is an online farmers market where small, artisan food producers and growers can sell their products. Foodies in Florida, say, can order raw, handcrafted pepperjack cheese from Traver, Calif., or organic, fair-trade coffee truffles from Boulder, Colo.

What a great idea! And why not take it one step further? Farmers could band together and form large organizations–call them “corporations”–to grow and distribute mass quantities of food. Retail operations could be set up in every town; they would be sort of super farmers markets, or “supermarkets” for short. Soon everyone everywhere would be able to buy local food from all over the world!

Being one, I’m always willing to cut entrepreneurs – particularly the the ones who charge ahead and then look around to see who came with them – just a little bit more slack … still, this does border on the silly.

Of course silly can be great, especially if you remember to laugh.

Categories: Editorial Tags: ,

With the RDBMS Ailing … What's Next?

January 22, 2009 2 comments

not-dead-yet.jpg

Over the past couple of years we’ve come across application after application where the biggest block to being able to utilize a cloud (public or private) has been the relational database. This usually goes hand in hand with an inability to scale.

In a very (Monty) Python-esque manner RDBMS proponents proclaim “I’m not dead yet”, and the thing is that they’re absolutely right … but they are beginning to ail a bit.

There are many reasons for this, but mostly the cracks in the RDBMS monolith have a very simple explanation – the relational database is only one – albeit one powerful and long-lived – storage abstraction., and is not the best choice for many common, everyday problems.

We Can Do Better
In other words, there are many common, everyday problems in which the data can be effectively stored, managed, and retrieved in other abstractions …

… perhaps abstractions that fit the problems themselves much better, and may well be far more “cloud friendly”.

If you’ll grant me that for now (and we will get back to this question in future posts), then the next question is inevitable:

for those applications where the RDBMS is not the best choice, what else should I consider?

A Great Question
As life would have it, I think for anyone thinking about cloud-friendly storage abstractions this is the real question. Several friends (including @KentLangley and my colleague @msgroner) pointed out an excellent overview post by last.fm founder Richard Jones. From the intro:

Perhaps you’re considering using a dedicated key-value or document store instead of a traditional relational database. Reasons for this might include:

   1. You’re suffering from Cloud-computing Mania.

   2. You need an excuse to ‘get your Erlang on’

   3. You heard CouchDB was cool.

   4. You hate MySQL, and although PostgreSQL is much better, it still doesn’t have decent replication. There’s no chance you’re buying Oracle licenses.

   5. Your data is stored and retrieved mainly by primary key, without complex joins.

   6. You have a non-trivial amount of data, and the thought of managing lots of RDBMS shards and replication failure scenarios gives you the fear.

At any rate the post by Jones is a good place to get started on non-relational basic name-value stores.

This whole business of storing data in cloud-scale apps has become one of my favorite area for discussion – I can promise you more posts to come!

Categories: Editorial Tags:

Critical Mass … or Why I am Excited About 2009

January 19, 2009 1 comment

1_thumb_Morpho-peleides,-Common-morpho-cropped.png

Awhile back my colleague Sam organized a “Cloud Forecast 2009″ podcast, which was a lot of fun. Enough fun that we’re investing a bit in new audio gear and will turn out some more installments … we’ll keep you posted.

In any case, the occasion of that recording got me to thinking a bit, and with the Christmas and New Year’s holidays now well behind us, and the de-facto SuperBowlholiday looming (I was really glad to see Kurt Warner earn another shot at the title – the 2000 SuperBowl win by the Rams remains one of my favorite sports memories), I finally realized my theme for 2009, one that had been gnawing on me for some time and is now very clear:

I am excited.

I am excited about cloud computing, including progress in everything from virtualization to real-big-public-clouds with cool new storage facilities, from billion-core processors to dirt-cheap multi-terabyte drives … honestly, who can’t get excited about a terabyte for less than $100?

Even the debate over the shape, color, texture, and full extent of cloud computing is energizing. Of course, my favorite part of the debate is that it’ll be sorted out in the marketplace, and I like our chances … after all, does anyone need cloud-enabled applications?

I think so.

microsoft-arc-portable-notebook-mouse.jpg

I am particularly excited about the level of market awareness amongst all sorts of folks that we’re talking with, about real customers doing real stuff with real clouds – public, private, and a mixture of both.

Agree with me or not, the development of private clouds is a great boon to enterprises and government agencies everywhere, and looks to be terrific for us in 2009.

Closer to home I am excited about some really cool stuff that we’re building into our products here at Appistry. Great stuff, and can’t wait until we can start talking more …

Heck, I am even excited about my new mouse! (every once in a while Microsoft does something really great, and this is one of them).

Critical Mass
There are a lot of individual reasons to be excited, true enough. Yet the real reason to be excited runs far deeper.

After years and years and years of tons of folks making progress in all things computing, slavishly building bigger and better nets; faster and cheaper processors, beau coup rotgut-cheap drives, processors, and more; fundamentally new business models that enable some serious capital-expenditure-avoidance; cloud-friendly ways to build scalable, reliable apps; and much, much more … I think that we’re finally here.

We’ve reached critical mass.

250px-Klein_bottle.jpg Critical mass as in all of computing is going to go Klein-bottle on us, turning inside out in many dimensions, imitating the caterpillar and coming out as something fundamentally new on the other side.
All of the right factors have come together … economic, technological, conceptual … probably even cultural.
Time to help create the Fourth Age of Computing, time to see just what is possible.
Like I said, this is a great time to be in the computing biz.
Butterfly image courtesy of The Butterfly House, a really cool contrast to the prevailing mainstream of a Midwestern winter … plus it’s close to where I live. If you’re ever in St. Louis, please go enjoy the Butterfly House. If I see you, I’ll buy you a beer!

Categories: Editorial Tags: ,

Meltdown 2008, Part 1 – How I Learned to Love Chaos

October 10, 2008 Leave a comment

I remember the first few days of what became Appistry very well. Not simply well as in “cool we’re starting something that’ll change the world” well, but really, really well as in “day 2 of our awesome new venture was … 9/11”.

800px-Wtc-2004-memorial.jpg

Not just some 9/11, the 9/11.

So as our team watched that morning in the same horror and disbelief that is etched so indelibly in our collective cortex, we were faced with a very practical question – what do we do now?

Sure we had a pretty strong hunch that we could actually make the world of commodity infrastructure safe, easy, and cheap for enterprise software. Probably even creating the most reliable computing infrastructure in the planet, and sure we had been able to raise seed funds fairly easily, but that was yesterday, even early this morning … what about today?

What about today, in a world where the markets weren’t just jittery, they were closed. Banks weren’t being sold, they were closed. Air travel wasn’t expensive and late, it was closed.

I even had a check for more than $200,000 in my pocket from an eager investor, except there was only one problem – the entire US financial system was on indefinite holiday.

Would we ever be able to raise another nickel? Would anyone ever care about what we were about to build?

A Few Initial Steps
As we slowly awoke to that new reality, with so many unanswerable questions, we took care of a few first things first. A few of us drove my brother down to Marine Corp 4th Division HQ in New Orleans, where he reported for the first of three post-9/11 tours – all combat, from mine clearing in Afghanistan to quelling instability in Fallujah – with a wife and seven kids at home (a cool story, but really best saved for another day).

We finished building our commodity development systems (hey, we’re seriously hardcore about making sure that we can use the simplest, least expensive computers for anything) and screwing together our desks, and went about all of the other time honored rituals of the New Venture that are so comforting, at least to the entpreneureus serialius.

A little slower than usual at first, sure … but we gradually picked up steam. Before we knew it we had answered that initial question about what to do next – we built our new technology, built our new company, made the vision real. – in short, we kept that original goal, the same audacious vision.

Sure we did some things differently than we’d planned – we had a little longer to incubate the fundamental technologies, we made even better use of funds that we’d ever thought possible – but the bottom line is that we still moved forward, still strove relentlessly to make that vision real.

And it worked well. Really, really well.

737px-Airplane_vortex_edit.jpg

Meltdown 2008
Which brings us to the interesting times in which we find ourselves now – financial commotions in every corner, gloom and doom on the street corner for free.

Should we grab everything we hold dear and run for the hills? Just give up and go back to school, waiting for some sort of “better times”?

Well that’s certainly one course, but I think there is a better way – a far better way. Let’s start by thinking about how the average enterprise (other than say, the unhappy folks at Uber-Leveraged-Bad-Debt LLC) is likely to respond to this disappointing news chaos.

The Big Picture
There are two macro-level trends at this time. First, there will be a natural tendency to become more operationally focused, to think less about some entirely new capability and more about doing what you do today better. Historically this is at a maximum level in the earliest portions of a downturn, when organizations are still coming to grips with the life in which they are now immersed.

Like a boxer who’s just taken an unexpected gut shot, most organizations will step back, shake their heads and begin to think about how best to (tentatively) take the next few steps.

Once they wake up from the shock, the fog clears and they realize – “Oh right … I’m still here and there’s lot’s to do” – the second macro trend starts to become apparent. In particular, each organization will begin to think clearly about the choices before them and how best to move confidently forward.

It is within this trend that companies will begin to move beyond coping with the problem and start to think very hard about how to drive cost out of their business while increasing core capabilities. And they’ll be looking to do this at as close to rock bottom prices as possible.

146-Backpack.JPGThese kinds of economic stress can actually become a great equalizer, at least in the sense that the rules are changing for everybody and so much is up for grabs. Imagine officials stopping a World Cup match or a baseball playoff game and ordering every player to wear 75 kg backpacks for the remainder. On top of that they decide that remaining games will be played in formal attire and kick out a few of the remaining teams for good measure.

The result would be chaotic, to say the least.

While all analogies have their limitations (and this might have more than most!), you get the idea – those who can adapt to the new rules fastest and first are going to win. Even more to the point, in the real life uncertainty in which we all find ourselves, some of the newer technologies are crucial to the quest.

In a few days I’ll post Meltdown Part 2, which is focused on what an enterprise can do with technology to thrive in these times.

Categories: Editorial Tags:

Parallelization: Multi-Core, In a Cloud, Here or There, Anywhere

October 3, 2008 Leave a comment

(with apologies to the good Dr. Seuss on the title – sorry, I just couldn’t help it)

banner-1.gifLast week I was privileged to participate in a panel on Parallel Computing at emTech08, a cool event put on by the MIT Technology Review publication.

The participants included

and me (go ahead and give me a hard time, I can take it).

The range of discussion was interesting, since the panel included perspectives more rooted in multi-core (Ghuloum and Leiserson), mass ‘o machines (Reed, Snir), and more of a uniform view of both broad classes (me, though I think that’s may be shared by some of the other folks as well, at least a bit).

In addition, the panel was a mix of research and practical applications, which probably tended to color much of the discussion. All in all it made for cool (and hopefully not too boring for the audience!) conversation.

This is one panel that I probably would have much rather had in private, definitely accompanied by some really good adult beverages, but unfortunately we were constrained to an hour on a stage … and (at least for me) that hour passed by pretty fast.

An hour was probably enough time to begin to name a couple of the larger issues, but definitely not time enough for too much more.

Still, it did get me to thinking a bit …

A Few of the Issues
There were (and are) many more of course, but here are a few of the more dominant themes and issues that we discussed …

Market Pull. Whether it’s the inability of the processor manufactures to build individually faster cores at any price we could stomach (hence the advent of multi-core), or the advent of practical clouds (both public and private) opening up the prospects of deploying REALLY BIG apps on LOTS of VMs, the market is clearly demanding new solutions to creating parallelized apps. No question about it.

Complexity is Bad. There was a general agreement that complexity is, well … complex and generally toxic to effective development of parallel apps. Some folks had more of a stomach for complexity than others, but all in all many of the efforts are trying to fundamentally simplify the developer’s task.

Need for New Abstractions. The Complexity Problem is not going to be solved by wishful thinking alone, no matter what Oprah says (sound bite alert). Hence everything from new functional languages like F#, Erlang, Scala, to frameworks like map-reduce, to data-driven reliable service abstractions like our own application fabric are in play as ways to simplify.

Uncovering Inherent Data Orthogonality. I’ve gradually come to the opinion that some very high percentage of the apparent data dependencies that are anathema to effective parallel processing are not truly in the original problem. Rather, they are false dependencies, ones that we have inflicted on ourselves for no particularly good reason other than the tools, methodologies, or just bad habits that we bring to bear on our work.

(btw, don’t press me on a precise percentage or I’ll be forced to make something up here)

We’ve seen this with customers, and the more I look at new problems and how they are solved in most enterprises today, the more I see a big, massive goo of false dependencies.

Fix those, and we have a crack at effective parallelization in many cases.

Where This is Going
I am very optimistic about progress in helping developers create actual parallel applications that can be used in the enterprise, in production solving problems about which people actually care.

The population of these well-done apps is going to be increasing dramatically in the months and years to come, which is a good thing … a very good thing.

The timing couldn’t be better … in truth, I don’t think we really have much of a choice.

We are very active in this space, and I have a particular interest in the “false dependency” problem. I’m sure I’ll be posting more on this in the future.

But It Goes To 11 …

September 9, 2008 Leave a comment

… or a small adventure with the laws of physics and the vagaries of press releases.

I have been paying a lot more attention to the world of things that fly around the Earth since we have been working with the great team of folks over at GeoEye these past few years.

Before I go any further, I want to throw out my high-fives and share in the elation over the successful launch of GeoEye-1 this past Saturday.

This is a great event for all around, and I think is a harbinger of a whole new class of imagery-enabled, imagery-consuming services.

I’m working on a couple of posts that will explore the confluence of these sort of uber-pipes of fresh, interesting data with our sort of arbitrarily scalable, reliable cloud-based apps. So many possibilities … but, as happens so often in my life I first went off on a slight detour …

o3b-spacevehicle.jpg

Something Odd
While thinking about these posts this little item caught my eye:

Google Invests in Satellite Broadband Startup

Ok, this all looks pretty cool – a new crack at using satellites to make it easier to get the ‘net out to remote parts of the world.

Their distribution model even looks pretty smart (wholesaling to ISPs), and their bandwidth admirable (claimed 10Gbs – cool!).

But then I got down to this statement:

O3b Networks uses parabolic antennas, which reduce latency.

Huh?!?

There’s nothin’ the antenna shape can do to change the fact that at the speed of light it still takes about 246 msec (about 1/4 of a second) to traverse the 23,000 miles UP and then the 23,000 miles DOWN to / from the satellite.

But It Goes to 11?
So I thought maybe this was just something lost in the translation to press release. So I went to their site, and found this statement:

O3b Networks’ system virtually eliminates the delay of standard GEO satellites by reducing the round-trip transmission time from over ½ second to just 1/10 of a second. The reduced round-trip delay creates a web experience closer to terrestrial systems such as DSL or Optical Fiber.

Ok, I had to admire their persistence. So, either they had actually figured out how to jump past the speed of light, or maybe the satellites were a lot closer to the ground.

Bingo!

From a story in NetworkWorld today

… O3b will be able to offer the same capacity for $500 or less by using different, cheaper medium-earth orbit (MEO) satellites.

Geosatellites orbit the earth at an altitude of 22,500 miles, while MEO satellites are around 5,000 miles. The latency, or the time it takes for a signal to make a loop between earth and the satellite, can be upwards of 600 milliseconds for a geosatellite because it is further out. For a MEO, latency is around 120 milliseconds, close to that of a fiber network

Darn, that actually makes sense … well, back to work on figuring out how to go faster than the speed of light …

Actually, I meant to say back to work on uber-scalable, cloud-based apps that deliver on the promise of cool new stuff like GeoEye-1!

Categories: Editorial Tags: ,
Follow

Get every new post delivered to your Inbox.