The CTO of Amazon.com just did a presentation with customers on Amazon’s Web services. Worth looking, I think, at a paper he referenced by John Hagel (here) and John Seely Brown on the topic of on-demand resources. It’s here (PDF), and looks like interesting reading. (Hagel was author a few years back of net.gain, which was an effort to talk about how businesses could use online community - I wasn’t that impressed by it, but he seems to have continued from there). Otherwise, the presentation was pretty much an infomercial (they had to rearrange this morning’s programme when one of the speakers pulled out - she was getting death threats on her blog, about which more later; they filled in at least part of the time with a rehash of yesterday’s Wall Street stuff).

WG

I saw a piece a year or two back - in Business Week, before it shopped all its European customers the hideous digital edition - about “dark nets”, smaller, private networks of people who know each other and swap copyrighted content illegally. Because they’re private, the activity can’t be monitored the way P2P networks can. And so on. It all sounded terribly sinister until I realized that it’s jsut the stuff we all do.

I was reminded of this in the remaining session from yesterday’s O’Reilly Radar, which talked about Web 2.0 and Wall Street. Peter Bloom said that the big guns now have their own private exchanges, and an order only makes it out onto the open market if it can’t be matched immediately - that is, it’s not good enough for them. There are now 31 alternative trading systems dark pools of liquidity, and “14 families” (he didn’t list them all, but he did mention Citicorp, Morgan Stanley, and JP Morgan Chase) represent 95% of capital trading, and the market is driven on the buy side by hedge funds.  Also, in order to conceal what they’re doing, people with large orders break them up into tiny chunks - the average order size has declined to under 400 shares.

This is all of course huge change from as recently as the 1970s, when transactions were expensive (22 cents a share then, .02 cents a share now) and the talent, he said, was on the sell side - investment research was published and accessible to anyone. Now, research information has become a proprietary asset on the buy side.

Web 2.0 enables some interesting ideas around all this. For example, betting on the trader instead of the trade for short-term trading (the anti-Buffett approach). (Playing the man instead of the ball.)  Bloom mentioned a guy in Toronto who set up a system in which you start with $100. If your investments consistently pay off, he increases your capital allocation. But the point is “behavioral finance”, in which if someone’s a good trader you copy their trades; you don’t bothe r researching the securities they trade. He didn’t say this, but of course this is the promise managed funds have made for years - that their experts can produce consistently good results (which in general is not true - automated index funds generally do better). Bloom speculated that this approach could be extended to health care (treatments, outcomes, diseases) and consumer electronics buying - anything where people can benefit from collective intelligence. (I will refrain from mentioning collective stupidity; the idea behind this is to Darwinishly weed out the bad ideas and stupid choices in favor of the good ones.) He also mentioned valueinvestorsclub.com where you have to come up with good ideas that can be shared and rated on the site. In our conference bags are “dollars” for a project of his - an attempt to make a sort of eBay for charity funding.

Government is like the space program: it can’t afford to experiment. It’s hard to see how government could use these kinds of collective curation. But what about organised benefit fraud? governmentbenefitsclub.com?

wg

Look at this webcam at Stuttgart airport - you can control it, and see the results. This is The Transparent Society described in David Brin’s controversial book. Does sousveillance by “us” cancel out surveilllance from “them”, or does it compound the unexpected consequences?

See http://www.defeatingthehacker.com/videoham/

Invest in diesel generators. Why? According to Paul Kedrosky (venture capitalist) and Rich Miller (who keeps a blog about data centers!) the big constraints on building data centers isn’t any of the obvious things like availibility of contractors or whatever but that there is an 18-month lead time at the moment on 2megawatt diesel generators - and you have to have them as power backups. Or buy land where energy is plentiful and cheap, because that’s where all these big companies are going to need to build their data centers. Ten years from now, the big names in energy will be companies you’ve never heard of now. GE will survive, though, because it has a long history of survival.

Economics encourage people to waste things that are cheap. Currently, what’s cheap is processors. So people put in thousanjds of things, only to discover that the power to run them isn’t cheap - over the last 3-4 years power consumption per rack has gone up 3-4x. Plus the numbers keep going up - Google expects to have 800,000 blades by 2011. And even if you start installing more energy-efficient stuff now (Intel and AMD are both making pitches that their dual-core chips are more power-efficient than the older generation) it will be years before all the legacy stuff is replaced. (He likened this to old cars on the highway; but in fact the cars of the late 1970s and early 1980s were often more fuel-efficient than today’s fashionable SUVs. I have lots of friends that get worse gas mileage in their 5yo cars than I do in my 20yo Nissan Prairie.) It strikes me as ironic that just as everyone in the UK is getting all fired up about being green and energy efficiency they are all at the same time installing air conditioning and leaving more and more computers switched on all the time. One point Alec Proudfoot (of Google) made is that a lot of savings can be achieved by improving energy efficiency - eg, an ordinary PC PSU is about 70 percent efficient. Increase that to 85 percent, and you can achieve megawatt savings without turning anything off.

Which reminds me that yesterday over breakfast UK journalist Jon Honeyball told me that there is an incredible vulnerability in London relating to the path the pipes take to the main area where everyone’s disaster recovery backups are stored. Kind of like the fact that the main pipe carrying water to LA goes over the San Andreas Fault.

All of which goes to show that the systems we’re building are vulnerable to all sorts of non-obvious constraints. Like the piece Simson Garfinkel wrote in Wired some years back about “Fifty Ways to Kill the Net”. Number one was: “Buy ten backhoes.”

wg

Lunchtime conversation: UPS has decided its drivers are not going to turn left any more. (Apparently they haven’t backed up - reversed - for years.) Saving money through geographical planning. GPS systems don’t know about the amount of time a left turn takes compared to a right one (reverse as needed for UK).

This segued into a conversation about business models and how people decide what to charge for. For example, said one guy, take all those sites that give you, say, 10 wiki pages free but charge for an account that gives you up to 100 wiki pages. Well, wiki pages are text, and they don’t take up much space or bandwidth, so what’s being charged for isn’t actually resource-expensive. Why not find things users really want that *do* cost something to provide? (Which also then prevents, say, Google from coming along and stomping all over you like Bambi Meets Godzilla and promising everybody 1,000 free wiki pages just to scoop up all the traffic.)

Same guy also made the point that often there are hidden costs we don’t register. For example: processing power is cheap. Cheap does not equal easy. “So’s walking, but it still takes a long time.” You add cheap processors - and also expensive communications and integration problems. Plenty of obvious gotchas here for government computing.

wg

A significant problem for organizations (and governments) is that data is discrete. eg, a Las Vegas casino wants to know if someone they’ve already thrown out is making a new hotel reservation under another identity; or a care agency doesn’t want to place a child in a home with a violent criminal. The inability to link these things is enterprise amnesia. (Something the UK govt currently has a lot of.)

Jeff Jonas works on Non-Obvious Relationship Awareness - trying to make those links.  One point he makes is that as you add data eventually you get to a tipping point where the data starts to collapse. That is, say you have records of Wendy Grossman, Wendy M. Grossman, and various misspellings of those. You don’t know at the moment whether these are all the same person, but as you keep adding records eventually you add one that links some of those misspellings so that these apparently distinct people begin to collapse into a single person. This is what alarms privacy advocates; and yet, says Jonas, there are more and more increasingly cheap weapons that can do more damage than a nuclear weapon.

Jonas also makes the point that fabricated identity is harder to find than a stolen one because in the case of identity theft someone complains. This is the thickness of people: someone in their 50s who’s never owned a car or house, had a job, and has no educational rcords is much more likely to be fabricated. This person is thin, in data terms. Of course, this also might be simple lack of documentation…

wg

Another example of data you might expose to the world comes from Attention Trust (its owner, Seth Goldstein, is talking about it now). This an effort to help people regain control of their own clickstreams - but at the same time they can share them with others, another example of the stuff I posted about last night.

For every explicit video you make about yourself, he’s saying right now, you generate a zillion implicit data points - metadata around yourself that for years companies have been able to take advantage of but you have not. This is therefore an attempt to make visible the data we throw off without realizing it. An interesting idea for making the impact of our using government eservices more transparent?

wg

Also over breakfast, someone I know from the WELL told me he’d just overheard from the next table: “Content providers should be paid for the content they create.” Immediate flashback to a WELL picnic 17 years ago, where another WELLbeing said exactly that to Howard Rheingold. This kind of thing happens all the time because every time a new wave of people gets interested in technology they “discover” issues that they’re sure no one has ever thought about before. The discussions we are having today about security, copyright, anonymity, integrity of data…have been had over and over again going back to before the Internet was even invented. I have a friend who says in school he was taught that the Greeks had figured out everything, and it’s only since the 1960s that we reject all historical authority.

The first part of this morning’s O’Reilly Radar talks about the fact that due to liability issues teens today don’t get to use heavy machinery or discover talents they might have for actually building things. Designers now often don’t actually know how to make things.

Dale Dougherty fleshed this out with some discussion of geeks having fun - making railway cars out of bicycles and old carts, solar-powering a shipyard, making all kinds of very strange things out of old bicycle parts - not to be green, but just because they love bicycles. Brian Warshawsky then demonstrated a wind-up power source his company, Potenco, is working on to power the one-laptop-per-child. It looks exactly like a yoyo.

wg

Tim O’Reilly: Where people are having fun is where the nexus of the future is happening.

(Why we should talk to hackers.)

wg

Over breakfast this morning, a former UPS guy now working in government told the story of a friend who decided for various reasons that he’d rather store his data backups on hard drives than on optical media - CDs/DVDs. He cornered manufacturers to find out how long he could expect his data to last if the hard drives were left powered down. No one could really tell him, but eventually he forced the answer that possibly it would be all right as long as he turned them on once a year. Hard drive manufacturers don’t think about this kind of use; they think of their products as being designed to be powered up all the time.

In a large system, how many decisions are made to use technology pieces in ways that were never intended by their manufacturers and that could affect the integrity of these systems?

wg