Moore's Law and Structural Deepening

Apropos Tyler Cowen's book The Great Stagnation: How America Ate All The Low-Hanging Fruit of Modern History,Got Sick, and Will (Eventually) Feel Better.

Matthew Yglesias points to Kevin Drum pointing out that:

World-changing inventions just don’t come around all that often, and when they do it takes a long and variable time for them to become integrated enough and advanced enough to have an explosive economic effect. Steam took the better part of a century, electrification took about half that, and computers — well, we don’t really know yet. So far it’s been about 60 years and obviously computers have had a huge impact on the world. But I suspect that even if you put the potential of AI to one side, we’re barely halfway into the computer revolution yet. To a surprisingly large extent, we’re still using computers to automate stuff we’ve always done instead of actually building the world around what computers can do.

This reminds me of W. Brian Arthur's book The Nature of Technology and Arthur's idea of structural deepening (as described here).

Yglesias has his own theory that advances in computer technology have in fact been "un-optimally" rapid. We have come to expect that new and improved computer technologies will arrive so soon that it isn't really worth refining an existing paradigm. Therefore structural deepening hasn't really happened in Arthur's sense of the term. There has instead simply been a succession of straightforwardly better technologies.

The quantum leap intelligence problem: are posthumans ineffable?

There is an idea prevalent in transhumanism that when posthumans or strong AI finally develop they will be to us as we are to the beasts that perish. They will be so much more intelligent that we will be incapable of understanding them.

They will possess super-science mojo and will live in mathematically optimal blocks of matte computronium, and they will have awesome tech that we puny baselines won't be able to distinguish from magic. They will be ineffable and godlike and we simply won't be able to understand them or their motives.

(Aside: for the sake of brevity for the remainder of this post I will refer to "them and us" to distinguish baseline humans from posthumans - not because I don't buy into the whole Kurzweil machine-human merger, but just because it's easier to write about)

I disagree with this idea of posthuman ineffability.

The idea suggests that there are other ways of being intelligent (i.e. possessing a highly accurate model of the outside universe and a highly accurate model of yourself and your fellows, thereby enabling self-reflection, communication, and culture ¹) that are an entire quantum-leap above human intelligence such that we won't be able to comprehend them or their actions.

Michael Anissimov has written an interesting article making the point that human beings are dumb. In fact we possess only the bare minimum of intelligence required to create the civilization we have now.

Michael Anissimov makes some good points in this article on the current state of human intelligence:

Hey, human philosophers — I’ve got some bad news. It turns out that Homo sapiens probably isn’t the qualitatively smartest possible being.

...

How do I know? Well, most other members of the genus Homo had plenty of time to build agricultural civilizations, but they were too unintelligent to get off the ground. Homo sapiens was just barely smart enough to do the trick. And like a self-replicating machine that moves from 99.9% closure to 100% closure, the payoff was big.

I agree with this as far as it goes. All it took to develop complex social technologies like language and complex physical technologies like bows and arrows was a small increment in intellectual capacity.

We made that quantum leap from animal to human around 100 000 years ago: in the intervening period we haven't evolved a great deal (in fact, some say we've stopped evolving at all).

Ergo we are possessed of the bare minimum intellect required to sustain and develop technological civilization.

Anissimov uses this as an argument in favour of the idea that there there are many more superior modes of intelligence that we haven't yet developed or encountered, or in his words:

The apparent magnitude of our accomplishments, including those of Einstein, is merely a side-effect of how low our standards are. To another species on another world whose intelligence was crafted in the furnace of selection pressures more intense than ours, quantum mechanics is obvious from the get-go. The only thing funnier about how dumb we are to take so long to figure it out is our self-importance at having finally figured it out.

This is where I disagree with Anissimov. I think that the miniscule quantum leap between pre-human animals and human beings is a one-time event. Improvement is certainly possible, but to claim that there is some qualitatively and quantitatively different perspective on the universe that is definitively superior in every measurable dimension to human thought and would result in beings that we are incapable of understanding is incorrect.


Here's why:

1. Human knowledge and understanding does not progress wholly through deductive reasoning or pure cognition. In fact a large amount of human knowledge and understanding comes from what Nassim Nicholas Taleb calls stochastic tinkering, and Eric Beinhocker calls deductive tinkering. Trial and error and accident has contributed enormously to the development of human knowledge. Presumably a posthuman would make mistakes: otherwise how would it learn? And if it doesn't learn how does it grow and develop?
   
2. I agree with Kevin Kelly that there is a fallacy in the idea of "thinkism." Thinkism is the idea that it is possible for a mind, completely ignorant of the workings of the physical universe, to consider a few small objects, like a rock, flower, a feather, and a model of a galaxy embedded in amber, and then use these items to deduce the workings of the physical universe without any recourse to experiment. It could well be that there are other universes with different physical laws that could generate those items and without recourse to experiment how would this mind know which universe it lived in?
   
3. We will share the same universe (they may go elsewhere, of course, but the chances are baselines will stay here): as such this posthuman entity will be subject to all the usual laws of entropy, conservation of energy, gravity and whatnot. As such this posthuman will need things and do things that are explicable to us. Only by creating a solipsistic alternate-reality computer bizarro world could a posthuman behave in a completely ineffable fashion: and even then a posthuman would still be subject to the axioms of a given logical or mathematical environment. N-incomplete problems would remain so.
   

Looking at point 1 "how does a posthuman learn" suggests an interesting counterargument: "posthumans develop in a way that doesn't involve learning, they use something different and ineffable."

The problem with this counterargument is that what I'm arguing is empirically refutable: the existence of a truly ineffable posthuman entity is something that is observable, so my point can be refuted by the observation of one posthuman entity whose motives and actions we do not understand. However transhumanist thinkers can continue asserting that true posthumans are by definition ineffable until the end of time. I predict that as posthumans emerge and their actions are studied they will eventually always be found to be explicable by baseline humans (if weird and peculiar - see below).

I agree that there are almost certainly better modes of intelligence, but I disagree with the idea that these modes of intelligence will ever be wholly incomprehensible to baseline humans.

They may be faster, cleverer, wittier, more attractive, stronger, longer-lived, instatiated within superior hardware, and better at poker - but it doesn't mean baseline humans would be incapable of understanding them.

The distinction between what I'm arguing and what Anissimov implies in his article is fairly subtle, and I could be accused of nit-picking, but I think it's important that we realise that there is no reason to assume posthumans will be completely and utterly ineffable to us, at least not if they want to survive IRL (which they may not).

That humans have accomplished what we have says more about the power of the evolutionary methods of stochastic tinkering combined with occasional deductive reasoning than it does about the brilliance of human intellect: and this is exactly what Anissimov is saying and why I agree with the premise of his article.

But once you possess culture (what Ian Stewart and Patrick Cohen call extelligence, or what Richard Dawkins might term "a memetic environment"), and a reasonable means of manipulating the universe it doesn't matter how "smart" you are. Trial and error and learning and robust heuristics take care of the rest.

I believe that some posthumans will be pretty weird, some may be charismatic, some may be frightening. But we can get to where they are, they are post-humans and they took a path we will be able to follow. Because of this and for the reasons given above I don't believe posthumans or posthuman civilization will ever be truly ineffable.



1: In fact it could be that this superior intelligence works on a completely different basis to creating a highly accurate model of the universe and the self, and works on some other basis that we can't comprehend. This non-intelligent "intelligence" would be truly ineffable and would completely disprove my point if it actually was superior in every possible way to human intelligence.

Further reading: if you do understand precisely what I'm trying to say I should mention that it isn't original, Greg Egan argues something similar in the opening chapter of Schild's Ladder.

Which programming language am I?

For years now I've been tinkering around with computer programming languages.

Back when I was thirteen I bought a copy of C for Dummies in my local Oxfam shop. I learned a little but gave up half way through.

More recently I've been reading Sams Teach Yourself Java 6 by Rogers Cadenhead and Laura Lemay. I also have an up-t0-date copy of Sams Teach Yourself C++ Liberty Jones on my desk.

Another textbook that's floating around somewhere is PHP 6 and MySQL 5 Visual Quickpro Guide by Larry Ullman.

I'm also learning Javascript for one of my OU courses at the moment.

In short I've made my usual mistake of not focusing on one language and letting myself get distracted by other projects.

The motivation behind my interest has always been my desire to more fully understand the computer I use so much.

Cory Doctorow commented, in this review of his recent book, that:

"Even if you only write code for one day, one afternoon, you have to do it. Computers can control you or they can lighten your work — if you want to be in charge of your machines, you have to learn to write code."

Which is good advice, I guess. Chris Applegate has written a brief guide as to how you might start off, and recommends Python as a good place to start:

If you work in any information industry, or are thinking about a career in it, learn to code. And by code I don’t mean learn something hardcore like Java or C++, or even learn a full programming language (as you’ll see below). But it means getting above the usual abstractions you see - your web browser, Word, Excel - and getting involved at a deeper level, get to appreciate what the data it is you’re reading and realise it’s not just something to look at.

Python (along with Perl) is one of the few languages I haven't dickered around with at some stage.

I think that the important thing is that I focus on one language at a time.

Finance and computers: an analogy

A computer takes digital data and transforms it in fairly basic ways. Adding, subtracting, logical statements etc.

Computers are powerful tools because we can make them perform more complex tasks than these basic functions by building new layers of abstraction on top of these basic processes.

Machine code can be used to make a compiler for the C programming language, which can in turn be used to make an operating system, on top of which can run applications, which can be used in ever more complex and elaborate ways.

Abstraction and "higher order" properties are important in computing, but when the same ideas are applied to finance things go all gooey.

The basic unit of economic interaction is not a bit or a logic gate; it is a human being, an absurdly complicated thing, and one that we don't fully understand.

In computing, logic gates and machine code are fairly simple. Because they are simple and well understood, we can build a layer of abstraction on top of them and rely on them to function correctly whilst we pursue higher order things (like writing and reading blogs).

This is what SF writer Iain Banks calls the "dependency principle" - complex software is based on a simpler layer beneath it, which is in turn based on an even simpler layer beneath it, until you get out of software and into the bare metal.

In the recent economic troubles - the credit crunch caused by the insidious spread of bad mortgage debt and the fact that banks now don't know how much these assets are worth (if anything) - can be thought of in similar terms to the structure of software.

A layer of abstraction is based on a simpler substrate: derivatives that are based on the risk of a given mortgage defaulting.

The difference between finance and computers is the layer beneath the abstraction isn't straightforward and predictable.

The Heavens align...

I finally decided to bite the bullet and buy a new PC (with Vista, natch --- read the tagline) and now this happens.

ION: I've been reading Neal Stephenson's classic essay In the Beginning was the Command Line.

Let me just review my real reasons for not just using Ubuntu:

1. I use Dreamweaver and Photoshop, and I see no reason why I should install a completely different operating system just so I can run two of the three applications I actually use in WINE.
2. That's it.

What I really need to do is buy a new hard drive for the 'ol lappy and fire up Ubuntu on that.

Aha. A plan.

New PC

I have a new PC. It is joygasmically fast. It is one of these:


The keyboard supplied didn't work, irritatingly, but the company are sending a new one and I bought a cheap one for £7.96 from Tesco for the time being.

[image from Arbico]

And You're Only Just Realising This?

There comes a point in every man's life when he realises that he is almost certainly never going to create one of the fundamental circuit elements of electronics.

Also: one of the things that struck me as odd about the recent discovery/invention was how old-fashioned a discovery it seems.

We are no longer used to "fundamental" breakthroughs in areas other than the biological sciences, as Charles Stross comments in this interview:

"We seem these days to be seeing new ground-breaking theoretical developments at a rate of one every six months to a year: breakthroughs on the same order as general relativity or quantum theory. (You don't see such breakthroughs routinely in physics, which is a relatively mature field, but if you look into the biological sciences equivalent breakthroughs appear to be coming thick and fast.)"

There is something wonderfully retro-1950s-buttoned-down-white-labcoat-brylcreme- and-horn-rimmed-glasses about the invention/discovery of the "memristor."

Sadly my knowledge of electronics is ever-so-slightly too limited to truly grasp the theoretical implications of this. However the practical implications look extremely interesting:

"Today, most PCs use dynamic random access memory (DRAM) which loses data when the power is turned off.
But a computer built with memristors could allow PCs that start up instantly, laptops that retain sessions after the battery dies, or mobile phones that can last for weeks without needing a charge."

I spend at least ten minutes every morning waiting for my PC to power up at work (yes I know I could agitate for a better PC... but [deleted due to imminent curtailment of career prospects - free speech go hang]).

Imagine all the time you've spent waiting for a PC to power up: adding up all those two to three minute gaps could make a lot of difference in the world. You probably wouldn't even notice power cuts.

Of course my reading of this is that "instantly" means within a second or two and that the computer would retain the current session.

Anyway there's another thing off my list of things to do before I die...

C'est la vie.

The Asus Eee, Google and Cloud Computing

I saw an Asus Eee in an electronics shop today. It is qualitatively different from any laptop I have ever seen. It has the feel of a child’s plaything, and carries the same air of cheapness.

As Charles Stross says, the laptop and PC are both heading for a period of commoditisation. As the cost of laptops, PCs, and consumer electronics in general start to fall they will become as expendable as pocket calculators and digital watches.

Aside from the size, the two things that struck me about the Asus Eee are how quickly it starts up, and the operating system. The OS is some Linux derivative. Because the Eee has a solid-state hard-drive and no disk drive it has no moving parts (except for the hinge and keyboard) and as such immediately feels less fragile than a normal laptop. There are no air vents either. Altogether it gives the impression of something you can pour tea on and drop on the floor and it will still work.

The OS is pretty straightforward. There is no messing around with desktop metaphors. There is a simple tab-based menu system with applications grouped into web, office, games, education etc. A great deal of the Eee functionality is based on connections with the Internet, and specifically with the web.

As I said, the laptop restarts in seconds. One of the single biggest causes of minor stress in my life is slow computers. It is a joy to finally discover one that starts in less than ten seconds.

These days most laptop/PC functionality for the casual user is connected with the Internet. Even office programs are now being brought online. Games are now played online. Programs can be developed online (I can’t actually bring an example of this to mind but I know they are there).

Would I buy an Asus Eee? Yes. If I had the disposable income, and if I actually needed one, and if I didn’t know that before my current lappy conks out there will be many cheaper and better laptops in the same league of cheapness and niceness as the Eee.

Google and Cloud Computing

If the Eee teaches us that laptops will become (even) cheaper and (more) ubiquitous then other movements in the world of technology show us even more about the nature of the world ahead.

One of the big buzzwords of recent months has been the idea of “cloud computing” where instead of running programs on a box on your desk, you just enter the data and the commands and they are actually processes in a big “cloud” out on the Internet. Eee-style computers will presumably one day be connected to the Internet via high-bandwidth links and farm out surplus processing to large servers many kilometres away.

Companies like Google have recently been moving towards support of this sort of computing. The Register article where I first noticed this is here. Those at The Register use the charming phrase data smelters (coined here) to describe the vast powerhouses of computation that exist for the purpose of swapping bits in the service of the Net.

The problem for Google is that what they really have (and all they’ve ever really had) is a good search algorithm, a superb brand, and a lot of computers. The search techniques that made Google so successful have now been copied by other search engines. The brand remains as strong as ever, and will be crucial to Google’s long term survival.

The last point: the lot of computers, gives us our glimpse into the future, not just of Google, but of computing in the second quarter of the 21^st century. Vast data-warehouses connected via high-bandwidth links to thin clients like the Asus Eee.

Moore’s Law will probably chunter along for another few decades and then we’ll be left with ludicrously cheap laptop/mobile form-factor devices equipped with prodigious memory and processing power themselves, but connected via high-speed links to processing yards many orders of magnitude more powerful.

Every individual on the planet will own one of these cheap laptop/mobile form-factor devices but only the largest companies and states will be able to pay the huge costs of running the data centres.

Of course this could all be complete rubbish. It could be that swarm computing networks will emerge that reject the centralised client-server model in favour of a more egalitarian “flat” model without hierarchies.

Anyway: I’m willing to bet an Asus-Eee-equivalent of 2018 (i.e. £50 or thereabouts) that the next major upgrade of the MacBook family will involve a solid-state hard-drive.

Tom out.

UPDATE 21/03/2008: Goodness me I am such a fool. The most recent update of the MacBook family, the MacBook Air, does have a solid-state hard-drove option.

del.icio.us and 6pli

I've applied the ubercool data visualisation tool 6pli to my del.icio.us account: it's pretty awesome, and give me a new appreciation for how my interests overlap and relate to each other.


The results can be viewed for yourself here.

Ubuntu Update

This is by way of being an update, rather than a full blown review.

There are many aspects of Ubuntu that I like and there are some I dislike. The ones I like are:

1. There is no need for tedious and memory-consuming virus scans.
2. When new hardware is plugged in Ubuntu waits for me to do something about it, instead of trying (and failing) to be helpful by providing Autorun features as Vista does.
3. Change is good. It's refreshing to use something other than Windows.
4. There seems to be much greater scope for personalisation than with Windows.
   

All these things said, for the average and casual user there isn't a great deal in any of this. Most of the above points are more to do with the general crapness of Windows rather than anything good about Linux/Ubuntu.

Things I don't like about Ubuntu:

1. After dual booting Ubuntu with Vista I can no longer disable my Synaptics touchpad.
2. Ubuntu does not include native support for DVDs or mp3 players. I know that this is a niggle and shouldn't be a big obstruction, but for the casual and lazy user (like myself) it is just irritating.
3. New things scare me.
   
4. Ubuntu defaults to being so like windows that there doesn't really seem to be much point.
   
5. I've been prodding the bash shell or whatever it's called and it's all very oldschool and cool but to be honest I don't want to have to learn a whole new language just to get my PC to work when a GUI would do. And yes, I know that I can do pretty much everything through the GUI but I'm lazy.

OK - my conclusion so far is that there really isn't much point to Linux. If it's ever going to go mainstream it will be through things like the Asus Eee, which Charles Stross comments on at length here.

I'm still having trouble understanding the ubiquity of Microsoft Office in business, when OpenOffice is free and does exactly the same thing (at least as far as 90% of corporate users are concerned).

I think when I buy a new PC or laptop the first thing I'll do is install Ubuntu and use it from day one. This way I will avoid falling into the habit of using Windows.

Ubuntu Linux

As of today I am officially an Ubuntu Linux user. So far everything works very well.



Hooray!

Your move, Mr Doctorow...