irrationality

24 June 2019

Superintelligence, Rationality, and the Race to Save the World

Filed under: AGI, books, irrationality, risks — Tags: CFAR, Eliezer Yudkowsky, Less Wrong, rationality, Superintelligence, Tom Chivers — David Wood @ 11:45 pm

What the world needs, urgently, is more rationality. It needs a greater number of people to be aware of the mistakes that are, too often, made due to flaws and biases in our thinking processes. It needs a community that can highlight the most important principles of rationality – a community that can help more and more people to learn, step-by-step, better methods of applied rationality. And, critically, the world needs a greater appreciation of a set of existential risks that threaten grave consequences for the future of humanity – risks that include misconfigured artificial superintelligence.

These statements express views held by a community known sometimes as “Less Wrong” (the name of the website on which many of the key ideas were developed), and sometimes, more simply, as “the rationalists”. That last term is frequently used in a new book by science writer Tom Chivers – a book that provides an accessible summary of the Less Wrong community. As well as being accessible, the summary is friendly, fair-minded, and (occasionally) critical.

The subtitle of Chivers’ book is straightforward enough: “Superintelligence, Rationality, and the Race to Save the World”. The race is between, on the one hand, the rapid development of technology with additional capabilities, and on the other hand, the development of suitable safety frameworks to ensure that this technology allows humanity to flourish rather than destroying us.

The title of the book takes a bit more explaining: “The AI Does Not Hate You”.

This phrase is a reference to a statement by one of the leading thinkers of the community in question, Eliezer Yudkowsky:

The AI does not hate you, nor does it love you, but you are made of atoms which it can use for something else.

In other words, the existential risk posed by artificial superintelligence isn’t that it will somehow acquire the human characteristic of hatred, but that it will end up following a trajectory which is misaligned with the best interests of humanity – a trajectory that sees humans as a kind of irrelevance.

To be clear, I share this worry. I’ve given my reasons many times on this personal blog, and I wrote up my own analysis at some length in chapter 9, “Towards abundant intelligence”, in my most recent book, “Sustainable superabundance”. My ideas have been shaped and improved by many things I’ve learned over the years from members of the Less Wrong community. Indeed, my presentations about the future of AI generally include several quotations from Yudkowsky.

However, these ideas often cause a kind of… embarrassment. Various writers on AI have poured scorn on them. Artificial superintelligence won’t arrive any time soon, they assert. Or if it does, it will be easy to keep under human control. Or if it transcends human control, there’s no reason to be alarmed, because its intelligence will automatically ensure that it behaves impeccably towards humans. And so on.

These critics often have a second string to their analysis. Not only do they argue for being relaxed about the idea of existential risks from superintelligence. They also argue that people who do worry about these risks – people like Yudkowsky, or Oxford University’s Nick Bostrom, or Stephen Hawking, or Elon Musk – are somehow personally defective. (“They’re egotistical”, runs one complaint. “There’s no need to pay any attention to these people”, the critics continue, “since they’re just philosophers, or mathematicians, or physicists, or business people, etc, rather than being a real AI expert”.)

At an extreme, this set of criticisms expresses itself in the idea that the Less Wrong community is a “cult“. A related objection is that a focus on humanity’s potential extinction is a distraction from much more pressing real-world issues of the present-day and near-term future – issues such as AI algorithms being biased, or AI algorithms stirring up dangerous social divisions, or increasing economic inequality, or disrupting employment, or making weapons more dangerous.

It’s in this context that the book by Chivers arrives. It tackles head-on the controversies around the Less Wrong community – controversies over its ideas, methods, aims, and the lifestyles and personalities of many of its leading figures. It does this carefully and (for the most part) engagingly.

As the book proceeds, Chivers gives voice to the various conflicting ideas he finds in himself regarding the core ideas of the Less Wrong community. My own judgement is that his assessments are fair. He makes it clear that, despite its “weird” angles, the community deserves more attention – much more attention – for its core ideas, and for the methods of rationality that it advocates.

It’s a cerebral book, but with considerable wit. And there are some touching stories in it (especially – spoiler alert – towards the end).

The book provides the very useful service of providing short introductions to many topics on which the Less Wrong community has written voluminously. On many occasions over the years, I’ve clicked into Less Wrong material, found it to be interesting, but also… long. Oh-so-long. And I got distracted long before I reached the punchline. In contrast, the book by Chivers is divided up into digestible short chunks, with a strong sense of momentum throughout.

As for the content of the book, probably about 50% was material that I already knew well, and which gave me no surprise. About 30% was material with which I was less familiar, and which filled in gaps in my previous understanding. That leaves perhaps 20% of the content which was pretty new to me.

I can’t say that the book has made me change my mind about any topic. However, it has made me want to find out more about the courses offered by CFAR (the Center For Applied Rationality), which features during various episodes Chivers recounts. And I’m already thinking of ways in which I’ll update my various slidesets, on account of the ideas covered in the book.

In summary, I would recommend this book to anyone who has heard about Less Wrong, Eliezer Yudkowsky, Nick Bostrom, or others in the extended rationalist community, and who is unsure what to think about the ideas they champion. This book will give you plenty of help in deciding how seriously you should take these ideas. You’ll find good reasons to counter the voices of those critics who seek (for whatever reasons) to belittle the Less Wrong community. And if you end up more worried than before about the existential risks posed by artificial superintelligence, that’s no bad thing!

PS1: For a 10 minute audio interview in which Tom Chivers talks about his book, visit this Monocle page.

PS2: If you want to see what the Less Wrong community members think about this book, visit this thread on the Less Wrong site.

Comments (1)

7 August 2015

Brave new world – bold new adaptation

Filed under: futurist, happiness, irrationality, theatre — Tags: Aldous Huxley, Brave New World — David Wood @ 9:05 am

Q: What do the following cities have in common: Northampton, Edinburgh, Oxford, Nottingham, Cheltenham, Wolverhampton, Darlington, Blackpool, and Bradford?

A: They’re the locations which have theatres featuring in the forthcoming tour of a bold new production of Aldous Huxley’s Brave New World.

“Brave New World” is a phrase that frequently enters discussions about the future. Even people who have never read Huxley’s book – or people who have long forgotten the precise contents – recognise the phrase as a warning about the future misuse of technology. In Brave New World, people lead lives that are… comfortable, even blissful, but which lack authentic emotional experience. As a result, technology leads to a curtailment of human potential. Overall, humanity is diminished in that Brave New World, despite the gadgetry and convenience of that future society.

The version of Brave New World that’s about to go on tour has a script by Dawn King, is directed by James Dacre, features original music from These New Puritans, and is produced by Touring Consortium Theatre Company. The cast includes Sophie Ward, Abigail McKern, William Postlethwaite, Gruffudd Glyn, Olivia Morgan and Scott Karim.

I found out about this forthcoming tour a couple of months ago, when I was asked to come to speak, as a futurist, to representatives of the different theatres which would be hosting the play. Could I provide some perspective on the play, and why it is particularly relevant today?

I took the chance to read the script, and was struck by its depth. There are many layers to it. And despite Huxley having written the novel as long ago as 1931, it has many highly contemporary themes. So I was happy to become involved.

The team at Touring Consortium Theatre Company filmed what I had to say. Here are some short extracts:

Are we nearer to a Brave New World than we think? The pace of change is accelerating.

Factors that will shape the next 10-20 years.

Technologies from Brave New World that are almost within our grasp

Some of the social changes we’ve seen that are eerily close to what Aldous Huxley predicted in 1931.

What questions does Brave New World pose for today’s society?

Note: for the touring schedule – from 4 Sept to 5 Dec 2015 – see this listing.

To read more about Brave New World from a transhumanist perspective, see this website by philosopher David Pearce.

Leave a Comment

26 September 2013

Risk blindness and the forthcoming energy crash

Filed under: books, carbon, chaos, climate change, Economics, irrationality, politics, risks, solar energy — David Wood @ 11:28 am

‘Logical’ is the last thing human thinking, individual and collective, is. Too compelling an argument can even drive people with a particularly well-insulated belief system deeper into denial.

The Energy of Nations: Risk Blindness and the Road to Renaissance, by Jeremy Leggett, is full of vividly quotable aphorisms – such as the one I’ve just cited. I see Jeremy as one of the world’s leading thinkers on solar energy, oil depletion, climate change, and the dysfunctional ways in which investment all-too-frequently works. The Observer has described him as “Britain’s most respected green energy boss”. A glance at his CV shows an impressive range of accomplishments:

Jeremy Leggett is founder and chairman of Solarcentury, the UK’s fastest growing renewable energy company since 2000, and founder and chairman of SolarAid, an African solar lighting charity set up with 5% of Solarcentury’s annual profits and itself parent to a social venture, SunnyMoney, that is the top-selling retailer of solar lights in Africa.

Jeremy has been a CNN Principal Voice, and an Entrepreneur of the Year at the New Energy Awards. He was the first Hillary Laureate for International Leadership on Climate Change, chairs the financial-sector think-tank Carbon Tracker and is a consultant on systemic risk to large corporations. He writes and blogs on occasion for the Guardian and the Financial Times, lectures on short courses in business and society at the universities of Cambridge and St Gallen, and is an Associate Fellow at Oxford University’s Environmental Change Institute.

On his own website, The triple crunch log, Jeremy has the following to say about himself:

This log covers the energy-, climate-, and financial crises, and issues pertinent to society’s response to this “triple crunch”…

Let me explain why am I worried about oil depletion, climate change, and dysfunctional investment.

I researched earth history for 14 years, and so know a bit about what makes up the climate system. I researched oil source rocks for several of those years, funded by BP and Shell among others, and I explored for oil and gas in the Middle East and Asia, so I have a background in the issues relevant to peak oil. And more recently I have been a clean-energy entrepreneur and investor for more than decade, as founder of a solar energy company and founding director of a Swiss venture capital fund, so I have seen how the capital markets operate close to. That experience is the basis for my concerns…

Many of the critics who comment on my blogs urge readers to discount everything I say because I am trying to sell solar energy, and so therefore must be in it for the money, hyping concerns about climate change and peak oil in the cause of self enrichment. (As you would). They have it completely the wrong way round.

I left a lucrative career consulting for the oil industry, and teaching its technicians, because I was concerned about global warming and wanted to act on that concern. I joined Greenpeace (1989), on a fraction of my former income, to campaign for clean energy. I left Greenpeace (1997) to set up a non-profit organisation campaigning for clean energy. I turned it into a for-profit company (1999) because I came to the view that was the best possible way I could campaign for clean energy – by creating a commercial success that could show the way. The company I set up gives 5% of its operating profit to a charity that also campaigns for clean energy, SolarAid. All that said, I hope Solarcentury makes a lot of money. It won’t have succeeded in its mission if it doesn’t. I’m hoping fewer people will still want to discount my arguments, knowing the history.

Today marks the UK availability of his book, The Energy of Nations. Heeding its own advice, quoted above, that there are drawbacks to presenting arguments in an overly rational or compelling format, the book proceeds down a parallel course. A large part of the book reads more like a novel than a textbook, with numerous fascinating episodes retold from Jeremy’s diaries.

The cast of characters that have walk-on parts in these episodes include prime ministers, oil industry titans, leading bankers, journalists, civil servants, analysts, and many others. Heroes and villains appear and re-appear, sometimes grown wiser with the passage of years, but sometimes remaining as recalcitrant, sinister (yes), and slippery (yes again) as ever.

A core theme of the book is risk blindness. Powerful vested interests in society have their own reasons to persuade public opinion that there’s nothing to worry about – that everything is under control. Resources at the disposal of these interests (“the incumbency”) inflict a perverse blindness on society, as regards the risks of the status quo. Speaking against the motion at a debate, This House Believes Peak Oil Is No Longer a Concern, in London’s Queen Elizabeth II Congress Centre in March 2009, in the aftermath of the global financial crisis brought on by hugely unwarranted over-confidence among bankers, Jeremy left a trenchant analogy hanging in the mind of the audience:

I explain that those of us who worry about peak oil fear that the oil industry has lapsed into a culture of over-exuberance about both the remaining oil reserves and prospects of resources yet to be turned into reserves, and about the industry’s ability to deliver capacity to the market even if enough resources exist.

Our main argument is that new capacity flows coming onstream from discoveries made by the oil industry over the past decade don’t compensate for depletion. Hence projections of demand cannot be met a few years hence. This problem will be compounded by other issues, including the accelerating depletion of the many old oilfields that prop up much of global oil production today, the probable exaggeration by OPEC countries of their reserves, and the failure of the ‘price-mechanism’ assumption that higher prices will lead to increased exploration and expanding discoveries…

In conclusion, this debate is all about the risk of a mighty global industry having its asset assessment systemically overstated, due to an endemic culture of over-optimism, with potentially ruinous economic implications.

I pause to let that sentence hang in the air for a second or two.

Now that couldn’t possibly happen, could it?

This none too subtle allusion to the disaster playing out in the financial sector elicits a polite laugh from the audience…

Nowadays, people frequently say that the onset of shale oil and gas should dissolve fears about impending reductions in the availability of oil. Jeremy sees this view as profoundly misguided. Shale is likely to fall far, far short of the expectations that have been heaped on it:

For many, the explosive growth of shale gas production in the USA – now extending into oil from shale, or ‘tight oil’ as it is properly known – is a revolution, a game-changer, and it even heralds a ‘new era of fossil fuels’. For a minority, it shows all the signs of being the next bubble in the markets.

In the incumbency’s widely held view, the US shale gas phenomenon can be exported, opening the way to cheap gas in multiple countries. For others, even if there is no bubble, the phenomenon is not particularly exportable, for a range of environmental, economic and political reasons

This risk too entails shock potential. Take a country like the UK. Its Treasury wishes actively to suppress renewables, so as to ensure that investors won’t be deterred from bankrolling the conversion of the UK into a ‘gas hub’. Picture the scene if most of the national energy eggs are put in that basket, infrastructure is capitalised, and then supplies of cheap gas fall far short of requirement, or even fail to materialise.

As the book makes clear, our collective risk blindness prevents society as a whole from reaching a candid appraisal of no fewer than five major risks facing us over the next few years: oil shock, climate shock, a further crash in the global financial system, the bursting of a carbon bubble in the capital markets, and the crash of the shale gas boom. The emphasis on human irrationality gels with a lot of my own prior reading – as I’ve covered e.g. in Our own entrenched enemies of reason, Animal spirits – a richer understanding of economics, Influencer – the power to change anything, as well as in my most recent posting When faith gets in the way of progress.

The book concludes with a prediction that society is very likely to encounter, by as early as 2015, either a dramatic oil shock (the widespread realisation that the era of cheap oil is behind us, and that the oil industry has misled us as badly as did the sellers of financial hocus pocus), or a renewed financial crisis, which would then precipitate (but perhaps more slowly) the same oil shock. To that extent, the book is deeply pessimistic.

But there is plenty of optimism in the book too. The author believes – as do I – that provided suitable preparatory steps are taken (as soon as possible), society ought to be able to rebound from the forthcoming crash. He spends time explaining “five premises for the Road to Renaissance”:

The readiness of clean energy for explosive growth
The intrinsic pro-social attributes of clean energy
The increasing evidence of people power in the world
The pro-social tendencies in the human mind
The power of context that leaders will be operating in after the oil crash.

But alongside his optimism, he issues a sharp warning:

I do not pretend that things won’t get much worse before they get better. There will be rioting. There will be food kitchens. There will be blood. There already have been, after the financial crash of 2008. But the next time round will be much worse. In the chaos, we could lose our way like the Maya did.

In summary, it’s a profoundly important book. I found it to be a real pleasure to read, even though the topic is nerve-racking. I burst out laughing in a number of places, and then reflected that it was nervous laughter.

The book is full of material that will probably make you want to underline it or tweet an extract online. The momentum builds up to a dramatic conclusion. Anyone concerned about the future should make time to read it.

Not everyone will agree with everything it contains, but it is clearly an honest and heartfelt contribution to vital debates. The book has already been receiving some terrific reviews from an interesting variety of people. You can see those, a summary, Chapter One, and links for buying the book here.

Finally, it’s a book that is designed to provoke discussion. I’m delighted that the author has agreed to speak at a London Futurists event on Saturday 5th October. Please click here for more details and to RSVP. This is a first class topic addressed by a first class speaker, which deserves a first class audience to match!

Leave a Comment

17 September 2013

When faith gets in the way of progress

Filed under: aging, alienation, culture, death, faith, irrationality, rejuveneering, religion, SENS, solar energy, terrorism, UKH+ — David Wood @ 3:50 pm

Is it good that we grow old, weak, disease-prone, and eventually succumb, dead, to the ravages of aging?

The rise and fall of our health and vigour is depicted in this sketch from leading biogerontology researcher Alex Zhavoronkov:

This diagram is taken from the presentation Alex made at a London Futurists event on 31st August. Alex used the same slide in his presentation, several days later, to the SENS6 conference “Reimage aging” at Queens’ College, Cambridge.

My impression from the attendees at SENS6 that I met, over the four days I spent at the conference, is that the vast majority of them would give a resounding ‘No’ as the answer to the question,

Is it good that we grow old, weak, disease-prone, and eventually succumb, dead, to the ravages of aging?

What’s more, they shared a commitment that action should be taken to change this state of affairs. In various ways, they described themselves as “fighters against aging”, “healthy longevity activists”, and as “campaigners for negligible senescence”. They share an interest in the declaration made on the page on the SENS Research Foundation website describing the conference:

The purpose of the SENS conference series, like all the SENS initiatives, is to expedite the development of truly effective therapies to postpone and treat human aging by tackling it as an engineering problem: not seeking elusive and probably illusory magic bullets, but instead enumerating the accumulating molecular and cellular changes that eventually kill us and identifying ways to repair – to reverse – those changes, rather than merely to slow down their further accumulation.

This broadly defined regenerative medicine – which includes the repair of living cells and extracellular material in situ – applied to damage of aging, is what we refer to as rejuvenation biotechnologies.

This “interventionist” approach, if successful, would lead to a line, on the chart of performance against age, similar to that shown in the bright green colour: we would retain our youthful vigour indefinitely. Mechanisms supporting this outcome were explored in considerable technical details in the SENS6 presentations. The SENS6 audience collectively posed some probing questions to the individual presenters, but the overall direction was agreed. Rejuvenation biotechnologies ought to be developed, as soon as possible.

But not everyone sees things like this. SENS6 attendees agreed on that point too. Over informal discussions throughout the event, people time and again shared anecdotes about their personal acquaintances being opposed to the goals of SENS. You can easily see the same kind of negative reactions, in the online comments pages of newspapers, whenever a newspaper reports some promising news about potential techniques to overcome aging.

For example, the Daily Mail in the UK recently published a well-researched article, “Do lobsters hold the key to eternal life? Forget gastronomic indulgence, the crustacean can defy the aging process”. The article starts as follows:

They are usually associated with a life of gastronomic indulgence and heart-stopping excess. But away from the dinner table, lobsters may actually hold the secret to a long, healthy — and possibly even eternal — life.

For this crustacean is one of a handful of bizarre animals that appear to defy the normal aging process.

While the passing years bring arthritis, muscle loss, memory problems and illness to humans, lobsters seem to be immune to the ravages of time. They can be injured, of course. They can pick up diseases. They can be caught and thrown into a pot, then smothered in béchamel sauce.

But rather than getting weaker and more vulnerable over the years, they become stronger and more fertile each time they shed their shells.

The typical lobster weighs 1 to 2 lb. But in 2009, a Maine fisherman landed a colossus of 20 lb, which was estimated to be 140 years old. And that isn’t even the oldest lobster found so far. According to Guinness World Records, a 44 lb leviathan was caught in 1977, with claws powerful enough to snap a man’s arm.

The species belongs to an elite group that appears to be ‘biologically immortal’. Away from predators, injury or disease, these astonishing creatures’ cells don’t deteriorate with age…

For healthy longevity activists, there was lots of good news in the article. This information, however, was too much for some readers to contemplate. Some of the online comments make for fascinating (but depressing) reading. Here are four examples, quoted directly from the comments:

How would humankind cope with tens of millions of extremely old and incredibly crabby people?

People have to die and they’re not dying quickly enough. Soon the earth will run out of water and food for the ever increasing masses.

These “researchers” should watch Death Becomes Her

The only guarantee of eternal life is to read your Bibles. Though even if you don’t, eternal life of another kind exists, though it’s not particularly appealing: “And the smoke of their torment ascendeth up for ever and ever” (Rev 14:11).

To be clear, the goal of project such as those in the SENS umbrella is to extend healthy lifespans (sometimes known as “healthspans”) rather than simply extending lifespans themselves. Rejuvenation technologies are envisioned to undo tendencies towards unwelcome decrepitude, crabbiness, and so on.

As for the reference to the 1992 Hollywood film “Death Becomes Her” featuring Meryl Streep and Goldie Hawn in a frightful “living dead” immortality, I’ll get back to that later.

The question of potential over-population has a bit more substance. However, the worry isn’t so much the number of people on the earth, but the rate at which everyone is consuming and polluting. With potential forthcoming improvements in harnessing solar energy, we’ll have more than enough energy available to look after a planet with 10 billion people. Arguably the planet could sustain at least 100 billion people. (That argument is made, in a well-balanced way, by Ramez Naam in his recent book “The infinite resource” – a book I thoroughly recommend. I’ve also covered this question from time to time in earlier blogposts – see e.g. “Achieving a 130-fold improvement in 40 years”.)

However, I believe that there are deeper roots to the opposition that many people have to the idea of extending healthy lifespans. They may offer intellectual rationalisations for their opposition (e.g. “How would humankind cope with tens of millions of extremely old and incredibly crabby people?”) but these rationalisations are not the drivers for the position they hold.

Instead, their opposition to extending healthy lifespans comes from what we can call faith.

This thought crystallised in my mind as I reflected on the very last presentation from SENS6. The speaker was Thomas Pyszczynski of the University of Colorado, and his topic was “Understanding the paradox of opposition to long-term extension of the human lifespan: fear of death, cultural worldviews, and the illusion of objectivity”.

The presentation title was long, but the content was clear and vivid. The speaker outlined some conclusions from decades of research he had conducted into “Terror Management Theory (TMT)”. I’ve since discovered that the subject of “Terror Management Theory” has its own article in Wikipedia:

Terror management theory (TMT), in social psychology, proposes a basic psychological conflict that results from having a desire to live but realizing that death is inevitable. This conflict produces terror, and is believed to be unique to humans. Moreover, the solution to the conflict is also generally unique to humans: culture. According to TMT, cultures are symbolic systems that act to provide life with meaning and value. If life is thought meaningful, death is less terrifying. Cultural values therefore serve to manage the terror of death by providing life with meaning…

pyszczynski

Here’s the “paradox” to which Pyszczynski (pictured) referred: people oppose the idea that we could have longer healthy lives, because of the operation of a set of culture and philosophical ideas, which were themselves an adaptive response to the underlying fact that we deeply desire indefinitely long healthy lives. So the opposition is self-contradictory, but the people involved don’t see it like that.

For all of history up until the present age, the idea of having an indefinitely long healthy life was at stark variance to everything else that we saw around ourselves. Death seemed inevitable. In order to avoid collapsing into terror, we needed to develop rationalisations and techniques that prevented us from thinking seriously about our own finitude and mortality. That’s where key aspects of our culture arose. These aspects of our culture became deeply rooted.

Our culture operates, in many cases, below the level of conscious awareness. We find ourselves being driven by various underlying beliefs, without being aware of the set of causes and effects. However, we find comfort in these beliefs. This faith (belief in the absence of sufficient reason) helps to keep us mentally sane, and keeps society functional, even as it prepares us, as individuals, to grow infirm and die.

In case any new ideas challenge this faith, we find ourselves compelled to lash out against these ideas, even without taking the time to analyse them. Our motivation, here, is to preserve our core culture and faith, since that’s what provides the foundation of meaning in our lives. We fight the new ideas, even if these new ideas would be a better solution to our underlying desire to live an indefinitely long, healthy life. The new ideas leave us with a feeling of alienation, even though we don’t see the actual connections between ideas. Our faith causes us to lose our rationality.

Incidentally, similar factors apply, of course, when other things that have profound importance to us are challenged. For example, when we think we may lose a cherished romantic partner, we can all too easily become crazy. When your heart’s on fire, smoke gets in your eyes.

It turns out that Aubrey de Grey, the chief science officer of SENS, has already written on this same topic. In chapter two of his 2007 book “Ending aging”, he notes the following:

There is a very simple reason why so many people defend aging so strongly – a reason that is now invalid, but until quite recently was entirely reasonable. Until recently, no one has had any coherent idea how to defeat aging, so it has been effectively inevitable. And when one is faced with a fate that is as ghastly as aging and about which one can do absolutely nothing, either for oneself or even for others, it makes perfect psychological sense to put it out of one’s mind – to make one’s peace with it, you might say – rather than to spend one’s miserably short life preoccupied by it. The fact that, in order to sustain this state of mind, one has to abandon all semblance of rationality on the subject – and, inevitably, to engage in embarrassingly unreasonable conversational tactics to shore up that irrationality – is a small price to pay….

Aubrey continues this theme at the start of chapter three:

We’ve recently reached the point where we can engage in the rational design of therapies to defeat aging: most of the rest of this book is an account of my favoured approach to that design. But in order to ensure that you can read that account with an open mind, I need to dispose beforehand of a particularly insidious aspect of the pro-aging trance: the fact that most people already know, in their heart of hearts, that there is a possibility that aging will eventually be defeated.

Why is this a problem? Indeed, at first sight you might think that it would make my job easier, since surely it means that the pro-aging trance is not particularly deep. Unfortunately, however, self-sustained delusions don’t work like that. Just as it’s rational to be irrational about the desirability of aging in order to make your peace with it, it’s also rational to be irrational about the feasibility of defeating aging while the chance of defeating it any time soon remains low. If you think there’s even a 1 percent chance of defeating aging within your lifetime (or within the lifetime of someone you love), that sliver of hope will prey on your mind and keep your pro-aging trance uncomfortably fragile, however hard you’ve worked to convince yourself that aging is actually not such a bad thing after all. If you’re completely convinced that aging is immutable, by contrast, you can sleep more soundly.

Another speaker from the final session of SENS6, Mair Underwood of the University of Queensland, provided some timely advice to the SENS6 community, that dovetails well with the discussion above. Underwood’s presentation was entitled “What reassurances do the community need regarding life extension? Evidence from studies of community attitudes and an analysis of film portrayals”. The presentation pointed out the many ways in which popular films (such as “Death Becomes Her”, mentioned above) portray would-be life extensionists in a bad light. These people, the films imply, are emotionally immature, selfish, frustrated, obstructive, and generally unattractive. This is the pro-death culture at work.

To counteract these impressions, and to help free the broader community from its faith that aging and death are actually good things, Underwood gave the following advice:

Assure that life extension science, and the distribution of life extension technologies, are ethical and regulated, and seen to be so

Assuage community concerns about life extension as unnatural or playing god

Assure that life extension would involve an extension of healthy lifespan

Assure that life extension does not mean a loss of fertility

Assure the community that life extension will not exacerbate social divides, and that those with extended lives will not be a burden on society

Create a new cultural framework for understanding life extension.

This advice is all good, but I suspect that the new few years may see a growing “battle of faiths”, as representatives of the old culture fight harder in opposition to the emerging evidence that we we are on the point of possessing the technological means to extend human healthspans very significantly. This is a battle that may need more tools, to influence the outcome, than mere hard-honed rationality. At the very least, we’ll need to keep in mind how culture works, and the ways in which faith draws strength.

Follow ups: Several forthcoming London Futurists meetups address topics that are directly relevant to the above line of thinking:

Futurism, Spirituality, and Faith, in Birkbeck College on Saturday 21st September, discusses ways in which committed technoprogressives can best interact with faith-based movements, without these interactions leading to fruitless irrationality and loss of direction
Projects to accelerate radical healthy longevity, a Google Hangout On Air (HOA) on Sunday 29th September, features a panel discussion on the question, “What are the most important ongoing projects to accelerate radical healthy longevity?”
Futurists discuss The Transhumanist Wager, with Zoltan Istvan, another Google HOA, on Sunday 20th September, reviews a recently published novel about a possible near-future scenario of a growing battle between the old human culture and an emerging new culture that favours indefinitely long healthspans.
Finally, if you’re interested in the question of whether solar energy will be able, as I implied above, to address pending shortages in global energy supplies, even as human population continues to increase, you should make it a priority to attend the London Futurists event on Saturday 5th October, The Energy of Nations, with Jeremy Leggett. The speaker on this occasion is one of the world’s foremost authorities on solar energy, oil depletion, climate change, and dysfunctional investment. The topic of the best energy systems for the decades ahead is, alas, another one in which faith tends to subvert reason, and in which we need to be smart to prevent our thinking being hijacked by adverse factors.

For more information about the evolution of London Futurists, you can take a peek at a new website which is in the process of being implemented, at http://londonfuturists.com/.

Comments (1)

22 February 2013

Controversies over singularitarian utopianism

Filed under: alienation, change, chaos, climate change, complacency, Events, futurist, irrationality, politics, predictability, pseudoscience, risks, Singularity, UKH+, usability, vision — David Wood @ 3:47 pm

I shouldn’t have been surprised at the controversy that arose.

The cause was an hour-long lecture with 55 slides, ranging far and wide over a range of disruptive near-future scenarios, covering both upside and downside. The basic format of the lecture was: first the good news, and then the bad news. As stated on the opening slide,

Some illustrations of the enormous potential first, then some examples of how adding a high level of ambient stupidity might mean we might make a mess of it.

The speaker was Ian Pearson, described on his company website as “futurologist, conference speaker, regular media guest, strategist and writer”. The website continues, boldly,

Anyone can predict stuff, but only a few get it right…

Ian Pearson has been a full time futurologist since 1991, with a proven track record of over 85% accuracy at the 10 year horizon.

Ian was speaking, on my invitation, at the London Futurists last Saturday. His chosen topic was audacious in scope:

A Singularitarian Utopia Or A New Dark Age?

We’re all familiar with the idea of the singularity, the end-result of rapid acceleration of technology development caused by positive feedback. This will add greatly to human capability, not just via gadgets but also through direct body and mind enhancement, and we’ll mess a lot with other organisms and AIs too. So we’ll have superhumans and super AIs as part of our society.

But this new technology won’t bring a utopia. We all know that some powerful people, governments, companies and terrorists will also add lots of bad things to the mix. The same technology that lets you enhance your senses or expand your mind also allows greatly increased surveillance and control, eventually to the extremes of direct indoctrination and zombification. Taking the forces that already exist, of tribalism, political correctness, secrecy for them and exposure for us, and so on, it’s clear that the far future will be a weird mixture of fantastic capability, spoiled by abuse…

There were around 200 people in the audience, listening as Ian progressed through a series of increasingly mind-stretching technology opportunities. Judging by the comments posted online afterwards, some of the audience deeply appreciated what they heard:

Thank you for a terrific two hours, I have gone away full of ideas; I found the talk extremely interesting indeed…

I really enjoyed this provocative presentation…

Provocative and stimulating…

Very interesting. Thank you for organizing it!…

Amazing and fascinating!…

But not everyone was satisfied. Here’s an extract from one negative comment:

After the first half (a trippy sub-SciFi brainstorm session) my only question was, “What Are You On?”…

Another audience member wrote his own blogpost about the meeting:

A Singularitanian Utopia or a wasted afternoon?

…it was a warmed-over mish-mash of technological cornucopianism, seasoned with Daily Mail-style reactionary harrumphing about ‘political correctness gone mad’.

These are just the starters of negative feedback; I’ll get to others shortly. As I review what was said in the meeting, and look at the spirited ongoing exchange of comments online, some thoughts come to my mind:

Big ideas almost inevitably provoke big reactions; this talk had a lot of particularly big ideas
In some cases, the negative reactions to the talk arise from misunderstandings, due in part to so much material being covered in the presentation
In other cases, Isee the criticisms as reactions to the seeming over-confidence of the speaker (“…a proven track record of over 85% accuracy”)
In yet other cases, I share the negative reactions the talk generated; my own view of the near-future landscape significantly differs from the one presented on stage
In nearly all cases, it’s worth taking the time to progress the discussion further
After all, if we get our forecasts of the future wrong, and fail to make adequate preparations for the disruptions ahead, it could make a huge difference to our collective well-being.

So let’s look again at some of the adverse reactions. My aim is to raise them in a way that people who didn’t attend the talk should be able to follow the analysis.

(1) Is imminent transformation of much of human life a realistic scenario? Or are these ideas just science fiction?

The main driver for belief in the possible imminent transformation of human life, enabled by rapidly changing technology, is the observation of progress towards “NBIC” convergence.

Significant improvements are taking place, almost daily, in our capabilities to understand and control atoms (Nano-tech), genes and other areas of life-sciences (Bio-tech), bits (Info-comms-tech), and neurons and other areas of mind (Cogno-tech). Importantly, improvements in these different fields are interacting with each other.

As Ian Pearson described the interactions:

Nanotech gives us tiny devices
Tiny sensors help neuroscience figure out how the mind works
Insights from neuroscience feed into machine intelligence
Improving machine intelligence accelerates R&D in every field
Biotech and IT advances make body and machine connectable
…

Will all the individual possible applications of NBIC convergence described by Ian happen in precisely the way he illustrated? Very probably not. The future’s not as predictable as that. But something similar could well happen:

Cheaper forms of energy
Tissue-cultured meat
Space exploration
Further miniaturisation of personal computing (wearable computing, and even “active skin”)
Smart glasses
Augmented reality displays
Gel computing
IQ and sensory enhancement
Dream linking
Human-machine convergence
Digital immortality: “the under 40s might live forever… but which body would you choose?”

(2) Is a focus on smart cosmetic technology an indulgent distraction from pressing environmental issues?

Here’s one of the comments raised online after the talk:

Unfortunately any respect due was undermined by his contempt for the massive environmental challenges we face.

Trivial contact lens / jewellery technology can hang itself, if our countryside is choked by yoghurt factory fumes.

The reference to jewellery took issue with remarks in the talk such as the following:

Miniaturisation will bring everyday IT down to jewellery size…

Decoration; Social status; Digital bubble; Tribal signalling…

In contrast, the talk positioned greater use of technology as the solution to environmental issues, rather than as something to exacerbate these issues. Smaller (jewellery-sized) devices, created with a greater attention to recyclability, will diminish the environmental footprint. Ian claimed that:

We can produce more of everything than people need
Improved global land management could feed up to 20 billion people
Clean water will be plentiful
We will also need less and waste less
Long term pollution will decline.

Nevertheless, he acknowledged that there are some short-term problems, ahead of the time when accelerating NBIC convergence can be expected to provide more comprehensive solutions:

Energy shortage is a short to mid term problem
Real problems are short term.

Where there’s room for real debate is the extent of these shorter-term problems. Discussion on the threats from global warming brought these disagreements into sharp focus.

(3) How should singularitarians regard the threat from global warming?

Towards the end of his talk, Ian showed a pair of scales, weighing up the wins and losses of NBIC technologies and a potential singularity.

The “wins” column included health, growth, wealth, fun, and empowerment.

The “losses” column included control, surveillance, oppression, directionless, and terrorism.

One of the first questions from the floor, during the Q&A period in the meeting, asked why the risk of environmental destruction was not on the list of possible future scenarios. This criticism was echoed by online comments:

The complacency about CO2 going into the atmosphere was scary…

If we risk heading towards an environmental abyss let’s do something about what we do know – fossil fuel burning.

During his talk, I picked up on one of Ian’s comments about not being particularly concerned about the risks of global warming. I asked, what about the risks of adverse positive feedback cycles, such as increasing temperatures triggering the release of vast ancient stores of methane gas from frozen tundra, accelerating the warming cycle further? That could lead to temperature increases that are much more rapid than presently contemplated, along with lots of savage disturbance (storms, droughts, etc).

Ian countered that it was a possibility, but he had the following reservations:

He thought these positive feedback loops would only kick into action when baseline temperature rose by around 2 degrees
In the meantime, global average temperatures have stopped rising, over the last eleven years
He estimates he spends a couple of hours every day, keeping an eye on all sides of the global warming debate
There are lots of exaggerations and poor science on both sides of the debate
Other factors such as the influence of solar cycles deserve more research.

Here’s my own reaction to these claims:

The view that global average temperatures have stopped rising, is, among serious scientists, very much a minority position; see e.g. this rebuttal on Carbon Brief
Even if there’s only a small probability of a runaway spurt of accelerated global warming in the next 10-15 years, we need to treat that risk very seriously – in the same way that, for example, we would be loath to take a transatlantic flight if we were told there was a 5% chance of the airplane disintegrating mid-flight.

Nevertheless, I did not want the entire meeting to divert into a debate about global warming – “that deserves a full meeting in its own right”, I commented, before moving on to the next question. In retrospect, perhaps that was a mistake, since it may have caused some members of the audience to mentally disengage from the meeting.

(4) Are there distinct right-wing and left-wing approaches to the singularity?

Here’s another comment that was raised online after the talk:

I found the second half of the talk to be very disappointing and very right-wing.

And another:

Someone who lists ‘race equality’ as part of the trend towards ignorance has shown very clearly what wing he is on…

In the second half of his talk, Ian outlined changes in norms of beliefs and values. He talked about the growth of “religion substitutes” via a “random walk of values”:

Religious texts used to act as a fixed reference for ethical values
Secular society has no fixed reference point so values oscillate quickly.
20 years can yield 180 degree shift
e.g. euthanasia, sexuality, abortion, animal rights, genetic modification, nuclear energy, family, policing, teaching, authority…
Pressure to conform reinforces relativism at the expense of intellectual rigour

A complicating factor here, Ian stated, was that

People have a strong need to feel they are ‘good’. Some of today’s ideological subscriptions are essentially secular substitutes for religion, and demand same suspension of free thinking and logical reasoning.

A few slides later, he listed examples of “the rise of nonsense beliefs”:

e.g. new age, alternative medicine, alternative science, 21st century piety, political correctness

He also commented that “99% are only well-informed on trivia”, such as fashion, celebrity, TV culture, sport, games, and chat virtual environments.

This analysis culminated with a slide that personally strongly resonated with me: a curve of “anti-knowledge” accelerating and overtaking a curve of “knowledge”:

In pursuit of social compliance, we are told to believe things that are known to be false.

With clever enough spin, people accept them and become worse than ignorant.

So there’s a kind of race between “knowledge” and “anti-knowledge”.

One reason this resonated with me is that it seemed like a different angle on one of my own favourite metaphors for the challenges of the next 15-30 years – the metaphor of a dramatic race:

One runner in the race is “increasing rationality, innovation, and collaboration”; if this runner wins, the race ends in a positive singularity
The other runner in the race is “increasing complexity, rapidly diminishing resources”; if this runner wins, the race ends in a negative singularity.

In the light of Ian’s analysis, I can see that the second runner is aided by the increase of anti-knowledge: over-attachment to magical, simplistic, ultimately misleading worldviews.

However, it’s one thing to agree that “anti-knowledge” is a significant factor in determining the future; it’s another thing to agree which sets of ideas count as knowledge, and which as anti-knowledge! One of Ian’s slides included the following list of “religion substitutes”:

Animal rights, political correctness, pacifism, vegetarianism, fitness, warmism, environmentalism, anti-capitalism

It’s no wonder that many of the audience felt offended. Why list “warmism” (a belief in human-caused global warming), but not “denialism” (denial of human-caused global warming? Why list “anti-capitalism” but not “free market fundamentalism”? Why list “pacifism” but not “militarism”?

One online comment made a shrewd observation:

Ian raised my curiosity about ‘false beliefs’ (or nonsense beliefs as Ian calls them) as I ‘believe’ we all inhabit different belief systems – so what is true for one person may be false for another… at that exact moment in time.

And things can change. Once upon a time, it was a nonsense belief that the world was round.

There may be 15% of truth in some nonsense beliefs…or possibly even 85% truth. Taking ‘alternative medicine’ as an example of one of Ian’s nonsense beliefs – what if two of the many reasons it was considered nonsense were that (1) it is outside the world (the system) of science and technology and (2) it cannot be controlled by the pharmaceutical companies (perhaps our high priests of today)?

(5) The role of corporations and politicians in the approach to the singularity

One place where the right-wing / left-wing division becomes more acute in the question of whether anything special needs to be done to control the behaviour of corporations (businesses).

One of Ian’s strong positive recommendations, at the end of his presentation, was that scientists and engineers should become more actively involved in educating the general public about issues of technology. Shortly afterward, the question came from the floor: what about actions to educate or control corporations? Ian replied that he had very little to recommend to corporations, over and above his recommendations to the individuals within these corporations.

My own view is different. From my life inside industry, I’ve seen numerous cases of good people who are significantly constrained in their actions by the company systems and metrics in which they find themselves enmeshed.

Indeed, just as people should be alarmed about the prospects of super-AIs gaining too much power, over and above the humans who created them, we should also be alarmed about the powers that super-corporations are accumulating, over and above the powers and intentions of their employees.

The argument to leave corporations alone finds its roots in ideologies of freedom: government regulation of corporations often has undesirable side-effects. Nevertheless, that’s just an argument for being smarter and more effective in how the regulation works – not an argument to abstain from regulation altogether.

The question of the appropriate forms of collaborative governance remains one of the really hard issues facing anyone concerned about the future. Leaving corporations to find their own best solutions is, in my view, very unlikely to be the optimum approach.

In terms of how “laissez-faire” we should be, in the face of potential apocalypse down the road, I agree with the assessment near the end of Jeremy Green’s blogpost:

Pearson’s closing assertion that in the end our politicians will always wake up and pull us back from the brink of any disaster is belied by many examples of civilisations that did not pull back and went right over the edge to destruction.

Endnote:

After the presentation in Birkbeck College ended, around 40-50 of the audience regrouped in a nearby pub, to continue the discussion. The discussion is also continuing, at a different tempo, in the online pages of the London Futurists meetup. Ian Pearson deserves hearty congratulation for stirring up what has turned out to be an enlightening discussion – even though there’s heat in the comments as well as light!

Evidently, the discussion is far from complete…

Comments (2)

10 February 2013

Fixing bugs in minds and bugs in societies

Filed under: bias, change, culture, evolution, intelligence, intuition, irrationality, precautionary principle, Psion — David Wood @ 1:55 am

Suppose we notice what appears to be bugs in our thinking processes. Should we try to fix these bugs?

Or how about bugs in the way society works? Should we try to fix these bugs too?

As examples of bugs of the first kind, I return to a book I reviewed some time ago, “Kluge: The Haphazard Construction of the Human Mind”. I entitled my review “The human mind as a flawed creation of nature”, and I still stick by that description. In that review, I pulled out the following quote from near to the end of the book:

In this book, we’ve discussed several bugs in our cognitive makeup: confirmation bias, mental contamination, anchoring, framing, inadequate self-control, the ruminative cycle, the focussing illusion, motivated reasoning, and false memory, not to mention absent-mindedness, an ambiguous linguistic system, and vulnerability to mental disorders. Our memory, contextually driven as it is, is ill suited to many of the demands of modern life, and our self-control mechanisms are almost hopelessly split. Our ancestral mechanisms were shaped in a different world, and our more modern deliberative mechanisms can’t shake the influence of that past. In every domain we have considered, from memory to belief, choice, language, and pleasure, we have seen that a mind built largely through the progressive overlay of technologies is far from perfect…

These bugs in our mental makeup are far from being harmless quirks or curiosities. They can lead us:

to overly trust people who have visual trappings of authority,
to fail to make adequate provision for our own futures,
to keep throwing money into bad investments,
and to jump to all kinds of dangerous premature conclusions.

But should we try to fix these bugs?

The field where the term ‘bug’ was first used in this sense of a mistake, software engineering, provides many cautionary tales of bug fixing going wrong:

Sometimes what appears to be a ‘bug’ in a piece of software turns out to be a useful ‘feature’, with a good purpose after all
Sometimes a fix introduces unexpected side-effects, which are worse than the bug which was fixed.

I shared an example of the second kind in the “Managing defects” chapter of the book I wrote in 2004-5, “Symbian for software leaders: principles of successful smartphone development projects”:

An embarrassing moment with defects

The first million-selling product that I helped to build was the Psion Series 3a handheld computer. This was designed as a distinct evolutionary step-up from its predecessor, the original Series 3 (often called the “Psion 3 classic” in retrospect)…

At last the day came (several weeks late, as it happened) to ship the software to Japan, where it would be flashed into large numbers of chips ready to assemble into production Series 3a devices. It was ROM version 3.20. No sooner was it sent than panic set into the development team. Two of us had independently noticed a new defect in the agenda application. If a user set an alarm on a repeating entry, and then adjusted the time of this entry, in some circumstances the alarm would fail to ring. We reasoned that this was a really bad defect – after all, two of us had independently found it.

The engineer who had written the engine for the application – the part dealing with all data manipulation algorithms, including calculating alarm times – studied his code, and came up with a fix. We were hesitant, since it was complex code. So we performed a mass code review: lots of the best brains in the team talked through the details of the fix. After twenty four hours, we decided the fix was good. So we recalled 3.20, and released 3.21 in its place. To our relief, no chips were lost in the process: the flashing had not yet started.

Following standard practice, we upgraded the prototype devices of everyone in the development team, to run 3.21. As we waited for the chips to return, we kept using our devices – continuing (in the jargon of the team) to “eat our own dog food”. Strangely, there were a few new puzzling problems with alarms on entries. Actually, it soon became clear these problems were a lot worse than the problem that had just been fixed. As we diagnosed these new problems, a sinking feeling grew. Despite our intense care (but probably because of the intense pressure) we had failed to fully consider all the routes through the agenda engine code; the change made for 3.21 was actually a regression on previous behaviour.

Once again, we made a phone call to Japan. This time, we were too late to prevent some tens of thousands of wasted chips. We put the agenda engine code back to its previous state, and decided that was good enough! (Because of some other minor changes, the shipping version number was incremented to 3.22.) We decided to live with this one defect, in order not to hold up production any longer.

We were expecting to hear more news about this particular defect from the Psion technical support teams, but the call never came. This defect never featured on the list of defects reported by end users. In retrospect, we had been misled by the fact that two of us had independently found this defect during the final test phase: this distorted our priority call…

That was an expensive mistake, which seared a cautionary attitude into my own brain, regarding the dangers of last-minute changes to complex software. All seasoned software engineers have similar tales they can tell, from their own experience.

If attempts to fix defects in software are often counter-productive, how much more dangerous are attempts to fix defects in our thinking processes – or defects in how our societies operate! At least in the first case, we generally still have access to the source code, and to the design intention of the original software authors. For the other examples, the long evolutionary history that led to particular designs is something at which we can only guess. It would be like trying to fix a software bug, that somehow results from the combination of many millions of lines of source code, written decades ago by people who left no documentation and who are not available for consultation.

What I’ve just stated is a version of an argument that conservative-minded thinkers often give, against attempts to try to conduct “social engineering” or “improve on nature”. Tinkering with ages-old thinking processes – or with structures within societies – carries the risk that we fail to appreciate many hidden connections. Therefore (the argument runs) we should desist from any such experimentation.

Versions of this argument appeared, from two different commentators, in responses to my previous blogpost. One put it like this:

The trouble is that ‘cognitive biases and engrained mistakes’ may appear dysfunctional but they are, in fact, evolutionarily successful adaptations of humanity to its highly complex environment. These, including prejudice, provide highly effective means for the resolution of really existing problems in human capacity…

Rational policies to deal with human and social complexity have almost invariably been proved to be inhumane and brutal, fine for the theoretician in the British Library, but dreadful in the field.

Another continued the theme:

I have much sympathy for [the] point about “cognitive biases and engrained mistakes”. The belief that one has identified cognitive bias in another or has liberated oneself from such can be a “Fatal Conceit,” to borrow a phrase from Hayek, and has indeed not infrequently given rise to inhumane treatment even of whole populations. One of my favourite sayings is David Hume’s “the rules of morality are not conclusions of our reason,” which is at the heart of Hayek’s Fatal Conceit argument.

But the conclusion I draw is different. I don’t conclude, “Never try to fix bugs”. After all, the very next sentence from my chapter on “Managing defects” stated, “We eventually produced a proper fix several months later”. Indeed, many bugs do demand urgent fixes. Instead, my conclusion is that bug fixing in complex systems needs a great deal of careful thought, including cautious experimentation, data analysis, and peer review.

The analogy can be taken one more step. Suppose that a software engineer has a bad track record in his or her defect fixes. Despite claiming, each time, to be exercising care and attention, the results speak differently: the fixes usually make things worse. Suppose, further, that this software engineer comes from a particular company, and that fixes from that company have the same poor track record. (To make this more vivid, the name of this company might be “Technocratic solutions” or “Socialista” or “Utopia software”. You can probably see where this argument is going…) That would be a reason for especial discomfort if someone new from that company is submitting code changes in attempts to fix a given bug.

Well, something similar happens in the field of social change. History has shown, in many cases, that attempts at mental engineering and social engineering were counter-productive. For that reason, many conservatives support various “precautionary principles”. They are especially fearful of any social changes proposed by people they can tar with labels such as “technocratic” or”socialist” or “utopian”.

These precautionary principles presuppose that the ‘cure’ will be worse than the ‘disease’. However, I personally have greater confidence in the fast improving power of new fields of science, including the fields that study our mind and brain. These improvements are placing ever greater understanding in our hands – and hence, ever greater power to fix bugs without introducing nasty side-effects.

For these reasons, I do look forward (as I said in my previous posting) to these improvements

helping individuals and societies rise above cognitive biases and engrained mistakes in reasoning… and accelerating a reformation of the political and economic environment, so that the outcomes that are rationally best are pursued, instead of those which are expedient and profitable for the people who currently possess the most power and influence.

Finally, let me offer some thoughts on the observation that “the rules of morality are not conclusions of our reason”. That observation is vividly supported by the disturbing “moral dumbfounding” examples discussed by Jonathan Haidt in his excellent book “The Happiness Hypothesis: Finding Modern Truth in Ancient Wisdom” (which I briefly reviewed here). But does that observation mean that we should stop trying to reason with people about moral choices?

Here, I’ll adapt comments from my review of “The Moral Landscape: How Science Can Determine Human Values”, by Sam Harris.

That book considers how we might go about finding answers to big questions such as “how should I live?” and “what makes some ways of life more moral than others?” As some specific examples, how should we respond to:

The Taliban’s insistence that the education of girls is an abomination?
The stance by Jehovah’s Witnesses against blood transfusion?
The prohibition by the Catholic Church of the use of condoms?
The legalisation of same-sex relationships?
The use of embryonic stem cells in the search for cures of diseases such as Alzheimer’s and Parkinson’s?
A would-be Islamist suicide bomber who is convinced that his intended actions will propel him into a paradise of abundant mental well-being?

One response is that such questions are the province of religion. The correct answers are revealed via prophets and/or holy books. The answers are already clear, to those with the eye of faith. It is a divine being that tells us, directly or indirectly, the difference between good and evil. There’s no need for experimental investigations here.

A second response is that the main field to study these questions is that of philosophy. It is by abstract reason, that we can determine the difference between good and evil.

But Sam Harris, instead, primarily advocates the use of the scientific method. Science enters the equation because it is increasingly able to identify:

Neural correlates (or other physical or social underpinnings) of sentient well-being
Cause-and-effect mechanisms whereby particular actions typically bring about particular changes in these neural correlates.

With the help of steadily improving scientific understanding, we can compare different actions based on their likely effects on sentient well-being. Actions which are likely to magnify sentient well-being are good, and those which are likely to diminish it are evil. That’s how we can evaluate, for example, the Taliban’s views on girls’ education.

As Harris makes clear, this is far from being an abstract, other-worldly discussion. Cultures are clashing all the time, with lots of dramatic consequences for human well-being. Seeing these clashes, are we to be moral relativists (saying “different cultures are best for different peoples, and there’s no way to objectively compare them”) or are we to be moral realists (saying “some cultures promote significantly more human flourishing than others, and are to be objectively preferred as a result”)? And if we are to be moral realists, do we resolve our moral arguments by deference to religious tradition, or by open-minded investigation of real-world connections?

In the light of these questions, here are some arguments from Harris’s book that deserve thought:

There’s a useful comparison between the science of human values (the project espoused by Harris), and a science of diets (what we should eat, in order to enjoy good health). In both cases, we’re currently far from having all the facts. And in both cases, there are frequently several right answers. But not all diets are equally good. Similarly, not all cultures are equally good. And what makes one diet better than another will be determined by facts about the physical world – such as the likely effects (direct and indirect) of different kinds of fats and proteins and sugars and vitamins on our bodies and minds. While people still legitimately disagree about diets, that’s not a reason to say that science can never answer such questions. Likewise, present-day disagreements about specific causes of happiness, mental flourishing, and general sentient well-being, do not mean these causes fail to exist, or that we can never know them.
Likewise with the science of economics. We’re still far from having a complete understanding of how different monetary and financial policies impact the long-term health of the economy. But that doesn’t mean we should throw up our hands and stop searching for insight about likely cause and effect. The discipline of economics, imperfect though it is, survives in an as-yet-incomplete state. The same goes for political science too. And, likewise, for the science of the moral landscape.
Attempts to reserve some special area of “moral insight” for religion are indefensible. As Harris says, “How is it that most Jews, Christians, and Muslims are opposed to slavery? You don’t get this moral insight from scripture, because the God of Abraham expects us to keep slaves. Consequently, even religious fundamentalists draw many of their moral positions from a wider conversation about human values that is not, in principle, religious.” That’s the conversation we need to progress.

PS I’ve written more about cognitive biases and cognitive dissonance – and how we can transcend these mistakes – in my blogpost “Our own entrenched enemies of reason”.

Comments (3)

14 January 2012

Speaking of angels – visions of a world beyond

Filed under: books, irrationality, magic, paranormal, psychology — David Wood @ 1:03 am

How open-minded are you?

Suppose someone you’ve never met before takes a look at the palm of your hand, and shortly afterwards tells you surprising things about yourself – for example, about private issues experienced by your family, that no one else knows about. What would your reaction be?
Or consider the case of people apparently leaving their bodies, whilst near death, and travelling around the neighbourhood in an out-of-body experience, observing hidden details that could only be noticed by someone high up in the sky. Isn’t that thought-provoking?
Or what about reliable, trustworthy witnesses who return from spiritualist seances reporting materialisations and apparitions that the best conjurors of the day realise they could not possibly duplicate?
What about a president of the United States (Abraham Lincoln) who dreamed the details of his own death, in a precognition, several weeks ahead of that dreadful event?
What about someone who can cause the pages of a bible in another part of the room to turn over? Or pencils to rotate? Or solid steel spoons to bend and break?
Finally, what about a dog which springs to the window, seemingly knowing in advance that their owner has set off from work to return home, and will shortly be arriving at the house?

All these phenomena, and a lot more like them, are described in Professor Richard Wiseman’s recent book, “Paranormality: Why we see what isn’t there“.

At face value, these phenomena testify to the presence of powers far beyond the present understanding of science. They suggest the existence of some kind of angelic realm, in which information can travel telepathically, from one brain to another, and even backwards in time.

One common reaction to this kind of report is to cough in embarrassment, or make a joke, and move on to another topic.

Another reaction is to become a debunker. Indeed, Wiseman’s book contains some splendid debunking. I won’t spoil the fun by sharing these details here, but you can bear in mind the apparently miraculous feats demonstrated right in front of spectators’ eyes by magicians like Derren Brown or “Dynamo“. (As noted on his website, Wiseman “started his working life as a professional magician, and is a Member of the Inner Magic Circle”.)

However, “Paranormality” goes far beyond debunking. Although some of the apparently paranormal events do have mundane explanations, for others, the explanation is more wonderful. These explanations reveal fascinating details about the way the human mind operates – details that have only come to be understood within recent years.

These explanations don’t involve any actual transfer of disembodied thought, or any transcendent angelic realm. Instead, they shed light on topics such as:

Circumstances when the mind can become convinced that it is located outside the body
Ways to pick up subliminal cues, by which people “leak” information to one another via subtle movements
The sometimes spectacular unreliability of human memory
Cognitive dissonance – how people react when, on the surface, prophetic statements have proven false
The functioning of dreams, linked to sleep paralysis
Circumstances when people feel that there’s a ghostly presence
Purposeful movements made by the body, without the awareness of the conscious mind
Limitations in the mind’s concept that it has free will.

The book also retells some dramatic historical episodes. Some of these episodes were already familiar to me, from my days doing postgraduate research in the philosophy of science, when I looked hard and long at the history of research into the paranormal. Others were, I confess, new to me – including an account of Michael Faraday’s investigation of the mechanics behind table-turning at seances.

The book has many practical tips too:

How to develop the habit of “lucid dreams” (when you’re aware that you’re dreaming)
How to impress people that you can (apparently) read their mind and discern hidden depths of their character
How to distract an audience, so that they fail to notice what’s right in front of them
How to organise a group of people around a table, so that the table apparently starts moving of its own volition
How to avoid losing control of your mind in circumstances when powerful persuasive influences operate.

In other words, rather than dismissing instances of apparent paranormal occurrences as being inevitably misguided, Wiseman suggests there’s a lot to learn from them.

I expect to hear more of the same theme later today, at the “Centre for Inquiry UK” event “Beyond the Veil – a closer look at spirits, mediums and ghosts“. This is being held at London’s Conway Hall (one of my own favourite London venues). Richard Wiseman is one of the speakers there. The full agenda is as follows:

10.30 Registration (tickets will be available at the door)

11.00: Spirits on the brain: Insights from psychology and neuroscience – Chris French, Professor of Psychology and Head of the Anomalistic Psychology Research Unit at Goldsmiths, University of London

12.00: ‘Is there anybody there?’ – Hayley Stevens, a ghost hunter that doesn’t hunt for ghosts, who has been researching paranormal reports since 2005.

13.00: Lunch break

13.30: Mediums at Large – Paul Zenon, a professional trickster for almost thirty years, during which period he has appeared countless times as performer, presenter and pundit on numerous TV shows

14.00: Parnormality – Richard Wiseman, Professor for the Public Understanding of Psychology at the University of Hertfordshire

15.00: You Are The Magic – Ian Rowland, writer and entertainer with an interest in various aspects of how the mind works or sometimes doesn’t, who taught FBI agents how to be persuasive, and taught Derren Brown how to read fortunes

16.00: End

Postscript: Wiseman’s book contains a number of 2D barcodes. The book suggests that readers should point their smartphones at these barcodes. Their smartphones will then be redirected to short related movies on a special website, such as this one. It was a pleasant surprise to be reminded of the utility of smartphones while my mind was engrossed in reflections of psychology.

Leave a Comment

19 September 2010

Our own entrenched enemies of reason

Filed under: books, deception, evolution, intelligence, irrationality, psychology — David Wood @ 3:39 pm

I’m a pretty normal, observant guy. If there was something as large as an elephant in that room, then I would have seen it – sure as eggs are eggs. I don’t miss something as large as that. So someone who says, afterwards, that there was an elephant there, must have some kind of screw loose, or some kind of twisted ulterior motivation. Gosh, what kind of person are they?

Here’s another version of the same, faulty, line of reasoning:

I’m a pretty good police detective. Over the years, I’ve developed the knack of knowing when people are telling the truth. That’s what my experience has taught me. I know when a confession is for real. I don’t get things like that wrong. So someone who says, afterwards, that the confession was forced, or that the criminal should get off on a technicality, must have some kind of screw loose, or some kind of twisted ulterior motivation. Gosh, what kind of person are they?

And another:

I’m basically a moral person. I don’t knowingly cause serious harm to my fellow human beings. I don’t get things as badly wrong as that. I’m not that kind of person. So if undeniable evidence subsequently emerges that I really did seriously harm a group of people, well, these people must have deserved it. They were part of a bad crowd. I was actually doing society a favour. Gosh, don’t you know, I’m one of the good guys.

Finally, consider this one:

I’m basically a savvy, intelligent person. I don’t make major errors in reasoning. If I take the time to investigate a religion and believe in it, I must be right. All that investment of time and belief can’t have been wrong. Perish the thought. If that religion makes a prophecy – such as the end of the world on a certain date – then I must be right to believe it. If the world subsequently appears not to have ended on that date, then it must have been our faith, and our actions, that saved the world after all. Or maybe the world ended in an invisible, but more important way. The kingdom of heaven has been established within. Either way, how right we were!

It can sometimes be fun to observe the self-delusions of the over-confident. Psychologists talk about “cognitive dissonance”, when someone’s deeply held beliefs appear to be contradicted by straightforward evidence. That person is forced to hold two incompatible viewpoints in mind at the same time: I deeply believe X, but I seem to observe not-X. Most people are troubled by this kind of dissonance. It’s psychologically uncomfortable. And because it can be hard for them to give up their underlying self-belief that “If I deeply believe X, I must have good reasons to do so”, it can lead them into outlandish hoops and illogical jumps to deny the straightforward evidence. For them, rather than “seeing is believing”, the saying becomes inverted: “believing is seeing”.

As I said, it can be fun to see the daft things people have done, to resolve their cognitive dissonance in favour of maintaining their own belief in their own essential soundness, morality, judgement, and/or reasoning. It can be especial fun to observe the mental gymnastics of people with fundamentalist religious and/or political faith, who refuse to accept plain facts that contradict their certainty. The same goes for believers in alien abduction, for fan boys of particular mobile operating systems, and for lots more besides.

But this can also be a deadly serious topic:

It can result in wrongful imprisonments, with the prosecutors unwilling to face up to the idea that their over-confidence was misplaced. As a result, people spend many years of their life unjustly incarcerated.
It can result in families being shattered under the pressures of false “repressed memories” of childhood abuse, seemingly “recovered” by hypnotists and subsequently passionately believed by the apparent victims.
It can split up previously happy couples, who end up being besotted, not with each other, but with dreadful ideas about each other (even though “there’s always two sides to a story”).
Perhaps worst of all, it can result in generations-long feuds and wars – such as the disastrous entrenched enmity of the Middle East – with each side staunchly holding onto the view “we’re the good guys, and anything we did to these other guys was justified”.

Above, I’ve retold some of the thoughts that occurred to me as I recently listened to the book “Mistakes Were Made (But Not by Me): Why We Justify Foolish Beliefs, Bad Decisions, and Hurtful Acts”, by veteran social psychologists Carol Tavris and Elliot Aronson. (See here for this book’s website.) At first, I found the book to be a very pleasant intellectual voyage. It described, time and again, experimental research that should undermine anyone’s over-confidence about their abilities to observe, remember, and reason. (I’ll come back to that research in a moment). It reviewed real-life examples of cognitive dissonance – both personal examples and well-known historical examples. So far, so good. But later chapters made me more and more serious – and, frankly, more and more angry – as they explored horrific examples of miscarriages of justice (the miscarriage being subsequently demonstrated by the likes of DNA evidence), family breakups, and escalating conflicts and internecine violence. All of this stemmed from faulty reasoning, brought on by self-justification (I’m not the kind of person who could make that kind of mistake) and by over-confidence in our own thinking skills.

Some of the same ground is covered in another recent book, “The invisible gorilla – and other ways our intuition deceives us”, by Christopher Chabris and Daniel Simons. (See here for the website accompanying this book.) The gorilla in the title refers to the celebrated experiment where viewers are asked to concentrate on one set of activity – counting the number of passes made by a group of basketball players – and often totally fail to notice someone in a gorilla suit wandering through the crowd of players. Gorilla? What gorilla? Don’t be stupid! If there had been a gorilla there, I would have seen it, sure as eggs are eggs.

Chapter by chapter, “The invisible gorilla” reviews evidence that we tend to be over-confident in our own abilities to observe, remember, and reason. The chapters cover:

Our bias to think we would surely observe anything large and important that happened
Our bias to think our memories are reliable
Our bias to think that people who express themselves confidently are more likely to be trustworthy
Our bias to think that we would give equal weight to evidence that contradicts our beliefs, as to evidence that supports our beliefs (the reality is that we search high and low for confirming evidence, and quickly jump to reasons to justify ignoring disconfirming evidence)
Our bias to think that correlation implies causation: that if event A is often followed by event B, then A will be the cause of B
Our bias to think there are quick fixes that will allow significant improvements in our thinking power – such as playing classical music to babies (an effect that has been systematically discredited)
Our bias to think we can do many things simultaneously (“multi-task”) without any individual task being affected detrimentally.

These biases probably all were useful to Homo sapiens at an early phase of our evolutionary history. But in the complex society of the present day, these biases do us more harm than good.

Added together, the two books provide sobering material about our cognitive biases, and about the damage that all too often follows from us being unaware of these biases.

“Mistakes were made (but not by me)” adds the further insight that we tend to descend gradually into a state of gross over-confidence. The book frequently refers to the metaphor of a pyramid. Before we make a strong commitment, we are often open-minded. We could go in several different directions. But once we start down any of the faces in the pyramid, it becomes harder and harder to retract – and we move further away from people who, initially, were in the very same undecided state as us. The more we follow a course of action, the greater our commitment to defend all the time and energy we’ve committed down that path. I can’t have taken a wrong decision, because if I had, I would have wasted all that time and energy, and that’s not the kind of person I am. So they invest even more time and energy, walking yet further down that pyramid of over-confidence, in order to maintain their own self-image.

At root, what’s going wrong here is what psychologists call self-justification. Once upon a time, the word pride would have been used. We can’t bear to realise that our own self-image is at fault, so we continue to take actions – often harmful actions – in support of our self-image.

The final chapters of both books offer hope. They give examples of people who are able to break out of this spiral of self-justification. It isn’t easy.

An important conclusion is that we should put greater focus on educating people about cognitive biases. Knowing about a cognitive bias doesn’t make us immune to it, but it does help – especially when we are still only a few rungs down the face of the pyramid. As stated in the conclusion of “The invisible gorilla”:

One of our messages in this book is indeed negative: Be wary of your intuitions, especially intuitions about how your own mind works. Our mental systems for rapid cognition excel at solving the problems they evolved to solve, but our cultures, societies, and technologies today are much more complex than those of our ancestors. In many cases, intuition is poorly adapted to solving problems in the modern world. Think twice before you decide to trust intuition over rational analysis, especially in important matters, and watch out for people who tell you intuition can be a panacea for decision-making ills…

But we also have an affirmative message to leave you with. You can make better decisions, and maybe even get a better life, if you do your best to look for the invisible gorillas in the world around you… There may be important things right in front of you that you aren’t noticing due to the illusion of attention. Now that you know about this illusion, you’ll be less apt to assume you’re seeing everything there is to see. You may think you remember some things much better than you really do, because of the illusion of memory. Now that you understand this illusion, your trust your own memories, and that of others, a bit less, and you’ll try to corroborate your memory in important situations. You’ll recognise that the confidence people express often reflects their personalities rather than their knowledge, memory, or abilities… You’ll be skeptical of claims that simple tricks can unleash the untapped potential in your mind, but you’ll be aware than you can develop phenomenal levels of expertise if you study and practice the right way.

Similarly, we should also take more care to widely explain the benefits of the scientific approach, which searches for disconfirming evidence as must as it searches for confirming evidence.

That’s the pro-reason approach to encouraging better reasoning. But reason, by itself, often isn’t enough. If we are going to face up to the fact that we’ve made grave errors of judgement, which have caused pain, injustice, and sometimes even death and destruction, we frequently need powerful emotional support. To enable us to admit to ourselves that we’ve made major mistakes, it greatly helps if we can find another image of ourselves, which sees us as making better contributions in the future. That’s the pro-hope approach to encouraging better reasoning. The two books have examples of each approach. Both books are well worth reading. At the very least, you may get some new insight as to why discussions on Internet forums often descend into people seemingly talking past each other, or why formerly friendly colleagues can get stuck into an unhelpful rut of deeply disliking each other.

Leave a Comment

18 March 2010

Animal spirits – a richer understanding of economics

Filed under: books, Economics, irrationality, recession — David Wood @ 3:41 pm

It’s no secret that some of the fundamental assumptions of economic theory are faulty.

Specifically, the primary model in economics is that individuals invariably take actions which make good economic sense. The mythical “Homo economicus” (“Economic Man”) is motivated at all times:

To purchase goods and services that have lower cost;
To create goods and services that they can sell at higher price;
To minimise the amount of effort that they have to expend to create these goods and services.

Real world people, of course, deviate from this model in numerous ways. Lots of other things motivate us, beyond purely economic concerns.

Indeed, we can arrange human decisions on a two-by-two matrix:

On one dimension, decisions vary between economic motivations and non-economic movitations;
On the other dimension, decisions vary between rational and irrational.

Theories of classical economics take their lead from just one of the resulting four fields of life – the field of economic motivations that are pursued rationally. But what impact do the other three fields have on overall economic questions, such as booms and busts, inflation, employment, savings, and inequality?

Many classicial economists give the strong impression that these other three fields have limited impact – somehow their effects average out, or can be discounted. More recently, the rise of behavioural economics has challenged this conclusion, by increasingly providing evidence and analysis of factors such as:

Irrational biases in human decision making;
Herd mentality;
Limits of information;
The motivational importance of factors other than economic ones.

The best account I’ve encountered of this whole topic is the book “Animal Spirits: How Human Psychology Drives the Economy, and Why It Matters for Global Capitalism“.

This book was authored last year by two eminent economists, George A. Akerlof and Robert J. Shiller. Their phrase “Animal Spirits” is taken from Keynes – from a part of the thinking of Keynes that, they believe, has been too often neglected (even by people who describe themselves as followers of Keynes):

The markets are moved by animal spirits, and not by reason

(paraphrased from Keynes’ 1935 book “The General Theory of Employment Interest and Money”)

Akerlof and Shiller provide five chapters that explain each of five important contributors to “animal spirits”:

Confidence and Its Multipliers
Fairness
Corruption and Bad Faith
Money Illusion
Stories.

These explanations interweave many accounts of economic episodes over the decades, adding to the plausibility of the fact that these factors matter a great deal.

Next, Akerlof and Shiller show how considerations of these “animal spirits” provide deeper insight into each of eight key questions of economic theory:

Why Do Economies Fall into Depression?
Why Do Central Bankers Have Power over the Economy (Insofar as They Do)?
Why Are There People Who Cannot Find a Job?
Why Is There a Trade-off between Inflation and Unemployment in the Long Run?
Why Is Saving for the Future So Arbitrary?
Why Are Financial Prices and Corporate Investments So Volatile?
Why Do Real Estate Markets Go through Cycles?
Why Is There Special Poverty among Minorities?

To my mind, the analysis is devastating: any serious discussion of eonomics needs to take account of these findings.

Footnote: Amazon.com contains a whole series of nasty and devious reviews of this book. Don’t be misled by them! The motivations of the people writing these reviews would be a worthy subject for an analysis in its own right. There are other kinds of “animal spirits” afoot here.

Comments (6)

dw2

24 June 2019

Superintelligence, Rationality, and the Race to Save the World

7 August 2015

Brave new world – bold new adaptation

26 September 2013

Risk blindness and the forthcoming energy crash

17 September 2013

When faith gets in the way of progress

22 February 2013

Controversies over singularitarian utopianism

10 February 2013

Fixing bugs in minds and bugs in societies

14 January 2012

Speaking of angels – visions of a world beyond

19 September 2010

Our own entrenched enemies of reason

18 March 2010

Animal spirits – a richer understanding of economics

Pages

Recent Posts

Archives

Recent Comments

Categories

Email Subscription