Why I stopped working on the Bongard Problems

First, what are the Bongard problems? You can best understand the answer by clicking on the link above, but to save you from going on a tangent, I’ll simply say here that Bongard problems are visual puzzles: you look at something like the drawing with the 6+6 boxes below, and you try to figure out why the six boxes on the left have been separated from the six boxes on the right. What is it that the six boxes on the left have in common?

In other words, the goal of my research was not to write simply a program that solves Bongard problems, but to write a program that implements some fundamental principles of cognition, which help it exhibit — in a rudimentary way — some aspects of human behavior, or of human-like thinking.

Okay. So where are the ethical issues in all that?

They’re in the remote possibility of building intelligent machines that act, and even appear, as humans. If this is achieved, eventually intelligent weapons of mass destruction will be built, without doubt. That’s what I would like to explain below.

Now, picture this kind of robot in the not-so-remote future. Imagine that she (or he, or it) has enough computational power in the computer situated in her skull (or in her thorax, doesn’t matter where) to allow her to behave naturally like a human being. And also imagine that in her belly she harbors not guts (which would be useless to her), but a small nuclear bomb.

Impossible? Why not? I’m not talking about current technology, by which nuclear weapons look like monsters. I’m talking future technology, when downscaling everything appears reasonable.

(Note: A reader raised the objection that there is a limit to how small nuclear weapons can become due to the well-known critical mass for radioactive material to start nuclear chain reactions, and the rest of the appurtenances that a nuclear bomb must have in order to function as a controlled bomb. I don’t dispute all this. Although we can’t predict the technology of the future on the basis of what we know at present, let us suppose that planting a nuclear bomb in a womb will indeed remain an impossibility forever. But I don’t mean to narrow the scope of what I am talking about to nuclear weapons. It could be biological weapons: a “bomb” that spreads deadly viruses, without even a need for an explosion, but working merely by the actroid’s coughing or sneezing. So when you read about “nuclear weapons” in this article, please interpret that as a generic term for “weapons of mass destruction”.)

Let’s see... I suspect you don’t really object that this is a plausible scenario. What you really believe (or maybe just hope) is that it will be us, our side, our army that will acquire such marvelous weapons. The enemy won’t have them, and so we, with our superior technology, will emerge victorious and live happily ever after, having crushed the barbarians. Yey!

It is typically Americans who display this attitude regarding hi-tech weapons. (If you are an American and are reading this, what I wrote doesn’t imply that you necessarily display this attitude; note the word “typically”, please.) The American culture has an eerily childish approach toward weapons, and also some outlandish (but also child-like) disregard for human life. (Once again, you might be an intelligent, mature American, respecting life deeply; it is your average compatriot I am talking about.) Here is what an American journalist wrote in Washington Post, on May 6, 2007:

“So where does the air vehicle called the Predator [i.e., a flying robot] fit? It is unmanned, and impressive. In 2002, in Yemen, one run by the CIA came up behind an SUV full of al-Qaeda leaders and successfully fired a Hellfire missile, leaving a large smoking crater where the vehicle used to be.”

Yes, just as you read it: a number of human beings were turned to smoke and smithereens, and this pathetic journalist, whoever he is, speaking with the mentality of a 10-year-old who blows up his toy soldiers, reports in cold blood how people were turned to ashes by his favorite (“impressive”, yeah) military toys. Of course, for overgrown pre-teens like him, the SUV was not full of human beings, but of “al-Qaeda leaders”, of terrorists, sub-humans who aren’t worthy of living, who don’t have mothers to be devastated by their loss. Thinking of the enemy as subhuman scum to be obliterated without second thoughts was a typical attitude displayed by Nazis against Jews (and others) in World War II. (The full article is here, and explains how soldiers become sentimentally attached to their robots, extensions of their teenage-time toys, obviously ascribing to them a higher value than human life; the above quoted passage appears on the 3rd page.)

If this attitude were marginal among Americans, if the above story were a fluke, I wouldn’t worry at all. Any moron can say anything they like in a free society, and even have their moronic thoughts appear in print. The problem from my point of view is that I’ve seen the above attitude again and again in the years that I lived in the U.S.A. Once, the janitor of the building where I used to do my research, having just learned some sad news about American soldiers killed in Iraq, wondered in a discussion with me: “Why don’t we just nuke ’em all? Just turn the damn desert into glass and be done with those ___” (I don’t remember what adjective he used). You might think the janitor wasn’t very sophisticated in his approach toward war or human lives. But a few days later I was reading another article on a web site, of which unfortunately I didn’t save the address, that was reporting about a similar issue as the one above: how to use robots to enter caves (it was known that the al-Qaeda leader, Osama bin Laden, was hiding in caves at the Afghanistan–Pakistan border back then), search for terrorists, and blast the place, terrorists, robot, and all; to “smoke them out of their holes”, as that pinnacle of wisdom, the American president, said immediately after the 9/11 attacks.

So: how nice it would be to have “actroids” pregnant with nuclear or biological bombs, right? Perhaps “nuctroids”, how about that? Of course, only we would know they are actually nuctroids. To the terrorists they would pass for normal people.

How immature must a person be to believe something like that! Think of nuclear weapons. When they first appeared on the scene, in the second half of the 20th century, originally only five nations possessed them: the U.S.A., the U.K., France, the Soviet Union, and China (the victors of WWII). Gradually, more countries entered the nuclear club, some of them openly (India, Pakistan), others secretly (Israel). Now every pariah state can have their nuclear toys, or dream of acquiring them. At the time of writing the present, there is a strong fear in the international community that Pakistan’s American-supported dictator will be overthrown by extremist Islamists, and the nuclear bombs that Pakistan possesses will fall into the hands of terrorists. It is no secret that Iran, an avowed enemy of the U.S.A., is planning to build its own nuclear weapons. Turkey, now an ally of the U.S., is planning to build its own “energy-only” nuclear factories; but after one or two decades Turkey might turn into a hub for radical Islamism due to its gradually changing demographics, and so in the future we might have another Iran-like nuclear-power wannabe there. So by what stretch of the imagination and crooked logic will it become impossible for pariah states, or even individuals, to possess and command “nuctroids” in the foreseeable future?

Technology spreads. It’s not something that can be confined within national borders. Especially now, when we talk about globalization, we must understand that knowledge “goes global” too, and this includes the specialized knowledge needed to build an extra-small weapon of mass destruction, or a human-like deadly robot.

So how does working toward innocent projects such as the automation of Bongard problems tie in into all this?

As I explained earlier, it’s not just the automation of Bongard problems that’s involved. It’s about the automation of cognition. Anyone who works toward making machines intelligent, and especially wanting machines to “come alive”, must understand the grave ethical issues involved in such an endeavor. Consider the following email message sent by a student at Indiana University (IU, the academic institution where I did my Ph.D.) in 2008 (my emphasis):

Hello everyone,

  The IU Robotics Club is having its first meeting this Thursday,
January 17th. We are a group of undergraduates and graduates hailing
from many fields with a common interest in all robotics, automata,
synthetic life and artificial intelligence. We encourage people to make
stuff come alive, whatever they decide that means.

Does anyone at IU realize the ethical issues that these kids are toying with? Is it really more important to be concerned with cloning and stem cell research? Does it not matter at all that these kids, or maybe their children, might be turned to a loose collection of quantum particles some time in the not-so-remote future by the fruits of their own toy-making? Or is it that what causes the indifference is the remoteness of the future, whereas other ethical issues in science are present here-and-now? But don’t the seriousness of the nuctroid threat and its logical inevitability make any impression on anyone?

      About your concerns about automated weapons, I see your point and I agree that there are plenty of bad possibilities on the horizon.  However, I see it as a kind of race between those bad prospects, on the one hand, and on the other hand, the unification of humanity via things like the Web and cell phones and email and so forth.  It seems to me that the tighter we are all tied together into one big net, so that we all share one common fate, the lower the probability of global conflict, and the more cooperation there will be between people (and peoples).  I hope that the world’s shrinking will occur more rapidly than the progress of weapons research.  One never knows, of course.

Right, one never knows. I am not expressing absolute certainties here, even when I talk about the “inevitability” of an event (see above). I thought about my ex-advisor’s idea, but my conclusions were not as optimistic as his. Specifically, I don’t think that globalization will turn the Earth into a big, happy village. Far from it. The financial distance between the richest rich and the poorest poor worldwide seems to be increasing, decade after decade: whereas affluent societies become more affluent, not only the poor remain equally poor, but their numbers keep increasing. Affluent societies curb their birthrate, whereas poor ones keep it fixed and high. Poverty generates more poverty, in a vicious circle. The result of globalization can be curiosities such as TV antennas jutting out of straw-made huts in Africa; or bare-breasted, colorfully decorated black girls with a pair of earphones stuck in their ears and an MP3 player hung among their other ornaments. But globalization doesn’t improve the living standards of the poor. It merely makes it easier for the rich to plunder the natural resources of the poor, as the Bush Oil Wars in the Middle East prove, and as the indigenous tribes in Central and South America know all too well. Globalization sends the “culture” of the U.S.A. (“music”, Hollywood-like movies, fast-food eating habits, Western-like dress codes, etc.) to the rest of the world, but also sends natural resources and wealth from other nations to the U.S.A. This results in animosity of the have-nots against the have-lots. I don’t see how globalization will help ease the tension. If anything, I see globalization as a factor that exacerbates the tension and accelerates the time of an eventual global conflict, because through globalization the poor come into contact with some of the achievements of the rich, they learn about them, while still remaining poor, so they see what they’re missing and become jealous. In 1789, in France, the masses of poor revolted against the oligarchy of the aristocrats (and won). If trends continue as today, some time in the future the global village will explode in an analogous fashion, when the masses of poor nations will again request their fair share of the global pie from rich nations. Whether the explosion will happen before the development of nuctroids, I don’t know. But even if it happens during the interim when the affluent societies will have just developed and possess nuctroids whereas the poor will not have acquired them yet, such hi-tech weapons would be useless for the rich: you just can’t “nuke ’em all”. The mentality of “turning the desert into glass” can only be entertained by naïve minds, and result in their own — and our — demise. (I am not trying to express any superiority over the said janitor’s intelligence here; if anyone with his attitude can rationally argue against my claims, I’ll be happy to learn about such arguments.)

In the late 19th – early 20th century, with the anticipated spread of the use of electricity, some people were afraid that others would be electrocuted, so electricity was perceived as a public threat. Such worries, which appear even funny today, were not baseless or useless. It was because of such worries that measures were taken and technology developed that made it possible to build essentialy harmless electrical devices.

A related issue, specific to the people in the U.S.A., is that after the 9/11 attacks Americans went through a period of intense fear, an apprehension about anything that could upset their easy and cozy way of living. Now, after several years without another attack on U.S. soil, they have timidly started exiting from this period of apprehension, and the first articles that look at their “age of fear” with a critical eye (and even a sense of humor) have started appearing (e.g., read this one). The danger here is that they will experience what I call the “swing of the pendulum to the other extreme position”; i.e., just like when you release a pendulum from one extreme position it doesn’t go straight to the equilibrium point but swings all the way to the other extreme first, similarly, Americans might feel disdain for the kind of danger that I describe here, and treat it as just another one of those hateful scenarios that used to send chilling sparks of fear up and down their spines in the past. It’s a natural psychological reaction to try and turn one’s face away from what causes discomfort. But Americans can sense that this is not a case like those they’re familiar with, if they realize that the “reign of terror” was a cheap trick employed for years by their post-9/11 administrations in order to reduce civil liberties and pass antidemocratic policies with greater ease. I am not a member of their administration, not even an American. I am speaking as a person concerned about fellow people and the future of humanity as a whole.

Research that aims toward making machines appear human must be marked as highly dangerous, or ethically suspicious at least. Such research should not be funded. Note, I am not advocating an enmity toward all research in artificial intelligence and cognitive science; only a discouragement of the research that explicitly leads to the development of machines that can deceive humans, and pass as humans. To have computers that can compose high-quality music, for instance, or translate between languages, is not dangerous. Nor is it dangerous to have self-aware, conscious machines. If anything, a self-aware machine that places a high value on its preservation, and on the preservation of humanity, is probably more difficult to persuade to go and explode itself among people — Islamist suicide bombers notwithstanding. This last thought implies that sophistication is probably a desired attribute of machines: the more self-conscious, the less of a nuctroid threat; but self-conscious they must be by law, not by the goodwill of the free market; and self-conscious means human-like in mind, not in form and external appearance. We do need robots that work for us, but not robots that trick us into misidentifying them.

High school children, undergraduates, graduates, and in general all people involved in the educational systems of the U.S.A., Europe, Japan, Australia, and elsewhere, must abandon their naïve attitude of “Let’s make stuff come alive”, and become aware of the seriousness of such an endeavor. Children cannot discover the seriousness of this matter by themselves, so it is up to the academic institutions to educate their students and take appropriate measures. If universities, such as my own IU, can be so serious about the ethics of procedures that involve psychological experiments on human subjects and even on animals (the bureaucracy they impose is amazing), it is high time they become even more serious about what machines their students in artificial intelligence and cognitive science are experimenting on.

Americans should grow up and abandon their juvenile-minded treatment of weapons, high technology, and the value of “non-American human life” (which, sadly, to many of them is synonymous with “lowlife”). This is the hardest part of my proposal. One can’t just tell an entire culture to do this, don’t do that. In this case matters are complicated by the existence of an elite rich class in the U.S.A., in the interests of which is to keep the public uninformed, having my janitor’s “Let’s nuke ’em all” attitude, because the lack of public awareness increases the short-term benefits of the rich class (they support wars, which help them manufacture, advertise, and sell more weapons abroad, for instance). This is compounded by the American myth that it doesn’t matter if you are poor, because if you’re capable enough you can raise your social status all the way to the top. Having believed this myth for centuries, the poor among the Americans don’t mind much being kept at bay (i.e., being poor and thus staying at my janitor’s level of political and educational sophistication) because the notion that anyone can rise to the top sweetens the pill and makes it more palatable. I must note here, I believe the so-called American dream of “rags-to-riches” is a myth for at least two reasons: first, it does matter if you are born Black, Hispanic, etc. — after all, what is the percent of non-white billionaires in the U.S.A.? And second, it is analogous to winning the jackpot in a lottery: yes, you can be the sole lucky winner, and you can even nurture your ego by thinking that winning is not a matter of mere luck, but the existence of the jackpot itself presupposes a vast number of losers; what are the chances that you’ll have the guts and wits to be the sole winner and rise high in social status? (And is it really so attractive to live in a society with a handful of winners and millions of losers, resulting in social conflicts and civil unrest?)

According to a survey that appeared in a magazine sometime between 2003 and 2006, of which I regret to have not written down the source (it was intimated to me by some friends of mine), it can be inferred that 17% of Americans believe that they belong to the top 1% of financial status in the U.S.A. The corresponding percent among the British is around 3%. When Americans were asked if they hoped whether they would reach the top 1% some time in their lives, the percent of those who thought so doubled (34%). This means that 16% of Americans delude themselves about their present situation, and a whopping 33% (one third!) delude themselves about their future situation. I consider that poll to be very revealing regarding American self-delusions; if anyone knows its source, please let me know. (Special thanks to a reader, Walter van Huissteden, who brought to my attention this article, which reports various opinion poll results that seem very similar to those I refer to here; the original source of my data seems to be a July 2000 Gallup poll.)

For the above reasons, I realize the tremendous difficulty of talking to an entire culture. So, the only course of action that seems conceivable to me is to raise their awareness by having as many voices as possible talk about this issue in unison, and do this particularly in the most anarchic, relatively uncensored-from-above medium of communication that humanity has ever known, the Internet.