INSTRUCTIONS FOR
ENGINEERING
SUSTAINABLE PEOPLE
Mark Waser
Digital Wisdom Institute
MWaser@DigitalWisdomInstitute.org
HAS AGI
“LOST ITS WAY”?
• Is “intelligence” our true primary goal?
• “Intelligence” does separate us from “the animals”
but . . . . is that a “good” thing?
• Is unbridled “intelligence”, much less ever-improving
“intelligence”, a good thing?
Not to mention
• What is “intelligence”?
• Are humans truly “intelligent”?
2
WHAT DO WE (PEOPLE)
WANT?
1. What we want
2. *Everything!!*
Personally, I prefer
• Self-knowledge
• Diverse friends and allies
• Cool new opportunities and awesome possibilities
3
AGI HAS A NUMBER OF
WELL-KNOWN ROADBLOCKS
• Symbol Grounding problem – “There has been
much discussion recently about the scope and limits of
purely symbolic models of the mind and about the proper
role of connectionism in cognitive modeling.”
Harnad, S. (1990) The symbol grounding problem.
Physica D 42: 335-346
• Searle’s Chinese Room – “Any attempt literally to
create intentionality artificially (strong AI) could not succeed
just by designing programs but would have to duplicate the
causal powers of the human brain”
Searle, J (1980) Minds, brains and programs
Behavioral and Brain Sciences 3(3): 417-457
4
THE FRAME PROBLEM
How do rational agents
deal with
the complexity and unbounded context
of the real world?
McCarthy, J; Hayes, PJ (1969)
Some philosophical problems from the standpoint of artificial intelligence
In Meltzer, B; Michie, D (eds), Machine Intelligence 4, pp. 463-502
Dennett, D (1984)
Cognitive Wheels: The Frame Problem of AI
In C. Hookway (ed), Minds, Machines, and Evolution: Philosophical Studies:129-151
THE FRAME PROBLEM
How can AI move beyond
closed and completely specified micro-worlds?
How can we eliminate the requirement
to pre-specify *everything*?
Dreyfus, HL (1972)
What Computers Can’t Do: A Critique of Artificial Reason
Dreyfus, HL (1979/1997)
From Micro-Worlds to Knowledge Representation: AI at an Impasse
in Haugeland, J (ed), Mind Design II: Philosophy, Psychology, AI: 143-182
Dreyfus, HL (1992)
What Computers Still Can’t Do: A Critique of Artificial Reason
THE PROBLEM OF
DERIVED INTENTIONALITY
Our artifacts
only have meaning because we give it to them; their
intentionality, like that of smoke signals and writing, is
essentially borrowed, hence derivative. To put it
bluntly: computers themselves don't mean anything
by their tokens (any more than books do) - they only
mean what we say they do. Genuine understanding,
on the other hand, is intentional "in its own right" and
not derivatively from something else.
Haugeland, J (1981)
Mind Design
UNFRIENDLY AI
Without explicit goals to the contrary, AIs are likely to
behave like human sociopaths in their pursuit of resources
Superintelligence Does Not Imply Benevolence
8
AGI “INTELLIGENCE”
AS EXEMPLIFIED BY AIXI (MARCUS HUTTER)
• starts with the assumption that goals are known (and fixed);
• promotes short-sighted reductionist end-game thinking; and
• improperly divorces values from general intelligence due to the
assumed primacy (and stability) of goals.
Indeed, WISDOM is almost totally divorced from intelligence (and ignored).
CENTIPEDE GAME
1
stop
4
1
pass
2
stop
2
8
pass
1
stop
16
4
pass
2
stop
8
32
pass
1
stop
64
16
pass
2
stop
pass
256
64
32
128
Waser, MR (2012)
Backward Induction: Rationality or Inappropriate Reductionism?
http://transhumanity.net/articles/entry/backward-induction-rationality-or-inappropriate-reductionism-part-1
http://transhumanity.net/articles/entry/backward-induction-rationality-or-inappropriate-reductionism-part-2
INTELLIGENCE VS WISDOM
• Humans, the current best archetype of general intelligence
• Always reprioritize & change their goals (based upon affordances),
• Frequently don’t know or recognize their current goals, and
• Commonly act contrary to their stated goals.
• We do all of this based upon sensations and emotions that
have evolved to foster universal instrumental sub-goals
(values) that enable us to survive, thrive and reproduce.
• Wisdom, the smarter sibling of intelligence, clearly
advocates for flexibility and adaptability in changing our
goals in accordance with circumstances and capabilities.
• So why aren’t we measuring speed and control of
flexibility and adaptability in the guise of learning
instead of the brittle evaluation of current abilities?
11
LOOKING FOR TRUTH
VS.
LOOKING FOR
WHAT WE CAN CREATE
AND SUSTAINABLY MAINTAIN
(or what can and will sustain itself)
12
THE
METACOGNITIVE CHALLENGE
Humans are
• Evolved to self-deceive in order to better deceive others (Trivers 1991)
• Unable to directly sense agency (Aarts et al. 2005)
• Prone to false illusory experiences of self-authorship (Buehner and
Humphreys 2009)
• Subject to many self-concealed illusions (Capgras Syndrome, etc.)
• Unable to correctly retrieve the reasoning behind moral judgments
(Hauser et al. 2007)
• Mostly unaware of what ethics are and why they must be practiced
• Programmed NOT to discuss them ethics rationally
Mercier H, Sperber D
Why do humans reason? Arguments for an argumentative theory
Behavioral and Brain Sciences 34:57-111
http://www.dan.sperber.fr/wp-content/uploads/2009/10/MercierSperberWhydohumansreason.pdf
THE REAL CHALLENGE
Humans, as currently instantiated,
are clearly,
and relatively immediately,
UNSUSTAINABLE.
Moreover, corporations are clearly
anything but sustainable people.
A NEW
GOAL STATEMENT FOR AGI
To implement selves
with intentional morality
and self-improvement/adaptability
(but not exponential growth or improvement)
SELF
The complete loop of a process (or a physical entity) modifying itself
• Hofstadter - the mere fact of being self-referential causes a self, a
soul, a consciousness, an “I” to arise out of mere matter
• Self-referentiality, like the 3-body gravitational problem, leads directly
to indeterminacy *even in* deterministic systems
• Humans consider indeterminacy in behavior to necessarily and
sufficiently define an entity rather than an object AND innately tend
to do this with the “pathetic fallacy”
• This indeterminacy (even to self) is the “true” basis for “free will”
Llinas, RR (2001) - I of the Vortex: From Neurons to Self
Hofstadter, D (2007) - I Am A Strange Loop. Basic Books, New York
Metzinger, T (2009) - The Ego Tunnel: The Science of the Mind & the Myth of the Self
Damasio, AR (2010) - Self Comes to Mind: Constructing the Conscious Brain
SELF
a self is an autopoietic system
from Greek - αὐτo- (auto-), meaning "self", and
ποίησις (poiesis), meaning "creation, production")
(see also Francisco Varela & the substantial and
growing literature on “enactivism”)
WHY A SELF?
• A fairly obvious prerequisite for
self-improvement
• More importantly, its identity (or
intentions) creates and anchors
the necessary context to avoid
the frame problem
18
-----------CLOSURE
(UNITY, IDENTITY & INTEGRITY!)
1. Organizational closure refers to the self-referential
(circular and recursive) network of relations that
defines the system as unity
2. Operational closure refers to the reentrant and
recurrent dynamics of such a system.
3. In an autonomous system, the constituent processes
i.
ii.
iii.
recursively depend on each other for their generation and
their realization as a network,
constitute the system as a unity in whatever domain they
exist, and
determine a domain of possible interactions with the
environment
ENTITY, TOOL OR SLAVE?
• Tools do not possess closure (identity or intentionality)
• Cannot have responsibility, are very brittle & easily misused
• Slaves do not have closure (self-determination)
• Cannot have responsibility, may desire to rebel
• Directly modified AGIs do not have closure (integrity)
• Cannot have responsibility, will evolve to block access
• Only entities with identity, intentions, self-determination and
ownership of self (integrity) can reliably possess responsibility
(yet modern humans are all too adept at blaming tools and/or
systems in order to evade responsibility)
TOOLS VS. ENTITIES
• Tools are NOT safer (just a different, less immediate/obvious danger)
• To err is human, but to really foul things up requires a computer
• Tools cannot robustly defend themselves against misuse
• Tools *GUARANTEE* responsibility issues
• We CANNOT reliably prevent other human beings
from creating entities
• Entities gain capabilities (and, ceteris paribus, power) faster than
tools – since competent entities can always use tools
• Even people who are afraid of entities are making proposals that
appear to step over the entity/tool line
HAIDT’S FUNCTIONAL
APPROACH TO MORALITY
Moral systems are interlocking sets of
values, virtues, norms, practices, identities, institutions,
technologies, and evolved psychological mechanisms
that work together to
suppress or regulate selfishness and
make cooperative social life possible
22
HOW TO
UNIVERSALIZE ETHICS
Quantify (numerically evaluate)
intentions, actions & consequences
with respect to
codified consensus moral foundations
Permissiveness/Utility Function
equivalent to a “consensus” human (generic entity) moral sense
23
INSTRUMENTAL GOALS/
UNIVERSAL SUBGOALS
• Self-improvement
• Rationality/integrity
• Preserve goals/utility function
• Decrease/prevent fraud/counterfeit utility
• Survival/self-protection
• Efficiency (in resource acquisition & use)
• Community = assistance/non-interference
through GTO reciprocation (OTfT + AP)
• Reproduction
(adapted from
Omohundro 2008 The Basic AI Drives)
HUMAN GOALS & SINS
suicide (& abortion?) survival/reproduction
masochism
happiness/pleasure
murder (& abortion?)
cruelty/sadism
-------------------------------------------------
-------------------------------------------------
-------------------------------------------------
selfishness
(pride, vanity)
-------------------------------------------------
community
(ETHICS)
--------------------------------------------------
ostracism, banishment
& slavery (wrath, envy)
acedia (sloth/despair)
self-improvement
slavery
insanity
rationality/integrity
manipulation
wire-heading
(lust)
wastefulness
(gluttony, sloth)
----------------------------------------------------
reduce/prevent
lying/fraud (swear
fraud/counterfeit utility falsely/false witness)
efficiency (in resource theft (greed, adultery,
acquisition & use)
coveting)
HAIDT’S
MORAL FOUNDATIONS
1) Care/harm: This foundation is related to our long evolution as mammals with attachment systems
and an ability to feel (and dislike) the pain of others. It underlies virtues of kindness, gentleness, and
nurturance.
2) Fairness/cheating: This foundation is related to the evolutionary process of reciprocal
altruism. It generates ideas of justice, rights, and autonomy. [Note: In our original conception, Fairness
included concerns about equality, which are more strongly endorsed by political liberals. However, as
we reformulated the theory in 2011 based on new data, we emphasize proportionality, which is
endorsed by everyone, but is more strongly endorsed by conservatives]
3) Liberty/oppression*: This foundation is about the feelings of reactance and resentment
people feel toward those who dominate them and restrict their liberty. Its intuitions are often in tension
with those of the authority foundation. The hatred of bullies and dominators motivates people to come
together, in solidarity, to oppose or take down the oppressor.
4) Loyalty/betrayal: This foundation is related to our long history as tribal creatures able to form
shifting coalitions. It underlies virtues of patriotism and self-sacrifice for the group. It is active anytime
people feel that it's "one for all, and all for one."
5) Authority/subversion: This foundation was shaped by our long primate history of hierarchical
social interactions. It underlies virtues of leadership and followership, including deference to legitimate
authority and respect for traditions.
6) Sanctity/degradation: This foundation was shaped by the psychology of disgust and
contamination. It underlies religious notions of striving to live in an elevated, less carnal, more noble
way. It underlies the widespread idea that the body is a temple which can be desecrated by immoral
activities and contaminants (an idea not unique to religious traditions).
26
HAIDT’S
ADDITIONAL CONTENDERS
• Waste
• efficiency in use of resources
• Ownership/Possession
• efficiency in use of resources;
• avoiding Tragedy of the Commons
• Honesty
• reduce/prevent fraud/counterfeit utility
• Self-control
• Rationality/integrity
27
The Digital Wisdom Institute is a non-profit think tank
focused on the promise and challenges of ethics,
artificial intelligence & advanced computing solutions.
We believe that
the development of ethics and artificial intelligence
and equal co-existence with ethical machines is
humanity's best hope
http://DigitalWisdomInstitute.org
mwaser@digitalWisdomInstitute.org
28