The Surprisingly Difficult Science of Beautiful Paintings Derek J. SMITH, C.Eng., C.I.T.P. High Tower Consultants Limited Cardiff Metropolitan University (Retired) smithsrisca@btinternet.com http://www.smithsrisca.co.uk A presentation to Thursday 26th July 2012 ABOUT THE AUTHOR • During the 1980s Derek Smith worked for British Telecom, Cardiff, where he specialised in the design and operation of very large CA-IDMS "semantic network" databases. • Between 1991 and 2010 he taught psycholinguistics and neuropsychology to the Speech and Language Therapy students at Cardiff Metropolitan University. • He is currently working with International Software Products, Toronto, on "Project Konrad", an artificial consciousness project using a CA-IDMS platform. WHY WE ARE HERE TODAY Here is a critically acclaimed painting. In a moment you will find out if you love it or hate it, or whether you are going to wonder what all the fuss is about, but there is no single science to explain this (or any other) moment of artistic appreciation. HOPEFULLY MANY OF YOU WON'T HAVE SEEN THIS PICTURE BEFORE. WHETHER YOU LIKE IT OR NOT, ASK YOURSELF WHY, AND GIVE YOURSELF A SCORE HOW MUCH. Francis Bacon HATE IT LOVE IT "Portrait of Lucien Freud" [ONE 0 10 OF THREE] (1952) PLAN OF THE PRESENTATION • Section 1 examines why vision evolved and how the eye feeds information to the visual areas of the brain. • Sections 2 to 4 then introduce some of the specific problems raised when the visual object happens to be a work of art. • Finally, Sections 5 to 7 report how these problems have helped the author design a robot brain capable of human-like experience, and suggest how that robot brain might, in turn, teach us more about the human experience of art. SECTION 1 THE VISUAL SYSTEM AND SURVIVAL HOW VISION BEGAN, HOW IT WORKS, WHY IT DOES WHAT IT DOES THE VERTEBRATE NERVOUS SYSTEM HOW IT ALL BEGAN This is a Pikaia, a wormlike creature from 530 MYA, known only from the fossil record. It has segmented blocks of muscle and has evolved a nerve cord to coordinate their contraction and relaxation. HEAD TAIL Pikaia, or creatures like them, are the evolutionary ancestors of all modern vertebrates, including humans. | ---------------------- 50mm -------------------- | THE VERTEBRATE NERVOUS SYSTEM HOW IT ALL BEGAN As new animals evolved, this nerve cord gradually thickened at the head end to form a brain. Visual information was processed in an area known as the "optic tectum" [solid blue]. The surface layer, or "cortex", of the "cerebrum" [light blue] provided additional general intelligence when called for. THE VERTEBRATE NERVOUS SYSTEM HOW IT ALL BEGAN As evolution continued, brains became larger and more powerful, and many functions of the optic tectum were taken over by the cerebral cortex. THE VERTEBRATE NERVOUS SYSTEM HOW IT ALL BEGAN The underlying structure of all these nervous systems is the same. Note that vision is just one cortical specialism amongst many. SMELL AND TASTE CONSCIOUSNESS, COMMAND, AND CONTROL HABIT, INSTINCT, AND EMOTION BALANCE AND REFLEXES BODILY SENSES INSTRUCTIONS TO MUSCLES VISION AND HEARING THE VERTEBRATE NERVOUS SYSTEM HOW IT ALL BEGAN It is common practice to refer to all mental processing above the level of instinct and reflex as "cognition", and to the science thereof as "cognitive science". SMELL AND TASTE KNOWING HABIT, INSTINCT, AND EMOTION BALANCE AND REFLEXES BODILY SENSES INSTRUCTIONS TO MUSCLES VISION AND HEARING THE VERTEBRATE NERVOUS SYSTEM HOW IT ALL BEGAN NOTE HOW VISUAL INFORMATION GETS SQUEEZED KNOWING UPWARDS THROUGH THE HABIT, INSTINCT, LOWER SYSTEMS AND ON ITS WAY TO EMOTION BECOMING CONSCIOUS. SMELL AND TASTE BALANCE AND REFLEXES VISION AND HEARING BODILY SENSES INSTRUCTIONS TO MUSCLES THE VERTEBRATE NERVOUS SYSTEM HOW IT ALL BEGAN NOTE ALSO HOW THE HIGHER SYSTEMS ARE NOT ALWAYS NEEDED .... KNOWING HABIT, INSTINCT, AND EMOTION BALANCE AND REFLEXES BODILY SENSES INSTRUCTIONS TO MUSCLES SMELL AND TASTE VISION AND HEARING THE VERTEBRATE NERVOUS SYSTEM THE VISUAL SYSTEM IN HUMANS Here is how the visual system feeds information into the human cerebrum. What happens after it arrives is far less clear because the mind's higher functions are still poorly understood. ? ? ? ? ? ?? THE VERTEBRATE NERVOUS SYSTEM HOW IT ALL BEGAN Because they are more technically precise, we shall be using a lot of engineering diagrams like this towards the end of the presentation. ? ? ? CORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS SENSORY INPUT EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SO WHAT HAPPENS AS THE LOWER VISUAL PROCESSES FEED INFORMATION THROUGH TO THE HIGHER ONES? THE VISUAL SYSTEM AND SURVIVAL SEARCHING FOR PREDATORS SEE ACTIVITY #1 IN YOUR ACTIVITIES PACK "The Hidden Tiger" Donald Rust (2011) THE VISUAL SYSTEM AND SURVIVAL FINDING A MATE THESE TWO MALES ARE DOING ARTISTS OFTEN HUMAN THEIR BEST TODEPICT BE VISUALLY INSTINCTIVE "BODY LANGUAGE" IN ATTRACTIVE TO POTENTIAL THEIR WORKS ..... BREEDING PARTNERS. THE HUMAN VISUAL SYSTEM IS SIMILARLY INVOLVED IN INTERPRETING OTHER PEOPLE'S BODY LANGUAGE. "The Flirt" Eugen von Blaas (1904) THE VISUAL SYSTEM AND SURVIVAL SEARCHING FOR OBJECTS SEE ACTIVITY #2 IN YOUR ACTIVITIES PACK This act of recognition will come very quickly once you have sorted the "figure" from the "ground". THE VISUAL SYSTEM AND SURVIVAL THE GESTALT LAW OF SIMILARITY Visual input is organised for us long before it becomes conscious. Indeed the organising actually determines what we become conscious of. RELAX AND LET YOUR VISUAL SYSTEM TELL YOU WHAT'S GOING ON HERE THE VISUAL SYSTEM AND SURVIVAL THE GESTALT LAW OF CONTINUITY THE VISUAL SYSTEM AND SURVIVAL THE GESTALT LAW OF CLOSURE THE VISUAL SYSTEM AND SURVIVAL THE GESTALT LAW OF CLOSURE Artists like to take advantage of the Gestalt Laws. Upon close inspection this is an impossible scene. However the visual system simply fills in the gaps, ignores the inconsistencies, and decides for us what is out there. Rene Magritte "Free Rein" (1965) THE VISUAL SYSTEM AND SURVIVAL THE GESTALT LAW OF COMMON FATE Your experience of this scene will change in an instant once things start to move, because things which move together get processed together. THE VISUAL SYSTEM AND SURVIVAL FACE RECOGNITION Another thing the visual system is good at is recognising people. SEE ACTIVITY #3 IN YOUR ACTIVITIES PACK THE VISUAL SYSTEM AND SURVIVAL FACE RECOGNITION AND CARICATURE THE VISUAL SYSTEM AND SURVIVAL BUT HOW LITTLE WE REALLY SEE PLAY "WHODUNNIT" VIDEO (WCRS) THIS SHORT VIDEO INCLUDES 21 DELIBERATE VISUAL TRICKS AND INCONSISTENCIES - HOW MANY CAN YOU SEE? THE VISUAL SYSTEM AND SURVIVAL VISUAL SCANNING SEE ACTIVITY #4 IN YOUR ACTIVITIES PACK THE VISUAL SYSTEM AND ART VISUAL SCANNING Our eyes build up what we see from a succession of very brief "fixations". The scanning process can nowadays be monitored using special glasses, as shown below. This technique allows us to study the points of relative interest in an artwork ..... Betz, Engelbrecht, Klein, and Rosenberg (2010) THE VISUAL SYSTEM AND ART VISUAL SCANNING Recent research* suggests that the visual system carries out two distinct scanning processes in tandem. Firstly there is a snap decision as to the GIST of the scene at hand. Secondly a slower item-by-item process scans each figure for its DETAIL. It's three actors on a film set *Locher, Krupinski, Mello-Thoms, and Nodine (2008) This guy in the foreground seems to be talking to me rather than the others in the picture. The guy in black's holding something up but I can't make it out. The girl's etc., etc. THE VISUAL SYSTEM AND ART VISUAL SCANNING Here are some more examples of rapid gist processing. Note how quickly you recognise the basic scenario ..... Note howanyone quicklynotice you But did can the identify scene blue aCar No?as a this or a that. SO MUCH FOR THE VISUAL SYSTEM STUDIED IN ISOLATION. BUT IT'S WHEN VISION AND EMOTION GET TOGETHER THAT THINGS START TO GET REALLY COMPLICATED. SECTION 2 ART AS FELT EMOTION A BRAIN OF MANY SELVES ART AS FELT EMOTION BEAUTY AS "AGREEABILITY" This picture is an "abstract" - just swirls of paint, which you may or may not find "agreeable". THIS IS PUZZLING, BECAUSE THE VISUAL SYSTEM ITSELF CONTAINS NO OBVIOUS PLEASURE CENTRES!! Jackson Pollock "Number 8" [DETAIL] (1949) ART AS FELT EMOTION BEAUTY AS "AGREEABILITY" IN FACT A PICTURE IS NOT ART UNTIL THE VISUAL SYSTEM HAS DELIVERED ITS FINDINGS TO A DIFFERENT BRAIN SYSTEM, NAMELY THE ONE RESPONSIBLE FOR OUR EMOTIONS, FOR ONLY THEN CAN WE LIKE OR DISLIKE IT!! ART AS FELT EMOTION MacLEAN'S (1952) "TRIUNE" BRAIN Understanding forms and scenes, felt emotion, thinking, and the will Emotional activation and instinctive behaviour; visual scanning and organisation Sensations and actions linked as Reflexes ART AS FELT EMOTION MacLEAN'S (1952) "TRIUNE" BRAIN FELT EMOTION EMOTION EYES SENSATION One of the reasons science does not yet understand art is that these three brain systems are responding to different things, and in different ways. ART AS FELT EMOTION MacLEAN'S (1952) "TRIUNE" BRAIN FELT EMOTION EMOTION EYES SENSATION This is easy to demonstrate whenever our higher self feels something which our lower selves have already started to respond to. ART AS FELT EMOTION MacLEAN'S (1952) "TRIUNE" BRAIN FELT EMOTION EMOTION EYES SENSATION This is why animals can get by without higher selves at all! ART AS FELT EMOTION THE DIFFERENT EMOTIONS There is a wide range of emotions hard-wired into us, including those shown here. :-) ;-) :-( :-{ {:-() SO LET'S HAVE A QUICK LOOK AT HOW ARTISTS DOWN THE AGES HAVE PLAYED ON OUR EMOTIONS ..... ART AS FELT EMOTION RITUAL AND MYSTERY IN ART Artist(s) unknown Venus of Willendorf (25,000 BC) Artist(s) unknown Lascaux II Cave Painting (15,000 BC) ART AS FELT EMOTION HAPPY ART Leonardo da Vinci "Mona Lisa" (ca 1503) Lucien Freud "Woman Smiling" (1959) ART AS FELT EMOTION ANGRY ART 3D-Edddy "Anger" (2004) EYEBROWS DOWN TEETH BARED EYES WIDE ART AS FELT EMOTION SCARY ART (ROMANTIC GOTHIC) Caspar David Friedrich "The Abbey in the Oakwood" (1810) SOMBRE COLOURS MOODY ATMOSPHERE MONKS AND COFFINS ART AS FELT EMOTION SCARY ART (MODERN GOTHIC) DISTURBING ART IS OFTEN BASED ON CONFLICTING EMOTIONS, AS SHOWN HERE. POSITIVE EMOTION FOR THE CHILD BUT NEGATIVE EMOTION FOR THE GIST Zhang Peng "Untitled" (ca. 2010) ART AS FELT EMOTION EXPERIMENTAL ART Yang Zhichao "Planting Grass" (2002) EVOKES NEGATIVE EMOTIONS ASSOCIATED WITH BODILY INJURY SECTION 3 SCENES AS STORIES THE EXCEPTIONALLY DIFFICULT SCIENCE OF INTERPRETING VISUAL DRAMA SCENES AS STORIES COMIC STRIP VERSUS CARTOON Stories are easy to tell pictorially if you are allowed to use lots of images, as in a comic strip ..... SCENES AS STORIES COMIC STRIP VERSUS CARTOON ..... but it is harder to make a single picture self-explanatory. Faced with this problem, cartoonists often combine simple images with supporting captions. Only then do we get the joke ..... Tim Harries SCENES AS STORIES "THE PREGNANT MOMENT" When storytelling in art you need to concentrate on what is known as "the pregnant moment", a single image which will convey your story to an audience, given what that audience already knows. This sort of "narrative art" is easy if your audience already knows the story, for then you can have a one-image narrative, as in the following example of religious art ..... SCENES AS STORIES "THE PREGNANT MOMENT" Caravaggio "Supper at Emmaus" (1601) WELL-KNOWN STORY WELL-KNOWN CHARACTERS HEAVY ON SYMBOLISM [CHECK OUT THE BASKET] Read the full story at Wikipedia SCENES AS STORIES SEEING AND UNDERSTANDING But if you don't already know the story you have to work it out for yourself from scratch. What are the important unseens here, for example? THIS, TOO, IS PUZZLING, BECAUSE THE VISUAL SYSTEM ITSELF CONTAINS NO NONVISUAL INTERPRETATION CENTRES!! SCENES AS STORIES SEEING AND UNDERSTANDING INSTEAD THE VISUAL SYSTEM HAS PASS INFORMATION TO THE HIGHER INTELLECT, AND THIS, IN TURN, OFTEN FINDS IT DIFFICULT TO WORK OUT WHAT HAPPENED. Did she kill him, perhaps? Or did he die from an illness? Or commit suicide? And who said he was dead anyway - maybe they've both just woken up with hangovers ..... SCENES AS STORIES SEEING AND UNDERSTANDING Typically the intellect needs to establish WHO did WHAT, to WHOM, with WHAT, WHERE, WHEN, and WHY. Just like in the game CLUEDO or our earlier WHODUNNIT. SCENES AS STORIES SEEING, UNDERSTANDING, AND FEELING The intellect also wants to know if we are reacting emotionally, i.e., HOW WE FEEL, because it will often have been the artist's conscious intent to disturb us. Here's another of Zhang Peng's deliberately disturbing images ..... Zhang Peng "Untitled" [CROPPED] (ca. 2010) SECTION 4 GOOD ART / BAD ART WHAT, THEN, IS TRUE ART? GOOD ART / BAD ART DOES ART HAVE TO BE LIFELIKE, OR JUST PLEASING ON THE EYE? As we have already seen, some sensations are soothing and pleasant in themselves even though they serve no particular purpose. Some good art, in other words, does not have to be about anything! GOOD ART / BAD ART DOES ART HAVE TO BE LIFELIKE, OR JUST PLEASING ON THE EYE? Nor does artistic skill mean being true to life, even when an artwork is about something. This formative example of "Cubism" is imprecise, primitive, disjointed, and lacks perspective, but many find its colours and shapes pleasing to the eye. Pablo Picasso "Les Demoiselles d'Avignon" (1907) GOOD ART / BAD ART IS IT TRUE TO LIFE? Similarly this 1952 caricature must have been good because a set of three sold for £23 million in February 2011. Francis Bacon "Portrait of Lucien Freud" [ONE OF THREE] (1952) GOOD ART / BAD ART IS IT TRULY BEAUTIFUL? Or perhaps the secret of good art is to pick a beautiful subject to start with. Some people rate this portrait as the most beautiful painting ever. Gottfried Helnwein "The Murmur of the Innocents (11)" (2009) GOOD ART / BAD ART DOES IT INSPIRE ME? Others look for art to inspire them spiritually and reinforce their faith. Check it out online - it's full of symbolism! Leonardo da Vinci "The Last Supper" [RETOUCHED] (1498) GOOD ART / BAD ART IS IT "POLITICALLY CORRECT"? Others use it to control what we think. George Grosz "The Hero" (1936) Nazi Propaganda (same era) ART AS FELT EMOTION EXPERIMENTAL ART Others believe that art's primary task is to make you stop and think - to challenge your assumptions about yourself and the world. GOOD ART / BAD ART IS IT JUST "COLLECTIBLE"? Others like to possess beauty in preference to cash. In 2011 this Cézanne sold as an investment to a private collection for over $250 million. Paul Cézanne "The Card Players" (1895) GOOD ART / BAD ART OR DOES IT JUST HAVE TO BE AWESOME? And many just like to stand in awe, reminded by the artist of the raw beauty of nature. This beauty is often impossible to describe in words. James Ward "Gordale Scar" (1814) GOOD ART / BAD ART CAN I EVEN DESCRIBE WHAT I FEEL? INDESCRIBABLY BEAUTIFUL ART IS KNOWN TECHNICALLY AS "SUBLIME". MANY ARTISTS STRIVE TO PRODUCE SUBLIMITY RATHER THAN JUST BEAUTY. Here's another example. Caspar David Friedrich "Wanderer above a Sea of Fog" (1818) A PAINTING CAN BE PRETTY, OR LIFELIKE, OR POLITICALLY ACCEPTABLE, OR PLEASANTLY SCARY, OR INDESCRIBABLY AWESOME, OR WHATEVER, AND EACH OF THESE RESPONSES ACTIVATES OUR BRAINS IN DIFFERENT WAYS! NO WONDER, THEN, THAT ART THEORY FINDS IT SO DIFFICULT TO EXPLAIN WHAT MAKES ART GOOD! GOOD ART / BAD ART WHY SUCH A PROBLEM? BECAUSE ART EXPERTS ARE NOT BRAIN EXPERTS ..... GOOD ART / BAD ART WHY SUCH A PROBLEM? ..... AND EVEN IF THEY WERE THEY WOULDN'T UNDERSTAND THE UNCHARTED AREAS OF THE BRAIN ..... ? ? ? ? ? ?? GOOD ART / BAD ART WHY SUCH A PROBLEM? ..... ESPECIALLY WHEN OUR FEELINGS NEED TO BE PUT INTO WORDS. EYES SUBLIME EMOTION FELT EMOTION EMOTION SENSATION OUR OWN RESEARCH IS TRYING TO SOLVE SOME OF THESE PROBLEMS USING COMPUTER SIMULATION SOFTWARE. WE BELIEVE THAT IF WE COULD GET A MACHINE TO GO THROUGH THE MOTIONS OF APPRECIATING ART WE SHOULD HAVE A BETTER IDEA WHAT GOES ON IN HUMANS. GOOD ART / BAD ART WHAT MIGHT ROBOTS MAKE OF ART? Here's Robby the Robot (from the 1956 movie "Forbidden Planet") looking at our masterpiece of sublime art. WHAT WILL IT TAKE FOR "HIM" TO SEE MORE THAN JUST PIGMENTS ON A CANVAS, AND TO BE STRUCK INSTEAD BY THEIR DEEPER MEANING? There's a lot of light blues and greys, with some dark green in the middle. BUT BEFORE WE CAN ANSWER THIS QUESTION WE NEED SOME IDEA WHAT ROBOTS ARE AND WHAT THEIR BRAINS ARE CAPABLE OF. YOU MAY JUMP STRAIGHT TO SECTION 6 IF YOU ALREADY KNOW THE HISTORY HERE. SECTION 5 A SHORT HISTORY OF ROBOTS FROM "AUTOMATICITY" TOWARDS "AUTONOMY" A SHORT HISTORY OF ROBOTS EARLY AUTOMATA Here's an automaton from around 2000 years ago. The word comes from the Greek for "self activating", and gives us the modern words "automatic" and "automation". These toys for the rich and powerful were worked by hidden pipes and linkages underneath! THEY MOVED FOR THEMSELVES, BUT HAD NO MIND. A SHORT HISTORY OF ROBOTS EARLY MECHANICAL PROGRAMMING Here's a typical pin-cylinder music box. This one plays a tune on bells. See how the pins "know" the tune in advance and play it by each activating a corresponding hammer. This is an early instance of "preprogramming". BUT, BEING ENTIRELY AUTOMATIC, MUSIC BOXES HAVE NO MIND. A SHORT HISTORY OF ROBOTS EARLY CALCULATING MACHINERY In 1495 the Italian painter-inventor Leonardo da Vinci demonstrated an automaton in the shape of a knight in arms. Worked by pulleys and wires, this automaton could perform a range of simple human-like movements. This 2005 replica is in a gallery in Germany, IT MOVES A BIT, BUT IT HAS NO MIND. A SHORT HISTORY OF ROBOTS EARLY CALCULATING MACHINERY Similar technology was used in early calculating machines. This one's from the German philosopherengineer Gottfried Leibniz in 1673. It's driven by the handle front left, IT DOES ARITHMETIC FASTER THAN WE CAN, BUT IT HAS NO MIND. A SHORT HISTORY OF ROBOTS MORE MECHANICAL PROGRAMMING The French inventor Jacques de Vaucanson built this automaton duck in 1739 to amuse a rich patron. Its inner clockwork permitted it to make lifelike movements, including feeding; also, as the appropriately placed arrow shows, going to the toilet. BUT IT HAS NO MIND. A SHORT HISTORY OF ROBOTS MORE MECHANICAL PROGRAMMING In the late 18th century, the weaving industry found a way to automate their looms by scaling up earlier "music-box" technology. They brought preprogramming into industry! This is the Joseph Jacquard punched card control system, on a 5 metre tall continuous loop, one per loom. BUT THE LOOMS HAVE NO MIND. A SHORT HISTORY OF ROBOTS MORE CALCULATING MACHINERY In 1822 the British inventor Charles Babbage designed a giant calculating machine called a "difference engine". Unfortunately, the machine took a long time to build, and remained incomplete until the Science Museum finished the job for him in 2000. BUT THE MACHINE HAS NO MIND. A SHORT HISTORY OF ROBOTS FUTURIST THEATRE AND ART In 1914 the American author L. Frank Baum published a story entitled "Tik-Tok of Oz", in which a mechanical man shares adventures with Dorothy, one of the "meat people" [i.e., humans]. "The Wizard of Oz" (1939) L. Frank Baum "Tik-Tok of Oz" (1914) A SHORT HISTORY OF ROBOTS FUTURIST THEATRE AND ART In 1921 the Czech writer Karel Capek staged a play entitled "Rossum's Universal Robots", set in a world increasingly taken over by artificial workers he called "robots". The new name was an instant success. BAUM'S TIN MAN AND CAPEK'S ROBOTS HAD MINDS OF THEIR OWN, BUT ARE WORKS OF FICTION. NOBODY HAS YET BEEN ABLE TO BUILD ONE FOR REAL. A SHORT HISTORY OF ROBOTS ELECTRONIC CALCULATION In 1931, the Welsh physicist Charles Wynn-Williams devised electronic circuitry for counting. This enabled engineers to do away with cogwheels once and for all, even though the early circuitry was bulky. BUT SUCH CIRCUITS HAVE NO MIND. A SHORT HISTORY OF ROBOTS ELECTRONIC CALCULATION Between 1934 and 1938 the German Engineer Konrad Zuse built the Z1, a programmable electromechanical binary computer. This is Zuse, later in life, standing by a museum replica of his invention. THE MACHINE COMPUTES WELL ENOUGH, BUT HAS NO MIND. FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE In 1939, the Westinghouse Corporation exhibited Elektro the Motoman and Sparky the Dog at the New York World's Fair. Powered by a primitive electromechanical brain, Elektro could produce a number of simple output behaviours - movement and speech - to simple spoken commands. BUT LIKE A RADIOCONTROLLED TOY IT HAD NO MIND. FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE By 1943 electronic computers were processing binary signals according to preprogrammed logic. This is a replica of Colossus, the top-secret code-breaking machine built at Bletchley Park during WW2. IT'S AN ELECTRONIC BRAIN BUT IT HAS NO MIND. FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE Robots - some good, some bad have since starred in many movies. This is Pris, from the 1982 movie "Blade Runner". She's less obviously artificial than others, thanks to a synthetic skin. She also feels emotion and ponders her own existence. BUT AGAIN IT'S ONLY A WORK OF FICTION. "I think, therefore I am" FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE Similarly, this is Johnny Five, from the 1986 movie "Short Circuit". He looks bright enough, but he was worked almost entirely by remote control. JUST CLEVER PUPPETRY AGAIN! NOTE THE PATTERN WHICH IS EMERGING - MACHINES WITH MINDS ARE (TO DATE) ONLY WORKS OF FICTION! FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE This, on the other hand, is Kismet, a state-of-the-art university research project. Kismet can mimic a number of human emotional expressions, and is especially good at those involving eyebrows! Kismet may display facial emotions but has no higher functions system to feel those emotions. IT'S MIND IS THEREFORE ONLY ABOUT AS COMPLEX AS THAT OF A MOUSE. FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE Meanwhile computers have been getting smaller and smaller and faster and faster since Colossus' time ..... FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE This is the Honda-Kawasaki ASIMO. It can walk, has colour vision, face recognition, gesture recognition, and speech, and can even obey simple commands. But it has little intelligence, and no free will at all! IT LACKS THE AUTONOMY NEEDED TO BE ANYTHING MORE THAN AN AUTOMATON. Autonomy = flexibility and self-determination Automaton = automaticity and inflexibility FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE And this is Valerie, a robot receptionist under trial at Carnegie Mellon University in the US. She's going to be as near lifelike as they can make her! She'll even remember what you talked about last time you visited! SOCIAL LANGUAGE IS CERTAINLY A MAJOR MIND SKILL. SADLY PROGRAMMERS HAVE BEEN WORKING ON IT FOR 60 YEARS WITH ONLY VERY LIMITED SUCCESS. FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE So when all is said and done, the problem with robot mind lies in writing the necessary software not in giving it a brain or a body. PROCESSING HARDWARE PROCESSING SOFTWARE SENSOR HARDWARE [STILL IN ITS INFANCY!!!] EFFECTOR HARDWARE THE NEXT DIAGRAM SHOWS HOW FAR SCIENCE HAS GOT / NOT GOT WITH PROGRAMMING THE MIND AFTER 2000 YEARS OF TRYING. NOTE CAREFULLY WHERE THE PROBLEMS REMAIN. CHESS AND SIMILAR PROBLEM SOLVING UNDERSTANDING THINKING FELT EMOTION AUTONOMY AND SELF-REFERENCED WILL AND CONSCIOUS EXPERIENCE ? ? ? CORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL DIRECTED ATTENTION SIMPLE MACHINE EMOTIONAL STATES DRIVES AND INSTINCTS TRANSLATION CONCEPTS SENTENCE ANALYSIS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PATTERN RECOGNITION VOICE RECOGNITION WILLED BEHAVIOURS PRIMARY SENSORY ANALYSIS SENSORY INPUT EYES ARTIFICIAL RETINA REFLEX LEVEL SIMPLE BIONICS VOICE MOTOR SCHEMA MANAGEMENT SENTENCE SYNTHESIS MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT HAND MECHATRONICS VOICE SYNTHESIS SECTION 6 INTRODUCING KONRAD, THE SOFTWARE MIND PUTTING OUR SEVERAL SELVES TOGETHER INTRODUCING KONRAD THE ROBOT AWAITING A BODY • Project Konrad is a collaboration between the author and International Software Products, Toronto, and is so named as a tribute to the aforementioned Konrad Zuse. INTRODUCING KONRAD THE ROBOT AWAITING A BODY CORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS SENSORY INPUT EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES At the moment Konrad exists only in software. IT'S A ROBOTIC MIND WITHOUT A BODY - JUST A VOICE (WHICH YOU'LL BE HEARING SHORTLY) AND VAST AMOUNTS OF PRINT-OUT. MUSCLE OUTPUT We use the Kirshenbaum Phonetic Alphabet to encode output speech - click for details its v'eri 'pliziN on Di: Aii. t'u: 'pliziN in 'f'akt. its biyond d@skripSun O@lmO'st. 'Oka2i 'wot 'naU. INTRODUCING KONRAD THE ROBOT AWAITING A BODY The software presently runs on an IBM z196 mainframe, a very fast computer with a memory so large we are unlikely ever to fill it. INTRODUCING KONRAD THE ROBOT AWAITING A BODY CORTICAL LEVEL HIGHER COGNITION PERCEPTIONS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT SIGHTS AND SOUNDS MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT WE WERE PARTICULARLY INTERESTED IN HOW VISUAL INPUTS BECOME PERCEPTIONS AND FELT EMOTIONS IN MACHINES. INTRODUCING KONRAD THE ROBOT AWAITING A BODY ? ? ? CORTICAL LEVEL HIGHER COGNITION PERCEPTIONS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FEELINGS EMOTIONAL STATES MOTOR SCHEMA MANAGEMENT DRIVES AND INSTINCTS FIGURE-GROUND / DRAMATIC GIST REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT SIGHTS AND SOUNDS MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT WE THEREFORE SIMULATED WHAT WAS KNOWN ABOUT THE LAYOUT OF THE MIND'S HIGHER FUNCTIONS, AND TOOK EDUCATED GUESSES AT THE MANY UNKNOWNS. INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT SIGHTS AND SOUNDS MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SPEECH SOUNDS WE ALSO GAVE KONRAD A SIMULATED "WILL", THAT IS TO SAY A POINT AT WHICH ITS THOUGHTS COULD BECOME VOLUNTARY ACTIONS. INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT SIGHTS AND SOUNDS MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SPEECH SOUNDS WE ALSO ALLOWED KONRAD'S ACTIONS TO GENERATE NEW INPUTS, THUS CREATING CYCLES OF COGNITION. [ ] IT HAS BEEN KNOWN SINCE THE 1950s THAT THE MIND CYCLES ABOUT 7 TIMES A SECOND. INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT SIGHTS AND SOUNDS MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MORE IMPORTANTLY WE ALSO ALLOWED KONRAD TO TAKE ITS OWN VERBAL OUTPUTS AS SILENT INPUTS. INNER SPEECH MUSCLE OUTPUT SPEECH SOUNDS NB: YOU CAN'T PROPERLY STUDY INNER SPEECH IN HUMANS BECAUSE IT ALL TAKES PLACE SO QUICKLY ..... INTRODUCING KONRAD THE ROBOT AWAITING A BODY SEE ACTIVITY #5 IN YOUR ACTIVITIES PACK ..... THIS IS A SHAME BECAUSE BEING ABLE TO LISTEN TO ONESELF THINK IS A MAJOR MIND SKILL, AND ONE WE SUSPECT IS HEAVILY INVOLVED IN RESPONDING TO ARTISTIC BEAUTY. TRY TO LISTEN TO YOURSELF THINKING AS YOU WORK OUT WHAT THIS ARTIST HAD IN MIND, BECAUSE IT WILL HELP YOU WITH THE NEXT FEW SLIDES. René Magritte "Not to be Reproduced" (1937) INTRODUCING KONRAD THE ROBOT AWAITING A BODY So can the Konrad software do better than Robby the Robot if presented with our example of sublime art? CAN IT SEE MORE THAN JUST COLOURS ON A CANVAS? "This is a classic of Romantic Sublime, and it sends shivers up my spine." CAN A MACHINE FEEL, AND - MORE IMPORTANTLY - CAN IT FEEL ITSELF FEELING AND THEN TALK ABOUT THAT EXPERIENCE? INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION THE TEST REPORTED HERE RAN ON 16TH MAY 2012. WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS THE SOFTWARE TOOK 24.1 SECONDS OF CYCLICAL COGNITION TO PRODUCE FOUR SHORT SENTENCES OF INNER SPEECH AND 726 PAGES OF EXPLANATORY PRINT-OUT. http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS TO START WITH WE SIMULATED THE VISUAL SCANNING OF THE TARGET ARTWORK, FEEDING IN CODED DETAILS OF FIGURE AND GROUND AS DESCRIBED EARLIER. SPEECH SOUNDS SCIENCE DOES NOT KNOW THE PRECISE <CODE> USED HERE, SO WE USE <ENGLISH>. <GROUND[PICTURE FRAME] GROUND[SKY/MOUNTAIN] FIGURE[MAN]> http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY <PAINTING IN GALLERY [MAN UP MOUNTAIN]> PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS THIS ENABLED A RAPID JUDGEMENT ON GIST TO BE MADE AND THEN RETAINED IN SHORTTERM MEMORY [SHOWN HERE IN BLUE], PART OF A UNIQUE INDUSTRIAL STRENGTH CA-IDMS "SEMANTIC NETWORK" MEMORY DESIGNED BY THE AUTHOR. http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY <PAINTING IN GALLERY [MAN UP MOUNTAIN]>. PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS WE THEN SIMULATED THE ADDITIONAL JUDGEMENT THAT THE TECHNICALITIES OF THE PAINTING - ITS COLOURS AND FORMS - WERE INTRINSICALLY AGREEABLE. NICE <COLOUR-FORM BALANCE> http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY <PAINTING IN GALLERY [MAN UP MOUNTAIN]>. <VERY PLEASANT> PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES WITH THE GIST AND THE RAW EMOTION NOW SIDE BY SIDE, WE SIMULATED THE COMPOSITE JUDGEMENT THAT WE "LIKED" THE OBJECT AT HAND. MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS NICE WE ARE NOW FEELING OUR FEELINGS!! <COLOUR-FORM BALANCE> http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY <PAINTING IN GALLERY [MAN UP MOUNTAIN]> <VERY PLEASANT> PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL NICE DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS WE HAVE TO SHOW THE EMOTION TWICE, BECAUSE THIS IS HOW THE BRAIN IS ORGANISED LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]> <IT'S VERY PLEASING ON THE EYE> <VERY PLEASANT> PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES SIMULATED THOUGHT PROCESSES THEN GENERATED THE DEEP SENTENCE "IT'S VERY PLEASING ON THE EYE", AND APPROVED SAME FOR SILENT OUTPUT. MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SCIENCE DOES NOT KNOW THE SPEECH SOUNDS PRECISE <CODE> USED HERE EITHER, SO AGAIN WE USE <ENGLISH>. http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]>. <IT'S VERY PLEASING ON THE EYE> <VERY PLEASANT>. PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS THE DEEP SENTENCE WAS THEN RE-CODED INTO SPEAKABLE SOUNDS, SHOWN HERE USING THE KIRSHENBAUM PHONETIC ALPHABET. SPEECH SOUNDS < its v'eri 'pliziN on Di: Aii > http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]>. <IT'S VERY PLEASING ON THE EYE> <VERY PLEASANT>. PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS YOU CAN EITHER LEARN TO READ THIS PHONETIC CODE ITSELF, OR ELSE LISTEN TO IT USING COMMERCIAL SPEECH SYNTHESIS PACKAGES SUCH AS ESPEAK [MORE ON THIS]. SPEECH SOUNDS < its v'eri 'pliziN on Di: Aii > http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY <PAINTING IN GALLERY [MAN UP MOUNTAIN]>. <IT'S VERY PLEASING ON THE EYE> PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL NICE DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS MANY DIFFERENT CODING SYSTEMS ARE INVOLVED IN THE COGNITIVE CYCLE. COGNITIVE SCIENTISTS HAVE BARELY STARTED TO DECODE THESE AND NOBODY WILL UNDERSTAND THE MIND UNTIL THEY DO! < its v'eri 'pliziN on Di: Aii > http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]>. <IT'S VERY PLEASING ON THE EYE> <VERY PLEASANT>. PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS THE SURFACE SENTENCE WAS THEN RE-CYCLED AS INNER SPEECH INPUT, AND PASSED UPWARDS THROUGH THE AUDITORY SYSTEM. SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT <IT'S VERY PLEASING ON THE EYE> SIGHTS AND SOUNDS SPEECH SOUNDS . < its v'eri 'pliziN on Di: Aii > http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]>. <IT'S VERY PLEASING ON THE EYE> <VERY PLEASANT>. PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS THE SYSTEM HAS NOW HEARD WHAT IT THINKS, LEAVING IT FREE TO THINK SOME MORE ABOUT IT. WE BELIEVE THAT ALL COMPLEX REASONING DEVELOPS CYCLICALLY IN THIS WAY. SPEECH SOUNDS http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL WHO? WHAT? WHY? ETC. HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]>. <IT'S VERY PLEASING ON THE EYE> <VERY PLEASANT>. PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS THE SYSTEM THEN LOOKED FOR WORDS TO EXPLAIN WHY THE PAINTING WAS AS PLEASING AS IT CLEARLY WAS [LIKE YOU WERE DOING DURING ACTIVITY #5]. HOWEVER COGNITIVE SCIENTISTS HAVE LITTLE IDEA HOW THIS PROCESS WORKS. http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY ..... BUT BECAUSE THE PAINTING HAD BEEN DELIBERATELY SELECTED TO BE SUBLIME, NO SUCH EXPLANATION COULD BE FOUND. "It's just a man up a mountain so why should I like it so much?" INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]> <TOO PLEASING IN FACT> <VERY PLEASANT> PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES A SECOND DEEP SENTENCE WAS THEN GENERATED, AND DULY RECODED INTO THE KIRSHENBAUM PHONETIC ALPHABET, AS BEFORE. MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS < t'u: 'pliziN in 'f'akt > http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]> <TOO PLEASING IN FACT> <TOO PLEASING IN FACT> <VERY PLEASANT> PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES AGAIN THIS OUTPUT WAS RECYCLED AS INNER SPEECH ..... MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS < t'u: 'pliziN in 'f'akt > http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL WHO? WHAT? WHY? ETC. HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]> <TOO PLEASING IN FACT> <VERY PLEASANT> PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES ..... AGAIN THE SYSTEM LOOKED UNSUCCESSFULLY FOR WORDS TO EXPLAIN WHY THE PAINTING PLEASED IT ..... MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY ..... AND AGAIN IT FAILED. "I, too, want to stand at the top of the world and wonder what it's all about." "Is this what it means to be human, perhaps?" INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL NICE DIRECTED ATTENTION <PAINTING IN GALLERY [MAN UP MOUNTAIN]> <IT'S BEYOND DESCRIPTION ALMOST> <VERY PLEASANT> PSYCHOLOGICAL ACTS WILLED BEHAVIOURS CONCEPTS ADMITTING DEFEAT, THE SOFTWARE THEN PRODUCED A THIRD DEEP SENTENCE, AND DULY RECYCLED IT AS INNER SPEECH ..... SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS < its biyond d@skripSun O@lmO'st > http://www.smithsrisca.co.uk/DEMOS/2A158.pdf INTRODUCING KONRAD THE ROBOT AWAITING A BODY PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL NICE DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS FEELINGS NICE EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL INNER SPEECH ..... BEFORE DECIDING THAT IT WAS FINALLY TIME TO MOVE ON. MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SENSORY INPUT SIGHTS AND SOUNDS SPEECH SOUNDS < 'Oka2i 'wot 'naU> http://www.smithsrisca.co.uk/DEMOS/2A158.pdf SECTION 7 IN CONCLUSION ..... ART THEORY AWAITS BETTER COGNITIVE THEORY ART - A SURPRISINGLY DIFFICULT SCIENCE THE BIG PROBLEM - TOO MANY AREAS OF STUDY!! THE ARTWORKS THEMSELVES (Art History) SO THE PEOPLE WHO NEED THE HELP DON'T GET IT. BRAIN FUNCTIONS ( Neurophysiology / Endocrinology ) ART AS EXPERIENCE (Aesthetics) ART AS DRAMA (Dramatic Theory) BRAIN STRUCTURES (Neuroanatomy) THE EXPERIENCE (Phenomenology) THE WORKING PRINCIPLES (Mental Philosophy / Experimental Cognitive Science / Artificial Intelligence and Robotics) THE MEMORY USED (Database Theory / Neurochemistry) THE ANIMAL BEHAVIOURS (Comparative Zoology / Cognitive Palaeontology) THE CODES USED (Psycholinguistics [esp. Pragmatics]) ART - A SURPRISINGLY DIFFICULT SCIENCE BY TREATING THE BIOLOGICAL MIND AS AN ENGINEERING PROBLEM, THIS SOFTWARE EXPERIMENT HAS HELPED US BETTER UNDERSTAND HOW ARTWORKS ARE VISUALLY PROCESSED. ? ? ? CORTICAL LEVEL HIGHER COGNITION AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST PRIMARY SENSORY ANALYSIS SENSORY INPUT EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT ART - A SURPRISINGLY DIFFICULT SCIENCE WE ARE, FOR EXAMPLE, MORE CONFIDENT THAT THE MIND WORKS CYCLICALLY, CONSTANTLY SAMPLING THE PERCEPTUAL AND EMOTIONAL WORLDS ..... PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT SIGHTS AND SOUNDS MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SPEECH SOUNDS ART - A SURPRISINGLY DIFFICULT SCIENCE ..... AND CONSTANTLY UPDATING A NETWORK MEMORY MODEL OF WHAT IS GOING ON IN THOSE WORLDS. PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT SIGHTS AND SOUNDS MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SPEECH SOUNDS ART - A SURPRISINGLY DIFFICULT SCIENCE WE ARE ALSO MORE CONFIDENT THAT A PAINTING'S GIST EMERGES VERY RAPIDLY WHILST THE SUPPORTING DETAILED SCANNING TAKES LONGER; FURTHERMORE THAT MUCH CAN BE ACCOMPLISHED USING THE GIST ON ITS OWN. PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT <PAINTING IN GALLERY [MAN UP MOUNTAIN]> SIGHTS AND SOUNDS SPEECH SOUNDS ART - A SURPRISINGLY DIFFICULT SCIENCE WE ARE ALSO MORE CONFIDENT THAT THE LOWER REACHES OF OUR MINDS RESPOND EMOTIONALLY WITHOUT CONSULTING OUR HIGHER MINDS, THIS BEING HOW THINGS HAPPEN IN ANIMALS. PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT NICE SIGHTS AND SOUNDS MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SPEECH SOUNDS ART - A SURPRISINGLY DIFFICULT SCIENCE THIS LOWER PROCESSING OFTEN INFLUENCES HOW WE INTERPRET THE NARRATIVE DRAMA IN A PICTURE ..... PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELINGS EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL PRIMARY SENSORY ANALYSIS SENSORY INPUT MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT ALERT!! SCARY SIGHTS AND SOUNDS SPEECH SOUNDS ART - A SURPRISINGLY DIFFICULT SCIENCE ..... THE TRULY MAGICAL MOMENT BEING WHEN OUR HIGHER MIND COMES TO FEEL ITS OWN EMOTIONAL RESPONSE. PERCEPTIONSCORTICAL LEVEL PSYCHOLOGICAL HIGHER COGNITION ACTS AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST FEELS NICE FEELINGS IS NICE MOTOR SCHEMA MANAGEMENT REFLEX LEVEL MOVEMENT UNITS EMOTIONAL STATES DRIVES AND INSTINCTS PRIMARY SENSORY ANALYSIS SENSORY INPUT <PAINTING IN GALLERY [MAN UP MOUNTAIN]> <IT'S VERY PLEASING ON THE EYE> SIGHTS AND SOUNDS <VERY PLEASANT> LOWER MOTOR SUBROUTINES MUSCLE OUTPUT SPEECH SOUNDS SO CAN WE SAY MORE PRECISELY WHAT MAKES AN ARTWORK BEAUTIFUL AT ONE EXTREME, OR DISTURBING AT THE OTHER? ART - A SURPRISINGLY DIFFICULT SCIENCE WE THINK SO. THE KEY MOMENT SEEMS TO BE WHEN WE SENSE THAT AN ARTWORK IS BEAUTIFUL, OR SCARY, OR WHATEVER, BUT CANNOT IMMEDIATELY EXPLAIN WHY. FROM THAT MOMENT ONWARDS THE OBJECT OF STUDY BECOMES NOT JUST THE ARTWORK BUT ALSO ITS ASSOCIATED EMOTIONS. ART - A SURPRISINGLY DIFFICULT SCIENCE THE FACT THAT THE HIGHER BRAIN IS NOW STUDYING THE LOWER BRAIN AS WELL AS THE ARTWORK PUSHES IT TO ITS LIMITS! AND AT THE SAME TIME, EMOTIONALLY COMPLEX ENCOUNTERS ARE NATURALLY MEMORABLE ..... ART - A SURPRISINGLY DIFFICULT SCIENCE WHICH IS WHY WE LIKE A GOOD LAUGH ..... ..... OR A GOOD GHOST STORY ..... ..... OR A DANGEROUS TRICK AND HAVE WE FOUND THE MIND'S "INNER EYE", THAT IS TO SAY, THE BIT WHICH DOES THE EXPERIENCING? ART - A SURPRISINGLY DIFFICULT SCIENCE NO. THE SYSTEM HAS MERELY BEEN MOVING SHORT FRAGMENTS OF <CODED> INFORMATION FROM ONE MEMORY LOCATION TO ANOTHER, BECAUSE THAT IS WHAT COMPUTERS ARE GOOD AT. IT MAY EVEN BE THAT THE INNER EYE PROBLEM MAY NEVER BE SOLVED! ART - A SURPRISINGLY DIFFICULT SCIENCE NEVERTHELESS WE DO HAVE A BETTER IDEA ABOUT WHAT MIGHT BE GOING ON DURING COGNITION ..... WHAT NEEDS TO BE MOVED FROM WHERE TO WHERE IN WHAT ORDER WHAT <CODES> ARE USED ALONG THE WAY ART - A SURPRISINGLY DIFFICULT SCIENCE IN OTHER WORDS WITH SOMETHING AS COMPLEX AS THE MIND, WHERE SCIENTIFIC DATA ARE OTHERWISE SO SCARCE, IT HELPS TO HAVE A WORKING MODEL TO STUDY. ? ? ? CORTICAL LEVEL SCIENTIFIC GUESSWORK HIGHER COGNITION AUTONOMOUS WILL DIRECTED ATTENTION WILLED BEHAVIOURS CONCEPTS SUBCORTICAL LEVEL SECONDARY SENSORY ANALYSIS FIGURE-GROUND / DRAMATIC GIST SCIENTIFIC DATA PRIMARY SENSORY ANALYSIS SENSORY INPUT EMOTIONAL STATES DRIVES AND INSTINCTS REFLEX LEVEL MOTOR SCHEMA MANAGEMENT MOVEMENT UNITS LOWER MOTOR SUBROUTINES MUSCLE OUTPUT ART - A SURPRISINGLY DIFFICULT SCIENCE AND ABOVE ALL IT HELPS TO HAVE NEW MINDS AND YOUNGER SCIENTISTS INVOLVED IN THE DEBATE, WHICH IS WHY WE ALL CAME HERE TODAY. THE END ANY QUESTIONS? ? ? ? Copyright Notice: This material was written and published in Wales by Derek J. Smith, CEO of High Tower Consultants Limited, Chief Designer on the Konrad artificial consciousness project, and Editor of the Smithsrisca series of cognitive science study resources. It forms part of a multifile e-learning resource, and subject only to acknowledging Derek J. Smith's rights under international copyright law to be identified as author may be freely downloaded and printed off in single complete copies solely for the purposes of private study and/or review. Commercial exploitation rights are reserved. The remote hyperlinks have been selected for the academic appropriacy of their contents; they were free of offensive and litigious content when selected, and will be periodically checked to have remained so. Prior scholarship is acknowledged throughout, and precise references may be obtained from the author if needed. Internet images have been used in good faith on the understanding that they were in the public domain when imported. Copyright © 2012, High Tower Consultants Limited. Publication was by PowerPoint presentation on 26th July 2012. This online version comes to you for follow-up private study. FROM AUTOMATA TO ARTIFICIAL INTELLIGENCE In 1942 the American science fiction writer Isaac Asimov published a short story entitled "Runaround", in which he coined the word "robotics" for the science of designing, building, and operating robots. The hero of the tale is a robot named Speedy, who initially has trouble when the three "Fundamental Rules of Robotics" produce conflicting behaviours, but eventually - and of its own volition - saves the lives of its human masters. BY MAKING ITS OWN CONSIDERED CHOICES SPEEDY SEEMS TO HAVE A MIND, BUT ONLY IN FICTION.