Why Art is Needed for Software Development Le développement de logiciel, est-ce un art ? Gregor v. Bochmann School of Information Technology and Engineering (SITE) University of Ottawa September 26, 2002 My personal background • MSc in physics and mathematics; PhD in theoretical physics; experience in computer programming; changed to computer science in 1971 (after PhD), professor at University of Montreal for 25 years • People important for me – My mother was high-school teacher for music and visual arts (my father, biologist, died in 1942 during the war) – The grandfathers of my father were both well-known artists – My high-school physics and mathematics teachers – My piano teacher • Hobbies – Travelling, hiking, observing nature – Playing music: chamber music, orchestra – Setting up an inventory of the paintings of my great-grandfather (see http://beethoven.site.uottawa.ca/MalerBochmann/ ) About this talk • Theme: Art artisan, artist Software Development software engineer Historical development – artisanal economy – first industrial revolution: energy and automation – second industrial revolution: information processing Artisans, artists, software engineers – similarities and differences: the importance of inventions The activities of a software engineer Automation of software engineering – research objectives – intrinsic limitations (the importance of inventions) Some comments on my research contributions Conclusions Art Software Development artisan, artist software engineer What do these people do ? – build something useful – Artisan (shoe maker makes shoes) – Engineer (mechanical engineer designs a bridge) – learn the rules of the profession – Shoe maker: how to make shoes – Engineer: physical laws concerning statics and dynamic mechanics – Artist (composer): rules of harmony – produce something beautiful – elegant shoes – nice looking bridge – beautiful music – Artists: convey a sentimental or social message – Mozart’s piano concert or Zola’s book Economy based on artisans Example of a shoe maker – builds something useful – notes the client’s requirements: size, style of shoes, desired color, etc. – follows the rules of the profession – selects the material to be used for different parts of the shoe – knows how to fabricate these parts – knows the best order in which to assemble them (this may be adapted to the particular circumstances – inventions) – produces something beautiful – adds on some ornamentation – inventions (as requested by the client and/or according to his own invention and temperament) Typically, each object is individually designed and fabricated First industrial revolution Characteristics • • • Energy replaces man or horse power Chemical engineering: better materials Automated manufacturing (e.g. assembly lines for cars, automatic weaving, etc.) People’s activities: 1. Design product to be manufactured (inventions) 2. Design and build machines for manufacturing (inventions) 3. Assist the machines on the assembly line Note: This is boring ! -- Why ? Typically, many identical objects are manufactured for each design Second industrial revolution (IT) Relative production cost decreases first unit – – – – – – additional unit House Plane Vinyl music album MP3 music album Income tax program New e-commerce appl. Software and digital media: no copying costs Consequences and open questions • Only activity (1) remains (to design the product -- need for inventions) • Free copying (e.g. Napster) vs. Copyright – or ”Open Source Software” ?? – Providing useful software components without remuneration • Globalization and uniformization … – or sub-cultures ?? • Do we need more software engineers ? – No: software for all useful applications will soon be available – Yes: the power of computer and communication hardware increases exponentially over time (Morse’s law); therefore many new applications will be possible in the future and they must be developed Back to: art – science – engineering useful – rules – beautiful – message • Simplicity – Beauty (or Triviality ?) – – – – Nature Mathematics Paintings: too simple becomes trivial Science: to find simple explanations for nature’s behavior; this is never trivial – Engineering: simple, elegant designs are easier to verify, build • Rules vs. Freedom – The free space allows the expression of the artist’s message – This is the space for inventions » Bach’s “Inventions“ » Communication through human or computer language » Engineering: design choices – Filling the free space with random choices (e.g. automatic art) Shoe Maker vs. Softw. Engineer – builds something useful • notes the client’s requirements: size, style of shoes, desired color, etc. • notes the requirements of the system to be built – follows the rules of the profession • • • • • • selects the material to be used for different parts of the shoe selects software components to be re-used knows how to fabricate these parts knows techniques for software design, implemention and testing knows the best order in which to assemble them follows a suitable SE process – produces something beautiful • adds on some ornamentation (as requested by the client and/or according to his own invention and temperament) • tries to built the system in a simple manner (as far as possible) • a program can be elegant, beautiful ! ? • where requirements leave freedom, designs a nice user interface and user options SE : an artisanal profession ? . . . as indicated by above comparison • Most software has too many bugs !! • Much software is created – without a well-defined development process – without the use of automated tools • Sometimes, older disciplines look down to SE and computer science, which is a relatively young discipline – Each car manufacturer has a well-defined and optimized manufacturing process – There is automation not only in the manufacturing process, but also during the design (e.g. simulation tools, static analysis packages, etc.) • Much of my research: towards automation of SE • Can the SE process be completely automated ? Activities An activity (in general) processing input output IT activities Algorithm: “process or rules for calculation” (Concise Oxford Dict.) Program: algorithm coded in a language executable by a computer, or automatically translatable by a compiler into executable code Example: The activity of sorting Fred Tom Alice Suzanne 12 000 7 000 10 000 5 000 input sorting algorithm output Alice Fred Suzanne Tom 10 000 12 000 5 000 7 000 Requirements (properties of output, depending on input): (1) output list is sorted by names (increasing order) (2) the set of output lines should be equal to the set of input lines Example of a sorting algorithm start create empty output list is input list empty ? yes no extract an element from the input list; call it E insert E into the output list at the appropriate place end Activities of a programmer-analyst Requirements Requirements input input Find (invent) appropriate algorithm and describe it Find appropriate decomposition, reusing if possible existing modules output output Algorithm A B C Requirements input Algorithm or composition Algorithm (in executable specification language) input Verification (does the algor. or composition output satisfy requirements ?) Translation from high-level language to executable code output “correct” or “in error” executable code Some comments Requirements input Find (invent) appropriate algorithm and describe it Algorithm output input if possible existing modules reusable components distributed applications, communication protocols Try to establish Find appropriate decomposition, reusing Requirements Can these activities be automated? A B output C Requirements input output Algorithm or composition Algorithm (in executable specification language) “correct” or “in error” Verification (does the algor. or composition satisfy the required properties ?) input Automatic translation from high-level language to executable code Compiler executable code output Can one find algorithms for automating the verification process ? (see next slides) Completely automated: Compilers exist for programming languages and certain executable specification languages The “verification” activity Requirements input Algorithm or composition including algorithms Verification (does the algorithms satisfy the required properties ?) output “correct” or “in error” Can the solution of this problem be automated ? Can one find an algorithm that solves this problem ? Two approaches: proving – testing - try to make a proof of the correctness - execute the given algorithms with various inputs (test inputs) and check whether the obtained outputs satisfy the requirements Verification: … towards a proof • Different levels of formalization – How are the requirements and the algorithm defined ? • Intuition, informal description / documentation, formal specification • Undecidable problems – Important result of computer science: Certain problems are undecidable (it is known that no algorithm exists that would solve them !!) • The verification problem is undecidable • Simplified cases of the verification problem are decidable – e.g. a proof (prepared by human) of the correctness of some algorithm may be verified by a proof checking algorithm Verification: … through testing Approach: To execute the program (or to simulate the execution of an algorithm) with particular inputs and to verify that the obtained outputs satisfy the requirements Difficulty: If one does not test for all possible input sequences (exhaustive testing) there is no assurance that all errors are found. Exhaustive testing is normally not possible. Example: Test of a telephone switch supporting N lines (the case of M simultaneous connect requests to various destinations): I = (N)M (number of cases) M=N = 4: I = 44 = 256 M=N=10: I = 1010 M=N=20: I = 2020 = 1026 Note: Many practical problems have an exponential complexity, that is, the execution time depends exponentially on the size of the input data (here the size of the switch) Formal methods and related tools Specification languages human creativity Find (invent) appropriate algorithm and describe it Requirements input Algorithm output Find appropriate decomposition, reusing Requirements A B if possible existing modules input output C based on predicate calculus: e.g. Z, for assertions about inputs and outputs, invariant assertions (for requirements) based on state machines (e.g. SDL) or process algebras (e.g. LOTOS); executable specification languages (for algorithms) Algorithms for the derivation of protocol specifications Equation solving (given A and C, find B) Tools for verification Requirements “correct” or “in error” Verification input Algorithm or executable program (does the algorithm satisfy the required properties ?) output Semi-automatic theorem provers Automated tools for exhaustive exploration of reduced finite state models (model checking) Simulated execution environments for composed systems Tools for test development with guaranteed coverage criteria (non-exhaustive) My own contributions (in red) Development of state machine languages, including standardization of the Estelle language (similar to SDL and UML state charts): definition, execution environments, testing, application to communication protocol standards Seminal work on protocol derivation and equation solving Methods and tools for test development (in the context of an industrial research chair at the University of Montreal with Hewlett-Packard, 1989-97) Now: Also applications in the area of distributed multimedia systems (teleconferencing, IP telephony) and photonic networks Specification languages based on predicate calculus: e.g. Z, for assertions about inputs and outputs, invariant assertions (for requirements) based on state machines (e.g. SDL and Estelle) or process algebras (e.g. LOTOS); executable specification languages (for algorithms) Algorithms for the derivation of protocol specifications Equation solving (given A and C, find B) Tools for verification Semi-automatic theorem provers Automated tools for exhaustive exploration of reduced finite state models Simulated execution environments for composed systems Tools for test development with guaranteed coverage criteria (non-exhaustive) Conclusions • Nature of software development – To build according to requirements (somethings useful) – To follow rules of best practices; note that most activities cannot be completely automated since the problems are either undecidable or of exponential complexity – To invent new solutions – freedom of choice – creativity – de la programmation considérée comme un des beaux-arts (Pierre Lévy, 1992) • Need for software engineers with – good communication skills to understand and to explain the problem and its solution – creativity to find appropriate system architectures, components and algorithms ( t( this is an Art )t ) – sense of rigor to verify that the constructed software satisfies the requirements and does not contain errors ( an • Ongoing research tries to further automate the SE process, but it is not clear how far this process may go, given the inherent limitations