From: AAAI Technical Report SS-02-08. Compilation copyright © 2002, AAAI (www.aaai.org). All rights reserved. Functional and Formal Reasoning in Architectural Sketches Ellen Yi-Luen Do DesignMachineGroup Department of Architecture,Universityof Washington Seattle, WA98195-5720 ellendo@u.washington.edu arguments hold true especially in the domain of architecture wherethe tool of the trade is drawing(Doand Gross 2001). Weare interested in building computational tools to support design reasoningthroughthe interface of freehand sketching. In this paper, we first report our research on empiricalstudies of design drawingsin both the functional and formal aspects. Wethen describe somecomputational tools built to support designing. Finally, weconcludewith discussionsandpossible future researchdirections. Abstract Architectsuse different symbols andconfigurations in their drawingsto explorealternatives and to communicate with each other. Theyuse sketchesto performfunctional and formalreasoningin their designprocess.For example, when thinking about spatial arrangements,they drawbubble diagrams;whenthinkingaboutnaturallighting, theydrawa sunsymbolandlight rays. Thispaperreports the findings fromempiricalstudiesof drawingsandreportsthe software systems that were implementedto support intention inference and automatedactivation of knowledge-based designtools to supportdesign. Studies Why Study Design Drawing of Design Sketches Wehave conductedseveral empirical studies on design and drawingto determine whether, and to what extent, it is possible to infer, interpret, or evenguess whata designer was thinking about by looking at the drawingshe has made. Weare interested in howdrawings get made, and what specific knowledgeand reasoning process they represent. Specifically, we are concernedwith the thought processes that underlie the operations comprising a drawing. Therefore,these empirical studies were conductedfocusing on design problem solving with form or function, separately. The studies include 1) data analysis of 62 architecture students’ concept diagramming,2) video transcripts and protocol analysis of four architects conductingdesignof an architect’s office, and 3) a retrospective analysis of pavilion house design that was carried over the period of 15 years byan architect. In the early design process, designers drawdiagramsand sketches to explore ideas and solutions. Drawingsare mental tools. Architects sketch to help themselvesto see and understand the form they work with, and to communicatewith others. Despite the fact that these drawings may seem crude, they are valuable in understanding how designers work because each of them serves its purposein a workof importance.Eachline in a drawingplays a role. It couldrepresent an outline of a form or a path of force (e.g., wind, rain, or light ray). Architecturaldesign deals with both formand function. The activity of drawing includes both seeing and thinking about the subject being represented. Design drawing is an iterative and interactive act involving recording ideas, recognizingfunctions and meaningin the drawings, and finding new forms and adapting them into the design (Goldschmidt 1991; Sch6n and Wiggins 1992; Suwaand Tversky 1996). It serves as external symbol systems to facilitate thinking and support emergentideas (Goel 1995; Fish 1996; Mezughi1996). Recently we have seen research workand computational systems to support sketching in various domains(Hearst, Gross et al. 1996; Stahovich 1996; Landay and Myers 2001). For example,a user can sketch widget symbolsfor interface design (Landay1996)or military course of action planning (Forbus, Fergusonct al. 2001). Theyargued that electronic sketching tools provide designers focus on the task at hand,on spatial relations andstructure of the design instead of specific detailed look and feel of the drawing (Ferguson 1992; Landay and Myers 2001). These Functional Reasoning The first study used design stories and diagrams from a case-based design aid called Archie (Kolodner 1991; Domesbekand Kolodner 1992; Zimring, Do et al. 1995). The database library of Archie contains stories, problems and responses from post-occupancy evaluation data collected in field studies of ten courthousesand libraries. All related items are cross-linked. Eachparticipant were given the tasks of makingdiagramsfrom stories, writing stories fromgiven diagrams,pairing diagramsand stories, and commenting on existing Archiediagram-storypairs. The second study involved protocol analysis of four designers in action. The designers were first given a 37 programbrief of an architect’s office space design. After reading the design program,designers were asked to start with a newsheet of paper (or tracing paper) for each task and to focus on four different concerns in conceptual, schematicdesign. The tasks include 1) spatial arrangement, 2) lighting, 3) visibility andprivacy,and4) fitting a special piece of furniture into the design. Fromthe diagrammingexperiment (Do 1995) and design protocol analysis (Do 1997) we found that designers use graphic symbolsto represent certain physical objects and design tasks and concerns. For example, when thinking about spatial arrangementsof functional spaces, designer would draw bubble diagrams to represent the rooms and their connections. They also drew graphic symbols for furniture objects to put themselvesin the context of the design problems.Whenthinking about lighting concerns, a designer woulddrawa configuration consisting of an arrow penetratinga vertical rectangle, representinga light ray in a sectional view. Figure 1 showsexamplesof the diagrams drawnby the participants. Formal Reasoning The third study is a retrospective analysis of the pavilion house design drawing. Over the course of 15 years, the architect had been engagedin formal manipulationsof the designfor a residenceand archivedall the drawinginto six CD-ROMs. Welooked at 110 drawings selected by the architect and developeda coding schemeto classify these drawingsinto different categories. This study resulted a conceptual frameworkto account for connections among the drawings(Neiman,Doet ai. 1999). Ourstudy attempts to identify relationships betweendrawings in order to understandthe role of formal reasoningin a design process. Whatbeganas a thought experimentresulted in a range of plausible interpretations to account for what might have actually happened in the design process. The interpretations were madethrough several iterations of sorting, classification and coding. Theresults werelater comparedwith the designer’s retrospective examinationof the drawings. I.Iootrmnq~ (E7) (Ell Pipe Body box1|61 IF.d) Chirnneyh~ ’~F-l:nkl ~ eacmYm’*) R15) Hernial sgtlpa, ~===- ilk ~ me (E’I4! co~rnn~t) (EI~,)Ve~lcu window Wall(E2) Figure I. Drawingconventions:(a) bubblediagramfor spatial arrangement, (b) graphicsymbolsfor furniture layout, and(c) lightingconcernsin sectionalview. Designersalso wrotetext to label the functions or names of the space. Whenthinking about fitting a piece of furniture in a conference room, designer wouldnot only drawgraphic symbolsfor furniture but also dimensional markers and numbersto reason about dimensions. P 4* I0-40 bufferfor table 40*2=80 section Iff 3" lff-~T ffi 37’ = ....... !_ ©onfcrcflc¢ room ~.- ..-D 20’’10’ ,,2(X)~l. Figure 2. Dimensionalreasoning for spatial arrangement accordingto programsquare footage requirements(numbers, markersandcalculations) For example,a participating designer drewdimensional markswith 10’ intervals along the length of the site (three 1 O’s and a 7’). Thenhe checkedto see if the table couldfit into the conferenceroom(20"10). First he wrote downthe dimensionof the table (4"10), calculated and wrote down the answer40, and doubledit (80) for buffer space. Figure 2 shows the drawing and annotations of the reasoning process derivedfrom the verbal protocol. 38 Figure3. Principalarchitecturalelementsfor the PavilionHouse (withcodingfor elementtypes). Manyof the drawings are composedof more than one drawingand consist a compositionof plan, sectional, or isometric viewsof the floor plan, or study variations of a fagade. Wedivided the compositedrawingsinto individual drawings and coded them with unique identifiers and drawing properties. The schemecodes properties of the drawings such as the elements depicted as well as projection types and viewangles of the building. Figure 3 showsthe common architectural elements and their codes (in parenthesis, e.g., El represents a column, and E2 represent a wall). Figure 4 shows a drawing coded of elementtypes, location and mediumused. ID# PI-30 CPl-9a) P2-16 P2-15) Drawing Elements El, E2, E3, E4, ES, E6, E7, ES, E9, El0, El2, El2, El2, El2, El2, El3, El4, El5 Location /scale Medium pencil MI) ~ciiow, blue, red markers M4) Figure4. Adrawing in codedtable of differentclassifications. For any two drawings, the coding schemealso accounts for the transformations of the design elements such as geometrictranslations of shapes, locations and color. The letter codes E, L, T, and C correspond respectively to elementidentifiers, location identifiers, and transformation types, and C specifies the use of a color. D indicates the projection type, V the view direction, Mthe drawing medium,and I the designer’s self-described intention in makingthe drawing. For example,the expression E 16@L4->(T4+T I+T 17) ->@L9 indicates that the designelement#16(staircase) at location #4 (middle left) movesdown(transformation #4), moves right (transformation#1) and rotates 180" (transformation #17) to location #9 (lower right). If there is transformation (transformation #17) of the location (L9: lower right) of element #16 between two drawings, it wouldbe codedas E 16@L9->(T16) ->@]..9 The East Building was designed to extend space for programming and to act as complement to the West Building. It was a design task to provide not only organizationand functional needs,but also to fit in a noneregular (trapezoidal)site with strict restrictions of setback lines fromthe surroundingstreets and aesthetic principles (symmetry, axis, etc). .... .... ..,,j~" -" . ~ .... i~,e ¯ Tt’~! !i ~L~ : _ ..,.::]. _~-~.,~ ~._sdjo-o--b--ewjo~6 ~ ---~__ :’~i Figure 5 showsthree design drawings, the 3x3 location grid, and the expressionsof transformationsof the staircase between them. ’ L -..,.:.’ .,.:.’.. ¯ L] I; " " ~" ’~ ..... 11 e ........ 9,t, " ’t .I IF~I Figure6. Site plan for the EastBuildingNationalGalleryof Art (drawnafter (Galleries1978)) The early design concepts kept in Pei’s notebook or diary showsconcerns about the axis of the WestBuilding (Figure 6, left), and extendingit to divide the trapezoidal lot into severaltriangles, as shown in Figure7 (tight). E16 0L4 -> (T4+Tl+T17) -> 0L9 E16 0L9 -> (T16) -> 0L9 Figure5. Staircase(El6) in drawingI (left) is moved 0"4:down, TI: right, andTI7:rotate 180)to a differentlocation(from1.4 Lg) in drawingII (middle). The location of the staircase identicalbetween drawingII (middle)anddrawingIII (right) codedas transformation TI6 (no transformation). Drawing Evolution Figure7. Concept sketch,site analysis,and spatial arrangement (drawnafter (Galleries1978)) in Design Process The notion of design being the fusion of form and function is widelyacceptedand followedin the field of architecture and industrial design (Meikle 1979). The dictum ’Form follows Function’ was first introduced by the architect LouisSullivan (Twombly 1986) in 1896to explain that his building design follows natural law. Later Miesvan dcr Rohefurther argued that ’Form IS Function’, and many modernistarchitects and designersshare this notion. There are often debates about whetherform precedes function or vice versa (Janson 1982; Anderson1987). However,it’s the commontraining and education of architects that successful building should incorporate form and function together. The East Building of National Gallery of Art designed by I. M. Pei is a goodexampleof howconceptual drawing evolves to account for the form and functional arrangementsof the building. Figure 5 showsthe final drawingof the building on site. The original National Gallery of Art, or the West Building is a symmetrical, rectangular, neoclassical building with a rotunda in the middle. 39 Examining Pei’s design drawings from a formal perspective reveals that the shapes or the formare tightly connectedto the functionof the spaces. At first glance, the drawingsin Figure 8 sharesimilar shapes and structure. Theyall have triangle components.The three drawingson the right are actually fromfacing pagesof the notebook. o Figure 8. Designdrawingsfromthe design notebook.(after (Galleries1978)) The first sketch on the left is a very simple diagram showingtwotriangles, one isosceles and one right triangle The second sketch is muchmore elaborate. The motif of triangles has becomepatterns coveting several areas. The final building uses the triangle pattern for the space flame abovethe concoursethat connectsexhibition space and the Center for Advanced Study. The form of triangle serves the purpose of functional support for the frame. The third sketch is diagrammatic. Each circle represents an independent museumas noted in the notebook (Galleries 1978), and connected to the bottom right triangle with another triangle. The last sketch shows the symmetric feature of the WestBuilding, and each gallery has a prism shapeand is connectedby bars. Figure 9 contains a series of diagrams explaining the developmentof the East Building from preliminary concept stage to the final conception realized in the building. Besides obeying the setback guidelines and specific stipulations from the" urban agencies, Pei projected the longitudinal axis of the West Building to bisect the building lot. Usingthe axis as a guide, a symmetricalline that is diagonalon the site is drawnand dividesthe lot into twoessential spaces. Oneis the isosceles triangle, facing the West Building and echoing the symmetricalgeometry. Theother spaceis a right triangle. This formal arrangement solves the problemof the site; at the sametimeit provides functional space for two programmingrequirements. The isosceles triangle is usedfor exhibition galleries, and the right triangle is the Center for Advanced Study and other scholarly and administrative functions. These two functional spacesare connectedby a triangle concourse,an atrium with skylights. The three comersof the isosceles triangle becamegallery towers connectedby bridges. Figure 9. Design evolution: (1) axis extended from West Building,and setbacklines, (2) diagonaldivides2 spaces, isosceles triangle and a right triangle, (3) comersof isosceles triangle becameexhibition towers, and (4) the right triangle became the Centerfor Advanced Study.(alter (Galleries1978)) (right) derived from the 2Dplan (middle). Note also concept sketch on the left of Figure 10: though crude, several shapes and lines are identifiable. Thebottomright angle is circled and connected with bubbles for more developmentof the courtyard schemefor the Center of AdvancedStudy. Figure 11 shows a pair of the East Buildingdesign with similar shapes but different details. This is similar to the findings from the Pavilion House study. Designers move, rotate, and transform design elementsin their drawingto makeformal arrangements.It is worthnoting that in this case, the transformationof the drawingelementsinvolves functional arrangementsas well. (Wemaybe able to further verify this if we can capture design rationale and verbal protocols of the wholedesign process.) Figure11. Thelocationof the openingor courtyardfor the Center of Advanced Study(ocoupying the right triangle at the bottom) are differentin these twodesignsketches.Theopening onthe left schemeis on the south side, whilethe drawingonthe right has the opening fi’omthe slopeof the triangle.(after (Galleries1978)) Computational Tools for Sketching Stories of howthese drawings account for the design, and the solving of aesthetic and engineeringdecisions are abundant (Wiseman1990) (Art; Galleries 1978). descriptions of form and function in design development are usually inter-connected, Besides the evolution of architectural drawings in the design process, someother drawings also shed light on what actions designer take whenthinking about design. Figure10. (1) Concept sketchof the buildingwithfight triangle circled and called out. (2) Designof the Centerfor Advanced Studywithcourtyard.(3) Isometricdrawingof the volume for the Centerfor Advanced Study.(a~er(Galleries1978)) The first exampleis the moveor projection from twodimensional floor plan drawing to three-dimensional isometric drawing. Figure 9 showsa 3D isometric drawing 4O Recognizingthat it is possible to associate symbolsand spatial arrangements in a drawing with the designer’s intention, or task context, we have implemented computational tools to use freehand sketching as an interface to intelligent systemsfor design. Thesedesign tools include using diagrams for knowledgeand image retrieval, building performancesimulation, and threedimensional modelmakingfor early stages of design (Do 1998; Gross and Do 2000). At the most basic level of symbolic processing, our systemscapture stroke data fromthe tablet and use the pen path and stroke features (speed, comer,aspect ratio, ere) identify symbolsdrawnby the designers. A symbolcould be a single-stroke glyph, or consisted of multi-strokes glyphs. A low-level recognizer starts the processing and display recognition upona pen-up action. Designers can also turn off the recognition display, or ask the background processor not to resolve ambiguoussymboluntil further information is given that helps the programidentify the context. Resolving ambiguousintentions is necessary because the same drawing symbol could mean different things in different context (Gross and Do 1996). For example, a circle on a floor plan could mean a column while a circle outside and abovea building section with a line penetrating the building envelopewouldbe the sun and light ray. Similarly, a circle with alphabet neighbors is likely to be an alphabetOwhilea circle next to a numberis likely to be the numberzero. A higher-level recognizer deals with analysis of spatial relations amongdrawing symbols and the combinationthereof. For example, a big circle surroundedby small squares on a floor plan could meana dining table set. Figure 12 showsthat the wigglyline placed directly below an horizontal line can be recognized as a symbol for the "ground," thus set the contextinto ’section.’ Im m I--zN I M~~u~ ~ m, . kllkim k ° . m~ N D Figure 14. Whena ’monitor_glare’intention is recognized, (bottomleft) a message prompts the user (topleft) andit activates a WebBrowser(fight), requestsa keyword searchandthe search returnsa list of relevantinformation N~ Figure12. A"ground"symbolis composed of a horizontal line anda wiggly-line(left). Thenamesof the elementsare replaced by the newnameof the configuration(andsets the contextto ’section’) The Right-Tool-Right-Timemanageridentifies context based on existence of special symbolor configuration to trigger an intention recognizer. Oncedesign intention is recognizedin the drawing, the systemwouldautomatically activate the appropriate tools for the task at hand. For example,Figure 13 showsa visual analysis programcalled Isovist (Do and Gross 1997) can be activated once the Right-Tool-Right-Time programrecognizes the intention of ’view’. The viewpoint and lines are translated into the programfor simulation. Figure13. A ’view’intention is recognizedin a ’plan’ context (top) andthe drawing is translatedinto walllines (fromlines) a viewpoint(arrowsand circle) and sent to the Isovist visual simulationprogram(bottom). Another example uses the intention and context information derived from the drawingto activate keyword search on a web search engine or a database. Figure 14 showsthat an intention of "monitor_glare"is recognizedby seeing the configuration of a computer(monitor and hard drive) and a sun ray (sun symbolwith arrowindicates light direction). This intention activates the browserto launch keywordsearch using a website. The sameinteraction can be applied to other knowledge-basedsystems, a slide library, or a case library like Archie. 41 A common task in design is dimensional reasoning and calculation of area footage. Drawing symbols of dimensional markers and numeric operators can be recognizedas a context of ’calculation’ and thus activate a calculator for designer. Figure15 showssuch interaction. b~ O , ~mms mmml ~r.~m mam~ r Figure15. Aconfiguration of numbers andarithmeticmarks(left) is recognizedas an "adding" intention (middle) and RTRT activatesa calculator(right). The Back of an Envelopeproject and the Right-ToolRight-Time manager demonstrate that a pen-based, or calligraphic interface can be used in a variety of domainsin design process. Designers can use freehand sketches and diagrams to index and retrieve databases or to activate knowledge-based information systems and simulation programs.It is a frameworkof knowledgecapture. Onecan use the graphic recognition of the systemand add to the frameworkmore modules and functionality to support design. Belowwedescribe two newsketching interfaces for supportingformal and functional reasoningin architectural design: I) GIDA- GraphicsInterpreter of DesignActions, and 2) VRSketchpad- an interface for creating instant 3D worldsby sketching. GraphicsInterpreter of DesignActions Withthe coding schemedevelopedfor the Pavilion House analysis, we are currently working on a Graphics Interpreter of Design Action - GIDAto managedrawing comparisonand sorting of the diagrams. The GIDAsystem allows a user to diagramover a picture underlayof design drawingand to generate an analysis of the drawingitself and its relationships with other drawings. For example,a diagram’stopological and geometricrelations amongparts of the drawingcan be recorded and used in comparisonto another diagramtraced from a different design drawingto reveal the spatial transformations amongthe elements. Figure 16 showsa pair of design drawingstraced from the Pavilion Housefagade study. The GIDAsystem lays out a 3x3grid over the drawingand generates a list of occupied cell numbersfor each object. D element from first drawingto the second one(addition of grid cell 3). The table showsthat the Hoodelement in Drawing#1 is removedfrom Drawing#2. Likewise, the transformation of the ChimneyBoxis a movingdownfrom (4 5 4) to (7 4 5 8 VR Sketchpad In VRSketchpad(Do 2001) we developed a simple sketch recognition systemthat inferred three-dimensionalmodels fromtwo-dimensionalsketches. The idea is to use drawing to construct virtual built spaces. Our current prototype systemenables a designer to drawa floor plan with walls, columns, and furniture elements and the programproduces a 3D model in VRML(Virtual Reality Modeling Language).Figure 18 showsthat any froehand sketch can be convertedand extrudedas columnsand wall partitions. i’ ¯ | : F Figure16. GIDA’s locationidentifier (3x38rid) overtwodesign drawings In the exampleof Figure 16, Drawing#1 (Figure 16 left) has nine elements and Drawing#2 has eight. Each element has a position in the global coordinatesystemand a list of the local grid cell sequence.Figure 17 is a table showing the list of elements for each drawing and their correspondingcell sequence. Ekmtmlt Drawing#1 T~ekWall ChinmeyBox Chi~ Pipe V-wmd~,v I V-window 2 V-window 3 Hood H-~ip1 H-ship 2 H-strip3 H-stamp 4 (7456987) (,454) (14) (69) (51) (4 7) (6 3 2) (45 6) (456) Drazemg ~ (741236987) ~4587) (14) (369) Figure18. Curveshapesand lines (leR) are extrudedto make partitionsin a VRML world(right). The recognition engine of the systeminterprets drawing symbols(e.g., circles and lines) with designer intended object representations (columns, walls) and diagram configurations(e.g., dining table and chairs). A designer can also drawarrowsto indicate locations of interests and therefore define a viewingpath into the 3Dworld (encoded as viewpoints into a VRML world). A designer can also define a configuration of symbolsto represent furniture objects as shownin Figure 19. By sketching the furniture layout in diagrammatic form with arrows indicating specific viewpointsand view angles, designer can produce a 3Dscene in an instant and the visitors can follow the guidedpath to explorethe virtual space as designed. (456) (45 6) (456) (789) Figure17. State of drawingelementsrepresentedas list of cell sequence. Transformations between drawings can be inferred by comparingthe lists of the sameelements from different drawings. For example,the cell sequencelist of V-window 1 was changed from (6 9) to (3 6 9). The bounding sizes of the elementin the twodifferent drawingsare very close. This describesthe shifting-uptransformationfor this 42 4b ~r Figure19. Furniturelayoutsketch(TV,couch,dinningtable set, columns,andwalls) creates 3DVRML scene(righOwith objects. Discussion Sketchingis important in the early, conceptual stages of architectural design. Therefore,computationaltools should support sketching activities. The empirical studies examined intentions in and relations among design drawings. Our prototype systems demonstrate how freehand drawing interfaces can support analysis and design. Theseare all part of a larger researchagendaon the role of drawingfor design support. Manyissues are worth exploringfurther. The act of drawingis a form of design reasoning. An architectural design involves both functional and formal reasoning. Designdrawingsuse lines and geometricshapes to represent natural or manmadeartifact such as building components and plants, phenomenasuch as sound and light, humanbehaviorsuch as sight and circulation, as well as boundariesof spaces. Thestudies of design students on the conceptdiagramming exercise and the protocol analysis of architecture design reveals promising results for computational tools to support sketching. Wefound the designers drawdifferent symbolsand configurations when thinking about different functional design concerns. Onthe other hand, our exploratorystudy of the Pavilion Housebroadenedour understandingof the formal role that drawings play in design. Design sketches are made of geometricshapes. Notsurprisingly, aesthetic principles and formal expressions also influence howa designer works. From the study we found that a designer manipulates design objects through transforming shapes and locations, changingviewpoints, drawingtypes, and mediato explore design alternatives. Whenfunctional concerns are easy to solve, these form manipulations involve the search of implicit graphical law or convention. For example, the balance of the configuration, its symmetrical or axial considerations,aspect ratio of the object and proportion,all serve as criteria for design manipulation, or or moves. Transformations are applied to previous designs to generate alternatives and to predict the outcomesof new proposal. The designer manipulates the visualized representations to evaluate the consequencesof design moves. These manipulations are simply geometric transformations but in combination the process becomes complex.Wefound that designers play mental gameswith themselves. They play by defning rules, selecting strategies, making design movesfrom the rules, and evaluating and discovering the outcome. These design moveschange the shape, dimensions, orientations, and placementsof the design elements. If a design process always involves fusion and interchanging activities of both functional and formal aspects, then how might we find the appropriate interpretations of the design intentions expressed in drawing?Are there specific symbolsthat can be identified to indicate the right context?Theseries of design drawings of the East Building reveal that designer moveback and forth from functional to formal concerns fluently. The geometric shapes are manipulated first for formal arrangements such as symmetryand establishing axial lines. Thenthese shapes are read as symbolarrangements for functional spaces. The ingeniouspart of the design is that all problems, be they functional or formal, are all solved and evolvedthroughthese architectural sketches. In summary,we conclude that we can view the act of drawingas manipulationsof symbolsthat represent internal functional and formal knowledge. Understanding architectural sketches are not an easy test. Laboratory studies can isolate formal or functional problems separately. However,in a real life seeting, these issues mingle and are dealt with at the same time. To make computationaltools really support sketching activities in design, we would need to study more howdesigners move betweendifferent modesin a real design task. Observing and analyzing design protocols could probably help identifying howdesigner operates. Maybethe transitions amongvarious states, the sequence of drawing, are the external controls of the environment representing the mentalimage.It is clear that the activity of drawingcan not be detached from seeing, and thinking about the subject being represented. Wecan not drawan object or a scene unless wesee it before us whetherin real physicalworld,or in mind’seye. Manytimes, the drawingsare created from adopting and adapting objects from memory or imagination. Therefore, using drawing to accessing databases or knowledge repository is definitely an important issue. Analyzing and understanding what sketches represent is important. However,there is still a broadarea to explore in order to understandhowsketches are made, and why are they made, to support what activities. Acknowledgements Thematerial in this paper is baseduponworksupportedby the National Science Foundationunder Grant numbersIIS96-19856 and IIS-00-96138. The views and findings containedin this materialare those of the author and do not necessarily reflect the views of the National Science Foundation. References Anderson, S. 1987."TheFiction of Function."Assemblage 2:1831. Art, NationalGallery.http://www.nga.gnv/. Do,E. Y.-L. 1995. What’sin a diagramthat a computershould understand. In CAAD Futures ’95: TheGlobalDesignStudio, Sixth InternationalConference on Computer AidedArchitectural Design Futures. M. Tan and R. Teh. Singapore, National Universityof Singapore:469-482. Do, E. Y.-L. 1997. Computabilityof DesignDiagrams-- an empirical study of diagramconventionsin design. In CAAD Futures97, R. Junge.Munich,Kluwer:171-176. Do, E. Y.-L. 1998. The Right Tool at the Right Time Investigation of FreehandDrawingas an Interface to Knowledge BasedDesign Tools. Ph.D. Thesis, Atlanta, GeorgiaInstitute of Technology. Do, E. Y.-L. 2001. Graphics Interpreter of Design Actions: the GIDAsystem of diagram sorting and analysis. In CAAD Futures 2001. B. d. Vries, J. P. v. Leeuwen and H. H. Achten.Eindhoven, KluwerAcademicPublishers: 271-284. Do, E. Y.-L. and M. D. Gross 1997. Tools for Visual and Spatial Analysis of CADModels - implementing computer tools as a meansto thinking about architecture. In CAAD Futures 97. R. Junge. Munich,Kluwer: 189-202. Do, E. Y.-L. and M. D. Gross 2001. Thinking with Diagramsin Architectural Design. In Artificial Intelligence Review. A. BlackweU.Dordrecht, The Netherlands, AcademicPublishers: 135-149. Domeshek, E. and J. Kolodner1992. A case-based design aid for architecture. In Artificial Intelligence in Design’92. J. S. Gero. Dordrecht, Netherlands, KluwerAcademicPublishers. Ferguson, E. 1992. Engineering and the Mind’sEye. Cambridge, MA,MITPress. Fish, J. C. 1996. HowSketches Work- A Cognitive Theory for Improved System Design, Doctoral Thesis. Loughborough University of Technology. Forbus, K. D., R. W. Ferguson, et al. 2001. "Towards a Computational Model of Sketching." In Intelligent User Interfaces: 77-83. Galleries, AdamsDavidson1978. I. M. Pei &Partners, Drawings for the East Building, National Gallery of Art. Its Evolution in Sketches, Rendering, and Models1968-1978. Washington,D.C., Exhibit Catelog. Goel, V. 1995. Sketches of Thought. CambridgeMA,MITPress. Goldschmidt, G. 1991. "The Dialectics of Sketching." In Creativity ResearchJournal v.4(# 2): 123-143. Gross, M. D. and E. Y.-L. Do 1996. AmbiguousIntentions. Proceedings, ACM Symposiumon User Interface Software and Technology(UIST’96). Seattle, WA:183-192. Gross, M.D. and E. Y.-L. Do2000. Drawingon the back of an envelope. In Computersand Graphics, Calligraphy Interface. J. A. J. a. E. Glinert. NewYork,PergamonPress. 24: 835-849. Hearst, M. A., M.D. Gross, T. F. Stahovich, J. A. Landay.1996. "SketchingIntelligent Systems."IEEEIntelligent Systems13(3): 10-19. Janson, H. W. 1982. Form Follows Function - or Does It? Modernist Design Theory and the History of Art. The Netherlands, Maarsen. Kolodner, J. L. 1991. "Improving human decision-making throughcase-baseddecision aiding." ,41Magazine 12(2): 52-68. Landay, J. A. 1996. SILK: Sketching Interfaces Like Krazy. HumanFactors in Computing,CHI96. T. V. Program. Landay, J. A. and B. A. Myers2001. "Sketching Interfaces: TowardMore HumanInterface Design." IEEE Computer34(3): 56-64. Meikle,J. L. 1979. Twentiethcenturylimited: industrial designin America,1925-1939.Philadelphia, TempleUniversity Press. Mezughi, M. M. 1996. The Integral Role of Drawing in Architectural Conception.Ph.D. Thesis. GlasgowSchoolof Art, MackintoshSchoolof Architeeture, University of Glasgnw. 44 Neiman,B., E. Y.-L. Do, and M. D. Gross. 1999. Sketches and Their Functions in Early Design: A Retrospective Analysis of Two Houses. Design Thinking Research Symposium’99. W. Porter and G. Goldschmidt. SchSn, D. A. and G. Wiggins1992. "Kinds of Seeing and their functions in designing."DesignStudies 13(#2): 135- ! 56. Stahnvich, T. F. 1996.SketchIT:a Sketch Interpretation Toolfor Conceptual Mechanical Design. Artificial IntelLigence Laboratory. Ph.D. Thesis. Cambridge, MA,Massachusetts Institute of Technology. Suwa, M. and B. Tversky 1996. "WhatArchitects See in Their Sketches: Implications for Design Tools." Proceedings of ACM HumanFactors in Design’96 (CHI’96): 191-192. Twombly,R. C. 1986. Louis Sullivan : his life and work. New York, Viking. Wiseman,C.1990. I. M.Pei: a profile in Americanarchitecture. NewYork, H. N. Abrams. Zimring, C., E. Y.-L. Do, J Kolodner and E Domeshek.1995. Supporting Case-Study Use in Design Education: A Computational Case-Based Design Aid for Architecture. Computingin Civil Engineering, A/E/C Systems ’95. J. P. Mohsen.Atlanta, GA,AmericanSociety of Civil Engineers. II: 1635-1642.