Listoflectures
TDDA69DataandProgramStructure
DeclarativeProgramming
CyrilleBerger
1IntroductionandFunctionalProgramming
2ImperativeProgrammingandDataStructures
3Parsing
4Evaluation
5ObjectOrientedProgramming
6Macrosanddecorators
7VirtualMachinesandBytecode
8GarbageCollectionandNativeCode
9ConcurrentComputing
10DeclarativeProgramming
11LogicProgramming
12Summary
2/46
Lecturecontent
DeclarativeProgramming
Make
RegularExpressions
SQLAndDatabases
DeclarativeProgramming
StructuredQueryLanguage(SQL)
InterpretingSQL
3/46
Declarativeversusproceduralprogramming
DeclarativeProgramming
Expresseslogicofcomputation
withoutcontrolflow:
Declarativeknowledge:describes
whatistrue.
Whatshouldbecomputedandnothowitshould
becomputed.
E.g.√xisysuchthaty²=xandy≥0
Proceduralknowledge:howto
computewhatistrue.
Examples:XML/HTML,SQL,antlr4/
yacc,make/ants,RegExp/Template
Matching...
E.g.Analgorithmtocompute√xgivenx
5
6
InterpretingDeclarativeProgramming
Inimperativelanguages:
Aprogramisadescriptionofthecomputational
processes
Theinterpretercarriesoutexecution/evaluation
rules
Make
Indeclarativelanguages:
Aprogramisadescriptionofthedesiredresult
Theinterpreterfiguresouthowtogeneratethe
result
7
Make
Makefile
AMakefileismadeofasetofrules:
Makeisabuildautomationtoolswhichspecify
howtogenerateoutputfilesaccordingtoaset
ofrulesandasetofinputfiles
CommonlyusedunderUnixtobuildC/C++
program
Butcanbeusetocontrolgenerationof
anything,really(latex...)
Alternative:Ants,nmake,scons...
TARGETS:PREREQUISITES
RECIPE
TheTARGETSistheoutputfiles
ThePREREQUISITESisthelistoffilesthatyouneedto
generatetheTARGETS
RECIPEishowtogeneratetheoutputfromtheinput
Example:
.PHONYall
all:a.out
a.out:main.cpp
gccmain.cpp
9
HowtowriteaMakefileinterpreter(1/2)
10
HowtowriteaMakefileinterpreter(2/2)
Interpreter
Parsethemakefileintoasetofrules
defexectute_target(target,rules,phony):
forruleinrules:
if(rule[0]==target):
should_execute=False
forpinrule[1]:
execute_target(p,rules,phony)
if(notpinphonyandfile.date(p)>file.date(target)):
should_execute=True
if(should_executeandnotexecute(rule)):
raiseException('Failedtogeneratetarget:'+target)
if(notFile.exist(target)):
raiseException('Missingfile:'+target))
[TARGETS,[PREREQUISITES],RECIPE]
rules=[['all',['a.out'],''],['a.out',['main.cpp'],'gcc
main.cpp']
phony=['all']
if__name__=='__main__':
(rules,phony)=parse('Makefile')
execute_target(rules[0][0],rules,phony)
11
12
Tomakethingabitmorecomplicated...
Isitconvenienttowrite:
all:myprogram
myprogram:main.oa.ob.o
gccmain.oa.ob.o-omyprogram
main.o:main.c
gccmain.c-omain.o
a.o:a.c
gcca.c-oa.o
b.o:b.c
gccb.c-ob.o
Itisnicertowrite:
all:myprogram
myprogram:main.oa.ob.o
gcc$^-o$@
%.o:%.c
gcc-c$<-o$@
RegularExpressions
Rulesaredefined
withtemplate
matching
13
BasicRegularExpression
RegularExpression
Concatenation:aabaab
Aregularexpression(regex)describesaset
ofpossibleinputstrings
Theycanbeusedformatchingstringsand
forsearch&replace
Theyarecommonlyusedforsimpleparsing
yes:'aabaab'
no:everyotherstring
Wildcard:.u.u.u.
yes:'cumulus','jugulum'...
no:'succubus','tumultuous'...
Unionaa|baab
yes:'aa','baab'
no:everyotherstring
Makefilerules
Processnaturallanguages
Fieldvalidation
...
Closureab*a
yes:'aa','aba','abba'...
no:'ab,'ababa'...
Parenthesesa(a|b)*aab
yes:'aaab','abbaab','ababaab'...
no:'abcaab','acabaab'...
15
Oneormorea(bc)+de
yes:'abcde','abcbcde'...
no:'ade','bcde'...
Range[A-Z][a-z]*
yes:'Capitalized','Word'...
no:'uncapitalized','wOrd'...
Exactlyk[0-9]{2}(0[0-9]|10|
12)([0-2][0-9]|30|31)-[0-9]{4}
yes:900431-3234...
no:902331-3234,900452-3234...
Negations[^aeiou]{6}
yes:rhythm
no:decade
16
Howtointerpretaregularexpression?
RegularExpressionsinPython
importre
p=re.compile('[a-z]+')
p.match('')
p.match('tempo')
Regularexpressionsareaconcise
waytodescribepatterns
Howtoimplement'p.match'?
useaderministicfiniteautomaton
(DFA)
17
18
InterpretaDeterministicFiniteAutomaton
DeterministicFiniteAutomaton(DFA)
ADeterministicFiniteAutomatonconsistsof
Takeawordcomposedoflettersin∑
DoesthewordmatchtheDFA?
Qafinitesetofstates
∑afinitesetofinputsymbols
q₀astartstate
Fasetoffinalstates
δatransitionfunctionfromQx∑->Q
Treatthewordasastreamofinputsymbols
qisthecurrentstate
Startinq=q₀
Givencthecurrentinputsymbol,thenq=δ(q,c)
Whennoinputsymbolsremain,ifq∈F,then
acceptotherwisereject
19
20
DFAandRegexp-Example
DFAandRegexp
Regexpsareaconcisewaytodescribeasetofstrings
DFAsaremachinetorecongnizewhetheragiven
stringisinagivenset
Theorem:foranyDFA,thereexistsaregular
expressiontodescribethesamesetofstrings,forany
regularexpression,thereexistsaDFAthatrecognize
theset
Conscequence:toimplementaregularexpression
matcher,buidaDFAandexecuteit
DFA:
Q₀andQ₂arefinalstates
Regexp:(ab*a)*
21
22
DatabaseManagementSystems
ADatabaseisacollectionoftable
Atableisacollectionofrecords
Arecordisarowwithavalueforeachcolumn
Acolumnhasanameandatype
SQLAndDatabases
TheStructuredQueryLanguage(SQL)isthemostwidelyused
programminglanguageforaccessingDBMS
24
SQLExamples
Tablecreation
createtablecitiesas
select38aslatitude,122aslongitude,"Berkeley"asnameunion
select42,71,"Cambridge"union
select45,93,"Minneapolis";
StructuredQueryLanguage(SQL)
Dataretrieval
select"westcoast"asregion,namefromcitieswherelongitude>=115union
select"other",namefromcitieswherelongitude<115;
25
StructuredQueryLanguage(SQL)
SelectingValueLiterals(1/2)
TheSQLlanguageisanANSIandISOstandard,but
DBMS'simplementcustomvariants
Aselectstatementcreatesanewtable,eitherfrom
scratchorbyprojectingatable
Acreatetablestatementgivesaglobalnametoatable
Lotsofotherstatementsexist:analyze,delete,explain,
insert,replace,update,etc.
Mostoftheimportantactionisintheselectstatement
Thecodeforexecutingselectstatementsfitsonasingle
sheetofpaper
Aselectstatementalwaysincludesacommaseparatedlistofcolumndescriptions
Acolumndescriptionisanexpression,
optionallyfollowedbyasandacolumnname
select[expression]as[name],[expression]as[name],;...
Selectingliteralscreatesaone-rowtable
select'abraham'asparent,'barack'aschild;
Theunionoftwoselectstatementsisatable
containingtherowsofbothoftheirresults
27
28
SelectingValueLiterals(2/2)
NamingTables(1/2)
select'abraham'asparent,'barack'aschildunion
select'abraham','clinton'union
select'delano','herbert'union
select'fillmore','abraham'union
select'fillmore','delano'union
select'fillmore','grover'union
select'eisenhower','fillmore';
SQLisoftenusedasaninteractive
language
Theresultofaselectstatementisdisplayedto
theuser,butnotstored
Acreatetablestatmentgivesthe
resultaname
createtable[name]as[selectstatement];
29
30
SelectStatementsProjectExistingTables
NamingTables(2/2)
createtableparentsas
select'abraham'asparent,'barack'aschildunion
select'abraham','clinton'union
select'delano','herbert'union
select'fillmore','abraham'union
select'fillmore','delano'union
select'fillmore','grover'union
select'eisenhower','fillmore';
Selectstatements:
SELECT[columns]FROM[table]WHERE[condition]ORDERBY[order]
Aselectstatementcanspecifyaninputtableusingafromclause
SELECTparent,childFROMparents;
Asubsetoftherowsoftheinputtablecanbeselectedusinga
whereclause
SELECTparent,childFROMparentsWHEREparent='fillmore';
Anorderingovertheremainingrowscanbedeclaredusingan
orderbyclause
SELECTparent,childFROMparentsORDERBYparent
Columndescriptionsdeterminehoweachinputrowisprojected
toaresultrow
SELECTchildFROMparentsORDERBYparent
31
32
JoiningaTablewithItself
JoiningTwoTables
TwotablesA&Barejoinedbyacommatoyieldall
combosofarowfromA&arowfromB
Twotablesmayshareacolumnname;dotexpressions
andaliasesdisambiguatecolumnvalues
createtabledogsas
select'abraham'asname,'long'asfurunion
select'barack','short'union
select'clinton','long'union
select'delano','long'union
select'eisenhower','short'union
select'fillmore','curly'union
select'grover','short'union
select'herbert','curly';
createtableparentsas
select'abraham'asparent,'barack'aschildunion
select'abraham','clinton'union
...;
select[columns]from[table]where[condition]orderby[order];
[table]isacomma-separatedlistoftablenameswith
optionalaliases
Selectallpairsofsiblings
selecta.childasfirst,b.childassecond
fromparentsasa,parentsasb
wherea.parent=b.parentanda.child<b.child;
Selecttheparentsofcurly-furreddogs
selectparentfromparents,dogswherechild=nameandfur='curly';
33
34
JoiningMultipleTables
Multipletablescanbejoinedtoyieldall
combinationsofrowsfromeach
createtablegrandparentsas
selecta.parentasgrandog,b.childasgranpup
fromparentsasa,parentsasb
whereb.parent=a.child;
InterpretingSQL
Selectallgrandparentswiththesamefuras
theirgrandchildren
selectgrandogfromgrandparents,dogsasc,dogsasd
wheregrandog=c.nameand
granpup=d.nameand
c.fur=d.fur;
35
NumericalExpressions
StringExpressions
InaSELECTstatement:
Stringvaluescanbecombinedto
formlongerstrings:
SELECT[expression]AS[name],...FROM[table]WHERE[expression]ORDERBY
[expression]
Expressionscancontainfunctioncallsand
arithmeticoperators
Combinevalues:+,-,*,/,%,and,or
Transformvalues:abs,round,not,Comparevalues:<,<=,>,>=,<>,!=,=
Examples:
SELECT'hello,'||'world';
->'hello,world'
Basicstringmanipulationisbuiltinto
SQL,butdiffersfromPython:
SELECTsubstr(s,4,2)||substr(s,instr(s,'')+1,1)from
phrase;
->low
SELECTabs(-2)
SELECTcount(*)FROMparents
37
38
TablerepresentationinPython
Anaivefirstimplementationofselect
Thenamedtuplefunctionreturnsanewsub-classoftuple
Onecorrect(butnotalwaysefficient)implementationof
selectusessequenceoperations
Exampleofquery:
fromcollectionsimportnamedtuple
City=namedtuple('City',['latitude','longitude','name'])
cities=[City(38,122,'Berkeley'),
City(42,71,'Cambridge'),
City(43,93,'Minneapolis')]
[city.latitudeforcityincities]
->[38,42,43]
SELECTname,60*abs(latitude-38)ASdistanceFROMcitiesWHEREname!='Berkeley';
Distance=namedtuple('Row',['name','distance'])
defcolumns(city):
latitude,longitude,name=city
returnDistance(name,60*abs(latitude-38))
defcondition(city):
latitude,longitude,name=city
returnname!='Berkeley'
forrowinmap(columns,filter(condition,cities)):
print(row)
->Row(name='Cambridge',distance=240)
->Row(name='Minneapolis',distance=300)
Attributenamesareaccessibleasthe_fieldsattributeof
aninstanceofCity
print(cities[0])
print(cities[0]._fields)
Output:
->City(latitude=38,longitude=122,name='Berkeley')
->('latitude','longitude','name')
39
40
SQLInterpretationExample
ASelectClass
TheSQLparsercreatesaninstanceoftheSelectclassforeach
selectstatement
SQLQueries:
CREATETABLEcitiesAS
SELECT38ASlat,122ASlon,'Berkeley'ASnameUNION
SELECT42,71,'Cambridge'UNION
SELECT45,93,'Minneapolis';
SELECT60*(lat-38)ASnorthFROMcitiesWHEREname!='Berkeley';
classSelect:
"""select[columns]from[tables]where[condition]."""
def__init__(self,columns,tables,condition):
self.columns=columns
self.tables=tables
self.condition=condition
self.make_row=create_make_row(self.columns)
defexecute(self,env):
"""Join,filter,andmaprowsfromtablestocolumns."""
from_rows=join(self.tables,env)
filtered_rows=filter(self.filter_fn,from_rows)
returnmap(self.make_row,filtered_rows)
deffilter_fn(self,row):
ifself.condition:
returneval(self.condition,row)
else:
returnTrue
InPython:
City=namedtuple('City',['lat','lon','name'])
cities=[City(38,122,'Berkeley'),
City(42,71,'Cambridge'),
City(43,93,'Minneapolis')]
s=Select('60*(lat-38)asnorth','cities','name!="Berkeley"')
forrowins.execute({'cities':cities}):
print(row)
41
CreatingRowClassesDynamically
42
JoiningRows
Eachselectstatementcreatesatablewithnewcolumns,
representedbyanewclass
Joiningcreatesadictionarywithallnamesand
aliasesforeachcombinationofrows
defcreate_make_row(description):
"""
Returnafunctionfromaninputenvironment(dict)toanoutputrow.
description--acomma-separatedlistof[expression]as[columnname]
"""
columns=description.split(',')
expressions,names=[],[]
forcolumnincolumns:
if'as'incolumn:
expression,name=column.split('as')
else:
expression,name=column,column
expressions.append(expression)
names.append(name)
row=namedtuple('Row',names)
returnlambdaenv:row(*[eval(e,env)foreinexpressions])
fromitertoolsimportproduct
defjoin(tables,env):
"""Returnaniteratoroverdictionariesfromnamestovaluesinarow."""
names=tables.split(',')
joined_rows=product(*[env[name]fornameinnames])
returnmap(lambdarows:make_env(rows,names),joined_rows)
defmake_env(rows,names):
"""Createanenvironmentofnamesboundtovalues."""
env=dict(zip(names,rows))
forrowinrows:
fornameinrow._fields:
env[name]=getattr(row,name)
returnenv
43
44
QueryPlanning
Conclusion
Themannerinwhichtablesarefiltered,sorted,and
joinedaffectsexecutiontime
Selecttheparentsofcurly-furreddogs:
DeclarativeProgramming
Make
RegularExpression
SQL
selectparentfromparents,dogs
wherechild=nameandfur='curly';
Fourdifferentpossibilities:
Joinallrowsofparentstoallrowsofdogs,filterbychild=nameandfur=
'curly'
Joinonlyrowsofparentsanddogswherechild=name,filterbyfur='curly'
Filterdogsbyfur='curly',joinresultwithallrowsofparents,filterbychild=
name
Filterdogsbyfur='curly',joinonlyrowsofresultandparentswherechild=
name
45
46/46