Listoflectures
TDDA69DataandProgramStructure
Macrosanddecorators
CyrilleBerger
1IntroductionandFunctionalProgramming
2ImperativeProgrammingandDataStructures
3Parsing
4Evaluation
5ObjectOrientedProgramming
6Macrosanddecorators
7VirtualMachinesandBytecode
8GarbageCollectionandNativeCode
9DistributedComputing
10DeclarativeProgramming
11LogicProgramming
12Summary
2/58
Howisaprograminterpreted?
Sourcecode
Parser
Parser
Treevisitor
AbstractSyntaxTree
Transformation
Generator
Sourcecode
Bytecode
VirtualMachine
Assembler
Assembly
Lecturecontent
MutableValues
Macros
Decorators
SyntacticMacros
MacrosinJavaScript
Implementation
Usecases
...
OperatingSystem
CPU
3/58
4/58
States
Anassignmentchangesthevalue
ofavariable
Allnamesareaffectedbythe
mutation
a=Object()
b=a
a.v=1
print(b.v)
MutableValues
6
Mutationwithinafunctioncall,always?(1/2)
Mutationwithinafunctioncall(1/2)
array=[1,2,3,4]
len(array)->4
f(array)
len(array)->2
deff(s):
s.pop()
s.pop()
(global)I
f:func(...)
(f)II
s->array
7
value=2
print(value)->2
f(value)
print(value)->2
deff(s):
s=1
(global)I
f:func(...)
(f)II
s:2
8
Mutationwithinafunctioncall,always?(2/2)
array=[1,2,3,4]
len(array)->4
f(array)
len(array)->4
deff(s):
s=list(s)
s.pop()
s.pop()
Mutationwithinafunctioncall(2/2)
(global)I
f:func(...)
(f)II
s:[1,2,3,4]
(global)I
array=[1,2,3,4]
len(array)->4
f()
len(array)->2
deff():
globalarray
array.pop()
array.pop()
f:func(...)
(f)II
9
Drawbacksofmutability
10
MutableDefaultarguments
deff(s=[]):
s.append(5)
return
len(s)
print(f())
print(f())
print(f())
pi=2
Whathappenifyoudothis?
deff(s=[]):
s.append(5)
returnlen(s)
print(f())
print(f())
print(f())
(global)I
f:func(...)
(f)II
s:[5]
11
s:[5,5]
(f)II
(f)II
s:[5,5,5]
12
Comparison(1/2)
Comparison(2/2)
InPython:
Valuecomparison
InJavaScript:
Valuecomparison
a=[1,2]
b=[1,2]
c=a
print(a==b)->true
print(a==c)->true
a=5
b="5"
Identitycomparison
console.log(a==b)->true
console.log(a===b)->false
Identitycomparison
print(aisb)->false
print(aisc)->true
13
Constantvalues(1/2)
14
Constantvalues(2/2)
['set'['x'10]]
defevaluate(node):
...
elifnode[0]=='set':
if(currentFrame.isConstant(node[1][0])):
throwError()
else:
currentFrame.set(
node[1][0],
evaluate(node[1][1]))
['set'['x'10]]
defevaluate(node):
...
elifnode[0]=='set':
currentFrame.set(
node[1][0],
evaluate(node[1][1]))
15
16
MutableFunctions
Afunctionwhose
behaviorvariesovertime
Example:
(global)I
withdraw:func(...)
defmake_withdraw(balance):
(make_withdraw)II
defwithdraw(amount):
balance=balance-amount
balance:25
returnbalance
returnwithdraw
withdraw=make_withdraw(100)
(withdraw)III
withdraw(25)->75
withdraw(25)->50
amount:25
withdraw(25)->25
Macros
17
Macros
Cpreprocessor(1/3)
Amacroisaruleorfunctionthat
mapaninputtoanoutput.
Example:Cpreprocessor,
decoratorsinPython,sweetjs,lisp
macro...
ThefirststepofcompilingaC
programis:
toreplacepreprocessingkeyword#include,
#define,#ifdef...
#include<stdio.h>
#defineMY_MACRO(x)(x)*(x)
...
andapply#definedmacros
19
20
Cpreprocessor(2/3)
Cpreprocessor(3/3)
#include<stdio.h>
#definePRINT(text) printf(text)
Ifnotcareful,theyleadtowrongorinconsistent
behaviour:
#defineoldfunc(a,b)newfunc1(a);newfunc2(b);
oldfunc(5,6)
=>newfunc1(5);newfunc2(6);
for(i=0;i<5;i++)oldfunc(5,6);
=>for(i=0;i<5;i++)newfunc1(5);
newfunc2(6);
#definemax(a,b)(a)<(b)?(b):(a)
max(f(1),f(2))
=>(f(1))<(f(2))?(f(2)):(f(1))
intmain(int_argc,constchar**_argv)
{
PRINT("HelloWorld!\n");
return0;
}
gcc-Etest.c-opreprocessed.c
21
22
Decorators
Decorators
Whataboutchangingthebehavior
ofafunctionwithoutchangingits
code?
24
Memoization(1/2)
Memoization(2/2)
Fibonacci:
Manuallyrewrittento:
deffib(n):
print(n)
ifn==0:
return0
elifn==1:
return1
else:
returnfib(n-2)+fib(n-1)
__fib_cache={0:0,1:1}
deffib_cached(n):
ifnin__fib_cache:
return__fib_cache[n]
else:
__fib_cache[n]=fib_cached(n-2)
+fib_cached(n-1)
return__fib_cache[n]
25
Memoizationdecorator(1/2)
defmemoization(func):
cache={}
defmemoized(*args):
ifargsincache:
returncache[args]
else:
val=func(*args)
cache[args]=val
returncache[args]
returnmemoized
fib=memoization(fib)
26
Memoizationdecorator(2/2)
Thefollowingis
@memoization
deffib(n):
returnnifn<2elsefib(n-2)+fib(n-1)
equivalentto
deffib(n):
returnnifn<2elsefib(n-2)+fib(n-1)
fib=memoization(fib)
27
28
NoconstantsinPython
Pythondoesnothaveconstants
ConstantsinPythonwithadecorator
defconstant(f):
deffget(self):
returnf()
returnproperty(fget,None)
class_Math(object):
@constant
defpi():
return3.141592653589793
math=_Math()
print(math.pi)->3.141592653589793
math.pi=2
print(math.pi)->2
Youcanfakethemwithproperties:
class_math:
defPI(self):
return3.141592653589793
pi=property(PI,None)
math=_math()
print(math.pi)
math.pi=2
print(math.pi)->3.141592653589793
math.pi=2->AttributeError:can'tsetattribute
29
Adjustabledecorators
30
Bound-checking
defbound_checking_decorator(min,max):
defmake_decorator(func):
defdecorator(x):
if(x<minorx>max):
raiseValueError()
returnfunc(x)
returndecorator
returnmake_decorator
@bound_checking_decorator(0,float('inf'))
deffib(n):
returnnifn<2elsefib(n-2)+fib(n-1)
Decoratorscantakearguments:
@decorator(args...)
deffunc(fargs...):
pass
isequivalentto
decorators(args)(func)
31
32
Chaindecorators
@bound_checking_decorator(0,float('inf'))
@memoization
deffib(n):
returnnifn<2elsefib(n-2)+fib(n-1)
SyntacticMacros
33
Syntacticmacros
Hygienicmacros
Hygienicmacrosaresyntacticmacros,wheremacrosanduser
environmentareseperate,sothatvariablecannotbecapturedby
macros
Cmacroworksatlexicallevel
SyntacticmacrosworksonAST
Exampleofcapture:
#include<stdio.h>
#defineADD(res,a,b,c)\
{\
intvalue=a;\
value+=b;\
value+=c;\
}
intmain(intargc,char**argv)
{
intvalue=0;
ADD(value,1,2,3);
printf("%i
",value);
return0;
}
MostlycommoninLisp-likelanguages
gcc-Etest2.c-opreprocessed.c
35
36
ASTManipulations
BuildingAST
importast
Givethepossibilitytowrite
functionsthattransforman
expressionintoanotherbeforeit
isevaluated
Macroexpansioncantakeplace:
node=ast.Expression(ast.BinOp(
ast.Num(2),ast.Mult(),
ast.Num(3)))
fixed=ast.fix_missing_locations(node)
Duringinterpretation
Duringcompilation
codeobj=compile(fixed,'<string>','eval')
print(eval(codeobj))
37
38
Incrementallnumbers
GetASTofafunction
importinspect
importast
defincrement(f):
source=inspect.getsource(f)
ast_tree=ast.parse(source)
ast_tree.body[0].decorator_list=[]
classT(ast.NodeTransformer):
defvisit_Num(self,node):
node.n+=1
returnnode
exec(compile(T().visit(ast_tree),__file__,mode='exec'))
returnlocals()[ast_tree.body[0].name]
deffunc(x):
returnx+1
print(func(1))
source=inspect.getsource(func)
ast_tree=ast.parse(source)
ast_tree.body[0].body[0].value.right.n=2
exec(compile(ast_tree,__file__,mode='exec'))
@increment
deffunc(n):
return(2+n)*3
print(func(1))
print(func(1))
39
40
Template(1/2)
Template(2/2)
defapply_template(template):
deft(f):
f_ast=ast.parse(inspect.getsource(f)).body[0]
body_node=f_ast.body[0]
template_ast=ast.parse(inspect.getsource(template))
template_ast.body[0].args=f_ast.args
classT(ast.NodeTransformer):
defvisit_Expr(self,node):
if(node.value.id=='__body__'):
returnbody_node
else:
returnnode
exec(compile(T().visit(template_ast),__file__,mode='exec'))
returnlocals()[template_ast.body[0].name]
returnt
defmy_template():
forxinrange(1,10):
__body__
returnv
@apply_template(my_template)
deffunc(v):
v=v*x
deffunc(v):
forxinrange(1,10):
v=v*x
returnv
41
42
WhenareASTmanipulationsuseful?
InPython,ASTmanipulationsareseldomused
Functionsdecoratorscovermostoftheneed
Butsometimemacroswillworkbetter
Ifthereisaneedtocontrolfunctionreturn
Toapplymodifieronclassesorexpressions(decoratorsonlywork
onfunctions)
Theycanbeusedforoptimization,suchastail-calloptimization
Whencallingafunction,allargumentsareexecuted
...onlyonce
Theycansavefunctioncall
MacrosinJavaScript
ButASTmanipulationsareobfuscated
PythonASTmoduleAPIcanchangefromversiontoversion
Nogoodsyntaxtodefinerules
43
sweet.js
Definingrules
macro<macroname>{rule{<ruledefinition>}=>{<output>},rule{...}...}
Example:
Doesnotexistsinthestandard
sweet.js(http://sweetjs.org/)isa
JavaScriptlibrarythatbrings
supportforhiegenicmacroto
JavaScript
macroidentity{
rule{
($x)
}=>{
$x
}
}
identity(42);
identity([1,2,3]);
45
46
Exampleofhygienicsyntax
macroswap{
rule{
($x,$y)
}=>{
vartmp=$y;
$y=$x;
$x=tmp;
}
}
Implementation
varfoo=100;
varbar=200;
vartmp='myothertemporaryvariable';
swap(foo,bar)
47
Compiler
Tokentree
Inscheme/lisp
Tokensfor'{42}'are['{','42',
'}']
Tokentreematchdelimiters:
lexer-(token)→reader-(s-expression)→
expander/parser-(AST)→
CommonlyinJavaScript
['{}',['42']]
lexer←(feedback/token)→parser-(AST)→
Feedbackisneededtodistinguishdivision(//)
fromregularexpressions(//x//).
Insweetjs
lexer-(token)→reader-(token-tree)→
expander/parser-(AST)→
49
Reader
50
Expander
Thereaderconvertastreamof
tokensintoatoken-tree
todistinguishbetweendivisionand
regularexpression
Theexpandervisitthetree:
Iftokenisanidentifierandtheitmatchesone
ofthemacro
Thencheckwhichruleapply
Thenapplythetransformationspecifiedinthe
rule
Ifprevioustokenisanoperator('+','-'...),then
'/'isthebeginingofaregularexpression
Ifprevioustokenisaparenthesis(')'),then'/'is
adivision,unlessbeforetheparenthesiswas
thekeyword'if'('if(x)/}/)
...
51
52
ClassinJavaScript/sweejs(1/2)
InJavaScript6
classPerson{
constructor(name){
this.name=name;
}
say(msg){
console.log(this.name+'
says:'+msg);
}
}
varbob=newPerson('Bob');
bob.say('Macrosaresweet!');
Usecases
InJavaScript<6
functionPerson(name){
this.name=name;
}
Person.prototype.say=
functionsay(msg){
console.log(this.name+
'says:'+msg);
};
varbob=new
Person('Bob');
bob.say('Macrosare
sweet!');
54
ClassinJavaScript/sweejs(2/2)
macroclass{
rule{
$className{
constructor$cparams$cbody
$($mname$mparams$mbody)...
}
}=>{
function$className$cparams$cbody
$($className.prototype.$mname
=function$mname$mparams$mbody;)...
}
}
sweet-contracts
Macrocanbeusedtoimplement
contract(sweet-contract):
function(Num)->Num
functiondbl(n){
returnn+n;
}
function({name:Str,age:Num},[Str,Num...])>(NumorBool)
functionavg_score(person,scores){
...
}
55
56
MarosAdvantages/Inconvenients
Conclusion
Advantages
Mutablevariablesandfunctions
Decorators
MacrosandASTtransformations
Allowtoextendthelanguage
Canhelpcodereusability
Incovenients
Needtobewelldesignedandwellsupported
bythelanguage
Introducedialects,whichmightmakethecode
hardertoread
Non-standard
57
58/58