53
Flanders Environment Report
M I R A - P E 2OO5
i
POLICY
EVALUATION
F landers E n v iro n m e n t R epo rt:
p o lic y e v a lu a tio n
C h a irm a n :
P r o je c tm a n a g e r:
Rudi Verheyen (IM)
M a rleen Van Steertegem
S e c re ta ry :
T e a m m e m b e rs :
P h ilip p e D’ H o n d t (VMM)
M yria m Bossuyt
M e m b e rs f o r th e F le m is h C o u n c il
C aro line De Geest
lo h a n B rouw ers
f o r S cie nce P o lic y :
H anne Degans
P atrick M eire (IM)
S ofie Janssens
Chris VinckierfK.LL Leuven)
S tijn O verloop
Bob Peeters
M e m b e rs o f th e B o a rd o f
Lisbeth S talpaert
S e c re ta rie s -g e n e ra l:
Barbara T ie le m an
Veerle Beyst (APS)
H ugo Van H ooste
Ludo V anongeval (AMINAL)
Jeroen Van Laer
Els van W alsum
M e m b e rs o f th e F le m is h N a tu re
a n d E n v iro n m e n t C o u n c il:
Luc Goeteyn (M iN a Council)
) a n Tu r f (Bond Beter Leefmilieu vzw)
M e m b e rs f o r th e F le m is h
S o c io -E c o n o m ic C o u n c il:
A n n e m ie B ollen (SERV)
Peter Van H um beeck (SERV)
In d e p e n d e n t e x p e rts :
R ikAm pe(Vito)
Jeroen Cockx (GMO)
Geert De B lust (IN)
M yria m Dumortier(NARA, IN)
Rudy H erm an (AWI)
Erika V ander P utten
114618
Flanders Environment Report:
policy evaluation
M IR A -P E 2OO5
P resented o n 21 O c to b e r 2005 t o Kris Peeters,
F le m is h m in is te r f o r P u b lic W o rk s , E n e rg y , th e E n v iro n m e n t a n d N a tu re
M a rle e n Van S te e rte g e m , fin a l editing
' A
V L I 2 (vzw)
VLAAMS INSTITUUT VOOR DE ZEI
F L A N D ER S M A R IN E IN S T IT U T E
Oostende - Belgium
VLAAMSE MILIEUMAATSCHAPPIJ
M arleen Van Steertegem (fin a l editing)
FLANDERS EN VIR O N M ENT REPORT: POLICY EVALUATION
M IR A -P E 2O O 5
72 pp., 24.5 cm
D/2006/6871/008
© 2006, Vlaamse Milieumaatschappij, Aalst (Belgium)
The use o f texts fro m the report w ith acknowledgement to the source
is encouraged.
Layout and cover design
SignBox, Peter De Roy 81 Betty Reyniers
Typesetting
Ourtype Versa
Linguistic correction
Stein Pée
Translation
Oneliner Language 81eBusiness Solutions
bvba
(Belgium)
Printed by
Sintjoris nv, Merendree (Belgium)
Contact
Flemish Environment Agency - m i r a
Van Beneden laan 3 4 ,
tel:
B -28 00
(v m m )
Mechelen, Belgium
+ 3 2 15 451 4 6 6 1 fax: + 3 2 15 4 33 2 80
mira(a)vmm.be
www.milieurapport.be
Printed on 50 %post-consumer recycled paper, chlorine-free
and w ith o u t optic bleach.
The illustratio ns on the cover and div id in g pages are adaptations o f pages
from the Linnean Herbarium a t the Swedish Museum fo r N atural History in
Stockholm.
M I RA - P E
2005
FOREWORD
‘ W h a t can w e learn fro m p o lic y e v a lu a tio n s ? ’ is th e tit le o f a n e w p u b lic a tio n o f th e
5
S w edish E n v iro n m e n ta l P ro te c tio n A gency, in w h ic h a ro u n d tw e n ty e n v iro n m e n ta l
p o lic y e v a lu a tio n s are re p o rte d and d iscu sse d 1. T h e t it le p e rfe c tly s u m m a rise s th e
i Swedish Environmental
p u rp o s e o f p o lic y e v a lu a tio n s : to learn fro m th e m ; to d is c o v e r m e ch a n ism s th a t w o rk
Protection Agency (2004)
o r id e n tify p ro b le m s in o r d e r t o solve th e m .
21 evaluationsw hat do they teach us?
B efore y o u lies th e la te s t p o lic y e v a lu a tio n re p o rt o f th e Fle m ish E n v iro n m e n t A gency,
Naturvdrdsverket, Stockholm
in s h o rt MIRA-PE 2005. T h e re p o rt o ffe rs an o v e rv ie w o f 24 e v a lu a tio n s tu d ie s o f th e
(www.naturvardsverket.se)
F le m ish e n v iro n m e n ta l p o lic y c a rrie d o u t d u rin g th e past tw o years. It in clu d e s:
in s tru m e n ts , p o lic y m easures, ta rg e t g ro u p s , e ffe cts, sid e e ffe cts, ju s tic e , k n o w le d g e ,
p o w e r, u n w illin g n e s s , a m b itio n ... In s u m m a ry , th e re p o rt is a b o u t (e n v iro n m e n ta l)
p o lic y and h o w th is p o lic y w o rk s (o r do es n o t w o rk ).
T his is th e second MIRA-PE re p o rt. It is its e lf p a rt o f a le a rn in g process in th e shape o f a
ste p -b y -s te p pla n . MIRA-PE 2003 c o n ta in e d an in tro d u c tio n to p o lic y con ce p ts and fo u r
in -d e p th e v a lu a tio n s o f aspects re la ted to th e w a te r and w aste p o licy. They w e re con ceived
a sa basis fo r a m o re exte nsive research p ro g ra m m e re g a rd in g p o lic y e v a lu a tio n o n th e o n e
hand and fo r th e c re a tio n o f an e xp e rtise n e tw o rk in Flanders o n th e o th e r. MIRA-PE 2005
b u ild s o n th is : it o ffe rs five in -d e p th e v a lu a tio n s w h ic h re fe r to five c o m p o n e n ts o f th e
p o lic y ch a in . B ut th is la te s t re p o rt goes even fu rth e r. An in v e n to ry o f th e key e v a lu a tio n
stu d ie s o n th e Flem ish e n v iro n m e n ta l p o lic y pro v id e s m o re in s ig h t in to th e flaw s o ft h e
system . M o re o ve r, less w e ll-k n o w n e n v iro n m e n ta l p o lic y areas are g ive n a tte n tio n as w e ll.
In a d d itio n , th e e x p e rtis e n e tw o rk is fu r th e r e x p a n d e d . T h e MIRA-PE process a n d (th e
p u b lic a tio n s o f) th e C entre o f E xpertise o n E n v iro n m e n ta l P olicy S ciences, a m o n g
o th e rs , have c o n trib u te d to a g re a te r fa m ilia rity w ith th e a c tiv itie s o f researchers fro m
d iffe r e n t research c e n tre s . M o re o v e r, th e se researchers w e re p ro v id e d w ith a fo r u m in
w h ic h to e x ch ange ideas an d c o o p e ra te . S om e e x p e rts c o m m itte d th e m s e lv e s as a u th o rs
and researchers o f a M IR A c h a p te r, o th e rs chose to p ro v id e th e MIRA-PE a u th o rs w ith th e
e sse ntial (q u a n tita tiv e and q u a lita tiv e ) d a ta. In fact, h ig h -q u a lity a nd o b je c tiv e p o lic y
e v a lu a tio n s re q u ire th e c o o p e ra tio n o f p e o p le in v o lv e d in th e p o lic y . T h e ir e n th u s ia s m
a nd e ffo rts are e x tre m e ly v a lu a b le , in s tru c tiv e a nd an exa m p le .
We w o u ld lik e to th a n k ev e ry o n e : th e MIRA te a m fo r its c o o rd in a tio n o f th is re p o rt, th e
a u th o rs fo r th e ir research and th e ir c o m p e llin g s ynth esis, th e re viso rs fo r th e ir o p in io n s
a nd s u g g e s tio n s and th e MIRA S te e rin g G ro u p fo r its c ritic a l re fle c tio n and c re a tiv ity .
O nce m o re , a h ig h -q u a lity re p o rt has been p ro d u c e d , th a n k s to th e e ffo rts o f m any.
It is said th a t e x p e rie n c e is th e b e st tea cher. T his is c e rta in ly tru e fo r re p o rtin g o n
e n v iro n m e n ta l p o lic y e v a lu a tio n . In o th e r c o u n trie s , m o re and m o re re p o rts o n p o lic y
e v a lu a tio n are p u b lis h e d . H ow eve r, even th e re th e tr a d itio n is lim ite d an d ‘ best
p ra ctic e s ' are scarce. M IR A b e lo n g s to th e (se le ct) c lu b o f lead ers in th is respect.
U n fo rtu n a te ly , lead ers o fte n have to search fo r th e rig h t w ay: n o ta n easy task. We
are h a p p y to ta k e o n th e c h a lle n g e .
Frank Van Sevencoten
RudiVerheyen
A d m in is tra to r-g e n e ra l
C h a irm a n o f th e M IR A
o f th e F lem ish E n v iro n m e n t A gency (VMM)
S teering C roup
September 2005
M I RA- PE
C O N T E N T S
2005
S u m m a ry
g
A bout
MIRA-PE
il
1
W h a t is MIRA-PE?
2
W h a t can y o u fin d in th is re p o rt?
12
3
W h a t is th e fu n c tio n o f MIRA-PE?
12
4
W h e re d o w e w a n t to g o?
5
W h a t are ty p ic a l e le m e n ts o f MIRA-PE?
6
W h o is in v o lv e d in MIRA-PE?
7
W h o is MIRA-PE a im e d at?
R eferences
ti
14
17
18
18
En v i r o n m e n t a l
T H E
15
M I C R O S C O P E :
In tro d u c tio n
p o l i c y
24
2003-2005
E V A L U A T I O N S
21
1
In v e n to ry
22
2
D iscussion
3
E v aluatio n s u m m a rie s
26
38
A nne x: In v e n to ry re fe re n ce lis t 2 0 0 3 -2 0 0 5
R eferences
R evisors
64
65
65
E xp la n a to ry lis t o f a b b re v ia tio n s
69
A b o u t th e F lem ish E n v iro n m e n t A gency
File o n Flanders
71
70
u n d e r
21
(PE s ta n d in g fo r 'P o lic y E v a lu a tio n ’). P olicy e v a lu a tio n is d e fin e d as fo llo w s : th e scie n ­
t if ic a n alysis o f a s p e c ific p o lic y th e m e w h ic h is e v a lu a te d o n th e basis o f c rite ria and
serves as a s ta rtin g p o in t f o r th e fo r m u la tio n o f re c o m m e n d a tio n s .
MIRA-PE 20 05 o ff e r s - a fte r a c h a p te r w ith m o re in fo rm a tio n o n th e ch a ra cte ristics o f th is
r e p o r t - a n o v e rv ie w o f th e e v a lu a tio n s tu d ie s o f(s u b )a re a s o f th e F lem ish e n v iro n m e n t
p o lic y p u b lis h e d b e tw e e n 1 July 2003 and 31 M ay 2005. W ith th is in v e n to ry , MIRA-PE 2 0 05
fu lfils th e o b je c tiv e o f b e in g a w o rk o f re fe re n ce w ith regard to p o lic y e v a lu a tin g s tu d ie s
in Flanders. A fte r a dis c u s s io n o f th e in v e n to ry , th e re is a s u m m a ry o f th e 24 p o lic y e v a lu ­
a tio n s s tru c tu re d in file s . T h e s u m m a ry s y s te m a tic a lly s h o w s th e m a tte r th a t was
re searched, th e m e th o d , th e fin d in g s and th e re c o m m e n d a tio n s o f t h e p o lic y e v a lu a tio n .
The d is c u s s io n o f t h e in v e n to ry in c lu d e s , in th is o rd e r: th e ty p e o f p o lic y , th e ty p e o f
in s tru m e n t, th e ty p e o f e v a lu a tio n , th e e n v iro n m e n ta l th e m e s (lin k e d t o th e Flem ish
R e g io n ’s e n v iro n m e n ta l s p e n d in g ), th e e v a lu a tio n c rite ria used, th e o r d e rin g p a rty and
th e e v a lu a tin g in s titu tio n s . O n e o f t h e fin d in g s is th a t a ll p o lic y and in s tr u m e n t typ e s are
researched m o re o r less e q u a lly . O n ly th e c o o rd in a tin g (e n v iro n m e n ta l) p o lic y is
researched less. S econ dly, a tte n tio n is p a id e s p e c ia lly t o e x -p o s t and e x -n u n c e va lu a ­
tio n s and process an d p ro d u c t e v a lu a tio n s . In g e n e ra l, e n v iro n m e n ta l th e m e s fo r w h ic h
a lo w e r b u d g e t is a v a ila b le are e va lu a te d less. O ne o f t h e e x c e p tio n s is th e w a te r p o lic y ,
w h ic h a b s o rb s m o re th a n h a lf o f th e e n v iro n m e n ta l b u d g e t, b u t is e va lu a te d re la tiv e ly
little . F u rth e rm o re , o n ly a lim ite d n u m b e r o f e v a lu a tio n s ta lk a b o u t e ffe ctive n e ss. The
g re a t m a jo rity o f t h e e v a lu a tio n s r e fe r to th e m u tu a l c o o rd in a tio n , in te g ra tio n , k n o w l­
ed ge an d c a p a c ity a m o n g th e p e o p le in v o lv e d in th e p o lic y e tc. M o s t p o lic y e v a lu a tio n s
w e re o rd e re d by th e g o v e rn m e n t. M o s t o f t h e researchers w h o ca rrie d o u t th e e va lu a ­
tio n s w o rk fo r a u n iv e rs ity .
T h e d is c u s s io n als o in v e s tig a te s w h e th e r th e re is a ny c o rre s p o n d e n c e b e tw e e n th e
in v e n to rie d p u b lic a tio n s w ith regard to m e th o d o lo g y o r c o n te n t, w h e th e r th e y are n e u ­
tra l and in d e p e n d e n t, w h e th e r th e y are o f h ig h q u a lity and w h ic h im p a c t th e y c o u ld have
o n th e fo r m u la tio n o f a p o lic y . From th e in v e n to ry , it b e com es cle a r th a t th e re are a
n u m b e r o f h o t to p ic s : several s tu d ie s researched e n fo rc e m e n t, th e in te rn a tio n a l e n v i­
ro n m e n ta l p o lic y a n d th e ro le o f t h e F le m ish R egion in it an d th e in te g ra tio n o f t h e v a ri­
o u s g o v e rn m e n t b o d ie s . T h e e ffe c t o f t h e s tu d ie s o n th e p o lic y is v a ria b le : w e s u sp e ct
th a t th e e v a lu a tio n s in itia te d by th e g o v e rn m e n t its e lf c o u ld have q u ite a b ig im p a c t,
p r o v id in g th e y are c a rrie d o u t by th e p e o p le in cha rge o f t h e p o licy.
2005
MIRA-PE is th e b ia n n u a l p o lic y e v a lu a tio n re p o rt o f th e Fle m ish E n v iro n m e n t A gency
M I RA- PE
Summary
About MIRA-PE
.. What is MIRA-PE?
MIRA-PE Is th e F le m ish E n v iro n m e n t A g e n c y ’s b ia n n u a l re p o rt o n th e e v a lu a tio n o f t h e
Fle m ish e n v ir o n m e n t p o lic y (PE stan ds fo r ‘ P olicy E v a lu a tio n ’).
P o licy e v a lu a tio n is d e fin e d as fo llo w s in th is re p o rt: th e s c ie n tific an alysis o f a s p e c ific
p o lic y th e m e w h ic h is e v a lu a te d o n th e basis o f c rite ria an d serves as a s ta rtin g p o in t
f o r t h e fo r m u la tio n o f re c o m m e n d a tio n s .
The c o n te n t o f MIRA-PE is d e te rm in e d by p o lic y th e m e s . A p o lic y th e m e is (p a rt o f a) p o licy
area (e.g. w aste p o licy) and can re fer to o n e o r m o re in s tru m e n ts a n d /o r o n e o r m o re
e n v iro n m e n ta l issues fo r w h ic h a po lic y is carried o u t.
MIRA
M oreover, it can be a b o u t th e agenda-setting, objectives,
p la n n in g , e x e c u tio n , e tc. o f th a t p o lic y . O th e r areas
MIRA stands for Flanders Environment Report ('Milieurapport
in v e s tig a te d by MIRA-PE are th e c o n s e q u e n c e s o f t h e
Vlaanderen' in Dutch). MIRA is coordinated by the Flemish
p o lic y o n th e b e h a v io u r o f t h e ta r g e t g ro u p s (o u tc o m e )
Environment Agency (VMM) which has hired a project team for
a n d /o r o n th e e n v ir o n m e n t (e n v iro n m e n ta l e ffe c t).
this purpose. MIRA’s mission has been determined by decree and
The c o m in g in to b e in g o f MIRA-PE is q u ite a lo n g story.
is threefold:
Even th o u g h a lre a d y in 1995 th e re g io n a l le g is la to r
• a description, analysis and evaluation o fth e existing state o f
o rd e re d th e e n v iro n m e n ta l re p o rtin g d e p a rtm e n t to
m ake p o lic y e v a lu a tio n s , it w as n o t u n til 2 0 0 0 th a t th is
ta s k w as re a lly s ta rte d . T h e re fo re , th e firs t e n v iro n m e n ­
ta l re p o rts d id c o n ta in c ritic a l analyses o f th e e n v iro n ­
the environment;
• an evaluation ofthe environmental policy conducted up to
now;
• a description o f the developments expected in the environ­
m e n t p o lic y , b u t th e y lacked in -d e p th e v a lu a tio n s . In
ment in case o f unchanged as well as amended policy in terms
M ay 2 0 0 0 , th e th e n m in is te r o f t h e E n v iro n m e n t asked
o f a number o f scenarios that are thought to be relevant.
th e c h a irm a n o f t h e M IR A s te e rin g g ro u p to re p o rt sys­
The study ofthe state o f the environment is presented in the
te m a tic a lly o n in -d e p th p o lic y e v a lu a tio n s . The s te e rin g
annual MIRA-T reports. In 2000, the first full scenario report
g ro u p o p te d fo r a s te p -b y -s te p p la n , s ta rtin g by d ra w in g
(MIRA-S 2000) was published. The first policy evaluation report
u p an in v e n to ry o f e x is tin g p o lic y e v a lu a tio n s and
(MIRA-PE) was published in |une 2003.
m e th o d s .T h e s te p -b y -s te p
p la n
had to
lead to th e
c re a tio n o f an e x p e rtis e n e tw o rk (see p a ragraph 4 W here
All publications, figures and indicators from MIRA can be consulted
d o w e w a n t to go?).
on the website www.milieurapport.be (available in English).
The results o f this inventory and o f some in-depth policy evaluations was the subject o f
th e firs t BE re p o rt in 2003.
T h e
g
o
v
e
r
n
m
e
n
t
a n d
I T S
I N S T R U M E N T S
A steering policy instrument denotes an instrument used by
is aimed at the creation o f a specific situation.
the government to persuade, force or encourage a target group
Examples: the construction o f sewers, the collection o f waste, the construction
or another authority to implement a measure or to enable itself
o f roads, the construction o f container parks, the plan tin g o f a row o f trees..,
to take measures.
Examples: Royal Decrees, decrees, covenants, licences, h ie s , subsidies,
awareness campaigns, p revention...
A direct policy instrument denotes an effective activity o f the
Supporting instruments help w ith the choice o f steering and
direct policy instruments.
Examples: planning, market creation, research and development,
m onitoring, inform ation m anagem ent...
government which intervenes in the physical environment and
2
.
What can you find
in this report?
T h is E nglish v e rs io n o f MIRA-PE c o n ta in s an in v e n to ry a nd analysis o f t h e e v a lu a tio n
s tu d ie s o f t h e F le m ish e n v ir o n m e n t p o lic y p u b lis h e d b e tw e e n 1 July 20 03 and 30 M ay
2005. T h e D u tch v e rs io n c o n ta in s an a d d itio n a l p a rt c o n s is tin g o f fiv e in -d e p th p o lic y
e v a lu a tio n s , se le cte d fro m th e in v e n to ry a n d d ra w n u p a c c o rd in g to a fix e d s tru c tu re .
These ch a p te rs have be en se le cte d in such a w ay th a t each o n e e va lu ates a d iffe re n t
e le m e n t in th e p o lic y c h a in .
3. What is the fu n c tio n
ofMIRA-PE?
W ith MIRA-PE 20 05, w e w a n t to c o v e r th re e d iffe re n t o b je c tiv e s : re p o rtin g o n
e n v ir o n m e n t p o lic y s tu d ie s , fo llo w in g u p th e e n v iro n m e n t p o lic y p la n n in g and
b u ild in g u p e x p e rtis e .
A W O R K OF REFERENCE
MIRA-PE 20 05 w a n ts to c o n s o lid a te its fu n c tio n o f a w o rk o f re fe re n ce . M o re th a n in
MIRA-PE 2003, w e o ffe r an o v e rv ie w , a sta te o f a ffa irs o f t h e e n v iro n m e n ta l p o lic y
e v a lu a tio n s in th e F le m ish R egion.
POLICY PLANNING
MIRA-PE a im s a t o ffe r in g a d d e d va lu e fo r th e fo llo w - u p o f t h e e n v iro n m e n ta l P olicy Plan
by c o m m e n tin g o n th e m easures in c lu d e d in th e pla n . T h e re fo re , th e re p o rt m u s t f it th e
p o lic y m a k e rs ’ k n o w le d g e re q u ire m e n ts as c lo s e ly as p o s s ib le , i.e . it m u s t be as p o lic y ­
re le v a n t as p o s s ib le . A m o n g o th e r th in g s , w e have a d ju s te d th e d iv is io n o f t h e p o lic y
e v a lu a tio n s as m u c h as p o s s ib le to th e la te s t E n v iro n m e n ta l P olicy Plan (M IN A -P la n 3).
MIRA-PE w a n ts t o h e lp s u p p o rt th e m a n a g e m e n t cycle as p re se n te d in th e M IN A -p la n ,
c o n c re te ly by re p o rtin g da ta fo r th e e v a lu a tio n o f t h e p o lic y ( fig u re 1).
Figure 1: M a n a g e m e n t cycle in M IN A -P la n 3 and th e place o f MIRA-PE in it
■>
Policy preparation
Policy evaluation
Policy implementation
MIRA-PE
Source: E n v iro n m e n ta l Policy Plan 2003-2007, p. 322
L E A R N I N G P ROCESS
MIRA-PE 20 05 a im s a t c o n tr ib u tin g to th e le a rn in g process re la te d to p o lic y e v a lu a tio n .
T h is le a rn in g process, w h ic h is an u p c o m in g success in th e Fle m ish p o lic y p ra ctice and
esp e cia lly in th e e n v ir o n m e n t a d m in is tra tio n , p ro v id e s re fle c tio n o n th e p o lic y a n d raises
a n u m b e r o f q u e s tio n s . Are w e o n th e rig h t tra ck? W h a t m e th o d s are u se fu l? Q u e s tio n s
can also be asked re g a rd in g th e use fu ln e ss o f t h e p o lic y e v a lu a tio n s . Can th e p o lic y
e v a lu a tio n s p re s e n te d he re o n ly be a p p lie d to th e ir s p e c ific o b je c t? O r can c o n c lu s io n s
an d m e th o d s also be used in o th e r p o lic y areas?
N o t s u rp ris in g ly , each p o lic y th e m e has a n u m b e r o f s p e c ific ch a ra cte ristics w h ic h m ake
g e n e ra lis a tio n s to o th e r p o lic y areas d iffic u lt. For in sta n ce , each p o lic y area has its o w n
h is to ry , acto rs, s p e c ific in s titu tio n s , le g is la tio n , p o litic a l s e n s ib ilitie s , etc. Flowever,
th e m e th o d s , e v a lu a tio n c rite ria and a s s u m p tio n s in th e p re s e n t re p o rt can be a p p lie d
to o th e r (e n v iro n m e n ta l) p o lic y areas. Lessons w e can d ra w fro m p o lic y e v a lu a tio n s in a
p a rtic u la r p o lic y area m ay also be v a lid fo r o th e r p o lic y areas. T h e re fo re , th e in fo rm a tio n
in th is re p o rt can be used to a n s w e r th e fo llo w in g ‘ u n iv e rs a l’ p o lic y q u e s tio n s :
• Fiow can w e d e te rm in e th e e ffe c t o f a p ro g ra m m e o n th e ta rg e t g ro u p ?
• W h a t d iffic u ltie s can a c c o m p a n y th e d e v e lo p m e n t o f a re g io n -o rie n te d in te g ra te d
p o lic y and h o w can w e m a n a g e th e m ?
•
H o w can w e p re d ic t w h e th e r a re o rg a n is a tio n o f services m ig h t be u se fu l?
•
H o w can th e o b se rva n ce o f law s be g u a ra n te e d o n c e th e y have e n te re d in to force?
•
H o w can E u ro pean (in te rn a tio n a l) le g is la tio n be tra n s p o s e d su cce ssfu lly in to th e
F le m ish c o n te x t? W h a t are th e c o n d itio n s f o r success?
• H o w can th e im p le m e n ta tio n o f a p ro g ra m m e be o p tim iz e d ? W h a t are th e ty p ic a l
p ro b le m s ?
• W h a t are success fa c to rs in th e in te g ra tio n o f p o lic y levels (e.g. fro m E u ro pean to
Fle m ish , re g io n a l to lo cal le ve l)?
• H o w can E uro pean d ire c tiv e s be tra n s p o s e d c o rre c tly fro m a legal p o in t o f vie w ?
4. Where do we want to go?
For th e re a lis a tio n o f its s ta tu to ry p o lic y e v a lu a tio n m is s io n , MIRA w o rk s w ith a s te p -b y ste p p la n . A firs t s te p co n s is ts o f d ra w in g u p an in v e n to ry o f e x is tin g research in Flanders
an d c o n tin u in g research o n c e rta in p o lic y th e m e s . These e v a lu a tio n s p ro v id e th e basis
fo r th e n e x t s te p , in w h ic h m o re e x te n s iv e and m o re g e n e ra l analyses and assessm ents
o f e n v iro n m e n ta l p o lic y areas are m a de. T his is p o s s ib le by m o b ilis in g and s tru c tu rin g
an e x p e rtis e n e tw o rk f o r p o lic y e v a lu a tio n s . T h is process is c u rre n tly in p rogre ss and
MIRA-PE 20 05 w a n ts to c o n trib u te t o it.
By m eans o f a m a trix w e o ffe r y o u a v isual o v e rv ie w o fw h ic h p o lic y th e m e s have been
tre a te d in MIRA-PE 20 03 and MIRA-PE 20 05 a n d , th e re fo re , w h ic h th e m e s s till ne ed to
be co v e re d (ta b le i) .T h e n u m b e r o f 'e m p t y ' c e lls speaks fo r its e lf. W e re fe ry o u to th e
in v e n to ry (n e x t c h a p te r) o f t h e p re s e n t re p o rt fo r fu r th e r c o m m e n ts o n th is m a trix and
th e c o rre s p o n d in g c o n c e p ts .
T a ble 1: M a trix o f e v a lu a te d p o lic y to p ic s in MIRA-PE 2 0 03 a n d MIRA-PE 2005
policy type*
environm ental topic
region
international
integration
target group
coordination
general
climate
waste
manure
nature
water
photochemistry
ozone layer
acidification
hazardous substances
soil
nuisance
fragmentation
’
region = re g io n -o rie n te d policy; in te rn a tio n a l = in tern a tio n a l policy; in teg ratio n = integrated g o v e rn m e n t policy; ta rg e t g ro u p = targ e t g ro u p policy;
c o o rd in a tio n = c o o rd in a tin g policy
T
P o l i c y
TO
T H E
y p e s
e v a l u a t i o n
o f
p o l i c y
e v a l u a t i o n
P o l i c y
a c c o r d i n g
e v a l u a t i o n
a c c o r d i n c
t o
t i m e
S U B IE C T
plan evaluation: review o f plans and/or goals before or after
policy
their implementation
■ ex-nunc evaluation: evaluation during implementation o f the
policy (also called ex-interim)
• ex-post evaluation: evaluation after implementation o f the
|
product evaluation: review o f the policy results
policy
ABOUT
MIRA-PE
5. What are typical elements o f
MIRA-PE?
T h e re p o rts o n th e S tate o f t h e E n v iro n m e n t in th e Fle m ish R egion (M IR A-T) are d ra w n
up o n th e basis o f in d ic a to rs an d have a fix e d fra m e w o rk fo r analysis: th e d is ru p tio n
ch a in o r DPSIR c h a in . MIRA-PE also c o n ta in s such re c u rrin g e le m e n ts , c o n c re te ly th e
p o lic y c h a in as a fra m e w o rk f o r analysis, th e e v a lu a tio n c rite ria and th e fix e d s tru c tu re .
First, w e w ill ta lk a b o u t th e p o lic y c h a in .
T H E P O L I C Y C H A I N AS A F R A M E W O R K
FOR ANALYSI S
The p o lic y c h a in in d ic a te s th e p o lic y process (fro m th e s ta rt o f t h e p o lic y u p to its
re su lts ). It a llo w s us to u n ra v e l th e c o m p le x ity o f ‘ th e ’ p o lic y and serves a sa g u id e fo r a
g o o d u n d e rs ta n d in g o f t h e p re s e n t re p o rt. Figure 2 visualise s th e p o lic y c h a in .
Fig ure 2: The p o lic y c h a in as a fra m e w o rk f o r an alysis fo r MIRA-PE
social and
socio­
p o litic al issues
econom ic
factors
European legislatio n
e n v ir o n m e n t d i s r u p t io n c h a in
p o lic y pro c e s s
plan
program m e
w ith
and
objectives
budget
operationalisation
Source: W ilkin so n (1999).
o u tc o m e
o utp ut
en fo rcem en t
b eh avio u r o f
targ et groups
en viron
m e n ta l effect
pressu re-s itu a
t i o n - im p act
im p lem en ta tion
2005
process evaluation: review o f policy processes, what concerns
the development o f the policy as well as its implementation
MIRA-PE
• ex-ante evaluation: evaluation before implementation o f the
environm ental processes
T
h e
p h a s e s
i n
t h e
p o l i c y
c h a i n
(
f i g u r e
2)
The chain starts with one or more social and political issues
This output is related to the environmental disruption chain or
which are put on the agenda o f the policy makers. This starts
DPSIR chain which is also displayed in figure 2. People adjust
a process, the policy process, which consists o f three phases.
their behaviour, which contributes indirectly to a decrease in
Each phase leads to a concrete result (displayed in a circle).
environmental pressure, leading in turn to an improved
The first phase is the planning phase. Solutions for the social
environmental situation and possibly havinga positive impact on
and political issues are designed, compared and discussed. This
people, nature and the economy. The output can also influence the
should normally result ina plan with clear objectives. In the
environment directly, without the need for a change in behaviour.
second phase, the operationalisation phase, the plans become
However, apart from the policy chain external factors play a role
more concrete: a programme is drawn up in keeping with the
budget in order to ensure optimal implementation with the
as well. These are harder to control with a policy but their
influence on environmental disruption is just as big - or even
available human and financial resources. After that, the plan is
(a lot) bigger. These factors must not be forgotten because they
implemented (phase 3). In the field, we need to see a concrete
demonstrate the government's fallibility when it comes to
output o f this: e.g. a planted row o f trees, a granted licence, a
controlling social processes and therefore the environment.
builtw ind park, an imposed fin e ...
By m eans o f a rro w s , fig u re 2 s ho w s th e causal c o n n e c tio n s b e tw e e n th e c o m p o n e n ts
o f t h e p o lic y a nd d is ru p tio n cha in s w ith o u t d e s c rib in g th e m e x p lic itly . T h is a p p ro a ch
can o n ly be used w h e n th e re is a p o lic y th e o ry w h ic h sh o w s causal re la tio n s (e.g. p o lic y
in s tr u m e n t w ith o u tp u t x leads to im p ro v e m e n t in th e w a te r q u a lity y, w h ic h re su lts in
a re covery o f th e fis h p o p u la tio n by z %). H ow ever,
th e se p o lic y th e o rie s are ra re ly a v a ila b le because th e
Results
o f
t h e
p o l i c y
u n d e rly in g p o lic y processes (fro m th e p o lic y c h a in ) and
th e s o c ia l a n d e n v ir o n m e n ta l p ro c e s s e s (fr o m th e
output: government contributions (e.g. amount o f subsidies,
e n v ir o n m e n ta l d is r u p tio n
granting a licence)
c h a in ) are in s u f fic ie n tly
k n o w n o r because th e re are to o m a n y u n c o n tro lla b le
e x te rn a l fa c to rs .
outcome: change in behaviour in the target groups (e.g.
local authorities, citizens, companies)
environmental effect: improvement / deterioration o f
I f a p o lic y e v a lu a tio n w a n ts to e x p la in th e appearance
the environment (and in a later stage nature) as a result o f
o r n o n -a p p e a ra n c e o f b e h a v io u ra l, e n v iro n m e n ta l o r
the policy (sometimes also called environmental impact)
e c o lo g ic a l e ffe cts, it n o rm a lly needs to o p e n a n d analyse
th e ‘ b la c k b o x e s ' o f th o s e b e h a v io u ra l, e n v iro n m e n ta l and e c o lo g ic a l processes.
T h e re fo re , in th e area o f e n v iro n m e n ta l an d n a tu re p o lic y , p o lic y e v a lu a tio n is n e a rly
alw ays a k in d o f process e v a lu a tio n .
EVALUATION CRITERIA
A n o th e r ty p ic a l a sp e ct o f p o lic y e v a lu a tio n s and MIRA-PE are c rite r ia . It is c le a r th a t a m e re
analysis o f a p o lic y th e m e is n o t an e v a lu a tio n . An e v a lu a tio n re q u ire s a ju d g e m e n t and
th is is m a d e u s in g c rite ria . The m o s t o b v io u s c rite rio n is e ffe ctive n e ss. T h is s h o u ld
p ro v id e an a n s w e r to th e q u e s tio n w h e th e r th e in s tru m e n t o r m ix o f in s tru m e n ts has
c o n trib u te d to an in te n d e d c ha nge in b e h a v io u r o r to an e n v iro n m e n ta l im p ro v e m e n t.
O th e r c rite ria are e ffic ie n c y (w as th e o u tp u t o b ta in e d w ith a m in im u m cost?) an d ju s tic e
(d o e s th e in s tr u m e n t e n s u re a fa ir d is tr ib u tio n o f p lea sure and p a in ?). T h e c rite ria esp e­
c ia lly fo c u s o n th e p ro d u c ts (re s u lts ) o f t h e p o lic y .
As d e s c rib e d , e x p la n a tio n s fo r th e re s u lts m u s t be s o u g h t in th e p re c e d in g processes.
cess an d fa ilu re . E v a lu a tio n c rite ria in c lu d e : c a p a c ity (d o th e a cto rs have th e c o rre c t and
Ev a l u a t i o n c r i t e r i a
................................................................................................
Criteria by which policy results are judged: effectiveness,
|
(d o th e a c to rs k n o w w h a t is e xp e cte d ? has th e re been
2005
s u ffic ie n t re source s to carry o u t th e ir task?), k n o w le d g e
MIRA-PE
T hose processes have to be e va lu a te d as w e ll, so th a t an e x p la n a tio n can be fo u n d fo r suc­
e n o u g h research in to w h ic h ta rg e t g ro u p s a n d /o r a lte rn a tiv e s are ne cessary? ), a c c e p ta tio n (is th e o b je c tiv e
efficiency, justice
m e n ts a c c e p te d ? ), in te g r a tio n (are th e o b je c tiv e s tu n e d in
Criteria by which policy processes are judged (and its results
explained): capacity, knowledge, acceptance, transparency,
sid e th e a d m in is tra tio n ? ).
integration, mutual coordination...
T h e R e g u la to ry M a n a g e m e n t U n it o f t h e F le m ish m in is tr y lis ts e ig h t q u a lity c rite ria fo r
fu tu re le g is la tio n (G u id e C h a ra cte ristics o f g o o d le g is la tio n , 2003):
■ necessary and e ffe c tiv e ;
• a p p ro p ria te and bala nced ;
• fe a s ib le and e n fo rc e a b le ;
■ la w fu l;
• c o h e re n t;
• s im p le , c le a r an d accessible;
• w e ll-fo u n d e d and n e g o tia te d ;
•
p e rm a n e n tly re le v a n t an d to p ic a l.
T h e re p o rt w ill re fe r to th e c rite ria an d h o w th e y are used a t re g u la r tim e s .
FIXED STRUCTURE
MIRA-PE n o t o n ly re p o rts o n e x is tin g p o lic y e v a lu a tio n s , b u t th e MIRA te a m s o m e tim e s
also in itia te s in -d e p th p o lic y e v a lu a tio n s . These p o lic y e v a lu a tio n s have to be pre p a re d
a c c o rd in g t o a fix e d s tru c tu re (ta b le 2).
T a b le 2: S tru c tu re o f an in -d e p th p o lic y e v a lu a tio n in itia te d by MIRA
Introduction
3
Analysis
1
Situation
4
Evaluation
2
Concepts & methods
5
Conclusions & recommendations
6.
Who is involved in MIRA-PE?
MIRA-PE 2 0 0 5 is th e re s u lt o f a c o o p e ra tio n b e tw e e n a u th o rs , re viso rs (le c tu re rs ) and th e
MIRA te a m . 116 e x p e rts fro m u n iv e rs itie s , research in s titu te s , in te re s t g ro u p s , a d m in is ­
tra tio n s , etc. have p u t th e ir e x p e rtis e a t th e d is p o s a l o f t h e p re s e n t re p o rt.
MIRA-PE
to each o th e r? are th e re o p p o s ite forces w ith in and o u t-
ABOUT
a cce pted ? are th e o rg a n is a tio n an d th e m ix o f in s tru ­
• T h e a u th o rs w rite th e ch a p te rs . W h e n p o lis h in g th e ir te xts, th e y ta k e in to a c c o u n t
th e re m arks and s u g g e s tio n s o f t h e readers a nd th e p ro je c t te a m .
• A t a greed tim e s , th e revisors u n d e rta k e a c ritic a l re v is io n o f t h e a u th o rs ’ w o rk .
They are m e n tio n e d a t th e e n d by n a m e and a ffilia tio n .
• T h e MIRA te a m o f t h e F le m ish E n v iro n m e n t A gency is re s p o n s ib le fo r th e fo llo w - u p
w ith re gard t o th e c o n te n t, th e c o o rd in a tio n and th e e d itin g o f e n v iro n m e n ta l re p o rts
an d e la b o ra te s c o n te n ts to g e th e r w ith th e a u th o rs . A fte r e d itio n by th e p r o je c tte a m ,
th e fin a l te x t is s u b m itte d to th e a u th o rs a nd th e re viso rs f o r a p p ro v a l.
• T h e s ta tu to r ily d e s c rib e d MIRA s te e rin g g ro u p is re s p o n s ib le fo r s te e rin g th e c o n te n ts
o f t h e re p o rtin g process.
7. Who is MIRA-PE aimed at?
T he re p o rt is a im e d a t p o lic y m a kers (m e m b e rs o f p a rlia m e n t, c a b in e t m e m b e rs , th o s e
re s p o n s ib le fo r p la n n in g ), p o lic y e x e c u to rs and o th e r p e o p le in v o lv e d in th e p o lic y
(e .g. ta rg e t g ro u p re p re s e n ta tiv e s ). T h ey are th e o n e s w h o have to ta c k le th e p ro b le m s
id e n tifie d by th e re p o rt, as w e ll as s tim u la te th e success fa c to rs. T h e r e p o r t’s
re c o m m e n d a tio n s can h e lp th e m w ith th e ir task.
M o re o v e r, o th e r p e rs o n s w ith an in te re s t in th e p o lic y (o r a c ritic a l v ie w o n it) m ay fin d
th e re p o rt u s e fu l as w e ll. A fte r a ll, a p o lic y has c o n se q u e n ce s fo r every c o m p a n y and
every c itiz e n .
An in tr o d u c tio n to th e c o n c e p ts , m e th o d s and a p p ro a c h o f p o lic y e v a lu a tio n s can be
fo u n d in c h a p te r i o f MIRA-PE 2003 (d o w n lo a d a b le a t w w w .m ilie u r a p p o rt.b e ).
R e f e r e n c e s
Gids Kenmerken van een goede regel­
Wilkinson D. (1999) The contribution
geving (2003) (Guide characteristics o f
o f EU monitoring and reporting on
good legislation). Kenniscel Wetsmatiging
'sound and effective’ environmental
(Regulatory Management Unit), Ministerie
policymaking. First draft paper for the
van de Vlaamse Gemeenschap, Brussel.
MINA-plan 3 (2003) Milieubeleidsplan
2003-2007 (Environmental Policy Plan).
Ministerie van de Vlaamse Gemeenschap,
EEA-project ‘Towards a new EU framework
for reporting on environmental policies
and measures’ (project manager M. T.
Ribeiro), z i September 1999,
http://reports.eea. eu.int/rem/en/
Brussel.
soundeffective.pdf.
Flemish environmental
policy 2003-2005
under the microscope:
24 evaluations
Barbara Tieleman, M IR A , VMM
INTRODUCTION
T h e in v e n to ry o f e n v iro n m e n ta l p o lic y e v a lu a tio n s (1996-2003) as a p a rt o f th e fir s t P olicy
E va lu a tio n R e p o rt (M l RA-PE 20 03) m e t w ith g e n e ra l a p p ro v a l. We w ill c o n tin u e o n th is
é la n in MIRA-PE 20 05 by p r o v id in g an o v e rv ie w o f th e p u b lis h e d p o lic y e v a lu a tio n s fo r
th e p e rio d 20 03 -2 0 0 5 .
T h e p u rp o s e o f th is in v e n to ry is t o re in fo rc e its fu n c tio n a s a re fe re n c e w o r k o n e n v iro n m e n ta l
p o lic y e v a lu a tio n s . In th is d o c u m e n t w e th e re fo re p re s e n ta n o v e rv ie w o f th e p o lic y
e v a lu a tio n s tu d ie s , p u b lis h e d b e tw e e n 1 July 2003 and 30 M ay 2 0 05 (p a rt 1, In v e n to ry ).
G eneral c o n c lu s io n s o n th e f u ll o v e rv ie w are d ra w n based o n p a rtic u la r ch a ra cte ristics.
These ‘ c o n c lu s io n s ’ are p re s e n te d in p a rt 2 'D is c u s s io n '. In p a rt 3 w e p ro v id e a s u m m a ry
o f each p u b lic a tio n . As regards th e in v e n to ry a n d th e d is c u s s io n , w e w e re in s p ire d by
a s im ila r in v e n to ry o f th e S w edish E n v iro n m e n ta l P ro te c tio n A gency, 2 0 0 4 . T h e exact
re fe re n c e o f each in v e n to ry p u b lic a tio n can be fo u n d in th e a n n e x t o th is ch a p te r.
i. Inventory
t . i . S E L E C T I O N OF T H E P U B L I C A T I O N S
In p re p a ra tio n o f th e in v e n to ry , w e e x p lo re d th e Fle m ish U n iv e rs itie s ' databases
(in c lu d in g s u p p o rt c e n tre s ), th e e n v iro n m e n ta l a d m in is tra tio n , th e p o lic y a d viso ry
agencies, th e a d v is o ry b o d ie s SERV and M iN a C o u n c il, th e Fle m ish U n io n fo r C ities
and M u n ic ip a litie s (VVSG) and th e e n v iro n m e n ta l lib ra rie s ’ n e tw o rk (F e in e t).
We c o n s u lte d a larg e v o lu m e o f in fo rm a tio n resources d is c u s s in g e n v iro n m e n ta l p o lic y ,
such as a rtic le s , re p o rts , b o o k s and o th e rs . The p u rp o s e o f th e in v e n to ry w as to p ro v id e a
c o m p le te lis t o f h ig h -q u a lity p o lic y e v a lu a tio n s . M o s t p u b lic a tio n s are e a sily c a te g o rise d
as p o lic y e v a lu a tio n s , b u t th is d o e s n ’t h o ld fo r a ll o f th e m .
W e th e re fo re used th e fo llo w in g g u id e lin e s as s e le c tio n c rite ria :
• A p o lic y e v a lu a tio n s h o u ld c o m p ly w ith th e c h a ra c te ris tic s o f th e d e fin itio n o f a p o lic y
e v a lu a tio n , i.e.
• be a s c ie n tific an alysis (n o p e rs o n a l o p in io n s , n o d e s c rip tio n s );
• have e n v iro n m e n ta l p o lic y as a c e n tra l th e m e (n o in v e s tig a tio n s w ith regard to an
e n v iro n m e n ta l p ro b le m w ith o u t c o n s id e ra tio n o f p o lic y aspects; n o in v e s tig a tio n s
w ith regard to o th e r p o lic ie s th a n th e e n v iro n m e n ta l p o lic y - in c lu d in g S u sta in a b le
D e v e lo p m e n t. E nergy a nd a g ric u ltu ra l p o lic ie s are th e re fo re e x c lu d e d fro m th e
in v e n to ry , un le ss th e y d eal w ith e n v iro n m e n ta l Issues);
• c o n ta in e v a lu a tio n c rite ria (e v a lu a tio n is re q u ire d , id e a lly based o n e x p lic it c rite ria .
So th e re can be n o p o lic y o r g u id e lin e dis c u s s io n w ith o u t e v a lu a tio n . A sa
re s u lt, m a n u a ls fo r m u n ic ip a litie s c o n ta in in g e x p la n a tio n s o n de crees o r
legal e x p la n a tio n s o f g u id e lin e s are n o t ta ke n in to c o n s id e ra tio n );
• c o n ta in re c o m m e n d a tio n s ;
• n o re c o m m e n d a tio n s (a d v is o ry papers) fro m a d v is o ry c o u n c ils . A lth o u g h th e y d o
c o n ta in c ritic a l analyses (p re d o m in a n tly e x -a n te e v a lu a tio n s ), th e y are n o t a p p ro p ria te
in an e la b o ra te c a s e s tu d y c h a p te r. M o re o v e r, it re m a in s u n c le a r w h a t s c ie n tific
a n alysis served as a basis f o r th e s e re c o m m e n d a tio n s ;
• n o artic le s , un le ss as research o r s c ie n tific analysis s u m m a ry. I f i t is u n c le a r w h e th e r th e
a rtic le is based o n s c ie n tific analysis (i.e . based o n research), i t is n o t in c lu d e d in th e
in v e n to ry . Ifo n th e o th e r hand i t is c le a rly based o n s c ie n tific analysis, th e n th e a rtic le
basis, in p a rtic u la r th e research and n o t so m u c h th e a rtic le itse lf, is in c o rp o ra te d ;
• th e p o lic y e v a lu a tio n s m u s t be o f a h ig h q u a lity . In th e lo n g te r m , MIRA-PE w a n ts t o be
an e x a m p le in th e fie ld o f h ig h -q u a lity p o lic y e v a lu a tio n s a t th e service o f p o licym a ke rs.
T h e re fo re p u b lic a tio n s la c k in g a c le a r an alysis fra m e w o rk w e re e x c lu d e d fro m th e
in v e n to ry . Flow ever, Flanders d o e s n o t have a lo n g s ta n d in g tr a d itio n o f p o lic y
e v a lu a tio n , w h ic h o b v io u s ly has re p e rc u s s io n s o n th e p ro d u c t q u a lity . C o n se q u e n tly,
th e in v e n to ry leaves s o m e ro o m fo r spe cia l cases: s tu d ie s w h ic h d e v ia te fro m th e
o rd in a ry g u id e lin e s , b u t w h ic h stan d o u t because o f th e ir re c o m m e n d a tio n s .
P o licy s u p p o rtin g s tu d ie s w e re n o t ta k e n in to a c c o u n t. T h e re is a fin e lin e b e tw e e n
‘ p o lic y s u p p o rtin g ’ and ‘ p o lic y e v a lu a tin g ' research th o u g h . D e p e n d in g o n th e research
o b je c tiv e , th e im p o rta n c e o f th e ‘ s u p p o rt’ and ‘ e v a lu a tio n ’ e le m e n ts w ill d iffe r. D iffe re n t
p o lic y s tu d ie s serve d iffe r e n t pu rpose s:
• m e a su re e ffe cts;
2005
• in te n s ify ‘th e le a rn in g ’ w ith in an o rg a n is a tio n .
MIRA-PE
• g a in s c ie n tific in s ig h t in to p o lic y processes;
The firs t ty p e a im s a t p r o v id in g 's c ie n tific in s ig h ts ’, th e s e c o n d a t ‘a c c o u n ta b ility ’ and
I
th e th ir d a im s a t 'r e fle c tio n ' o n th e p o lic y . The p o lic y e v a lu a tio n s are g e n e ra lly a im e d at
fV
‘ a c c o u n ta b ility ’, th e le g itim a tio n o f p o lic y cho ice s, p o lic y processes a nd p o lic y re su lts
W e lin k e d several c h a ra c te ris tic s to each in v e n to ry p u b lic a tio n . These ch a ra cte ristics
serve a s a basis fo r fu r th e r d is c u s s io n (p a rt 2).
The c h a ra c te ris tic 'e n v ir o n m e n ta l to p ic ’ re q u ire s a b r ie f e x p la n a tio n . For each p u b lic a tio n it
20 07), th e F le m ish E n v iro n m e n ta l P o licy P lan):
• c lim a te ;
• w aste ;
• n a tu re ;
• w a te r - s u r fa c e w a te ra n d d e s ic c a tio n q u a lity c lu s te rin g ;
• a c id ific a tio n ;
• ha zardo us substances;
• s o il;
• n u isa n ce ;
• fra g m e n ta tio n .
F u rth e rm o re , w e have c a ta lo g u e d each p u b lic a tio n a c c o rd in g to e va lu a te d 'p o lic y ty p e '.
T he M IN A -p la n 3 c la s s ific a tio n is la rg e ly m a in ta in e d .
• re g io n -o rie n te d p o lic y : a p o lic y w h ic h ta c k le s th e e n v iro n m e n ta l p ro b le m s lin k e d
to s p e c ific areas. A p o lic y w h ic h e n hance s th e q u a lity o f s p e c ific areas o f th e
e n v ir o n m e n t and w h ic h is re s p o n s ib le f o r t h e m u tu a l c o o rd in a tio n b e tw e e n
th e v a rio u s fu n c tio n s ;
• p o lic y a im e d a t ta r g e t g r o u p s : a p o lic y w h ic h in flu e n c e s th e b e h a v io u r o f sta k e h o ld e rs
w h o c o n tro l th e q u a lity o f th e e n v iro n m e n t. A p o lic y w h ic h s tim u la te s g o v e rn m e n t
c o o p e ra tio n and a sense o f re s p o n s ib ility a m o n g th e a fo re m e n tio n e d p a rtic ip a n ts ;
MICROSCOPE
• d e p le tio n o f th e o z o n e layer;
THE
• p h o to c h e m is try ;
UNDER
• e u tro p h ic a tio n ;
2003*2005
• g e n e ra l (p o lic y a im e d a t several to p ic s c o m b in e d o r a t ‘th e e n v ir o n m e n t’ in g e n e ra l);
POL I CY
is in d ic a te d w h ic h e n v iro n m e n ta l p ro b le m c o n s titu te s th e basis o f th e e va lu a te d p o licy.
T he fo llo w in g e n v iro n m e n ta l to p ic s are in c lu d e d (in s p ire d by th e M IN A -p la n 3 (2 0 0 3 -
ENVIRONMENTAL
1. 2 . O V E R V I E W OF P U B L I C A T I O N S
FLEMISH
(B o u ck a e rt, 1992).
• in te g r a te d g o v e r n m e n t p o lic y : a p o lic y a im e d a t th e o p tim is a tio n o f in te rn a l c o o p e ra tio n
and th e c o o p e ra tio n w ith o th e r p o lic y d o m a in s and m a n a g e m e n t levels;
• in te r n a tio n a l p o lic y ,
■ c o o r d in a tin g p o lic y : a p o lic y w h ic h is to fo rm a s o lid basis fo r th e p o lic y , th e p o lic y
processes an d th e p o lic y s tru c tu re s .
M a n y in v e n to ry p o lic y e v a lu a tio n s fo c u s o n o n e o r m o re s p e c ific p o lic y in s tru m e n ts .
W h e n e v e r th is is th e case, w e have in d ic a te d w h a t k in d o f in s tr u m e n t is in v o lv e d .
W e d is tin g u is h th e fo llo w in g p o lic y in s tru m e n ts :
• so cia l I c o m m u n ic a tiv e in s tru m e n ts ;
• e c o n o m ic in s tru m e n ts ;
• legal in s tru m e n ts ;
• s u p p o rtin g in s tru m e n ts ;
• d ire c t in s tru m e n ts ;
• e n fo rc e m e n t.
C o n tra ry to M IN A -p la n 3, e n fo rc e m e n t is n o t c a te g o ris e d as a legal in s tru m e n t.
E n fo rc e m e n t is an in s tr u m e n t used by th e g o v e rn m e n t, w h ic h m ay c o m p ris e d ire c t
an d a ll k in d s o f in d ir e c t p o lic y in s tru m e n ts and in th a t sense ‘tra n s c e n d s ’ tra d itio n a l
c la s s ific a tio n s . It is a ls o a sep arate p o lic y p ro b le m w ith sep arate p o lic y e v a lu a tio n
research.
S om e legal in s tru m e n ts (e.g. a p e r m it f o r c o m p a n y x) are w e ll-d e fin e d an d lim ite d to
t h e ir fie ld o f a p p lic a tio n , o th e r legal in s tru m e n ts are a ‘c o lle c tio n ’ o f m e asures and
in s tru m e n ts (e .g. th e m a n u re a c tio n plan o r th e w a ste im p le m e n ta tio n p la n ). T h e la tte r,
a lso ca lle d s u p p o rtin g p o lic y in s tru m e n ts , are lo o s e ly d e fin e d and have a w id e a p p lic a tio n
fie ld . It is a c o lle c tiv e te rm fo r p ro g ra m m e s a nd p ro je c ts in v o lv in g charges, s e n s itis a tio n ,
fra m e w o rk s tru c tu re etc. A p o lic y e v a lu a tio n in v o lv in g a s u p p o rtin g in s tr u m e n t and a
‘ s im p le ’ legal in s tr u m e n t w ill o b v io u s ly s tro n g ly d iffe r in scop e and fo cu s.
The c h a ra c te ris tic 'e v a lu a tio n ty p e ' in c lu d e s th e c o n c e p ts e x -a n te , e x -n u n c a n d e x -p o s t
e v a lu a tio n ; p la n , process, and p ro d u c t e v a lu a tio n (see c h a p te r ‘A b o u t MIRA-PE’).
T a ble 1 sh o w s o u r s e le c tio n o f p o lic y e v a lu a tio n s tu d ie s sin ce i July 2003 (24 p u b lic a tio n s ).
The p u b lic a tio n s w e re c la s s ifie d a c c o rd in g to e n v iro n m e n ta l to p ic , and th e n a c c o rd in g
to y e a r and nam e.
Table i: O v e rv ie w o f p o lic y e v a lu a tio n s , p u b lis h e d b e tw e e n i July 2003 and 30 M ay 2005
topic
Study in preparation o f the evaluation o f the federal environment-
policy
instrum ent
evaluation
type'
type2
type
target group
l/ec/so
ex ante
specific product policy (2003)
product
international
(2003)
ex nunc
process
|
Europeanisation o f Flanders: a screening o f the managerial structures
2005
A. Heyerick
General
N
E. Van Reybroeck, P. Bursens
General
Flemish structural consultation for sustainable development (2004)
Integration/
K. Bachus, H. Bruyninckx, M. Poesen-Vandeputte
coordination
process
Border crossing Flemish environmental policy (2004)
International
ex nunc
"
Follow-up o f environmental complaints by the Environmental
Integration/
Inspection: an efficient instrument for high supervision? (2004)
target group
en
process/
product
A. Monsieurs
General
Pilot project VLAREM supervision Klein-Brabant, Transportsector
ex nunc
target group
en
ex post
R. Segers
General
integration/
ex ante
in Flanders (2004)
coordination
process/
international
ex nunc
plan
P. Van Flumbeeck, 1. Dries, J. Larosse
General
The Europeanisation o f Flemish officials, members o f cabinet
M. Baetens, P. Bursens
General
Carte blanche or corset? Evaluation o f the Cooperation Agreement
integration
su/so
ex nunc
General
Enforcement: the missing link in the environmental policy cycle?
target group
en
ex post
plan
(2005)
T. De Bruyn, K. Bachus
General
The Implemented sustainable development policy (2005)
coordination
su
General
Environmental policy integration in Europe (2005)
integration
■
ex nunc
target group
su
ex post
process
European Environment Agency
Waste
Sector-specific implementation plans in the Flemish waste policy.
ex post
product
P. Delbaere, P. Dresselaers, N. Gouzée
product
Effectiveness analysis o f a planning Instrument (2003)
). Gysen, K. Bachus, H. Bruyninckx, L. Van Ootegem
Waste
Waste prevention actions and measures in secondary schools in
target group
di/so
ex post
product
Flanders (2003)
OVAM
Waste
Waste flows and waste policy in elementary schools in Flanders (2003)
OVAM
target group
di/so
ex post
product
MICROSCOPE
K.Bachus
THE
process
(2005)
UNDER
process
and politicians: opinion, expertise and attention (2005)
2003-2005
Linking Innovation Policyand Sustainable Development
POL I CY
process
(2004)
ENVIRONMENTAL
process
K. Geeraerts, P. Bursens, P. Leroy
General
ex ante
FLEMI SH
General
MIRA-PE
General
title + author) s)
topic
title + author(s)
Waste
Presti 4: involved in environmental protection - measuring
policy
instrum ent
evaluation
type'
type'
type
target group
so
ex post
the effects (2005)
product
A. Gommers, K. Couderé
Manure
Three-tier manure policy: evaluation and future exploration (2004)
target group
su
M. Vervaet, L. Lauwers, S. Lenders, S. Overloop
Nature
Nature, how? Investigation into the social framework for nature
ex nunc
process
target group
ex ante
preservation in Flanders (2004)
plan
T. Leroy, 1. Loots, P. Leroy
Nature
Report o f the Commission on the implementation o f Guideline
region
1
92/43/EC on the conservation o f natural habitats and o f wild flora
ex nunc
process
and fauna (2004)
European Commission
Nature
Nature report 2005. The condition o f nature in Flanders-figures
region/
for the policy (2005)
international/
process/
M. Dumortier, L. De Bruyn, M. Hens, J. Peymen, A. Schneiders,
target group
product
l/ec/so
ex post
T. Van Daele, W. Van Reeth, G. Weyembergh, E. Kuijken
Nature
Nature
Financial instruments for nature, forestand landscape (2005)
region/
B. Kayaerts
target group
Region-oriented nature policy: operationalityand implementation
international/
o f the Bird Guideline and Habitat Guideline (2005)
region
ec
ex post
1
ex nunc
product
process
A. Cliquet, G. Van Hoorick, |. Lambrecht, D. Bogaert
Water
Local water policy about to turn: interim evaluation o f the sub-basin
region/
functioning (2005)
integration
1
ex nunc
process
A. Crabbé
Water
Regional subsidies for municipal waste water treatment plants
integration
ec
in a European context (2005)
ex post
product
Court o f Audit
1 region = re g io n -o rie n te d policy; in te rn a tio n a l = in te rn a tio n a l policy; in teg ratio n = in tegrated g o v e rn m e n t policy; ta rg e t g ro u p = ta rg e t g ro u p policy;
c o o rd in atio n = c o o rd in a tin g policy
2 so = social in stru m e n t; I = legal in stru m en t; su = s u p p o rtin g in stru m en t; ec = e c o n o m ic in stru m en t; d i = dire c t in stru m e n t; en = e n fo rc e m e n t
2. Discussion
T h e in v e n to ry d is c u s s io n c o n s is ts o f e ig h t s u b s e c tio n s . T h e ty p e o f p o lic y , th e ty p e
o f in s tru m e n t, th e ty p e o f e v a lu a tio n , th e e n v iro n m e n ta l to p ic s (lin k e d t o b u d g e t),
e v a lu a tio n c rite ria , th e p rin c ip a l and th e e v a lu a tin g in s titu tio n s are discussed in th a t
o rd e r. In th e s ix th s u b s e c tio n w e w ill a n sw e r g e n e ra l q u e s tio n s raised by th e in v e n to ry ,
in w h ic h c o n c lu s io n s fro m p re c e d in g p a ragraph s are in c lu d e d . Finally, a m a trix visu a lise s
t o w h a t e x te n t th e p u b lic a tio n s p ro v id e an actu al e v a lu a tio n o f th e e n v iro n m e n ta l p o lic y
asa w h o le .
As th e n u m b e r o f e v a lu a tio n s is lim ite d , th e d is c u s s io n serves p r e d o m in a n tly as an
illu s tra tio n o f h o w th e se p o lic y e v a lu a tio n s to o k place. P u b lic a tio n s c o n ta in in g tw o
ca te g o rie s fo r o n e c h a ra c te ris tic c o u n t tw ic e .
MI RA- PE
2 . 1 . TYPE OF P O L I C Y
in v e n to ry p u b lic a tio n s e va lu a te a p o lic y s p e c ific a lly a im e d a to n e o r m o re ta rg e t g ro u p s .
2005
E va lu a tio n s o f c o o rd in a tin g p o lic ie s o c c u r less fre q u e n tly .
|
Figure i sh o w s th e d is tr ib u tio n o f th e p u b lic a tio n s a c c o rd in g to p o lic y typ e . M o s t
N
Figure 1: D is trib u tio n o f th e in v e n to ry p u b lic a tio n s a c c o rd in g t o p o lic y ty p e
FLEMI SH
16%
c o o rd in a tin g policy
reg io n -o rie n te d policy
ENVIRONMENTAL
22%
in teg rate d g o v e rn m e n t
policy
ta rg e t g ro up policy
in te rn a tio n a l policy
o f e v a lu a tio n s fo r c e rta in c a te g o rie s . A ll c a te g o rie s are e q u a lly im p o rta n t. It is u p to
UNDER
p o lic y m a k e rs t o m a ke th e rig h t cho ice s.
e v a lu a tio n d id n o t e v a lu a te any in s tru m e n ts , it w as a d d e d to th e ca te g o ry 'n o t
a p p lic a b le ’. Figure 2 s h o w s th e e x a m in e d in s tru m e n ts .
Figure 2: D is trib u tio n o f th e in v e n to ry p u b lic a tio n s a c c o rd in g to p o lic y in s tr u m e n t
6 social in stru m en t
4 eco n o m ic in stru m en t
e n fo rc e m e n t 3
5 legal in stru m en t
s u p p o rtin g in s tru m e n t 4
2 d irect in stru m en t
MICROSCOPE
A ll e v a lu a tio n s w e re c la s s ifie d a c c o rd in g to th e ty p e o f e v a lu a te d in s tru m e n t. I f an
THE
2 . 2 . TYPE OF P O L I C Y I N S T R U M E N T
n o t a p p lic ab le 7
2003-2005
W e d o n o t w a n t to m a ke re c o m m e n d a tio n s w ith regard to an increase in th e n u m b e r
POL I CY
37%
16 %
N early o n e fo u rth o f th e p u b lic a tio n s d o n o t e v a lu a te any in s tru m e n ts . In th e o th e r
e v a lu a tio n s th e re is lit t le fo c u s o n d ire c t in s tru m e n ts . T h a t is n o t s u rp ris in g since,
a m o n g s t o th e rs , in M IR A -T (th e F le m ish S tate o f th e E n v iro n m e n t-re p o rt), re p o rts o n
d ire c t in s tru m e n ts are p ro v id e d by th e s o -c a lle d response in d ic a to rs , u s u a lly backed
up by an an alysis o f th e e n v iro n m e n ta l s itu a tio n .
2 . 3 . TYPE OF E V A L U A T I O N
T h e d is tr ib u tio n o f th e p u b lic a tio n s a c c o rd in g to ty p e o f e v a lu a tio n is illu s tra te d by tw o
fig u re s . Figure 3 s h o w s h o w m a n y e v a lu a tio n s w e re c arried o u t b e fo re , d u rin g o r a fte r
th e co n c e rn e d p o lic y . Figure 4 s h o w s h o w m a n y p u b lic a tio n s c e n tre a ro u n d p o lic y pla n s,
p o lic y processes o r p o lic y re s u lts (p la n , process a n d p ro d u c t e v a lu a tio n re s p e c tiv e ly ).
A fte r e x a m in in g th e d is tr ib u tio n o f th e e n v iro n m e n ta l p o lic y e v a lu a tio n s p e r research
p e rio d (fig u re 2), w e see th a t re la tiv e ly fe w e x -a n te e v a lu a tio n s are ca rrie d o u t. T h is m ay
ch a n g e in th e fu tu re w h e n m o re re g u la tin g im p a c ta n a ly s e s (RIAs) w ill be c a rrie d o u t. As
o f r January 2 0 0 5 a RIA has t o be p e rfo rm e d fo r a ll ru le s and re g u la tio n s w ith re g u la tin g
e ffe c ts o n c itiz e n s , th e bu sin ess c o m m u n ity o r n o n - p r o fit o rg a n is a tio n s . It re m a in s to be
seen i f th e q u a lity o f th e RIAs w ill a llo w th e ir in c lu s io n in th e in v e n to ry . W e a lso p o in t
o u t th a t th e re are im p a c ta n a ly s e s w h ic h are n o t p o lic y e v a lu a tin g , b u t w h ic h m ig h t as
w e ll p ro v id e a fo u n d a tio n fo r fu tu re p o lic ie s .
C learly an e x -p o s t p o lic y e v a lu a tio n can p ro v id e n e w in s ig h ts f o r p o lic y d e v e lo p m e n t
an d th e re fo re fo r e x -a n te p o lic y e v a lu a tio n . H ow eve r, th is le a rn in g e ffe c t re q u ire s a
m in im u m o f e x -p o s t p o lic y e v a lu a tio n research w h ic h p ro v id e s s u ffic ie n t g e n e ric (so
n o t o n ly c a se -sp e cific) in s ig h ts a n d is in a d d itio n used fo r e x -a n te e v a lu a tio n s . From
th is p e rs p e c tiv e Flanders s till h a sa lo n g w a y to g o .
Fig ure 3: D is trib u tio n o f th e in v e n to ry p u b lic a tio n s a c c o rd in g to ty p e o f e v a lu a tio n
(a c c o rd in g t o research p e rio d )
e x-p o st ev a lu a tio n io
4 ex-ante evaluation
10 ex-nun c ev alu ation
Figure 4 c le a rly s h o w s th a t fe w e v a lu a tio n s e x a m in e th e p la n n in g phase o f th e p o lic y .
T h e n u m b e r o f e v a lu a tio n s th a t fo c u s o n p o lic y processes is s im ila r to th e n u m b e r
fo r p o lic y re s u lts o r p ro d u c ts .
Figure 4: Distribution o f the inventory publications according to type o f evaluation (by subject)
3 plan evaluation
p ro d u c t ev alu atio n 10
MIRA-PE
12 process evaluation
2005
I
N
FLEMI SH
W h e n w e c o n s id e r th e fa c t th a t th e ad v is o ry b o ards re g u la rly advise o n d r a ft decrees,
th e re are m o re e x -a n te an d p la n e v a lu a tio n s th a n appears fro m th e in v e n to ry - even
p o in te d o u t in pa ra g ra p h 1.1.
2.4 . EN V IR O N M E NT AL TOPICS
e v a lu a tio n . The e n v iro n m e n ta l p ro b le m s w e re lin k e d to clu s te re d e n v iro n m e n ta l to p ic s ,
Fig ure 5: D is trib u tio n o f th e in v e n to ry p u b lic a tio n s a c c o rd in g t o e n v iro n m e n ta l to p ic
UNDER
n a tu re 5
2003-2005
u sin g M IN A -p la n as a s ta rtin g p o in t. Figure 5 sho w s th e c o n s id e re d e n v iro n m e n ta l to p ic s .
POL I CY
For each p u b lic a tio n w e in v e s tig a te d th e e n v iro n m e n ta l p ro b le m s u n d e rly in g th e p o lic y
ENVIRONMENTAL
th o u g h th e re c o m m e n d a tio n s are o fte n to o lim ite d w ith regard to e v a lu a tio n , as w e
THE
w a te r 3
MICROSCOPE
w aste 6
2 c lim ate
T h e ‘g e n e ra l’ p o lic y e v a lu a tio n s c o n s titu te th e m a jo rity . T h is co m e s as n o s u rp ris e s ince
also p o lic y e v a lu a tio n s c o v e rin g several to p ic s are c a te g o ris e d as such . W aste and n a tu re
p o lic ie s are c le a rly in v e s tig a te d m o re fre q u e n tly ; th is d o e s n o t h o ld fo r w a te r p o lic y
(th re e p u b lic a tio n s ) o r m a n u re p o lic y (o n e p u b lic a tio n ). C lim a te aspects are discussed
in tw o case s tu d ie s . T h e o th e r to p ic s re m a in e d u n e x a m in e d . W e su sp e ct i t d e p e n d s
o n th e re la tiv e w e ig h t o f th e p o lic y d o m a in s , social and p o litic a l in te re s t, d e lica cies,
c o m p le x itie s an d th e a v a ila b ility o f th e resources fo r p e rfo r m in g e v a lu a tio n s .
T h e lin k b e tw e e n th e e v a lu a te d e n v iro n m e n ta l to p ic s and th e e x p e n d itu re o n th e se to p ic s
are p re s e n te d in ta b le 2. The data re g a rd in g e n v iro n m e n ta l e x p e n d itu re s can be fo u n d in
M IN A -p la n 3 (p. 352 a n d so fo rth ).
T a b le 2: L in k b e tw e e n th e F le m ish R e g io n ’s e n v iro n m e n ta l e x p e n d itu re (2 0 0 2 ) a n d th e e n v iro n m e n ta l to p ic s o f
th e in v e n to ry
ou r topic
topic classification
environm ental
share in total
nu m b er o f policy evalua­
classification
from M IN A -plan 3
expenditure
environm ental
tions from the inventory
(in 1000 euros)
expenditure (%)
regarding this topic
92 319
10.15
295
0.03
2
(7 %)
3.78
6
(20.5%)
general
integrated government policy,
12
(41.5 %)
instruments, region-oriented
policy, participants, international
policy, costs and financing
climate
climate
waste
waste products
34 342
manure
eutrophication
77 397
8.51
1
(3-5 %)
nature
biodiversity
127 872
14.06
5
(17-3 %)
water
surface water pollution, desiccation
507 386
55-79
3
(10.2%)
and integral water policy
photochemistry
photochemistry
772
0.08
0
(0%)
depletion o f
ozone
147
0.02
0
(0%)
the ozone layer
acidification
acidification
507
0.06
0
(0%)
hazardous
hazardous
905
0.10
0
(0%)
54 463
5-99
0
(0%)
0
(0%)
substances
substances
soil
soil
nuisance
noise, odour and light nuisance
556
0.06
fragmentation
fragmentation
3 832
0.42
0
(0%)
-
‘air in general’
8 490
0.93
0
(0%)
263
0.02
0
(0%)
909 546
100
29
(100%)
costs and financing
total
Source: M IN A -p la n 3
T o p ics th a t w e re n o t e v a lu a te d te n d t o h o ld a lim ite d share in th e e n v iro n m e n ta l
e x p e n d itu re s . W e a lso n o te th a t th e share o f e v a lu a tio n s o n n a tu re m o re o r less
c o rre s p o n d s to th e share o f n a tu re p o lic y in th e e n v iro n m e n ta l e x p e n d itu re (17.3 %
co m p a re d to 1 4 .0 6 % ). W e also fin d s o m e dis c re p a n c ie s w h e n w e lo o k a t e v a lu a tio n s
o f w a ste p o lic y , c lim a te p o lic y , w a te r p o lic y and th e g e nera l e v a lu a tio n s (in b o ld in th e
ta b le a b o v e ). For p u b lic a tio n s d e a lin g w ith w aste and c lim a te and fo r g e nera l e v a lu a tio n s ,
w e see th a t th e share o f e v a lu a tio n s is la rg e r th a n th e share o f th e se to p ic s in th e
e n v iro n m e n ta l e x p e n d itu re . In case o f a c lim a te p o lic y , th is can be p a rtly e x p la in e d by
th e fa c t th a t a large p a rt o f th e e xpenses fa lls u n d e r th e e n e rg y p o lic y (and th e re fo re
n o t u n d e r th e b u d g e t o f th e M in is try o f E n v iro n m e n t). In any case, w e w ill n o t m a ke a ny
sta te m e n ts re g a rd in g th e d is trib u tio n o ft h e resources allocated to each e n v iro n m e n ta l to p ic .
W ith regard to t h e w a te r p o lic y , w e w o u ld lik e to p o in t o u t th a t in p r o p o r tio n fe w e r
e v a lu a tio n s deal w ith w a te r p o lic y th a n th e s hare in th e e n v iro n m e n ta l b u d g e t (10.2 %
o p p o s e d to 55 .97% ).
2 .5. THE C RI T E R I A
In th e o ry th e p u rp o s e o f p o lic y e v a lu a tio n s is t o v e rify w h e th e r a s p e c ific p o lic y stays
o n ta rg e t (e ffe c tiv e n e s s ). D e liv e rin g ‘ p r o o f’ o f his e ffe c tiv e n e ss is p a rtic u la rly d iffic u lt.
c e rta in p o lic y is p e rfo r m in g w e ll o r n o t. Such c rite ria are fo r e x a m p le tra n sp a re n cy,
c o o rd in a tio n , in v o lv e m e n t, k n o w le d g e and capacity.
2005
The in v e n to ry s h o w s th a t e ffe c tiv e n e s s analyses are very rare (th e o n ly rig h tfu l
MIRA-PE
T h e re fo re an e v a lu a to r w ill o fte n use c rite ria w h ic h m a ke it p o ssib le to e x p la in w h y a
p u b lic a tio n s are Cysen e t al., 2003, OVAM, 2003a and 2 0 0 3 b and V ervaet e t a l., 2 0 0 4 ).
I
For a lm o s t a ll p o lic y e v a lu a tio n s , c rite ria are used a s a n e x p la n a tio n fo r an (u n)succe ssful
• k n o w le d g e (k n o w in g );
• w a n tin g ;
• le g itim a c y ;
• in te g ra tio n ;
• p a rtic ip a tio n ;
• le g a lity ;
These c rite ria w e re used f o r t h e e v a lu a tio n o f plan s, p ro d u c ts (p ro g ra m m e s , decrees,
la w s ...) as w e ll as f o r t h e e v a lu a tio n o f processes.
p a rtie s ) and p o lic y e v a lu a tio n s c o n d u c te d o n a pe rs o n a l in itia tiv e . ‘ P ersonal in itia tiv e ’
re fers t o e v a lu a tio n s in w h ic h th e p rin c ip a l and th e p ro v id e r are o n e and th e sam e
p e rso n o r o rg a n is a tio n .
M o s t p o lic y e v a lu a tio n s w e re o rd e re d by th e g o v e rn m e n t (16 p u b lic a tio n s ). The p u b lic
in s titu tio n s w h ic h o rd e re d the se p o lic y e v a lu a tio n s are th e e n v iro n m e n ta l a d m in is tra tio n
(A M IN A L, V M M , OVAM), th e C entre f o r E n viro n m e n ta l P olicy Sciences, th e C entre fo r
A d m in is tra tiv e O rg a n is a tio n in Flanders, th e d e p a rtm e n t o f Science and In n o v a tio n , th e
o ffic e o f t h e p u b lic p ro s e c u to r and th e provinces. T he m a jo rity o f th e ‘ pe rsona l in itia tiv e ’
p u b lic a tio n s have p rin c ip a ls th a t are m e m b e rs o f g o v e rn m e n t. OVAM has ta ke n th e
in itia tiv e tw ic e in th is way, fo r research c o n d u c te d by its o w n researchers. The C entre
fo r A g ric u ltu ra l e c o n o m y , th e Federal P la n n in g Bureau, th e E uropean C o m m issio n and
th e C o u rt o f A u d it have take n th e in itia tiv e on ce. O n ly th e p u b lic a tio n s o ft h e M iN a C ouncil
(Kayaerts, 2005) and th e K .U .Leuven (M onsieu rs, 2 0 04) are n o n -g o v e rn m e n t re la ted. We
w ill co m e back to th e in d e p e n d e n c e o f the se s tud ies (p aragraph 2.7).
MICROSCOPE
c a te g o rie s : p o lic y e v a lu a tio n s o rd e re d by th e g o v e rn m e n t (a nd ca rrie d o u t by e xte rn a l
THE
Figure 6 s h o w s th e in itia to r f o r t h e p o lic y e v a lu a tio n s . The d ia g ra m s h o w s tw o
UNDER
2.6. PRINCIPAL AND PROVIDER
2003-2005
• c o h e re n c e / s tru c tu re / c o o rd in a tio n .
P OL I CY
• c o m m u n ic a tio n ;
• c o o p e ra tio n ;
ENVIRONMENTAL
• ca p a c ity (b e in g a b le );
FLEMI SH
p o lic y , w it h o u t e v id e n c e o f e ffe c tiv e n e s s . Key c rite ria used in th e in v e n to ry p u b lic a tio n s
are:
Figure 6: D is trib u tio n o f t h e p u b lic a tio n s a c c o rd in g to p rin c ip a l
g o v e rn m e n t 16
8 personal initiative
T he fo llo w in g fig u re s h o w s th e p o lic y e v a lu a tio n u n d e rta k e rs (fig u re 7). H a lf o f th e p o lic y
e v a lu a tio n s fro m th e in v e n to ry seem to have been u n d e rta k e n by u n iv e rs itie s , fiv e o f
w h ic h to o k place w ith in th e scop e o f research c o n d u c te d by o n e o f t h e s u p p o rt cen tres.
Figure 7: D is trib u tio n o f t h e p u b lic a tio n s a c c o rd in g to p ro v id e r
9 g o ve rn m e n t
u n iversity 12
2 advisory board
1 en g in e e rin g agency
A side fro m th e large n u m b e r o f in v e s tig a tio n s u n d e rta k e n by u n iv e rs itie s , m a n y have
been c o n d u c te d by g o v e rn m e n t as w e ll (9 p u b lic a tio n s ): by Fle m ish s c ie n tific in s titu ­
tio n s , th e C o u rt o f A u d it, a m u n ic ip a lity , OVAM, th e E uro pean C o m m is s io n , th e
E uro pean E n v iro n m e n ta l A gen cy a n d th e fe d e ra l P la n n in g B ureau. T h e re m a in in g p u b li­
c a tio n s are u n d e rta k e n by th e M iN a C o u n c il, SERV a n d an E n g in e e rin g A gency.
2
. 7 - GENERAL A N D C O NCL US IVE Q U E S T I O N S
T h e p re v io u s pa ra g ra p h s d e s c rib e a p u b lic a tio n c la s s ific a tio n based o n va rio u s
ch a ra c te ris tic s (a c c o rd in g to e n v iro n m e n ta l to p ic , e v a lu a tio n ty p e , p o lic y typ e ,
MI RA-PE
in s tr u m e n t ty p e an d a c c o rd in g to p rin c ip a l and p ro v id e r). The analysis o f t h e
in v e n to rie d p u b lic a tio n s w ill be c o m p le te d by a s k in g several q u e s tio n s , fo c u s in g
o n aspects th a t w e re discussed e a rlie r in th is d o c u m e n t.
2005
C a n
t h e
e v a l u a t i o n s
b e
u s e d
f o r
En v i r o n m e n t a l
P o l i c y
P l a n
|
f o l l o w
- u p ?
ro
e n v iro n m e n ta l to p ic , p o lic y in s tru m e n ts , p o lic y ty p e , ta rg e t g ro u p ...). W e trie d to a lig n
th e c la s s ific a tio n as m u c h as p o s s ib le w ith th e se c a te g o rie s . I f th e p o lic y m a k e rs and th e
p o s s ib le th a t s o m e in v e n to ry e v a lu a tio n s have a lre a d y been used fo r th is p u rp o se .
A re
t h e r e
a n y
i n v e n t o r y
e v o l u t i o n s
c o m p a r e d
t o
W h e n w e lo o k a t th e e v a lu a tio n type s, it appears th a t o f t h e p o lic y e v a lu a tio n s b e tw e e n
p e rcentage s are 4 0 % , 4 8 % an d 12% re s p e c tiv e ly . A p p a re n tly th e p o lic y pla n s w e re
e va lu a te d to a lesser e x te n t. A p o s s ib le e x p la n a tio n fo r th is is th e fa c t th a t in MIRA-PE
w h ic h lead to an increase in th e p la n e v a lu a tio n s o f MIRA-PE 2003, se e in g th a t a d viso ry
UNDER
20 03 re c o m m e n d a tio n s fro m th e M iN a Board and SERV w e re a d ded to th e in v e n to ry ,
2003-2005
1996 and 20 03 (MIRA-PE 20 03, 57 p u b lic a tio n s ) 38 % w e re p ro d u c t e v a lu a tio n s , 37 %
w e re process e v a lu a tio n s and 25% w e re p la n e v a lu a tio n s . In MIRA-PE 20 05 th e se
POL I CY
M IR A -P E 2 0 0 3 ?
ENVIRONMENTAL
p e o p le in c ha rge o f t h e M IN A -p la n fo llo w - u p w is h to e v a lu a te th e p o lic y by m eans o f
an in -d e p th s tu d y , p o lic y e v a lu a tio n s are a v a ila b le fo r th is p u rp o s e (cf. in v e n to ry ). It is
FLEMI SH
M IN A -p la n c lu s te rs th e p la n n e d p o lic ie s based o n several fe a tu re s (a c c o rd in g to
b o a rd s e s p e c ia lly re fe r to p o lic y plans.
THE
W e n o te th a t th e e n v iro n m e n ta l to p ic s in th e p o lic y e v a lu a tio n re p o rt o f 2003 w e re
MICROSCOPE
cla ssifie d a c c o rd in g t o 'k e y ’ to p ic s . T o p ics such as e n v iro n m e n ta lly ha zardous
substances, a c id ific a tio n and p h o to c h e m is try w e re n o t in c lu d e d in th e analyses. The
c la s s ific a tio n in MIRA-PE 2003 w as as fo llo w s : m a n u re p o lic y e v a lu a tio n s (4 % ), n a tu re
(15% ), s o il (2 % ), w a ste (28 % ) , w a te r (9 %) and c lim a te (0 %). T h e e v a lu a tio n s th a t d id n ’t
fo c u s o n a s p e c ific to p ic o r th a t e va lu a te p o lic ie s w ith several to p ic s (ca te g o ry 'g e n e ra l’)
m a de up 41% o f t h e to ta l. In MIRA-PE 2005 th is is 3.5 % , 17.3%, 0 % , 2 0 . 5 % , 10.2% and
41.5% re s p e c tiv e ly (fig u re 4 ). 7% o f th e s e e v a lu a tio n s regard e v a lu a tio n s o n c lim a te
p o lic y . So th e share o f m o s t e n v iro n m e n ta l to p ic s has re m a in e d c o n s ta n t sin ce 1996.
T h e s hare o f s o il p o lic y e v a lu a tio n s has re duced an d was c o m p e n s a te d by an increase
in c lim a te e v a lu a tio n s . It has t o be n o te d th a t w e are d e a lin g w ith lim ite d case s tu d ie s
w h ic h o n ly d eal w ith m in o r fra g m e n ts o f t h e c lim a te p ro b le m s (H e ye rick, 2003,
De B ruyn & Bachus, 20 05).
A r e
t h e r e
t i o n s
a n y
s i m i l a r i t i e s
R E G A R D I N G
C R IT E R IA ,
b e t w e e n
DATA
t h e
i n v e n t o r y
C O L L E C T IO N
A N D
e v a l u a
­
M E T H O D ?
P aragraph 2.5 s h o w s th a t th e re are m a n y s im ila ritie s b e tw e e n th e c rite ria used by th e
a u th o rs f o r t h e p r e d ic tio n o r e x p la n a tio n o f p o lic y success o r fa ilu re (e ffe ctive n e ss).
C o o rd in a tio n , le g itim a c y , cap acity, k n o w le d g e , readiness and in te g ra tio n are c rite ria
re o c c u rrin g in v irtu a lly all e v a lu a tio n s . E v aluatio ns fo c u s in g o n e ffe c tiv e n e s s are th e
e x c e p tio n .
T h e m e th o d fo r da ta c o lle c tio n th a t is used m o s t fre q u e n tly is lite ra ry stu d y, s o m e tim e s
c o m b in e d w ith in te rv ie w s . Surveys a n d p a rtic ip a tin g o b s e rv a tio n are used f o r t h e
c o lle c tio n o f (q u a lity ) da ta (B aetens e t a l., 2005, M o n s ie u rs , 2 0 0 4 , OVAM , 2003a,
OVAM , 2 0 0 3 b , G o m m e rs e t a l., 20 03).
T h e re is n o sha red m e th o d f o r t h e actu al e v a lu a tio n . T h e re is n o o n e re cip e . W h ic h
m akes sense, because fo r each e v a lu a tio n s tu d y th e m e th o d s and te c h n iq u e s have to
be a d ju s te d to th e research q u e s tio n s . H ow ever, th e re are g e n e ra l m e th o d o lo g ic a l
re fe re n ce s and m a n u a ls . These are used an d m a de e x p lic it m o re o r less fre q u e n tly
d e p e n d in g o n th e q u a lity o f t h e stud y.
A re
OF
T H E R E
T H E
A N Y
S I M I L A R I T I E S
IN V E N T O R Y
W I T H
B ET W EE N
RESPECT
TO
T H E
PO LIC Y
E V A L U A T I O N S
C O N T E N T ?
Th e e v a lu a tio n s d iffe r s tro n g ly w ith regard to to p ic and scop e. H ow eve r, th e re are
e v a lu a tio n s w ith a s im ila r c o n te n t.
• The e v a lu a tio n s fo c u s in g o n ‘ e n fo rc e m e n t’ s h o w to be p ro b le m a tic , as (fo rw h a te v e r
reason) th e t o p ic re m a in s d e fe c tiv e and p o llu tin g c o m p a n ie s g e t o f f s c o t-fre e . The
re c o m m e n d a tio n s p u t fo rw a rd in th e se s tu d ie s are re quests f o r m o re p e rs o n n e l,
b u t ju s t as w e ll fo r a b e tte r fo llo w - u p an d fo r c rim in a l p ro s e c u tio n (M o n s ie u rs , 2 0 0 4 ,
Segers, 2 0 0 4 , De B ruyn & Bachus, 20 05).
• A lso in te rn a tio n a l p o lic y has been s c ru tin is e d in several e v a lu a tio n s ; th e re are so m e
c o m m o n c h a ra c te ris tic s t o be n o te d : w e n o te th a t (to o ) little a tte n tio n is g iv e n to
th e actu a l im p le m e n ta tio n . T h e re is t o o little c o n ta c t and c o o rd in a tio n b e tw e e n
d e p a rtm e n ts and th e v a rio u s (re g io n a l) g o v e rn m e n ts in B e lg iu m (B aetens & B ursens,
20 05, Van R eybroeck & B ursens, 20 03, G eeraerts e t al., 20 0 4 ).
• It is a lso re m a rk a b le t h a t s tu d ie s e x a m in in g th e e ffe c tiv e n e ss o f a ce rta in p o lic y to p ic
s h o w th a t it is c o m p lic a te d , as th e re is a lack o f o b je c tiv e s (H e ye rick, 2003) a n d /o r data
(H e ye ric k , 20 03, Gysen, 20 03). W h e n it w as p o s s ib le t o e va lu a te th e e ffe ctive n e ss, it
a p p e a re d t o have be en p o s s ib le because s u ffic ie n t data and cle a r o b je c tiv e s w e re a t
h a nd (O VAM , 2003a, OVAM, 2 0 0 3 b ). A nd in o n e in s ta n c e (V ervaet e t al., 2 0 0 4 ), th e
analysis s h o w e d th a t th e o b je c tiv e s w e re m e t, b u t th a t th e re was reason to b e lie ve
th a t th e o b je c tiv e s w e re d o u b tfu l.
• T h e e n v iro n m e n ta l in te g ra tio n in o th e r m in is trie s (h o riz o n ta lly ) o r a t o th e r levels
(v e rtic a lly ) is a h o t t o p ic as regards in v e n to rie s (E uropea n E n v iro n m e n ta l A gency,
2 0 05, Van H u m b e e c k e t a l., 2 0 0 4 , Bachus e t a l., 2 0 0 4 , D e lba ere e t al., 2 0 0 5 ). To
a c c o m p lis h b e tte r (e n v iro n m e n ta l) re sults, a s tro n g e r e n v iro n m e n ta l in te g ra tio n in
o th e r p o lic y d o m a in s seem s re q u ire d . A ll fo u r e v a lu a tio n s s h o w th a t an e n v iro n m e n ta l
p o lic y (a nd fin a lly s u s ta in a b le d e v e lo p m e n t) is o fte n im p le m e n te d 's e p a ra te ly ' fro m
o th e r d o m a in s , w h ic h leads to o p p o s ite g o a ls and in c o n s is te n t a c tio n s . F u rth e rm o re ,
th e in te g ra tio n o f s u s ta in a b le d e v e lo p m e n t o fte n appears t o be lim ite d to p ro m is in g
MIRA-PE
in te n tio n s ta te m e n ts , b u t d o e s n ’t m e e t th e d e m a n d s fo r im p le m e n ta tio n .
a r e
t h e
e f f e c t s
o f
t h e
p o l i c y
e v a l u a t i o n s
o n
t h e
p o l i c y
?
2005
W h a t
Th e in v e n to ry e v a lu a tio n s p ro v id e re c o m m e n d a tio n s . In th e o ry , th e y can th u s p re s e n t
|
in fo rm a tio n f o r t h e p re p a ra tio n (th e e x -a n te e v a lu a tio n s ), th e e la b o ra tio n (th e e x -n u n c
e v a lu a tio n s ) o r m o d ific a tio n (th e e x -p o s t e v a lu a tio n s ) o f t h e p o licy. U n fo rtu n a te ly it
h a p p e n s a fte r th e y are p u b lis h e d . In th e OVAM e v a lu a tio n s o n e ffe ctive n e ss o f w aste
FLEMI SH
re m a in s u n c le a r w h a t e ffe c ts th e e v a lu a tio n s have o n th e p o lic y . We d o n ’t k n o w w h a t
p o lic ie s in s c h o o ls (O VAM , 2003a, OVAM, 20 0 3 b ) it is s ta te d th a t th e re su lts w e re used as
e v a lu a tio n is tra d itio n a lly a c c o m p a n ie d by a re sponse fro m th e c o m p e te n t m in is te r.
Is th is s u ffic ie n t to g u a ra n te e a d e q u a te a c tio n s ? T h is re q u ire s an in v e s tig a tio n o f t h e
p rin c ip a ls a n d an a n s w e r to th e q u e s tio n 'h o w are th e e v a lu a tio n s used and w h y ? ’.
T h is c o u ld be th e s ta rt o f a le a rn in g process f o r t h e p ro d u c tio n o f g o o d p u b lic te n d e rs ;
e x p e c ta tio n s .
suspect th a t th e e v a lu a tio n s carried o u t by th e g o v e rn m e n t its e lf co u ld have a m a jo r
im p a ct, since th e y are carried o u t by th e p e o p le in charge o f th e po licy. It th u s appears
A c tio n Plan fo r S choo ls was set u p based o n th is research. The effects o f t h e p u b lic a tio n s
o f t h e C o u rt o f A u d it a nd o f t h e C entre fo r A g ric u ltu ra l E co n o m y are p o s s ib ly sm a lle r,
in s p ite o f in te re s tin g re su lts. Even so, such e v a lu a tio n s are use fu l as th e a u th o rs are n o t
b o u n d by d e lic a c ie s (re g a rd in g th e p rin c ip a l) and in th e o ry th e e v a lu a tio n is c o n d u c te d
in a n e u tra l way. T h e e ffe c ts o f t h e M iN a C o u n c il s tu d y (K ayaerts, 20 05) are m o re d iffic u lt
t o assess: p ro v id e d th a t it is c a rrie d by th e p o lic y m a k e rs and th a t th e s tu d y is w e ll
p u b lic is e d , th e a m b itio n , as it is phrased in th e s tud y, (n a m e ly, to d e liv e r a c o n trib u tio n
to th e d e b a te o n in s tr u m e n t o p tim is a tio n fo r o p e n space), can b e co m e a re ality.
F lo w
I N D E P E N D E N T
ARE
T H E
IN V E N T O R Y
PO LIC Y
E V A L U A T IO N S ?
We fin d th a t s o m e e v a lu a tio n s w e re c arried o u t in te rn a lly . In te rn a lly m eans th a t th e
e va lu a tin g o rg a n is a tio n is its e lf in v o lv e d in th e d e v e lo p m e n t o r im p le m e n ta tio n o ft h e
p o lic y th a t it is e v a lu a tin g . In fact, th is o rg a n is a tio n is th e n e v a lu a tin g its o w n fu n c tio n in g .
T h e re fo re th e n e u tra lity o f t h e p o lic y e v a lu a tio n s m ay be q u e s tio n e d . It s h o u ld be
nu anced th a t th e e v a lu a tio n can be c a rried o u t by a d e p a rtm e n t o f t h e o rg a n is a tio n
w h ic h is n o t in c ha rge o f t h e p e rfo rm a n c e o f t h e e v a lu a te d p o lic y and can a c tu a lly a c ta s
MICROSCOPE
A. M o n s ie u rs (K .U .L e u v e n ) p ro b a b ly re m a in lim ite d . A fin a l re p o rt a ttra c ts less a tte n tio n ,
THE
because th e a fo re m e n tio n e d c o n d itio n w as n o t m e t. T h e e ffe cts o f t h e fin a l re p o rt by
UNDER
th a t th e tw o OVAM e v a lu a tio n s o n w aste p re v e n tio n in sch o o ls h e lp e d shape th e po licy. An
2003-2005
The 'p e rs o n a l in itia tiv e e v a lu a tio n s ’ (8 p u b lic a tio n s ) have a d iffe re n t p o lic y e ffe ct: w e
P OL I CY
th e researchers c o u ld be g iv e n th e o p p o r tu n ity to c o rre c tly e s tim a te th e g o v e rn m e n t’s
ENVIRONMENTAL
a s ta rtin g p o in t f o r im p r o v in g th e p o lic y fo r th a t s p e c ific ta rg e t g ro u p . The C o u rt o f A u d it
a n e u tra l referee (as fa r as th is is p o ssib le fo r a g o v e rn m e n t in s titu tio n ). In te rn a l research
offe rs th e m a jo r ad vanta ge th a t researchers have ready access to th e necessary in fo rm a tio n
and a lre a d y have an in s ig h t in to th e processes. H o w e v e r th e in v o lv e m e n t a n d , in so m e
cases, years o f e x p e rie n c e in th e research fie ld c o u ld lead t o biased research. T h e re su lts
w ill th e re fo re have to be in te rp re te d w ith th e necessary c a u tio n . A lso e x te rn a l e v a lu a tio n
can be biase d, fo r e x a m p le w h e n e c o n o m ic in te re s ts are c o n s id e re d o r w h e n th e
in v o lv e m e n t w ith th e p rin c ip a l(s ) b e c o m e s to o close, th e re q u ire d o b je c tiv ity w ill
d isap pea r.
W h a t
i s
t h e
q u a l i t y
o f
t h e
i n v e n t o r y
p o l i c y
e v a l u a t i o n s
?
W e d id n o t g o in to d e ta il w ith regard to th e q u a lity o f t h e p re s e n t p u b lic a tio n s
(m e th o d o lo g ic a l accuracy, re a d a b ility , tra n s p a re n c y ...). In d e p e n d e n t a n d th o ro u g h
research w o u ld be re q u ire d . In o u rs e le c tio n w e d id e x c lu d e p u b lic a tio n s fo r w h ic h
th e an alysis fra m e w o rk c o u ld n o t be id e n tifie d .
In th is c o n te x t w e w o u ld lik e t o p o in t o u t s o m e d e fic ie n c ie s o f s o m e o f t h e in v e n to rie d
e v a lu a tio n s . In s o m e in sta n ce s th e re are n o e x p lic it research q u e s tio n s , c rite ria o r
re c o m m e n d a tio n s . T h is c o m p lic a te s n o t o n ly th e re a d in g o f t h e te x t, b u t a lso th e
re trie v a l o f t h e e s s e n tia l in fo rm a tio n (n a m e ly th e e v a lu a tio n o f th e c o n c e rn e d p o lic y
an d th e lessons t o be d ra w n fro m it).
M a n y p u b lic a tio n s d o n o t in c lu d e a s u m m a ry . O b v io u s ly , it is necessary to c a p tu re a ll
o f t h e nu ances in such a s tu d y - a s u m m a ry a lo n e can th e re fo re n o t s u ffic e - b u t w ill
th e p e o p le w h o w ill have to a ct o n th e p u b lic a tio n s ta k e th e tim e to g o th ro u g h th e
s o m e tim e s v e ry e la b o ra te research re p o rts ?
We are th e re fo re in fa v o u r o f p o lic y e v a lu a tio n s th a t are a c c o m p a n ie d by a s u m m a ry,
a c le a r s ta te m e n t (cf. 'd e c la ra tio n o f in te n t’ in th e C o u rt o f A u d it e v a lu a tio n s ) a lo n g
w ith e x p lic it research q u e s tio n s , e v a lu a tio n c rite ria and re c o m m e n d a tio n s .
2 . 8 . IN C O N C L U S I O N :
C O M P L E T I O N OF T H E M A T R I X
Th e actu a l p u rp o s e o f p o lic y e v a lu a tio n re p o rts - aside fro m p re s e n tin g a c u rre n t sta te o f
a ffa irs a nd an o v e rv ie w o f e n v iro n m e n ta l p o lic y e v a lu a tio n s - is t o in fo rm p o lic y m a k e rs
so as t o o ffe r a fo u n d a tio n fo r th e ir p o lic y (ch oices). T h is re p o rt c o n ta in s 24 p o lic y
e v a lu a tio n s , w h ic h a ll e va lu a te s o m e e n v iro n m e n ta l p o lic y a spect(s). The m a trix in
ta b le 3 s h o w s th a t s o m e 'g a p s have be en f ille d ’ (c o m p a re d to MIRA-PE 2 0 03), b u t th a t
a lo t o f w o rk is s till t o be d o n e t o c o v e r th e e n tire e n v iro n m e n ta l p o lic y . T h e d a rk e r
th e squ are (fro m y e llo w to g re e n to b lu e ), th e m o re p o lic y e v a lu a tio n s have been
c o n d u c te d .
T a ble 3: M a trix o f e v a lu a te d p o lic y to p ic s in MIRA-PE 2003 and MIRA-PE 2005
policy type*
environm ental topic
region
international
integration
target group
coordination
MIRA-PE
general
climate
2005
nature
I
water
fO
photochemistry
FLEMI SH
ozone layer
acidification
hazardous substances
fragmentation
* region = re g io n -o rie n te d policy; in te rn a tio n a l = in te rn a tio n a l policy; in teg ratio n
co o rd in a tio n = c o o rd in a tin g policy
in tegrated g o ve rn m e n t policy; targ e t grou p - targ e t gro up policy;
ENVIRONMENTAL
soil
POL I CY
2003-2005
UNDER
THE
MICROSCOPE
3. Evaluation summaries
T h is s e c tio n c o n ta in s th e s u m m a rie s o f t h e p o lic y e v a lu a tio n s in o u r in v e n to ry ( ta b le 1).
Each s u m m a ry c o n s is ts o f f o u r su b s e c tio n s :
• d e fin itio n o f p r o b le m and research q u e s tio n s o f t h e p o lic y e v a lu a tio n ;
• th e m e th o d s a nd c rite ria used;
• key fin d in g s a n d c o n c lu s io n s ;
• re c o m m e n d a tio n s .
W e d o n o t m a ke a n y s ta te m e n ts a b o u t th e q u a lity o f t h e p u b lic a tio n s . W h e n th e
re c o m m e n d a tio n s are very e la b o ra te , s p e c ific and d e ta ile d , w e a u to m a tic a lly re fe r to
th e actu a l p u b lic a tio n .
Study in preparation ofthe federal
environment-specific product policy
A. H e y e ric k • C entre fo r S ustainab le D e ve lop m ent, G hent U n iv e rsity (U C e nt), 2 0 0 3
Characteristics
E N V IR O N M E N T A L
T O PIC
general, cross-topic (waste, dimate)
PO LIC Y
TYPE
target group policy
Definition o f problem and research
Findings
questions
The first part ofthe research clearly lacks
This study examines the federal product
well-defined objectives, which makes it
pol icy. It consists o f two large sections. The
impossible to conduct an effectiveness
first question that is examined is whether a
evaluation. In the second part we found
product policy evaluation per se is in fact
the following:
possible. Secondly, three product policy
• Despite government efforts, the
TYPE
OF
PO LIC Y
IN S T R U M E N T
instruments will be evaluated: the
eco-label does not seem to be a success
legal, economic, and social policy
instrument
European eco-labels, the eco-bonuses and
on the Belgian market. A product
the implementation ofthe European
evaluation with effectiveness as its
TYPE
OF
E V A L U A T IO N
guidelines regarding fuel quality.
criterion is necessary in orderto figure
ex-ante evaluation, product evaluation
Method and criteria
out what is the cause o f this; here too
The author conducts a classic product
there's a lack o f objectives to carry out
evaluation. Ele tests the objectives against
the evaluation. On top ofthat, there are
the output, outcome and environmental
insufficient data with regard to the
effects. The author collects the qualitative
number o f companies with eco-label,
P R I N C I P A L
Federal Ministry o f Environment, product
policy department
and quantitative data from the involved
the number o f products with eco-label,
administration by document analysis.
the type o f products, the label
The criterion is effectiveness. For the
recognition by the consumer...
explanation o f pressure points and success
factors the following criteria are used:
• The evaluation o f the eco-bonuses is a
plan evaluation, because the legislation
clarity in the objectives, (communication)
is ready, but notyetapproved. The reuse
transparency and integration in and
objective was the driving principle in the
outside o f the policy domain.
composition ofthe original legislation.
During preparations and lobbying, the
standards (reduce C0 2) is hard to
of'reuse' and has to illustrate how the
law resulted ina mixture o f recycling and
quantify, seeing that there are many
generation o f waste can be avoided.
reuse objectives, with little coherence
more sources causing C0 2 emissions.
and sometimes even contradicting ele­
However, it should not spoil the effects
o f years o f separated collection by the
population. To achieve the environmental
fuel targets the importance ofthe
between 'recycling' and ‘reusing’.
approach a product policy by means o f a
instruments mix is referred to (e.g. a
The implementation ofthe European
plan and to clearly define the objectives
financial instruments help in the
guidelines regarding fuel quality is
and processes. A solid inventory system
transition from old to new product
2005
Recommendations
The time has come for policymakers to
MIRA-PE
ments. The population (one ofthe law’s
target groups) has trouble distinguishing
I
has to be set up. As regards the eco-
standards). Clearly, fuel standards alone
bonuses, the focus must shift back to reuse
are not sufficient for achieving the
environmental objective.
and has to be translated into quantifiable
targets. A mutual coordination has to be
organised. The driving environmental
achieved between the waste policy and the
objective is not met (i.e. to diminish C0 2
product policy. In addition, an information
emissions). The contribution offuel
campaign has to explain the exact meaning
ENVIRONMENTAL
obligations without major problems.
The inspection on the observance is well
POL I CY
E. Van R eybroeck, P. Bursens • Centre o f Expertise on A d m in is tra tiv e O rg a n isa tio n Flanders, C a th o lic U n iv e rs ity o f Leuven
(K .U . Leuven), 2003
UNDER
structures which have to translate the
European rules and regulations at a
This publication deals with the
Belgian level ('downloading') and must
Characteristics
ensure the Flemish input in the European
E N V IR O N M E N T A L
process ('uploading') play a central role
general
study on Europeanisation conducted at
in the report.
PO LIC Y
T O PIC
TYPE
international policy
Method and criteria
TYPE
OF
P O LIC Y
IN S T R U M E N T
For the Environment domain on the one
called ‘The Europeanisation o f Flemish
not applicable
hand, the intradepartmental structures
officials, members o f cabinet and
TYPE
OF
E V A L U A T IO N
are analysed (Coordination Committee
politicians: opinion, expertise and
ex-nunc evaluation, process evaluation
International Environmental policy (CCIM),
attention’ and will be discussed later on.
PRINCIPAL
Flemish Consultation International
Flemish government (Centre o f Expertise
The research question is h o w the Flem ish
Environmental policy (VOIM)...) and on
on Administrative Organisation Flanders)
m an ageria l system deals w ith the European
the other hand the interdepartmental
U n io n . This question will be answered
structures (e.g. between the departments
by an analysis o f the policy domains
Environment and Agriculture).
Environment, Agriculture, Education
Furthermore the expert committees, the
and Foreign Policy. The Flemish regional
council workgroupsand other consultation
MICROSCOPE
‘Europeanisation’ ofthe Flemish policy
structures. This is the first report on the
THE
Definition o f problem and research
questions
Organisation Flanders. A second part is
2003-2005
Europeanisation o f Flanders: a screening o f
the managerial structures
the Centre o f Expertise on Administrative
FLEMI SH
running smoothly. The target group was
well-prepared and could carry out the
platforms are discussed. The study is based
• Environment and Agriculture are
to set up a European cell within the
on interviews and empirical material
especially equipped for the preliminary
departments with clear, coordinating tasks
(files and reports). The criteria used in
policy stage: little attention goes to the
(this was already done forthe Environment
the analysis include transparency,
translation o f rules and regulations (the
department).
coordination, uniformity (of the formal
operational and implementation phase);
■ contact and uniformity between the
structures) and pro-activity.
structures and domains is limited.
Findings
Some key findings:
Recommendations
■ the sector 'Environment' has sufficient
Theauthors recommend developinga
formal tools for fulfilling European tasks
proactive strategy, to make the existing
compared to the sector 'Education1;
conversion database more accessible, and
Flemish structural consultation fo r sustainable
development
K. Bachus, H. B ru y n in c k x , M . P o e se n -V a n d e p u tte • HIVA, K .U .Leuven, 2004
Definition o f problem and research
Method and criteria
questions
This study provides an ex-ante evaluation
In the wake o f global attention for
o f a Flemish policy for sustainable
T O PIC
sustainable development (SD), also
development. The current situation is
general
Flanders is facing the challenge of
analysed, as well as the conditions for an
implementing a performant sustainable
efficient, performant sustainable
development policy. The question to be
development policy. This is done by a
answered is what such a Flemish SD policy
document analysis and interviews.
Characteristics
E N V IR O N M E N T A L
PO LIC Y
TYPE
integrated government and coordinating
policy
TYPE
OF
PO LIC Y
IN S T R U M E N T
not applicable
TYPE
OF
should look like. The study consists o f five
Furthermore a comparative study is set
research questions:
up with other countries. Case studies in
Canada, Germany, the Netherlands and
E V A L U A T IO N
• What are the current activities ofthe
the Belgian federal government serve as
ex-ante evaluation, process evaluation
Flemish government with relation to SD?
a foundation for the evaluation. At the
P R I N C I P A L
• What should the future SD
end o f the study, three round-table
Flemish government (AMINAL)
institutionalisation in Flanders look like?
conferences were held with the
• Is there any need for a Flemish SD
policymakers and stakeholders involved
decree?
to supply additional relevant information.
• Does Flanders need its own Flemish
Strategy for SD? If yes, in what form
The criterion is effectiveness (how can SD
and what is its relation with a possible
become effective in Flanders?). By means
national Strategy for SD?
ofthe criteria Political Support, Integration
• What kind o f (multi-stakeholder)
and Coordination, the authors offer several
participation goes hand in hand with
scenarios for implementing an effective SD
a Flemish SD strategy?
policy.
departments. Actions for SD initiated by a
provided that the Knowledge Cell
the key elements in a sustainable
vertical policy domain could meet with
Statutory Moderation keeps an eye on
development policy. Therefore, the
opposition from competing policy
the content o f the document. The
development o f a performant SD policy
domains.
authors ask not to allocate the 'SD
central element is the participation ofthe
Recommendations
domain but to the minister-president.
Flemish stakeholders through advisory
• The authors propose to set up a
A Strategy for Sustainable Development
should be based on this principle. Another
doesn’t affect other processes and
portfolio’ toa minister in a vertical policy
boards or networks. Some ofthe important
Coordination cell for Sustainable
peripheral conditions include available
Development. The coordination cell is
process. As a result all policy domains
budgets (SD should be more than words
to support the other policy domains in
should submit the concrete action
2005
for this is to get the policy accepted by all
Horizontal policy integration is one of
MI RA-PE
Findings
is required, with the emphasis on the
|
The involving o f Flemish stakeholders
these peripheral conditions have been
Parliament. The interdepartmental
could take place through an advisory
met. Furthermore, the changing cabinets
consultation should be ensured through
board. A special advisory board is not
causean instable climate for SD. So as to
a consultation committee.
recommended. The existing advisory
assure the horizontal integration, SD
• A Flemish Decree on SD is recommended,
boards SERV and MiNa Council could
provided that not everything is strictly
have advisory authority on the condition
department. The most important reason
defined, provided that the decree
that all groups are involved.
K. G eeraerts, P. B ursens, P. Leroy • Centre o f Expertise on E n viro n m e n ta l P olicy Sciences, U n iv e rsity o f A n tw e rp (U A ), 2 0 0 4
2003-2005
Border crossing Flemish environmental policy
POL I CY
should be the responsibility o f a horizontal
ENVIRONMENTAL
programs to the SD Coordination Cell.
these efforts and report to the Flemish
FLEMI SH
their efforts pertaining to SD; judging
platform. Documents show that not all o f
^
alone), long-term strategy and a political
UNDER
THE
Definition o f problem and research
towards the international forums; (2) what
is the coordination like within the Flemish
This publication deals with the way in
government; (3) what impact does Flanders
Characteristics
which the Flemish environmental
have on the international scene. The
E N V IR O N M E N T A L
government deals with the international
authors verify whether cooperation and
general
decision-making process pertaining to
mutual coordination are at hand and to
PO LIC Y
the environment. The research question
what extent every matter is arranged in a
international policy
is in w h ich w a y (a nd to w h a t extent) does
formal way.
TYPE
Flanders take p a rt in the re a lisa tio n o fth e
OF
T O P IC
TYPE
PO LIC Y
IN S T R U M E N T
not applicable
Method and criteria
in te rn a tio n a l e n v iro n m e n ta l policy. In the
TYPE
OF
E V A L U A T IO N
The authors have conducted this research
first place the formal and informal
ex-nunc evaluation, process evaluation
by means o f interviews and docu ment
participants, settings and (communication)
PRINCIPAL
analyses. A database was set up to offer
channels are described and analysed. An
Flemish government (Centre for
support to the research and in which all
evaluation will follow. Questions with
Environmental Policy sciences)
environmental agreements have been
regard to the subject: (1) what route included. The authors used an analysis
influence channel - does Flanders take
framework based on international
MICROSCOPE
questions
Findings
Recommendations
the classification ofthe Flemish preparation
In short, the authors note that the Belgian
The authors make several recommenda­
ofand participation in the international
institutional organisation creates a great
tions: importance o f federal loyalty; the
environmental policy: intra-stately versus
dependency between regional and federal
set-up o f a systematic consultation
extra-stately, cooperative versus non-
governments. This leads to a cooperative
procedure exceeding the boundaries
cooperative, direct versus indirect
attitude and respect towards the federal
between policy domains; the review of
literature. This analysis framework facilitates
participation and formal versus informal
government. An additional explanation lies
the CCIM; the set-up o f an effective interest
participation. The criterion used for the
in the fact that during the research period,
defence at the European Parliament... As
analysis evaluation is internal efficiency (of
the stakeholders had the same political
the recommendations are quite elaborate,
the processes) with the purpose o f achieving
colour. Flanders and Belgium have good
we would like to refer to the book itself
the best possible position in the international
international negotiating skills, but this
for a full summary with the authors’
policy. Subcriteria include amongst others
reputation is challenged byadefective
arguments.
the degree o f openness, precision,
implementation o f agreements.
cooperation and data transfer.
Follow-up ofenvironm ental complaints by the
Environmental Inspection: an efficient instrument fo r
high supervision?
A. M o n s ie u rs • F aculty o f Law , K.U . Leuven, 2 0 0 4
Characteristics
E N V IR O N M E N T A L
TO PIC
Definition o f problem and research
complaint handling could function as
questions
an efficient instrument for the high
This final report investigates the supervision
supervision).
o f class 2 and 3 companies by municipalities
Method and criteria
general
and the Environmental Inspection. The
The methods used are participating
P O LIC Y
TYPE
investigation starts from the observation
observation and a questionnaire for
target group policy and integrated govern­
that the supervision is rather deficient
municipalities, which tries to pinpoint their
ment policy
and that the municipalities - which are
enforcement problems. By means o f the
TYPE
OF
PO LIC Y
IN S T R U M E N T
responsible forthe supervision ofthe
case follow-up system, analysis is carried
enforcement
companies - do not always intervene when
out to see whether the Environmental
TYPE
OF
E V A L U A T IO N
required. In addition it seems that the
Inspection can guarantee a satisfactory follow
ex-nunc evaluation, process and product
Environmental Inspection offers inadequate
-up ofthe complaints. The evaluation is
evaluation
so-called 'high supervision’ with regard to
mainly performed based on the following
P R IN C IP A L
these companies. This results in a lack o f
criteria: efficiency, knowledge and capacity.
personal initiative (final report)
action against some nuisance generating
class 2 and 3 companies. The report analyses:
Findings
• the reasons for the deficient supervision
The author finds that municipalities intervene
by the municipalities and the bottlenecks
in companies only asa preventative
at the Environmental Inspection with
measure when environmental permits are
regard to the required high supervision;
requested. (The reason, according to the
■ a possible solution from the viewpoint of
the Environmental Inspection (i.e. the
investigation, is that enforcement is one
o f many tasks.) The questionnaire showed
that the municipalities have a wide range o f
municipalities do not get a warning from
This person should be an expert,
tasks to fulfil, a lack o f personnel and that
the Environmental Inspection (which
preferably someone with an
the complex environmental legislation is a
has to exert high supervision on the
independent status.
limiting factor to effective supervision. As a
municipalities) and the class 2 institutions
Also the E nviron m enta l Inspection
result, complaints for which the municipal­
remain uninvestigated and forthat matter
de p a rtm e n t should hire more personnel
ities are responsible, are forwarded to the
unpunished.
in orderto exert high supervision.
Environmental Inspection. The
Follow-ups are best performed through
Recommendations
Environmental Inspection therefore
a complaint handling system, which the
• To eliminate insecurity with regard
receives hundreds o f complaints regarding
municipalities and the Environmental
to the enforcement ofthe m u n icipal
topics which are not a priority within the
Inspection have access to. The
environmental officials (lack o f experience,
scope o f their primary inspection tasks. As
municipalities ought to be trained rather
no enforcement training, work load, no
it is, the Environmental Inspection already
than punished by the Environmental
systematic planning...), it is necessary to
lacks the personnel to effectively supervise
Inspection, seeing as in most cases the
hire an extra environmental official, for
class i companies, leave alone the
problem o f the municipalities is not
authorised supervision/ inspection only.
companies o f class 2. Therefore many
caused by unwillingness but by incapacity.
Pilot project VLAREM supervision Klein-Brabant,
‘Transportsector’
R. Segers • P u b lic a tio n b y th e m u n ic ip a lity P uurs
Definition o f problem and research
Method and criteria
questions
Through local visits, in fo m eetings a nd letters
This publication is the report o f a project
the target group becomes involved and
initiated by the Office ofthe Public
is aware o f what is happening when
E N V IR O N M E N T A L
Prosecutor. The project originated after
and where. The criteria ofthe project
general
a double observation: on the one hand
evaluation are 'a b ility ' (municipalities:
PO LIC V
companies did not comply with the
administrative follow-up; companies:
target group policy
environmental rules and regulations and
arranging o f the company premises and
TYPE
on the other hand municipalities were not
equipment), ‘g o o d w ill’ (the company
enforcement
well aware o f everything going on within
attitude) and ‘know led ge’ (inform
TYPE
Characteristics
OF
OF
T O P IC
TVPE
P O LIC Y
IN S T R U M E N T
E V A L U A T IO N
their territory. The author o f the document
companies about VLAREM). Additionally,
ex-post evaluation, process evaluation
summarises how the municipality(ies)
the cooperation between municipalities,
PRINCIPAL
Puurs (and Bornem and Sint-Amands) for
Courts o f Law, police and the Flemish
initiated by the Office ofthe Public
7 years tried to regulate, inform and
environmental authorities is discussed.
Prosecutor
technically support companies that did
Findings
not comply with environmental law, in
• A large number o f companies simply
cooperation with the Office ofthe Public
ignore the first warning notices.
Prosecutor, the police and the Environmental
• Administrative follow-up seems difficult:
Inspection department o f AMINAL. The
duringand afterthe project new companies
document provides an evaluation o f this
without a permit were added and
process, with the focus on the final results.
other companies had suspended their
Operators with good intentions expect
activities or were been taken over by
inspections to be a positive c o n firm a tio n
another company (without notifying
o f th e ir approach a nd investm ents.
the municipality).
The attitude ofthe operators has to
• Auto-control doesn’t seem to work for
change. The necessary information
most companies. In 19 cases there were
has been offered, especially to class 2
companies which, after 7 years o f
companies. Even so, many companies
project, did not succeed in acquiring
often do not even try to read or under­
a full permit for their activities. With
stand this information. The checklist
regard to the storage o f fuels and waste
was drawn up using clear and simple
products, no efforts were made by
wording so that this cannot be an excuse
most companies, while one quarter
nottoactupon it. The question rises
didn’t even own an oil secretor which
whetherthe future system o f’integral
happened to be one ofthe original
environmental conditions' can have
targets ofthe project.
a positive impact with thisoperator
■ Some fundamental shortcomings exist
that present a danger to the operator
attitude.
The use ofthe checklist is more
(fire, security) and the environment
convenient in the field than the official
(particularly regarding water and soil).
VLAREM texts. The developm ent o f
If nothing happens, the financial
the checklist seems to be a task fo r highe r
repercussions could be enormous.
g o vernm e nt; in th is w ay everyw here things
■ Some companies do not have an
adequate infrastructure; without
reconstruction works they cannot meet
the VLAREM criteria. Other companies
have problems because oftheir lack o f
know-how in this field; they don't have
anyone to supervise them and don’t
initiate any actions.
Recommendations
• Despite the offered information and the
favourable attitude ofthe Office ofthe
Public Prosecutor and the supervising
officials, for many years now some
companies do not take the inspections
seriously. Criminal prosecution is the only
option; otherwise projects will continue
to dragon. It seems reasonable to offer
companies first an administrative
option, except when dealing with acute
pollution or danger, but further followup should not be a permanent burden
on the municipal services.
are checked in the sam e m anner.
P. Van H u m b e e c k , I. D ries, ). Larosse • IW T Studies, 2 0 0 4
MIRA-PE
Linking Innovation Policy and Sustainable Development
in Flanders
2005
Findings
• At this point there is no integrated policy
I
Definition o f problem and research
questions
^
Innovation policy and sustainable
for sustainable development covering all
Characteristics
policy domains. Actions for sustainable
E N V IR O N M E N T A L
development are ad hoc and are specific
general
for each sector.
PO LIC Y
domains, and include a heterogeneous
■ Environmental policy, economic policy
TYPE
integrated government policy,
a coordinating policy
real connection at the moment, despite
TYPE
policymakers today are restrained by a
the (economic and ecological) advan­
not applicable
material and energy intensive industry
tages generated by cooperation and
TYPE
(and the resulting environmental
integration. During the government
ex-ante evaluation, process and plan
problems). The challenge remains to
period o f 1999-2004 an Environmental
evaluation
innovate such a system and to achieve a
Innovation Platform (MIP) was created.
P R IN C I PAL
knowledge-based economy. In order to
The Platform could enable a better
OESO and the Flemish government
do so, innovation policy and sustainable
synergy between innovation policy,
(Department Innovation and Science)
development policy have to go hand in
environmental policy and energy policy.
Recommendations
this is possible and what conditions have
In order to guarantee coordination
between innovation policy and environ­
mental policy, along with the integration
This ex-ante evaluation first o f all explores
o f sustainable development, actions have
how the sustainable development policy is
to be undertaken in the policy domains.
organised in Flanders. Secondly, the links
The MIP could enable the integration of
between the innovation policy and the
these initiatives. An important condition
sustainable development policy are
for achieving an innovating economy and
analysed. Finally, possible scenarios are
an innovating environmental policy is
investigated fora synergy between both
political support. Moreover, the ministries
policy domains. All o f this is done by
should share common objectives.
means o f a document analysis. The process
evaluation criterion is the integration
between both domains and the integration
within the existing vertical policy domains.
For this reason, coherence is examined in
the policy objectives, the political platform
as well as transparency.
E V A L U A T IO N
MICROSCOPE
Method and criteria
IN S T R U M E N T
THE
to be met.
OF
P O LIC Y
UNDER
hand. The authors further analyse how
OF
2003-2005
and technological innovation have no
present. This publication will analyse how
P OL I CY
o f these policy domains is rather limited at
ENVIRONMENTAL
group o f participants. The institutionalisation
T O PIC
FLEMI SH
development policy are relatively new.
They involve complex, horizontal policy
The Europeanisation o f Flemish officials, members o f
cabinet and politicians: opinion, expertise and attention
M. B aetens, P. Bursens • C entre ofE x p e rtis e on A d m in is tra tiv e O rg a n isa tio n Flanders, K.U . Leuven, 2005
Characteristics
E N V I R O N M E N T A L TOPI C
Definition o f problem and research
Evaluation criteria include knowledge,
questions
interest, integration (of organisations
The authors examined how ‘Europe’
inside and outside ofthe policy domain),
interrelates with the policy culture of
complexity, transparency and capacity.
general
the members o f cabinet, the officials
POL I CY TYPE
and the members o f parliament. The
Findings
The broad outline ofthe findings with
international policy
Europeanisation ofthat culture is
regard to environmental policy is as fol­
TYPE OF POL I CY I N S T R U ME NT
discussed, asking whether European
lows:
not applicable
TYPE OF E V A L U A T I ON
awareness is incorporated into the
government practice o f these policy
• There is strong focus on the policy
operators. This question is important since
preparation and the legal transposition
on the one hand it enables a correct and
o f European rules, with little attention
Flemish government (Centre o f Expertise
timely implementation o f European rules
for the application o f these rules
on Administrative Organisation Flanders)
and regulations (downloading) and on the
(operational aspect). The environmental
other hand it optimises the Flemish input
administration is even less concerned
ex-nunc evaluation, process evaluation
P R I N C I P A L
in the European process (uploading).
than the agricultural administration.
• The competence o f agricultural officials
Method and criteria
reaches beyond their own region much
By means ofa large-scale questionnaire
more than for environmental officials
Note: see also the publication
for members o f cabinet, officials and
'Europeanisation o f Flanders: a screening
(probably because 'agriculture' has only
members o f parliament, it is verified what
o fth e managerial structures’
recently been regionalised). Close ties
European related activities they perform,
with colleagues in the Walloon provinces
what their knowledge o f Europe is Iike,
are beneficial to the European policy
what theirthoughts on Europe are like,
process.
whether Europe is involved in their daily
• Contacts are predominantly established
lives and finally what the complexity o f
with the European Commission and also
the decision process is like in relation to
there, environmental officials seem to
European issues. Based on these data and
have fewer contacts with the members
fieldwork at the institutions, the
o f commission than agricultural officials.
'Europeanisation' ofthe policy operators
In general contacts with the European
(of cabinet, civil sen/ice and parliament)
members o f parliament are inadequate.
is measured.
• Onascaleofexpertise(howwelldoes
In order to evaluate these elements, the
one know Europe?) all participants attain
authors focus on four policy domains,
less than half o f the total possible score.
amongst which ‘the environment’.
The environmental officials score
significantly lower on the scale for
European expertise than the agricultural
members o f parliament.
expertise, attention ...). After all,
enthusiasm towards additional training.
A joint Belgian position is difficult
Europeanisation enhances the will
There is rather a lot o f support for
seeing the complexity ofthe Belgian
and capacity to participate in the
Europe, but still the officials and
institutional make-up. This is also
European policy process with strong
members o f cabinet remain critical
obviously a handicap for Belgium
opinions and an elaborate strategy
(much more so than members of
(and Flanders).
and to correctly apply the rules and
parliament). A lot o f attention goes to
regulations. We therefore refer to
2005
the European 'culture’ (knowledge,
MIRA-PE
except - paradoxical ly - among
officials. These show for example more
Recommendations
Europe (e.g. case follow-up in relation
the full report.
I
The authors give a lot o f recommendations
^
to the European rules and regulations),
(about 30 pages) aimed at reinforcing
FLEMI SH
ENVIRONMENTAL
Carte blanche or corset?
Evaluation ofthe Cooperation Agreement
K. Bachus ■H IVA, K .U .Leuven, 2004
Findings
■ Participation o f the municipalities in
important function In dealing with
environmental problems. The Flemish
the Cooperation Agreement is more
Characteristics
limited than before.
E N V IR O N M E N T A L
T O PIC
■ The conditions for financial aid to
general
government supports the municipalities
the municipalities are very strict.
PO LIC Y
and provinces in the development o f a
This sometimes causes problems
an integrated government policy
for municipalities.
TYPE
OF
P O LIC Y
IN S T R U M E N T
founding instrument
local) feei that they are not on the same
TYPE
ofthe instrument went exactly as planned.
wavelength.
ex-nunc evaluation, process evaluation
OF
E V A L U A T IO N
The object o f this interim evaluation was
The municipalities have to report on their
P R I N C I P A L
to identify those areas that need to be
accomplishments. These reports do not
Flemish government (AMINAL)
adjusted by the Flemish government.
always reflect the situation in the field.
Method and criteria
Recommendations
The evaluation was conducted by means
The investigation was initiated by
o f document analysis, interviews and
the Flemish government: the
participating observation. The criterion
recommendations are therefore intended
is effectiveness: not the achievement
for them. Recommendations are formulated
o f environmental targets, but ofthe
with regard to the relation between the
structuring objectives o f the Cooperation
Flemish Region and the municipalities. The
Agreement (e.g. good communication
Flemish Region should pay more attention
with the municipalities, support, planning,
to the weaker municipalities, provide more
number o f participating municipalities).
and better communication and organise
more frequent field visits. An internet
forum could be another interesting option
(e.g. to make people familiarwith practical
examples). Furthermore, efforts should
be made to embed the environmental
service in all the municipalities.
MICROSCOPE
study verifies whether the Implementation
THE
• Both government levels (Flemish and
TYPE
UNDER
local environmental policy. Cooperation
agreements are used for that purpose. This
2003*2005
The local government level fulfils an
P OL I CY
Definition o f problem and research
questions
Enforcement: the missing link in the environmental
policy cycle?
T. De B ruyn , K. B achus • M IR A -P E 2005,2005(D u tc h version)
Characteristics
E N V IR O N M E N T A L
T O PIC
general, cross-topic (waste, climate, water)
PO LIC Y
TV P I
target group policy
Definition o f problem and research
the traditional inspection systems will
questions
take care o f this. Paradoxically, all parties
This study focuses on enforcement.
indicate that it is precisely the traditional
Enforcement is required to ensure the
inspection systems that present a lot o f
observance o f environmental rules and
problems.
regulations. This aspect is often neglected
Recommendations
when drafting new rules and regulations.
The researchers offer two types o f
TYPE
OF
PO LIC Y
IN S T R U M E N T
enforcement
The researchers therefore consider h o w
recommendations. The first type refers to
the inclusion o f enforcem ent measures a t the
recommendations for policy preparation
TYPE
OF
E V A L U A T IO N
s ta rt o fth e policy process m ay influence the
and policy formulating administration, the
ex-post evaluation, plan evaluation
execution and enhance its effectiveness.
second type includes recommendations
P R I N C I P A L
Method and criteria
for the Flemish government in general
Flemish government (VMM)
The study centres around four case studies:
pertaining to the organisation o f
the discharge o f domestic waste water, the
enforcement. As regards the first type, It
separated collection o f industrial waste,
is recommended to prepare a 'manual'
the maintenance and inspection o f heating
with enforcement options and tips which
systems and the integral environmental
guarantee optimal enforcement. As far
conditions. The authors investigate
as the latter type is concerned, the
whether and how the policymakers took
researchers demand attention for
into account eleven criteria when drafting
communication between the public
their rules and regulations: these criteria
prosecutor and the Environmental
include acceptance by the target group,
Inspection, to explain the reasons for
costs and benefits in case o f non-
dismissal.
compliance, knowledge and transparency
ofthe rules, informal chance o f inspection,
chance o f sanctions, chance o f detection ...
Findings
The policymakers pay a lot o f attention
to voluntary compliance: through
Information and other instruments target
groups are involved in the policy process
so as to increase voluntary compliance.
However, policymakers are aware that
supervision and inspection are o f para­
mount importance. They assu me that
The implemented sustainable development policy
P. D elba ere , P. D resselaers, N. G ouzée • Federal P la n n in g B ureau, 2005
MIRA-PE
This publication contributes to the
transparency ofthe texts and plans,
commitment ofthe federal government
planning...
Characteristics
I
multiple ministries, coordination between
federal and regional level), participation,
2005
Definition o f problem and research
questions
E N V IR O N M E N T A L
TO PIC
Findings
implemented sustainable development
PO LIC Y
TYPE
Since the introduction ofthe first plan, at
policy on a federal level. The document
a coordinating policy
FLEMI SH
general
^
to perform an annual evaluation ofthe
least 56 % ofthe measures are now in
TYPE
OF
PO LIC Y
IN S T R U M E N T
the operational phase, 20 % are under
ofthe content ofthe two existing Plans
supporting instrument
preparation and 32% are being executed.
regarding Sustainable Development (2000-
TVPE
OF
E V A L U A T IO N
Nothing further is known with regard to
2004 and 2004-2008); (2) analysis and
ex-post evaluation, product evaluation
the other measu res as no other data are
evaluation ofthe processes in which the
P R I N C I P A L
kept by the reporters. The two federal plans
law
and the sector-specific policy plans ofthe
Strategy for Sustainable Development o f
federal government are overlapping.
policy levels other than the Belgian federal
government. During the investigation, a
Sustainable Development, but no further
plan evaluation o f the (kinds of) measures
suggestions are made to increase coherence.
taken, delayed, cancelled etc. was made.
The authors note five routes to
evaluation the process and output
improvement:
indicators are used (e.g. used budgets,
• improvements could be made with regard
number o f media citations, number o f
to the monitoring o f follow-ups forthe
reactions on the plan’s preliminary draft
Plan forSustainable Development;
and transparency ofthe plans).
• a guarantee for Belgian representation
on all national and international policy
Method and criteria
levels;
The various measures in the plan are
■ inthelongterm.aguaranteeforan
classified according to topic (poverty 8i
effective succession in time between the
health, energy...) and according to the type
instruments for sustainable development;
o f instrument (economic, communication,
■ the enhancement o f integration ofthe
voluntary...). The processes (and
thematic and sectoral policy;
participants) are described and analysed
■ enforcement ofthe knowledge basis
based on existing documentation
fora policy regarding sustainable
maintained by the Federal Planning Bureau.
development;
Criteria forevaluation include integration
■ themobilityoflargesocialgroups
(coordination o f the two plans, coordina­
and population by the federal Strategy
tion between the sectors distributed over
forSustainable Development.
MICROSCOPE
was examined as well. Forthe process
THE
Recommendations
UNDER
The coordination between the two plans
2003*2005
This is mentioned in the second Plan for
POL I CY
plans are generated and executed; (3) a
ENVIRONMENTAL
consists o f three sections: (1) evaluation
Environmental policy integration in Europe
E uro pean E n v iro n m e n t A gency ■O ffic e fo rO ffic ia lP u b lic a tio n s o fth e E u ro p e a n C o m m u n itie s ,2 0 0 5
Characteristics
E N V IR O N M E N T A L
TO PIC
general
PO LIC Y
TYPE
an integrated government policy
TYPE
OF
PO LIC Y
IN S T R U M E N T
not applicable
TYPE
OF
E V A L U A T IO N
ex-nunc evaluation, process evaluation
PRINCIPAL
personal initiative
Definition o f problem and research
o f Sustainable Development. Coordination
questions
between regions, ministries and policy
This publication o f the European
levels appears to be limited. The Belgian
Environment Agency evaluates the way
cooperation between the local level and
in which enviro n m e n ta l aspects are integrated
the Flemish level is mentioned as an
in the policies o f EU m em ber states. This
example (in the form o f a Cooperation
integration makes that environmental
Agreement). RIAs (regulating impact
problems are tackled pro-actively and
analyses) are conducted in 15 countries
systematically. Furthermore it enables
(including Belgium). However, we note
continuity, symbiosis and transparency
that they are not always o f a high quality
between policy domains and levels. Finally
and that they are not always used in a
'Integration o f environmental policy’ is
correct manner.
to ensure sustainable development.
Method and criteria
Recommendations
• A better coordination should be
The EU member states informed the
encouraged between the ministries
European Environment Agency o f the
and government levels pertaining
extent o f environmental integration
and sustainable development in their
to the environment.
■ Clear objectives forsustainable
administration by means o f a question­
development have to be defined,
naire. Additionally, documents from all EU
which are to be implemented by a
member states were analysed by experts.
coordinating ministry.
Findings
In general it appears that governments
have taken steps towards environmental
integration aiming at sustainable
development, but that a lot more can
be done. Most EU member states have
a Strategy for Sustainable Development,
but it remains unclear whether it is in fact
implemented. As far as the institutional
developments are concerned, most
EU member states have set upan
environmental cell within each ministry.
Belgium for example has an inter­
ministerial cell for the implementation
Sectoral implementation plans in the Flemish waste policy
Effectiveness analysis o f a planning instrument
J. Gysen, K. Bachus, H. B ru y n in c k x , L. Van O o te g e m • Centre o fE x p e rtis e on E n v iro n m e n ta l P olicy Sciences, K .U . Leuven, 2 0 0 3
Definition o f problem and research
actions and ideally resulting in an
questions
environmental effect. The researchers
This publication explores the ‘execution
reconstruct the chains by means o f logical
plan’ as a policy instrument, as it is used in
deduction, interviews with involved
E N V IR O N M E N T A L
the waste policy. First o f all the instrument
parties, policy theory, output, outcome
waste
itself is analysed (the functions o f the
and environmental indicators.
instrument and the question whether it
Characteristics
PO LIC Y
T O PIC
TYPE
target group policy
Findings
satisfies the conditions for success: critical
TVPE
OF
P O LIC Y
IN S T R U M E N T
The policy actions have not decreased
success factors). In addition the authors
supporting instrument
the organic-biological waste volume.
subject a specific implementation plan to an
TYPE
OF
E V A L U A T IO N
They did facilitate a maximal recuperation
effectiveness analysis (the implementation
ex-post evaluation, product evaluation
o f the waste products.
plan for organic-biological waste). The
PRINCIPAL
effectiveness analysis investigates a
Recommendations
Flemish government (Centre for
possible relation between the policy
The authors make some recommendations
Environmental Policy Sciences)
actions and the environmental impact and
with regard to the instrument itself: the
whether this relation shows any causality.
Method and criteria
model function has to be altered; the
government should communicate on
how the plan must be implemented and
(1) The instrument perse is assessed based
what resources must be provided for the
on critical success factors (conditions for
execution o f the implementation plan.
enhancing the instrument’s success) such
Moreover, the authors suggest collecting
as: good communication, security and
all data on output, outcome and impact,
support function (the target groups have
so as to enable a better evaluation o f the
to be involved and financial stimuli enable
instrument.
acceptance).
(2) The effectiveness analysis is based on a
methodology developed by the authors. As
it is extremely difficult to prove a correlation
between policy and environmental
condition, the researchers want to
reconstruct the relation(s) between
policy actions and their effects on the
environment step by step and then
the causes and consequences o f these
relations. In otherwords, causality chains
are constructed, starting from the policy
Waste prevention actions and measures in secondary
schools in Flanders
OVAM -O V A M , 2 0 0 3 a
Characteristics
E N V IR O N M E N T A L
P O LIC Y
OF
P O LIC Y
Findings
What is striking is that environmental
On the one hand, this study focuses on
protection systems in schools (where
how school boards adjust their policy to
teachers and students cooperate actively
waste
the changing Flemish waste rules and
according to a long-term vision) seem to have
TYPE
regulations (assess the outcome) and on
a considerable impact on the waste volume.
T O PIC
target group policy
TYPE
Definition o f problem and research
questions
IN S T R U M E N T
the other hand, it focuses on the effects
Communication is o f para mount importance.
o f a possibly structured environmental
An example o f a specific, successful measure
social-communicative and direct instru­
policy on the waste situation (assess the
could be a lunch box campaign combined
ment
environmental effects). The researchers
with ban on plastic and tin foil.
TYPE
OF
E V A L U A T IO N
want to identify the peripheral conditions
Recommendations
ex-post evaluation, product evaluation
in order for the schools to take successful
We have selected three ofthe most
PRINCIPAL
waste prevention measures. All o f this is
important recommendations:
personal initiative
examined by investigating the external
• Financial factors: the application o f
stimuli and internal characteristics o f
‘the polluter pays' principle will
individual schools. As regards the external
convince schools to take more action.
factors, in particular the municipal policy
However it needs to be stressed that by
and its impact on the schools' waste
avoiding waste, one can also save on
policies is further explored.
purchase and processing costs. This
Method and criteria
message should be conveyed to the
By means o f a questionnaire several
school boards in a clear manner.
variables are compared to each other and
■ External factors: the municipalities could
linked to assess the effectiveness o f the
provide stimuli, e.g. with regard to
measures. In this way the waste quantity
communication. The study shows that an
per fraction is compared to the type o f
active environmental service with clearly
school (ASO (General Secondary
targeted communication and school-
Education), TSO (Technical Secondary
oriented action can have stimulating
Education), industrial education) and
the actions undertaken by the schools
(e.g. a lunch box campaign).
effects.
• Internal factors: a well-defined
environmental protection system is very
effective: the distribution o f lunch boxes
The evaluation is based on many criteria.
and returnable glass bottles are concrete,
The principal criterion is effectiveness, with
effective measures. The extent o f student
explanatory notes on communication and
participation inthedevelopmentand
participation (participation ofthe
implementation o f these actions is the
students).
decisive factor for success.
OVAM • O VAM , 2 0 0 3 b
MIRA-PE
Waste flows and waste policy in elementaty schools
in Flanders
2005
Definition o f problem and research
the possible involvement o f the
covenant.
LrJ
municipalities in the environmental
This study presents an overview ofthe
|
questions
E N V IR O N M E N T A L
T O PIC
With regard to specific measures, we see
generated by elementary schools on an
waste
that the exclusion o f vending machines
annual basis. The data are a starting point
PO LIC Y
TYPE
target group policy
do not have a significant effect on the
external) that determine the size ofthe
TYPE
OF
P O LIC Y
IN S T R U M E N T
waste production. On the contrary we see
waste mountain. It is examined how some
social-communicative and direct
that a complete ban on specific packaging
very specific actions can possibly diminish
instrument
materials (e.g. aluminium and cardboard
the waste quantities.
TV P E O F
E V A L U A T IO N
ex-post evaluation, product evaluation
Method and criteria
especially in consultation with the parents
A questionnaire was forwarded to all
and students.
PRINCIPAL
Recommendations
the schools situated in the Brussels-Capital
Below we mention two striking
Region). The response was representative.
observations, which could be considered
as recommendations:
could be made on the waste quantities.
Note: see ‘Woste Prevention actions and
The compared criteria are analogous to
• Schools require active cooperation with
the ones ofthe research conducted in
municipalities: local officials should
secondary schools by OVAM (see above).
define specific cooperation actions.
THE
measures in secondary schools in
UNDER
By means o f output indicators, statements
2003-2005
personal initiative
Flemish elementary schools (except for
POL I CY
drink cartons) does have a real impact,
ENVIRONMENTAL
and the installation o f drinking fountains
for identifying the factors (internal and
FLEMI SH
waste quantities per waste fraction
Characteristics
Flanders' (OVAM, 2003a)
• A ban on certain packaging materials
Elementary schools collect more vegetable
has proven to be the most effective,
and fruit waste than secondary schools,
especially in consultation with parents
but generally speaking the waste quantity
and students. The exclusion o f vending
remains relatively limited in comparison
machines has no significant effects.
to the latter.
The internal factor which decreases waste
isagood environmental functioning
which is planned in detail and which works
continuously. Furthermore, we find that
schools which cooperate actively with
municipalities within the scope o f local
waste actions show very good results. What
is remarkable is that no effect is noted as to
MICROSCOPE
Sensitisation in itself is not sufficient;
Findings
Prestí 4: involved in environmental protection measuring the effects
A. G o m m e rs • Resource A nalysis, 2 0 0 3
Characteristics
E N V IR O N M E N T A L
PO LIC Y
T O PIC
OF
P O LIC Y
OF
questions
■ Not all objectives ofthe programme
Most companies produce waste during
the exercise o f their business activities.
To encourage these companies to take
TYPE
waste, water and energy reducing
IN S T R U M E N T
social-communicative instrument
TYPE
Findings
waste
target group policy
TYPE
Definition o f problem and research
E V A L U A T IO N
ex-post evaluation, product evaluation
P R IN C IP A L
Flemish government (OVAM)
measures, the government mobilises
several instruments. One ofthese
instruments includes the Presti
were clear.
■ Only a limited number o f companies
responded.
• The companies that responded have
a more positive attitude towards
environmental protection.
■ The environmental impact ofthe
programme (encouragingwaste
responding companies decreased
prevention actions). This study examines
more than that o f the non-participating
the effects ofthe programme on the
participating companies.
companies.
■ The participating companies are satisfied
with their participation.
Method and criteria
In order to discover the problems
Recommendations
experienced by programme representatives
Projects are best conducted in different
in-depth interviews were conducted. A
stages. Companies that have been involved
questionnaire was forwarded to the
in environmental protection fora longer
participating companies to collect data
time, could start in a higher stage.
on the effects ofthe programme. The
A system of'godparent companies’ could
companies not participatingwere
be an option to have experienced
questioned by means o f stratified
companies coach others.
sampling. Based on the questionnaire
some ofthe companies were selected
for in-depth interviews. The object was to
assess the effectiveness o f the programme.
M . V ervaet, L. Lauw ers, S. Lenders, S. O v e rlo o p • Centre fo r A g ric u ltu ra l E conom y, M in is try o fth e F lem ish C o m m u n ity , 2 0 0 4
MIRA-PE
The three-tier manure policy:
evaluation and future exploration
2005
Findings
questions
The authors examine whether the manure
I
Definition o f problem and research
tf
surpluses in Flanders can be eliminated.
Characteristics
manure surplus. This leads to excessive
This almost seemed to be the case in 2003.
E N V IR O N M E N T A L
nutrient losses towards the environment.
However, the authors do not wish to make
manure
Since 2000, the government has been
any statements about the effectiveness
PO LIC Y
trying to deal with these problems in the
ofthe manure policy. The authors in fact
target group policy
manure action plan (in casu MAP2bis). This
remain critical with regard to the correct
TYPE
plan starts from three tiers or 'measure
interpretation ofthe manure surplus. An
supporting instrument
packages’ (source approach, judicious
important part o f the manure surplus
TYPE
fertilisation and manure processing).
appears not to be 'internalised' by the
ex-nunc evaluation, product evaluation
For each o f these three tiers the following
policy. The government doesn't consider
P R IN C IP A L
targets have been set: the first two
one part o f the manure surplus, so that
personal initiative
effective or not. A more comprising
that a manure surplus in fact still exists.
are in fact effective p o licy in strum ents and
The authors verily whetherthe current
to w h a t extent they are coherent.
three-tier policy will in fact deal with a
'broader' interpretation ofthe manure
surplus.
This sheer product evaluation looks
into the effects in the field. Based on a
Recommendations
calculation model the sale at company
The authors list several recommendations,
owned sites, at third parties and manure
includinga plea foran efficient mineral
processing is simulated, starting from
inventory.
individual company data and the quantity
o f saleable manure. The criteria for this
study are coherence (between the measure
packages) and effectiveness. An implicit
evaluation ofthe objectives and their
reliability is conducted as well (i.e. how
the size ofthe manure surplus is to be
interpreted correctly).
MICROSCOPE
Method and criteria
THE
calculation ofthe nutrients losses shows
The authors verify whether the three tiers
E V A L U A T IO N
UNDER
responsible for at least 50 % thereof.
IN S T R U M E N T
2003*2005
25% and manure processing must be
P O LIC Y
P OL I CY
depending on the in te rp re ta tio n o fth e
m an ure surplus size, the policy can be called
OF
TYPE
ENVIRONMENTAL
measures have to contribute to the
elimination o f the manure surplus up to
OF
T O PIC
FLEMI SH
The Flemish Region struggles with a
Nature, how? Investigation into the social fram ework fo r
nature preservation in Flanders: partial assignment:
methodology fo r the evaluation o f a policy pertaining to
the social framework fo r nature preservation in Flanders
T. Leroy, I. Lo ots, P. Leroy • F aculty o f P o litic a l a n d S ocial sciences, U A , 2004
Characteristics
E N V IR O N M E N T A L
T O PIC
nature
PO LIC Y
TV P E
target group policy
TYPE
OF
PO LIC Y
IN S T R U M E N T
not applicable
TYPE
OF
E V A L U A T IO N
ex-ante evaluation, plan evaluation
PRINCIPAL
Definition o f problem and research
Findings
questions
• A framework policy (a systematic and
This publication makes an ex-ante
well-considered process aimed at
evaluation ofthe nature framework policy.
increasing social support for the nature
Currently there is no real framework policy
policy) cannot be a separate policy; it
and therefore it is not possible to conduct
must be a part ofthe integral nature
an ex-post evaluation. The study consists
policy. Only fragmentary elements with
o f two parts: a discourse analysis (opinion
regard to this can be found in policy
evaluation, visions, and hopes pertaining
documents and viewpoints ofthe
to framework) and an analysis o f specific
nature projects and the way in which a
framework is structured.
participants involved;
■ Nature policy requires links to other
policy domains. Nature dominance could
lead to a shrinking ofthe framework.
Flemish government (AMINAL)
Method and criteria
When nature must be exclusive, extra
This study is based on document analysis
framework attention will be required;
and interviews. The discourse analysis
• An exploration o f the social context
identifies viewpoints with regard to
(participants and their relations) is
framework. A separate analysis framework
important as the participants put their
Note: The firs t p a rt o f this study is
was used for the analysis o f nature
trust into the initiator and because it
published as 'Onderzoek naar het
projects. This analysis framework allows
draagvlak voor natuurbehoud in
the classification o f projects exceeding
facilitates an adequate framework
strategy.
Vlaanderen: deelopdracht: opstellen van
theirown situation. The framework refers
Recommendations
een k o rf van indicatoren voor het meten
to 3 questions: allocation (how much space
We refer to the findings above.
van het maatschappelijke draagvlak voor
natuurbehoud in Vlaanderen' by
M. DeZitter, T. Wymeersch, D. Bogaert,
A. Cliquet and published in Cheni:
Arteveldehogeschool and University
ofC hent, 2003. This research does not
evaluate policies, but is intended fo r
rendering operational the framework
policy, more specifically its monitoring.
and/or resources can be allocated to
nature?), location (what priority is given
to nature?) and joint use (function
interrelations?). The evaluation criterion
is 'clarity' in the discourse analysis (what
is the definition o f a framework?) and
'accessibility', 'mutual coordination
between the participants', 'acceptance
by participants’ in nature projects.
MI RA-PE
Repon ofthe European Commission on the
implementation ofGuideline 92/43/EC on the
conservation o f natural habitats and o f w ild flora
and fauna
2005
E uropese C o m m is s ie ■E C ,C O M (2 0 0 3 )8 4 5 ,2 0 0 4
Lo
Definition o f problem and research
■ No EU member state provides
information indicating the effects of
Characteristics
This publication examines the progress
the exceptions to the strict protection
E N V IR O N M E N T A L
for the species involved.
nature
ofthe implementation o f the Habitat
■ The reports ofthe EU member states
PO LIC Y
TYPE
effectively. The activities are inspected
are not very critical. In addition, the
a region-oriented policy
on a European level and on a EU member
reports vary significantly among
TYPE
state level.
OF
P O LIC Y
IN S T R U M E N T
EU member states, which makes
supporting instrument
comparison difficult.
TYPE
OF
E V A L U A T IO N
ex-nunc evaluation, process evaluation
Recommendations
on the reports ofthe EU member states.
The quality ofthe EU member state
As regards the European activities the
reports must improve. Methods have to
PRINCIPAL
personal initiative (European Commission)
authors start from commission reports.
be developed which enable in-depth
The paragraphs on EU member state
evaluation. The input ofthe European
Environmental Agency has to be
amplified for example.
EU member states are analysed by
The criterion is whether and how the
objectives ofthe European guideline are
implemented in the EU member states.
Findings
• Since 1999 there has been progress in
the selection o f regions, as required by
the Habitat guideline. Even so, we note
a delay in the planning. The Belgian lists,
with suggested regions for the Natura
2000 network are not exhaustive
enough, even though progress has been
made since 2002.
MICROSCOPE
experts and officials ofthe European
Commission.
THE
governmental and non-governmental
UNDER
progress were drafted by one or more
consultants. The national reports ofthe
2003-2005
The Implementation evaluation is based
POL I CY
Method and criteria
ENVIRONMENTAL
guideline and whether this is done
T O PIC
FLEMI SH
questions
Nature report 2005. The condition o f nature in Flanders figures fo r the policy
M. D u m o rtie r, L. De B ruyn, M . Hens, J. P eym en, A. S chne id ers, T. Van D aele, W. Van R eeth, G. W e ye m b e rg h
en E. K u ijk e n • Inst/tutefor Nature Preservation, 2005
Definition o f problem and research
Characteristics
E N V IR O N M E N T A L
T O PIC
nature
PO LIC Y
TYPE
monitoring, document research and
questions
interviews. The effectiveness ofthe policy
The Nature Report presents a current state
was examined in detail and whenever
o f affairs and an evaluation o f nature in
this was not possible, the progress was
Flanders, centred around specific topics.
evaluated.
It is impossible to summarise all findings
Recommendations
region-oriented policy, target group policy
o f this elaborate report. The report aims at
Because o f the diversity ofthe treated
TYPE
OF
PO LIC Y
IN S T R U M E N T
juridical, economic and social-communica­
TYPE
OF
policy support. By means o f indicators
policy topics in the Nature Report 2005,
the report paints a picture of how the
it is not possible to present all
tive instruments
environment is faring. Also the policy is
E V A L U A T IO N
described and incertain cases evaluated
recommendations. We therefore refer
to the full Nature Report.
process and product evaluation
PRINCIPAL
Flemish government. The submission o f a
Nature Report is an obligation by decree.
and knowledge (deficiencies) is(are)
indicated. Part 6 ofthe report'Protection
and recovery’ analyses and evaluates the
policy and its instruments explicitly. The
focus is on:
• international policy;
• VEN/IVON;
■ spatial policy;
- land acquisition;
• nature reserves;
■ forest reserves;
■ institutional instruments;
■ nature permit;
• water policy;
■ regional landscapes;
• cooperation with local governments.
In part 7 'Society' the nature framework
is presented as well as nature and
environment education.
Method and criteria
The report includes a number o f nature
indicators, output indicators, process
indicators and outcome indicators.
These indicators are collected through
Financial instruments fo r nature, forestand landscape
B. Kayaerts • M iN a C o u n c il, 2 0 0 5
Definition o f problem and research
clear and measurable objectives? Most
questions
instruments are a form o f compensation
This publication presents the financial
(e.g. compensation for property owners
Characteristics
instruments for implementing the nature
for the inclusion o f their lands in the
E N V IR O N M E N T A L
policy (in the broadest sense: aimed
VEN). The objectives are rarely SMART-
nature
at forests, landscape, nature and
formulated.
P O LIC Y
biodiversity). The evaluative extent o f
the study is discussed in the need fo r
• To what extent does the report focus
on the development ofthe financial
T O PIC
TYPE
region-oriented policy, target group policy
TYPE
OF
PO LIC Y
IN S T R U M E N T
o p tim is a tio n : the extent to which the policy
instruments for the coordination o f
economic instrument
draws attention to the various elements
the instrument in its spatial field of
TYPE
that impact the effectiveness o f the
application? Are the instruments
ex-post evaluation, process evaluation
instruments. But also other evaluation
sufficiently specifically oriented?
PRINCIPAL
aspects are discussed (see below). First
■ To what extent do the financial
an overview ofthe policy nature objectives
instruments fit in a region-oriented
is presented. The author discusses the
vision? Is sufficient attention paid to
strategic and operational targets. In the
the current or future spatial destination?
second last part the analysis focuses on
Is there any attention for management
whether the policy shows sufficient
continuity?
interest in the elements that impact
Finally the study focuses on the financial
effectiveness. The key criterion is
flow linked to the instruments.
coherence. The following evaluation
Method and criteria
questions will be answered:
The study examines the objectives, the
• To what extent does the policy focus
target groups and the instruments and
on the development o f clear strategic
verifies whether certain conditions for
objectives with sufficient structure?
achieving these objectives are met. The
■ To what extent is attention given to the
criteria are coordination, SMART content
development o f financial instruments
ofthe objectives and the involvement o f
for involving all relevant target groups
the target groups.
and the improvement o f the coherence
Findings
among the target groups?The analysis
The objectives are highly fragm en ted and
aims at the parties participating in the
d is trib u te d over several policy domains
'management'.
and the ope ra tio n a l objectives are n o t alw ays
■ To what extent does the policy focus
S M A R T-form ulated. The researcher
on coherence between the available
describes the target groups, followed by
financial instruments and the objectives?
a discussion ofthe 57 financial
To what extent does the policy focus on
OF
E V A L U A T IO N
personal initiative (MiNa Council)
instruments, each divided per target group
objectives (within ‘ nature’ between 'nature
Recommendations
and per objective, General observations
and landscape’ and between 'nature and
The preface mentions that the study aims
especially deal with subsidies; they are
agriculture’...) are not always geared to one
at delivering a contribution to the debate
strongly embedded in legislation and more
another. Furthermore, there is only consid­
on open space instrument optimisation.
than ha lfof the instruments are remark­
eration for the cooperation between the
The findings are therefore also recommen
ably young. The author finds that the
participants.
dations.
Region-oriented nature policy: operationality and
implementation ofthe Bird Guideline and the Habitat
Guideline
A. C liq u e t, G. Van H o o ric k , j. L a m b re c h t, D. B oga ert • M IR A -P E 2005,2 0 0 5 (D u tc h version)
Characteristics
E N V IR O N M E N T A L
T O PIC
nature
PO LIC Y
TYPE
a region-oriented policy
TYPE
OF
PO LIC Y
Definition o f problem and research
Findings
questions
■ As a result ofthe modifying decree of
This research evaluates the translation of
2002, the region-oriented obligations
the Habitat guideline and the Bird guide­
are translated to a reasonable extent into
line. Also the measures ensuing from the
the Flemish rules and regulations.
guidelines are analysed. The research ques­
tions are:
However, some legal problems still exist.
■ Few specific objectives have been includ­
ed in the policy plans (e.g. MiNa plan)
IN S T R U M E N T
• What are the specific region-oriented
juridical instrument
pertaining to particular protection areas.
obligations for Flanders under the
TYPE
OF
• Reserves appearto be particularly apt
E V A L U A T IO N
guidelines?
ex-nunc evaluation, plan and process eval­
uation
forthe protection o f critical species.
■ Did Flanders correctly transpose the
■ Theguaranteeforachievingthe
guidelines into legislation?
protection objectives on sites that
PRINCIPAL
• What instruments are provided and
Flemish government (VMM)
are not owned or managed by the
which ones are in fact implemented?
government or land management
■ What are the possible success and failure
associations, is inadequate.
factors?
■ The current (nature) permitting system
Method and criteria
does not sufficiently contribute to a
This research is based on document
differentiated policy, which is necessary
analysis. The instrument ‘nature guideline’
was evaluated by means o f interviews.
for the special protection areas.
■ Most specific plans for the special
protection areas are nature-oriented
First o f all, the transposition ofthe
plans. Their implementation was
European guidelines is examined. For each
instrument supporting the achievement o f
delayed. This means that the objective
with regard to the set-up the nature-
guideline objectives, the following criteria
oriented plans by the end o f 2007 for
o f analysis are used: legality, concreteness,
every area in the VEN, will not be
coherence and ambition. Whenever
achieved.
possible, effectiveness is also examined.
Recommendations
be allocated to nature organisation
• The protection objectives have to
and reserve formation.
be embedded in legislation.
• A systematic legal framework for
obligations has to be provided.
All interventions which may affecta
particular protection area must be
subjected to a permit.
2005
• The necessary financial resources must
MIRA-PE
the management and supervision
Management agreements are preferably
selective and differentiated.
I
FLEMI SH
Local water policy about to turn:
interim evaluation ofthe sub-basin functioning
ENVIRONMENTAL
A. Crabbé • Centre o f Expertise on E n v iro n m e n ta l P olicy Sciences, U A , 2003
evaluation criteria are used as starting
This study deals with a recent initiative for
points: the mobilising power o f the
Characteristics
an integrated water policy. Municipalities,
Flemish policy for the dynamicity ofthe
E N V IR O N M E N T A L
provinces, polders and waterworks are
sub-basin functioning, the transparency
water
stimulated by the Flemish Region to create
ofthe Flemish policy, the coordination
PO LIC Y
a more ambitious cooperation and more
ofthe sub-basin functioning (by the
region-oriented policy, integrated
TYPE
provincial councils), the legitimacy o f
government policy
the plan method and social participation
TYPE
are developed: Water policy plans which
in the policy planning.
juridical instrument
TYPE
OF
IN S T R U M E N T
E V A L U A T IO N
The ex-ante evaluation is concerned with
water policy. In this study an interim
ex-nunc evaluation, process evaluation
PRINCIPAL
plan delivery, the (un)certainty with regard
the processes which should lead to the
Flemish government (Centre o f Expertise
to the effects ofthe planning process,
development ofthe sub-basin manage­
on Environmental Policy Sciences)
the chance o f embedding the sub-basin
ment plans. The focus is on what has been
functioning, social participation in the
accomplished so far (ex post), as well as on
policy planning, mutual coordination
what is still to come (ex ante). The object
between the sub-basin functioning and the
ofthe study is to list critical failure and
basin functioning, mutual coordination
success factors which may impact the
with other arrangements in the water
development o f the sub-basin policy either
policy and coordination with other
in a positive or in a negative way.
arrangements outside ofthe water policy.
Method and criteria
Findings
This research is the result o f document
■ In all Flemish sub-basins the initiative
analyses, participating observation and
was taken to set up a water policy plan
interviews.
on a sub-basin level.
MICROSCOPE
the timeliness ofthe local water policy
analysis and evaluation are presented o f
THE
program actions and measures for future
P O LIC Y
UNDER
coherent water policy. On a scale o f 103
sub-basins, sub-basin management plans
OF
T O P IC
2003-2005
In the ex post evaluation the following
questions
POL I CY
Definition o f problem and research
■ The mutual coordination between
All initiatives to motivate provinces,
completed by the provisory planning of
municipalities, polders and waterworks
mid 2005.
sub-basin management planning and
to sub-basin functioning, have lead to
The future ofthe planning composition
other policies (e.g. nature policy,
a jumble o f financing channels and
on sub-basin level is unclear.
environmental planning) is not selfevident.
names, which does not enhance policy
There are no rules for rendering the
transparency.
policy in the traditional policy
The coordinating provincial councils
arrangements dependent from the
enable a positive dynamics in the
content ofthe sub-basin and basin
sub-basin functioning, but their effort
management plans (such as water course
is sometimes experienced as too
management, water purification and
dominant.
drinking water supply). This possibly
Most planning processes in the 103
has a negative impact on the execution
Flemish sub-basins will not be
o f the policy actions which are to be
Recommendations
The study provides three recommenda­
tions, three strategic objectives:
• to convince all stakeholders o f the added
value generated by cooperation;
• working on mutual coordination;
• optimising the sub-basin functioning
policy framework.
programmed.
Regional subsidies fo r municipal waste water treatment
plants in a European context
C o u rt o f A u d it • F lem ish P a rlia m e n t, P a rt 3 7 (2004-2005) N u m b e r 1,2005
Characteristics
E N V IR O N M E N T A L
T O PIC
Definition o f problem and research
the MiNa fund. The selection o f these
questions
documents was based on indications
The Court o f Audit evaluates in this
o f irregularities.
publication the organisation o f waste
The Court o f Audit defined efficiency,
(waste)water
water purification in Flanders (an
legality, economy and effectiveness as
P O LIC Y
TYPE
obligation resulting from the European
key criteria.
an integrated government policy
TYPE
OF
PO LIC Y
IN S T R U M E N T
economic instrument (subsidies)
TYPE
OF
E V A L U A T IO N
ex-post evaluation, product evaluation
PRINCIPAL
guideline on urban waste water). The
evaluation focuses on the task division
Findings
(municipalities, NV Aquafin and the
The Court o f Audit concluded that the
municipal government, in casu VMM),
Flemish Region does fulfil Its European
the financing channels, the subsidy
obligations, but not according to plan:
regulations and the KWZIs. The central
the region is still waiting for some large
personal initiative (one o f the tasks ofthe
research question is 'how does the Flem ish
investments by the municipalities in sewer
Court o f Audit is to perform effectiveness
Region fu lfil its European oblig a tio n s ? '
systems. These efforts are impeded by
factors such as unclear task division
analyses on its own initiative or by order o f
Method and criteria
the parliaments.)
between the Flemish Region and the
The Court o f Audit has screened the rules
municipalities, a deficient and outdated
and regulations and the parliamentary
planning, the undermining o f regional
documents and has conducted interviews
management by the municipal autonomy
involving the stakeholders. Furthermore it
and a lack o f optimal subsidy systems.
has examined some ofthe 2000-2003
documents at the water department and
Recommendations
the subsidy programme. If a project is
The Court o f Audit formulates
included in the programme only after
io recommendations:
approval ofthe preliminary draft, this
will enable the region to better estimate
the cost efficiency.
efforts with regard to task division
The region is to clearly indicate the
(municipal versus supra-municipal)
In any case the project’s cost efficiency
2005
projects and reasons for the subsidies.
and planning. The zoning and master
MIRA-PE
• The region is to continue its current
planningshould bring shed some light
is to constitute an important criterion.
The municipalities are to give account
I <Ó'
on the need for and requirements o f
collective and individual waste water
region obtains a clear picture ofthe
• The region must draw up a map o f all
executed works. If a municipality does
FLEMI SH
o f the received subsidies, so that the
treatment.
planned sewage works; not merely o f
if after the receipt o f an advance no
should enablea betterand coordinated
balance with explanations is requested,
planning and execution o f sewage works.
the region is to re-daim the non­
• For non-MiNa fund subsidised works,
accounted part ofthe received subsidy.
the region must be able to object to
advances for similar projects ofthe
fit in with the region's vision or which
same municipality.
The region is to observe all legal and
• The budget and additional budget
regulatory declarations o f the RIO
documents are to provide the Flemish
subsidies, including the obligation
Parliament with the required
Flemish minister ofthe Environment)
provide for the municipal sewage works
determines the subsidiary programme
and the involvement ofthe official
credits as required by decree. If a lack
o f credits is imminent, the region is to
take action.
• When works are combined, the current
subsidy rules and regulations impose
obligations on the municipalities, which
are hard to fulfil because they depend
on other organisations. Seeing as the
combined works make upa substantial
part o f the subsidy programmes, the
region is to consider a adjustment o f
these rules and regulations.
• The financial implications have to be
known to the region when setting up
MICROSCOPE
commission in the preparation thereof.
programmes is to match the available
THE
subsidy.
• The volume ofthe RIO subsidy
UNDER
that the Flemish government (not the
information on available credits which
2003-2005
do not meet the technical criteria.
P OL I CY
In such case the region may refuse
proposed sewage works which do not
ENVIRONMENTAL
not give proper account, in particular,
the ones that are being subsidised. This
Annex: Inventory
reference list 2003-2005
T h e f o llo w in g lis t c o n ta in s th e re fere nces o f t h e 24 p u b lic a tio n s fro m th e in v e n to ry .
Bachus K., Bruyninckx H. & Poesen-
DumortierM., De Bruyn L., Hens M.,
Kayaerts B. (2005) Financiële instru­
Vandeputte M. (2004) Vlaams structureel
Peymen |., Schneiders A., Van DaeleT.,
menten voor natuur, bos en landschap,
overleg voorduurzame ontwikkeling,
Van Reeth W., Weyembergh G. & Kuijken
MiNa-Raad, Brussel.
HIVA, K.U.Leuven, Leuven.
E. (red.) (2005) Natuurrapport2005.
Toestand van de natuur in Vlaanderen:
Bachus K., Vandeputte M., Sepelie R.,
cijfers voor het beleid. Mededeling van het
Vangeebergen B., Deraedt B., Mazijn B. &
Van Assche |. (2004) Evaluatie van de
Samenwerkingsovereenkomst, HIVA,
Leuven.
Instituut voor Natuurbehoud nr. 24,
Brussel.
L e royi, Loots 1. 8t Leroy P. (2004) Natuur,
hoe kan dat? Onderzoek naar het
maatschappelijk draagvlak voor natuur­
behoud in Vlaanderen, deelopdracht:
methodiek voor de evaluatie van beleid
European Environment Agency (2005)
inzake maatschappelijk draagvlak voor
Environmental policy integration in
natuurbehoud in Vlaanderen, Universiteit
Europe, Office fot the Official Publications
Antwerpen, Antwerpen.
Baetens M. & Bursens P. (2005) De
Europeaniseringvan Vlaamse
o fthe European Communities,
ambtenaren, kabinetards en politici:
opinie, expertise en aandacht, Steunpunt
Bestuurlijke Organisatie, Leuven.
Luxemburg.
Monsieurs A. (2004) De behandelingvan
milieuklachten door de Milieu-inspectie:
Europese Commissie (2004) Verslag van de
een efficiënt instrument voor de uitoefen­
Commissie over de uitvoering van
ing van hoog toezicht?, Faculteit
Richtlijn 92/43/EEG inzake de instand-
Rechtsgeleerdheid, K.U.Leuven, Leuven.
Cliquet A., Van HoorickG., Lambrecht]. &
Bogaert D. (2005) Gebiedsgericht natuur­
houdingvan de natuurlijke habitats en de
beleid: operationalisering en uitvoering
van de Vogelrichtlijn en de Habitat-
wilde flora en fauna, COM 845, Brussel.
richtlijn. Onderzoeksrapport MIRA-PE
Geeraerts K., Bursens P. & Leroy P. (2004)
2005, Vlaamse Milieumaatschappij, Aalst.
Vlaams milieubeleid steekt de grenzen
Crabbé A. & Leroy P. (2004) Op weg naar
over, Steunpunt Milieubeleidsweten­
schappen, Antwerpen.
vernieuwd lokaal waterbeleid.
Tussentijdse analyse en beoordelingvan
Gommers A. 81 Fernagut M. 8t Couderé K.
de ontwikkeling van waterbeheerplannen
(2003)
op deelbekkenniveau, Universiteit
Analysis, OVAM, Mechelen.
Antwerpen, Steunpunt
Milieubeleidswetenschappen, Antwerpen.
Effectmeting Prestí 4. Resource
Gysen J., Bachus K., Bruyninckx H. 81Van
Ootegem L. (2003) Sectorale uitvoerings­
De BruynT. & Bachus K. (2005)
plannen in het Vlaamse afvalbeleid.
Handhaving: de vergeten schakel in de
Effectiviteitsanalysevan een planningsin­
milieubeleidscydus? Onderzoeksrapport
strument, Steunpunt Milieubeleidsweten­
MIRA-PE 2005, Vlaamse Milieumaat­
schappen, Leuven.
schappij, Aalst.
OVAM (2003a) Afvalpreventie acties en
maatregelen in middelbare scholen in
Vlaanderen, OVAM, Mechelen.
OVAM (2003b) De afvalstromen en het
afvalbeleid in basisscholen in Vlaanderen,
OVAM, Mechelen.
Rekenhof (2005) Gewestelijke sub­
sidiëring van gemeentelijke waterzuiveringsinfrastructuur in Europees kader,
Publicaties Vlaams Parlement, Brussel.
Segers R. (2004) Pilootproject Vlarem
toezicht Klein Brabant, sectorTransport,
Uitgave gemeente Puurs, Puurs.
Van Humbeeck P., Dries I. & Larosse ).
(2004)
Heyerick A. (2003) Studie tervoorbereid-
Delbaere P., Dresselaers P. & Gouzée N.
ingvan de evaluatie van het federaal
(2005) Het gevoerde beleid inzake
milieugericht productbeleid - eind­
duurzame ontwikkeling, Federaal
rapport, Centrum voor Duurzame
Planbureau, Brussel.
Ontwikkeling, Gent.
Linking Innovation Policy and
Sustainable Development in Flanders,
IWT-studies, Brussel.
MIRA-PE
R e f e r e n c e s
R e v i s o r s
Europeaniseringvan Vlaanderen: een
Krís Bachus, HIVA, K.U.Leuven
|o Baele, Court o fA u d it
turen, Steunpunt Bestuurlijke
Bart De Peuter, Public M anagem ent
november 1992), Samson, Alphen aan den
Institute, K.U.Leuven
Vervaet M., Lauwers L., Lenders S. &
Rijn, 3-20.
MINA-plan 3 (2003) Milieubeleidsplan
mestbeleid: evaluatie en toekomst­
2003-2007, Ministerie van de Vlaamse
verkenning, Ministerie van de Vlaamse
Gemeenschap, Brussel.
Gemeenschap, Brussel.
Peter De Sm edt, APS, DepartmentAZF
lo ris De Smedt, Public M anagem ent
Institute, K.U.Leuven
Isabel D obbelaere, WES Research
& Consultancy
MIRA-PE 2003: Van Steertegem M. (red.)
Lydia Putseys, OVAM
PauISchreurs, IIVE
rapport Vlaanderen: beleidsevaluatie,
K arl Van B iervliet, Ecolas nv
P h ilip p e Van De Velde, OVAM
Erembodegem.
Mare Van Dyck, Resource Analysis nv
Swedish Environmental Protection
Agency (2004) 21 evaluations - what do
Stockholm. (http://www.naturvardsver-
Peter Van H um beeck, Study Office, SERV
|o ke Vandenabeele, SEEM, UA
P ieter Verbeek, Secretariat, M iNa Council
R udi Verheyen, Biology Departm ent, UA
THE
ket.se/bokhandeln/pdf/620-5425-2.pdf).
Study departm ent, ALT
UNDER
they teach us? Naturvärdsverket,
D irk Van G ijseghem , M o n ito rin g &
2003-2005
Vlaamse Milieumaatschappij,
P OL I CY
(2003) MIRA-PE2003: Milieu- en natuur-
ENVIRONMENTAL
Overloop S. (2004) Het drie-sporen-
FLEMI SH
Handboek beleidsvoering overheid (versie
Organisatie, Leuven.
I VO
Bouckaert G. (1992) Beleid en produc­
tiviteit. In: Depré R. & Van Beyien A. (red.),
doorlichtingvan de bestuurlijke struc­
2005
Van Reybroeck E. & Bursens P. (2003)
MICROSCOPE
Explanatory list o f
abbreviations
AMINAL: Administration o f Environment, Nature, Land and Water Management
2005
APS: Planning and Statistics Administration
M I RA- PE
ALT: Administration o f Agriculture
AWI: Administration o f Science and Innovation
69
CDO: Centre for Sustainable Development
CLE: Centre for Research on Agricultural Economics
EC: European Commission
EEA: European Environment Agency
EU: European Union
GMO: Regional Consultation Unit on the Environmental Policy
HIVA: Higher Institute for Labour Studies
INBO: Research Institute for Nature and Forest
IWT: Administration for Innovative and Scientific Research
K.U.Leuven: Catholic University o f Leuven
MiNa Council: Flemish Nature and Environment Council
MINA-plan: Environmental Policy Plan
MIRA: Flanders Environment Report
OVAM: Public Waste Agency o f Flanders
PE: Policy Evaluation
SERV: Flemish Socio-Economic Council
UA: University o f Antwerp
UCent: Ghent University
VEN: Flemish Ecological Network
VLAREM: Flemish Decree on Environmental Permits
VMM: Flemish Environment Agency
VVSG: Flemish Union for Cities and Municipalities
A b o u t the Flemish
Environm ent Agency
T h e F le m ish E n v iro n m e n t A gen cy (V M M ) is a Fle m ish p u b lic in s titu tio n th a t w o rk s f o r
th e F le m ish g o v e rn m e n t a nd serves its c itiz e n s . V M M m easures a n d checks th e q u a lity
o f t h e a ir an d th e w a te r, tra n s la te s its fin d in g s in to s c ie n tific re p o rts , fo rm u la te s
o p tim is a tio n re c o m m e n d a tio n s and s tim u la te s th e p o p u la tio n t o deal w ith th e
e n v ir o n m e n t in a s u s ta in a b le way. V M M c o lle c ts a levy o n w a te r p o llu tio n an d w a te r
a b s tra c tio n . F u rth e rm o re , V M M c o o rd in a te s th e Flanders E n v iro n m e n t R e p o rt (M IRA)
th a t deals w ith a ll aspects o f e n v iro n m e n ta l issues in Flanders fo r w h ic h p u rp o s e a
p ro je c tte a m is h ire d . V M M also plays an im p o rta n t ro le in sew erage p o lic y and w aste
w a te r tre a tm e n t. It is re s p o n s ib le fo r th e p la n n in g and th e in s p e c tio n o f t h e c o n s tru c tio n
o f sew erage system s a n d c o lle c to rs an d th e b u ild in g o f w aste w a te r tre a tm e n t p la n ts.
T h e ta sk o f t h e Flanders E n v iro n m e n t R e p o rt (M IR A) is th r e e fo ld :
• a d e s c rip tio n , an alysis a n d e v a lu a tio n o f t h e c u rre n t state o f t h e e n v iro n m e n t;
• an e v a lu a tio n o f t h e e n v iro n m e n ta l p o lic y c o n d u c te d u p t o th a t p o in t in tim e ;
• a d e s c rip tio n o f t h e e x p e c te d d e v e lo p m e n t o f t h e e n v ir o n m e n t by u n ch a n g e d p o lic y
an d by c h a n g e d p o lic y a c c o rd in g to a n u m b e r o fs c e n a rio s co n s id e re d re le va n t.
F u rth e rm o re , th e re p o rt m u s t be w e ll p u b lic is e d . M IR A p ro v id e s th e s c ie n tific
fo u n d a tio n fo r th e e n v iro n m e n ta l p o lic y p la n n in g in Flanders. The sta te o f t h e
e n v ir o n m e n t is p re s e n te d in th e a n n u a l M IR A-T re p o rts . In th e se re p o rts th e p o lic y m a k e r
an d th e c itiz e n fin d answ ers to q u e s tio n s a b o u t th e sta te o f t h e e n v iro n m e n t, w h a t
th e u n d e rly in g causes are and h o w th e e n v iro n m e n ta l s itu a tio n can be im p ro v e d .
The fir s t s c e n a rio re p o rt w as p u b lis h e d in 2 0 0 0 , MIRA-S 2 0 0 0 , th e n e x t e d itio n is
p la n n e d fo r 2 0 0 9 . The firs t p o lic y e v a lu a tio n re p o rt (MIRA-PE) a p p e a re d in June 2003,
th e seco nd e d itio n in th e a u tu m n o f 2005.
File on Flanders
The F lem ish R egion (F lande rs) is o n e o f t h e th re e B elgian re g io n s w ith its o w n
g o v e rn m e n t, p a rlia m e n t a n d a d m in is tra tio n . T h e o th e r tw o are th e B russels-C apital
fe d e ra l s ta te g iv in g th e re g io n s m o re and m o re p o w e rs. A p a rt fro m th e e n v iro n m e n t
2005
sta te s tru c tu re re fo rm s o v e r th e la st 30 years, B e lg iu m has been tra n s fo rm e d in to a
M I RA- PE
R egion and th e W a llo o n R egio n. C o m p ris e d in th e D u tc h -s p e a k in g p a rt o f t h e c o u n try ,
th e Fle m ish R egion has th e la rg e s t p o p u la tio n o f t h e th re e (5 8% ). A sa re s u lt o f v a rio u s
(th e s u b je c t o f th is re p o rt), th e Fle m ish g o v e rn m e n t is a lso c o m p e te n t fo r o th e r m a tte rs,
such as th e e c o n o m y , e m p lo y m e n t, e d u c a tio n and c u ltu re , fa rm in g , fo re ig n tra d e , la n d
p la n n in g , u rb a n d e v e lo p m e n t, h o u s in g , p u b lic w o r k s ...
Antwerp
■ Ghent
Hasselt
Louvain
■
I
Provincial capital
I Flanders
□
Brussels
□
Wallonia
Belgium
71
Flanders Enviro nm ent Report: Policy Evaluation
E n v iro n m e n ta l P olicy. H o w d o e s i t w o rk ? W h a t are th e re sults?
W h a t d o e s i t te a c h us? T h is re p o rt o ffe rs an o v e rv ie w o f
24 e v a lu a tio n s tu d ie s o f (su b)areas o f t h e F le m ish e n v ir o n m e n t
p o lic y . M e th o d s , in v e s tig a te d p o lic y areas, fin d in g s a n d
re c o m m e n d a tio n s o f a ll 24 e v a lu a tio n s tu d ie s are s u m m a ris e d ,
fo llo w e d by a d is c u s s io n o f t h e in v e n to ry as a w h o le .
MIRA-PE is th e b ia n n u a l p o lic y e v a lu a tio n re p o rt o f t h e F le m ish
E n v iro n m e n t A gency (PE sta n d s f o r ‘ P o licy E v a lu a tio n ’). It is
th e w o rk o f re fe re n c e w ith re gard to p o lic y e v a lu a tio n s tu d ie s
in th e F le m ish R egion o f B e lg iu m . The re p o rt is a im e d a t p o lic y
m a kers (m e m b e rs o f p a rlia m e n t, c a b in e t m e m b e rs , th o se
re s p o n s ib le fo r p o lic y p la n n in g ), p o lic y e xe c u to rs and o th e r
p e o p le in v o lv e d in th e p o lic y process. M o re o v e r, o th e r p e rsons
w ith an in te re s t in th e p o lic y ( o r a c ritic a l v ie w o n it) m ay fin d
th e re p o rt u s e fu l as w e ll. A fte r a ll, a p o lic y has c o n se q u e n ce s
fo r eve ry c o m p a n y and every c itiz e n .
A ll p u b lic a tio n s , fig u re s a n d in d ic a to r s fr o m M IR A c a n be
c o n s u lte d o n th e w e b s ite o f t h e F le m is h E n v iro n m e n t A g e n c y
w m v .m ilie u ra p p o rt.b e (a v a ila b le in E n g lis h ).