Widening Participation Evaluation considerations Tess Lea QEII Fellow


Widening Participation

Evaluation considerations



Tess Lea QEII Fellow

Department of Gender and Cultural Studies

Working with young or marginalised people


Comes down to:

› What do we want to do?

› How much time have we got?

› Who have we got?

› What resources do we have?

› Who is our target audience?

› Best imaginable outcome for results?

What’s the best approach?

Some techniques

› Interviews

› Observations

› Focus Groups

› Small questionnaires with crisp, manageable questions

› Review of documents, records, media

› Limited archival research

› Performance based methods

Interactive role playing

› Follow up (cohort) surveys to map changing decisions/perceptions

› Participant led data gathering

› Focus group

Economical but risk getting dominant ‘group think’

› Observations

less intrusive but takes longer saturation to get to patterns

› Interview format/interactive performance based

flexible and rich but time intensive

› Survey

larger potential population

useful for longitudinal or cohort analyses

not ideal for understanding detail or social complexity

Also dealing with questionnaire-fatigued target groups

Pros and cons


Sampling: size versus depth

› If want larger sample, must tame the monster: ask mainly yes/no questions; have only one open-ended question

› If want detail, limit the sample

› Most reliable method for young people is face to face interviews and participatory research

› In any case, design data analysis before you administer instrument of choice

Clarify what data you actually want and ask for it, not what you think should be asked

Involve target group in the revisions

Anticipate and design the data analysis tools (the database and coding systems) before going live

Some protocols

› Research shows that poorly worded questionnaires bore young people, or provokes conventional answers – needs to be attractive and easily interpreted

› Also important to avoid having it be another piece of school work

› True consent not possible in power structures

› Thus even more essential instruments are well designed

Conduct a small trial: test the questions – do people respond as you had hoped?

Ask young people how they found it. Gauge reactions and modify

Limit time required to 15 minutes or less

Be aware of biased terms – e.g. ‘assistance to the poor’ (positive) versus

‘welfare’ (negative)

Keep it uncluttered – less is more

Think about the order – make it a logical progression, not choppy


What are the right questions?

› What is the age group? Gender? Social background?

› What data story do you need to be able to tell and with what ambition

(For further funding? Publications?)

Can’t over-state need for clarity about purpose

› Vignettes of transformational stories are excellent

Write with journalistic flair

› Combine these with other impact data

The more ‘factual’ and repeatable over time, the better


Youth oriented evaluations that yield convincing data

› Given reality of need to maximise responses in shortest time frame and at least cost, how to get the holy grail of youth-centred evaluations that yield convincing data?

› Many interventions presume a deficit (in esteem, in awareness, in comfort) and rush to fill that, without any primary research

› Foregrounding the students and making the evaluations as studentcentred as the interventions vital especially given academic persona as another teacher/adult

› Be creative


Brainstorming time


Widening Participation

Are we making an impact? How known?

What is meant by/needed for sustainability?

Is it worth expanding/scaling?

What do we not want to be doing more of?

Brainstorming time

› Group discussion of project status in relation to the timelines and matching methods to questions to desired outcomes

› Contact details: tess.lea@sydney.edu.au

; 9351 6777