The Reinvention Centre for Undergraduate Research
Academic Fellowship
Evaluating and Reinventing Sociology’s Professional Skills Programme: Research Skills for
First Years
Analysis of First Year Student Questionnaires
105 questionnaires were completed and returned, of these there was a good rate of response to
those questions designed to generate quantitative data – ranging between 97% and 100%. The
response rate to the qualitative questions was much lower with under 50% of respondents giving
answers to some questions, responses to these questions also proved to be rather variable with
respondents interpreting the questions in very different ways so that similar points were raised in
different sections by different respondents.
Data from pre-coded questions was entered into SPSS and basic quantitative analysis performed,
qualitative data from open-ended questions was, where practicable, coded and also entered into
SPSS, where this was not practicable a general 'flavour' of responses is discussed within the
analysis.
Students overall view of PSP
The overall picture that emerges from the analysis of the data is that students are making selective
use of PSP – a large proportion of respondents referred positively to the guidance provided on
referencing and citation, while a significant number also found the help with essay structuring
advantageous (these numbers are only partially represented in the tables below as referencing
and essay structuring were referred to by different respondents at different points in the
questionnaire). At the same time over 50% regarded large parts or all of the programme in a
negative light (see Tables 1 & 12) – referring to material being at too low a level - “obvious”,
repeating existing knowledge from school, college or access courses or regarding the programme
overall as in some measure a “waste of time”. This can be related to the 40.8% of those who
commented on their effort in PSP who said that they only put effort into the programme where they
felt it to be beneficial, often with reference to skills necessary for writing class essays.
The Helpfulness of PSP so far
The majority of students found PSP either not very or not at all helpful, although a large minority
had also found it mostly helpful. The reasons for this are explored in the analysis of later sections
of the questionnaire.
Table 1 - How helpful have students found PSP so far?
Freque Perce Cumulative Percent
ncy
nt
Very
4
3.8
3.8
Mostly
45
42.9
46.7
Not
Very
51
48.6
95.2
Not At
All
5
4.8
100.0
Total
105
100.0
1
Clarity of Instructions and Guidance
Table 2 - How Clear did you find the instructions and guidance?
Frequency
Percent Valid
Cumulative
Percent Percent
Very
34
32.4
33.3
33.3
Mostly
55
52.4
53.9
87.3
Not Very 12
11.4
11.8
99.0
Not At
All
1.0
1.0
100.0
SubTotal 102
97.1
100.0
Missing
3
2.9
Total
105
100.0
1
As can be seen, 87% of respondents found the instruction and guidance either mostly or very
clear, of the 6 respondents who explained why they found the instructions not very or not at all
clear, 3 stated that they had found the instructions to be too obvious or basic, suggesting that
some of those who gave a negative response to this question were doing so as part of an
expression of general discontent with the PSP programme.
Table 3 - Reasons why instructions clear/not clear
Frequency
Percent
Valid
Percent
Cumulative
Percent
clear and easy to use
33
31.4
66.0
66.0
confusing
5
4.8
10.0
76.0
trouble accessing online
sources
2
1.9
4.0
80.0
too obvious/basic
5
4.8
10.0
90.0
other
5
4.8
10.0
100.0
Total answered
50
47.6
100.0
Missing
55
52.4
Total
105
100.0
As can be seen from Table 3, the vast majority of respondents who explained why the PSP
guidance was clear or not clear stated that the instructions and materials were clear and easy to
use, while a small number either found the instructions confusing or had experienced some
problems accessing online resources.
2
Relevance of PSP
Table 4 - How relevant is PSP to your academic modules?
Frequency Percent
Valid
Cumulative
Percent Percent
Very
10
9.5
9.7
9.7
Mostly
52
49.5
50.5
60.2
Not Very
35
33.3
34.0
94.2
Not At All
6
5.7
5.8
100.0
Sub Total
103
98.1
100.0
Missing
2
1.9
Total
105
100.0
When explaining how relevant they felt the programme to be to their modules many students
raised the usefulness of the referencing and essay writing skills covered. Students seemed to
respond to this question in a highly personalised way, often interpreting relevance as usefulness to
them and there were therefore many comments about the programme repeating knowledge that
students had acquired prior to coming to university and also about too much time being taken to
cover material.
Table 5 - How relevant is PSP to your academic modules * How useful have you found the
programme - Crosstabulation
How useful have you found the programme
How
Very
relevant is
PSP to your
Mostly
academic
modules
Not Very
Not At All
Total
Total
Very
Mostly
Not Very
Not At All
4
5
1
0
10
100.0%
11.6%
2.0%
.0%
9.7%
0
29
23
0
52
.0%
67.4%
45.1%
.0%
50.5%
0
8
23
4
35
.0%
18.6%
45.1%
80.0%
34.0%
0
1
4
1
6
.0%
2.3%
7.8%
20.0%
5.8%
4
43
51
5
103
100.0%
100.0%
100.0%
100.0%
100.0%
A Cramer's V value of 0.430 (p<.001) indicates that there is a moderately strong positive
association between how relevant students believe the programme to be and how useful they
have found it although the causal direction of this relationship is unclear.
3
Best and Least liked aspects of PSP
Table 6 - Best liked aspect of PSP
Frequen Perce
cy
nt
Valid
Percent
Cumulative Percent
Essay structuring
16
15.2
18.4
18.4
Referencing
39
37.1
44.8
63.2
Library use
5
4.8
5.7
69.0
clear material
5
4.8
5.7
74.7
answer related to SII more
generally
4
3.8
4.6
79.3
seminar methods
8
7.6
9.2
88.5
easy
1
1.0
1.1
89.7
helps to bridge college-uni
divide
2
1.9
2.3
92.0
note-taking
1
1.0
1.1
93.1
practical advice
2
1.9
2.3
95.4
other
4
3.8
4.6
100.0
Total answered
87
82.9
100.0
Missing
18
17.1
Total
105
100.0
A small number of students gave answers that appeared to relate to SII more generally when
answering what they liked best or least about the programme suggesting that some wither
misunderstood the purpose of the questionnaire or that they have failed to grasp the distinct
intentions and existence of PSP.
4
Table 7 - Least liked aspect of PSP
Frequency Percent Valid
Percent
Cumulativ
e Percent
not enough detail
2
1.9
2.9
2.9
repetition of existing
knowledge
9
8.6
13.2
16.2
repetitive nature of material 4
3.8
5.9
22.1
answer related to SII more
generally
8
7.6
11.8
33.8
organising notes
1
1.0
1.5
35.3
material at too low a level
13
12.4
19.1
54.4
too much time devoted to
PSP
2
1.9
2.9
57.4
seminar learning contract
1
1.0
1.5
58.8
general dislike of PSP
13
12.4
19.1
77.9
essay writing and planning
1
1.0
1.5
79.4
ordering of the programme
3
2.9
4.4
83.8
referencing and citing
1
1.0
1.5
85.3
library use
3
2.9
4.4
89.7
note-taking
4
3.8
5.9
95.6
other
3
2.9
4.4
100.0
Total answered
68
64.8
100.0
Missing
37
35.2
Total
105
100.0
As can be seen, there was a wide variety of least liked aspects of PSP, however just over 50%
expressed either a general dislike of PSP (19.1%) or that it was in some way at too low a level
(19.1%) and/or covered prior knowledge (13.2%). The comments by 3 respondents surrounding
the way the programme was ordered were reflected elsewhere in the questionnaire by other
respondents, with several comments overall about this – for example that by the time essay
structuring was covered students had already written one or 2 essays, or that the library
introduction occurred after they had already been using the library for some weeks.
5
Transferability of PSP Skills
Table 8 - How transferable are the skills developed in PSP?
Frequency
Percent
Valid Percent
Cumulative Percent
Very
3
2.9
3.0
3.0
Mostly
45
42.9
44.6
47.5
Not Very
48
45.7
47.5
95.0
Not At All
5
4.8
5.0
100.0
Total
101
96.2
100.0
Missing
4
3.8
Total
105
100.0
Qualitative responses to the question of how transferable the skills developed in PSP are
suggested that many students had not understood the meaning of 'transferable skills' or that they
had a very narrow view of such skills – for example there were numerous references to the
transferability of PSP skills to other undergraduate modules and also several comments along the
lines of “employers will not ask you to organise notes or how many points to put in each paragraph
of an essay”.
Time and Effort invested in PSP
Table 9 - How much time and effort have you invested in PSP?
Frequency Percent Valid
Percent
Cumulativ
e Percent
A lot
0
0.0
0.0
0.0
Some
34
32.4
33.0
33.0
Not Very
Much
62
59.0
60.2
93.2
None
7
6.7
6.8
100.0
Total
103
98.1
100.0
Missing
2
1.9
Total
105
100.0
It is to be noted that no students declared themselves to have invested “A lot” of effort in PSP,
although, as can be seen in Table 13, 2 felt that other members of their groups had invested a lot
of effort.
6
Table 10 - Reasons for time and effort spent
Frequency Percent Valid
Percent
Cumulative
Percent
repetition of existing
knowledge
5
4.8
10.2
10.2
put in effort where it was
felt to be beneficial
20
19.0
40.8
51.0
only made effort in
seminars
7
6.7
14.3
65.3
have done every
2
exercise/read every section
1.9
4.1
69.4
general dislike of PSP
programme
3
2.9
6.1
75.5
doesn't need much effort
2
1.9
4.1
79.6
other work takes priority
6
5.7
12.2
91.8
not used in seminars
3
2.9
6.1
98.0
other
1
1.0
2.0
100.0
Total answered
49
46.7
100.0
Missing
56
53.3
Total
105
100.0
Students' reasons for the time and effort they had spent on PSP echoed responses made
elsewhere in the questionnaire with over 40% commenting that they only put effort in where it was
felt to be beneficial – often such comments referred to the usefulness of the programme sections
on referencing and essay writing. A small but significant number of responses indicated that other
academic work took priority over PSP with several comments about needing to keep up with
reading for SII and other modules.
Table 11 - How useful have you found the programme * How much time and effort have you
invested in PSP - Crosstabulation
How much time and effort have
you invested in PSP
How useful
have you
found the
programme
Total
Total
Some
Not Very
Much
None
Very
3
8.8%
1
1.6%
0
.0%
4
3.9%
Mostly
14
41.2%
30
48.4%
1
14.3%
45
43.7%
Not Very
15
44.1%
31
50.0%
3
42.9%
49
47.6%
Not At All
2
5.9%
0
.0%
3
42.9%
5
4.9%
34
62
7
103
100.0%
100.0%
100.0% 100.0%
A Cramer's V value of 0.376 (p<.001) indicates that the positive association between students'
effort and how useful they found the programme is moderate in strength.
7
Table 12 - How much time and effort have you invested in PSP * How relevant is PSP to
your academic modules - Crosstabulation
How relelvant is PSP to your academic
modules
How much time
and effort have
you invested in
PSP
Some
Very
Mostly
Not Very
Not At All
3
20
9
1
33
30.0%
38.5%
27.3%
16.7%
32.7%
31
18
5
61
70.0%
59.6%
54.5%
83.3%
60.4%
0
1
6
0
7
.0%
1.9%
18.2%
.0%
6.9%
10
52
33
6
101
100.0%
100.0%
100.0%
Not Very Much 7
None
Total
Total
100.0% 100.0%
A Cramer's V value of 0.330 (p<.001) indicates that there is a moderate positive association
between how relevant students believe the programme to be and how much effort they have
invested in PSP
Table 13 - Perceptions of other students' level of investment in PSP
Frequenc Percen Valid
y
t
Percent
Cumulative Percent
A lot
2
1.9
2.0
2.0
Some
25
23.8
24.5
26.5
Not Very
Much
70
66.7
68.6
95.1
None
5
4.8
4.9
100.0
Total
102
97.1
100.0
Missing
3
2.9
Total
105
100.0
8
Table 14 - How much time and effort have you invested in PSP * Perceptions of other
students' level of investment in PSP - Crosstabulation
Perceptions of other students' level of
investment in PSP
How much time
and effort have
you invested in
PSP
Some
Not Very
Much
None
Total
Total
A lot
Some
Not Very None
Much
1
16
16
1
34
2.9%
47.1%
47.1%
2.9%
100.0%
1
9
49
2
61
1.6%
14.8%
80.3%
3.3%
100.0%
0
0
5
2
7
.0%
.0%
71.4%
28.6%
100.0%
2
25
70
5
102
2.0%
24.5%
68.6%
4.9%
100.0%
With a Cramer's V value of 0.34 (p<.005) there is a moderate strength of relationship between
student's perceptions of their own efforts and those of other group members, suggesting that
students on the whole perceive that others put in similar effort to themselves.
9
Tutor Approach to PSP
Table 15 - Tutor's approach to PSP
Freque Perce Valid
ncy
nt
Percent
Cumulative Percent
not enough
time/importance given to
PSP
5
4.8
6.0
6.0
takes the right amount of
time
8
7.6
9.5
15.5
takes it seriously
10
9.5
11.9
27.4
makes the best of it
despite student negativity
3
2.9
3.6
31.0
thorough
5
4.8
6.0
36.9
to the point
3
2.9
3.6
40.5
encouraging
3
2.9
3.6
44.0
enthusiastic
11
10.5
13.1
57.1
patient
1
1.0
1.2
58.3
gives 20 minutes a week to 5
it
4.8
6.0
64.3
negative attitude
4
3.8
4.8
69.0
laid back
4
3.8
4.8
73.8
uses most useful points
10
9.5
11.9
85.7
not much used in seminars 10
9.5
11.9
97.6
other
2
1.9
2.4
100.0
Total answered
84
80.0
100.0
Missing
21
20.0
Total
105
100.0
Students' responses to this question indicated a largely positive attitude towards seminar tutors.
The comments made also suggested that tutors took a flexible approach to the programme,
placing greatest emphasis and time upon those aspects that were perceived as most 'useful' to
students.
An analysis of the relationship between the students' description of their tutors' approach to PSP
and their seminar group showed there to be no significant relationship between the two variables.
Analysis of the relationship between seminar group and other variables, such as how helpful
students' had found the course, also showed no significant relationship.
Views of Changes to PSP
As can be seen there were varying views on the different suggested changes to PSP. Of the 102
respondents to these questions 79 (77%) wanted some form of change to the programme. As can
be seen from Tables 16-18, while making PSP optional was a popular suggestion, the ideas of
making it credit-bearing or timetabling it separately from SII were rejected by the majority of
10
respondents. Both a crosstabular analysis and a loglinear analysis of individuals' answers to all
three suggestions showed no significant pattern of answers, and the explanations of answers
showed a wide variety of views as to what should be be done to change PSP. 4 or 5 respondents
commented that if PSP was separately timetabled then students would be unlikely to turn up to it,
some coupled this comment with another statement that this would mean that students would miss
out on vital skills. A selection of comments made is given below.
Table 16 - View of making PSP Optional
Frequency Percent Valid
Cumulativ
Percent e Percent
Agree
56
53.3
54.9
54.9
Disagree
39
37.1
38.2
93.1
Don't Know 7
6.7
6.9
100.0
Total
102
97.1
100.0
Missing
3
2.9
Total
105
100.0
Table 17 - View of making PSP Credit-bearing
Frequency Percent Valid
Cumulative
Percent Percent
Agree
31
29.5
30.1
30.1
Disagree
66
62.9
64.1
94.2
Don't Know 6
5.7
5.8
100.0
Total
103
98.1
100.0
Missing
2
1.9
Total
105
100.0
Table 18 - View of making PSP separate from SII
Frequency Percent Valid
Cumulative
Percent Percent
Agree
32
30.5
31.1
31.1
Disagree
64
61.0
62.1
93.2
Don't Know 7
6.7
6.8
100.0
Total
103
98.1
100.0
Missing
2
1.9
Total
105
100.0
Comments about changes to PSP:
 Should be separate & compulsory & taught by professional
 necessary and more attention should be paid to it
 “It is a potentially valuable resource for students wider study and should be placed in its
own wider context”
11
“crucial to our learning therefore need to have a separate timetable slot for it”
 because it is important it should be credit-bearing
 important enough to be compulsory
More use to some than others
 those who need it could do it
some necessary skills taught – eg referencing
separate timetabling means some will miss out
hold a weekly optional session which individuals can sign up for
people would not turn up if timetabled separately
if it is compulsory then SII is the best place for it
Individuals should be assessed through a personal skills assessment at the beginning of
the year
I would benefit more from PSP if I invested more time in it
“Although sometimes annoying I think it can be helpful”
“I don't think things like listing abbreviations should be important enough to be considered
part of our degree” (ref. Credit for PSP)
“all these ideas would make the situation a lot better”
waste of time
 could probably self-teach from a handbook – not worth dedicating time to it xx
 does not warrant separate time
 “Just dont have PSP it is pointless it waste time from studying people wouldnt attend if
you really want to include it just give a handout for people to read if they want” [sic]
“Another separate lesson for PSP would add more work to one's schedule and could take
from other areas of study”
“Doesn't seem worth it if no credit is given for doing it”
students will miss out if it is made optional
easier to go through in own time than to timetable separately
“The skills outlined in the program are learnt gradually and cannot be taught formally. They
are learnt through mistakes and improvement eg. ESSAY WRITING”
“If there was credit attached than I would be happier doing the PSP as I would be gaining
more than knowledge. It would be better separate so it would give SII more individual
time.”



















Ruth Morton and Caroline Wright
January 2008
12