HPLINK
An Electronic Network Discussion Group for Health Promotion
Evaluation Report
March 8, 2002
Building Health Promotion Capacity
Saskatchewan Heart Health Program
in partnership with:
Health Canada
Saskatchewan Health
Heart and Stroke Foundation of Saskatchewan
Saskatchewan Heart Health Program
Department of Community Health and Epidemiology
Health Sciences Building
University of Saskatchewan
107 Wiggins Road
Saskatoon, SK S7N 5E5
E-mail: shhp@sdh.sk.ca
Table of Contents
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
About the SHHP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
About HPLINK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
SHHP: Building Health Promotion Capacity
Evaluation of HPLINK An Electronic Network Discussion Group for Health Promotion
About the SHHP
The Dissemination Phase of the Saskatchewan Heart Health Program - Building Health
Promotion Capacity (SHHP) is a five-year project funded by Health Canada, Saskatchewan
Health and the Heart and Stroke Foundation of Saskatchewan. The Dissemination Phase builds
on two previous Heart Health Program phases: the provincial Heart Health Survey undertaken in
1989-90, and the Community Demonstration Projects which operated from 1992-97. Lessons
learned from these and other experiences are being used to enhance health promotion in
Saskatchewan. The aims of this phase are to help health districts build their capacity to plan,
implement and evaluate health promotion activities, and to improve our understanding of the
capacity-building process. Capacity is defined as a set of knowledge, skills, commitment and
resources required by individuals and organizations to conduct effective health promotion.
SHHP interventions to build capacity include continuing education, consultation and networking
(electronic and face-to-face).
About this Document
This report summarizes the evaluation of HPLINK, an electronic listserv facilitated by the
SHHP.
About HPLINK
HPLINK was set up as an electronic discussion group for health promotion, primarily for health
promotion practitioners in Saskatchewan. It evolved from A Feasibility Study on Prairie
Health Promotion Electronic Networking carried out in 1998. Health promotion practitioners
felt that electronic communication would be a useful mechanism by which to obtain and share
information across geographic distances presented in the prairies. In late 1998, a Working Group
of seven Health Promotion Contacts (a person designated as a contact in each Health District for
health promotion work) and a Saskatchewan Health representative was struck to act as a pilot
group, to test the use of an e-mail group for communication. In the summer of 1999,
negotiations began with the University of Saskatchewan to be the host site for the listserv. At the
Health Promotion Summer School held at the University in August 1999, the listserv was
launched. In December of that year, archives were added to the service. HPLINK has been
functioning on a regular basis since that time.
HPLINK Evaluation - 3
SHHP: Building Health Promotion Capacity
The specific objectives of HPLINK have been to:
1.
mediate communication amongst practitioners of health promotion in Saskatchewan and
beyond;
2.
enable practitioners to communicate knowledge, insights, and creative solutions with
others; and
3.
share information and resources practitioners have found useful.
Methodology
Planning for an evaluation of HPLINK began in September 2000, one year after the launch of
HPLINK. A graduate student, Judith Wright, helped with the evaluation, particularly with
development of questionnaires, key informant interviews, and data analysis. She worked
together with Lori Ebbesen and Lorraine Khachatourians on this project. The interview
questions were guided by the questions from the HPLINK framework for evaluation.
The objectives of this evaluation included:
1.
2.
3.
4.
5.
6.
7.
reach multiple HPLINK stakeholders (subscribers, non-subscribers, listserv manager,
lurkers [those who are passive participants], key informants)
document developmental milestones
identify promising marketing strategies
assess/understand usage to date
identify reasons for non-subscription by Health Promotion Contacts (HPCs)
understand the contribution of HPLINK to increasing individual health promotion
capacity
explore future directions.
The following data gathering strategies were employed:
1.
2.
3.
4.
review of the number of postings, subscribers, non-subscribers
a content analysis of postings during three time periods
key informant interviews with members of the original Working Group, Health
Promotion Contacts (HPCs) who do and others who do not subscribe to HPLINK, those
who have left the list (un-subscribed), the list manager, and other key health promotion
practitioners in Saskatchewan
an on-line survey of current HPLINK participants.
A total of 13 key informants were interviewed (11 by phone, two in person) between December
2000 and March 2001, from a convenience sample of 14 contacts known to the SHHP (one nonresponse). Of the nine HPCs interviewed, three were former HPCs and six current HPCs in
HPLINK Evaluation - 4
SHHP: Building Health Promotion Capacity
Health Districts; they were HPLINK subscribers or former subscribers. Three additional key
informants who are not HPCs (two consultants in health promotion and one from outside the
health field) and who are current subscribers were interviewed. The HPLINK list manager was
also interviewed. Finally, three non-subscriber HPCs were interviewed.
The interview was piloted with three respondents. No changes were made to the format or the
interview guide (see Appendix 1). The interviews from the pilot were included in the key
informant interview results. The interview, on average, took 30 minutes, and each interview was
taped. Notes were transcribed from the tapes.
A survey was developed, based on the research questions and on results of the key informant
interviews, to send electronically to participants on HPLINK (see Appendix 2). The survey was
posted on a web site which was linked to the HPLINK e-mail message sent to the participants.
Responses were anonymous; the source e-mail address of respondents did not appear. The
survey was sent twice in June 2001, two weeks apart, in order to maximize response. The
majority of responses were received from the first posting. A total of 40 responses were
received, for a response rate of 35% (list membership at the time of posting was 114).
Findings
HPLINK Use
As of May 2001, there were 116 subscribers to HPLINK. At least 18% (22 of 118) of the current
subscribers appeared to be from within the Health Districts, including Health Promotion Contacts
(HPCs). Each Health District has a HPC; this group has been one of the targets of SHHP
interventions during the first half of the dissemination phase of the program. The majority of
HPCs were not subscribers; only nine of 33 HPCs (27%) were subscribed or have subscribed at
some point. At least two HPCs who were subscribers in 1999 were not current subscribers
suggesting they had left the list. It is difficult to categorize the remainder of the subscribers by email addresses alone, but they appear to include students, health promotion consultants/
practitioners, nutritionists, and health/social scientists attached to academic institutions.
This subscriber information was drawn from e-mail addresses only. Estimates were calculated
by comparing the HPLINK list of participants at the time of analysis (October 20, 2000) to a list
of HPC e-mail addresses (August 24, 2000). There may be other HPCs who were subscribing
under a global e-mail address (for example, che@sk.sympatico.ca). Other Health District
subscribers were identified by District initials incorporated in the e-mail addresses (for example,
abc@sdh.sk.ca). There may have been other District personnel who are subscribing under a
home-based or other e-mail addresses.
Content Analysis of HPLINK
HPLINK Evaluation - 5
SHHP: Building Health Promotion Capacity
Listserv entries were compared for the first three months of operation (September to December
1999) and for February, March and August 2000. In 1999, a total of 75 listserv entries were
posted (average of 25 per month). There were 263 entries made in 2000, averaging 22 per month
(range 8 - 34 per month). Sources of information (i.e. where the information was located or
originated) are shown in Table 1. National sources were the most common, such as Health
Canada, Canadian Policy Research Network, and sources in other provinces.
Table 1.
Sources of information
Source
1999 (% )
2000 (% )
National
31 (41)
28 (41)
International
22 (29)
23 (34)
Saskatchewan
20 (27)
15 (22)
Other
2 (3)
2 (3)
Total
75 (100)
68 (100)
Most of the postings were made by the list manager and increased in frequency between the two
time periods. This is reflective of additional time spent by the list manager in finding resources
to post on HPLINK. See Table 2 for frequency of postings by different sources.
Table 2.
Origin of postings
Origin of Postings
1 9 99 (% )
2000 (% )
List Manager
24 (32)
43 (63)
Health Districts/HPC s
18 (24)
7 (10)
Others*
58 (44)
21 (27)
Total
75 (100)
68 (100)
*Others
include ac adem ics, cons ultants, stud ents
The list manager sent primarily web resources and full text information. HPCs and Health
District subscribers tended to post mostly requests. The four topics most often addressed were
general health promotion, poverty, population health and health policy. See Table 3 for a
complete listing of topics.
Table 3: HPLINK posting topics, September to December, 1999, and
HPLINK Evaluation - 6
SHHP: Building Health Promotion Capacity
February, June and October, 2000
Top ic
1999
2000
Frequency
Percent
Frequency
Percent
General health promotion
15
20
9
13
Poverty
12
16
9
13
Tobacco related
9
12
1
1.5
Introduction
8
10.7
0
0
Public he alth
4
5.3
3
4
Child hea lth
3
4
2
3
Graduate programs
3
4
0
0
Education
2
2.7
0
0
W ome n s health
2
2.7
2
3
HIV/AIDS
2
2.7
2
3
Program evaluation
2
2.7
0
0
Repro ductive h ealth
2
2.7
3
4
Globalization
2
2.7
1
1.5
Employment
1
1.3
0
0
Health economics
1
1.3
0
0
Popula tion health
1
1.3
8
12
Environment
1
1.3
0
0
Menta l health
1
1.3
0
0
Cultural issues
1
1.3
0
0
Health policy
1
1.3
5
7
Injury
0
0
4
6
Health reform
0
0
3
4
Facilitation
0
0
3
4
Nutrition
0
0
3
4
Addictions
0
0
1
1.5
HPLINK Evaluation - 7
SHHP: Building Health Promotion Capacity
Top ic
1999
2000
Prima ry health
0
0
1
1.5
Rural he alth
0
0
1
1.5
Drug resistance
0
0
1
1.5
Health ethics
1
1.3
0
0
Humour
1
1.3
0
0
Comm unications technology
0
0
2
3
HPLINK
0
0
2
3
Diabetes
0
0
1
1.5
International development
0
0
1
1.5
75
97.3
68
98
Total
HPLINK Subscriber Estimations, April 2000 to January 2001
Subscriptions have grown steadily since the launch of HPLINK in August of 1999. More
subscribers are gained than lost each month (see Table 4). Based on the subscription list, there is
more unsubscribing and resubscribing activity among the Health Districts and HPCs than among
other subscribers.
Table 4: Subscription Activity, April 2000
April
May
June
January 2001
July
Aug.
Sept.
Oct.
Nov.
Dec.
Jan.
New/renew
subscribers
1
-
6
-
10
23
8
-
9
3
Lost subscribers
-*
-
4
-
14
2
1
-
3
1
Total subscribers
84
84
86
86
82
103
110
110
116
118
* (-) indicates number not available.
Other Observations
During the first three months of operation, the subject titles of at least 22% of the entries did not
HPLINK Evaluation - 8
SHHP: Building Health Promotion Capacity
identify the subject of the posting in a manner that would be convenient for searching the
archives (for example jobs, conferences and publications not specified, but identified as jobs
new publication, good stuff, or video ). Specificity of the subject line did increase to 96%
by October 2000.
There is little information available regarding use of the archives. The Directory Report from the
University of Saskatchewan Computing Services was requested for each month by the Technical
Manager. In the month of October 2000, there were 46 requests for /lists/ (not clear if these
were archives). It is not possible, either, when searching the archives, to see who made the entry
or the date of the message. All entries are simply identified as posted by the HPLINK Manager
and are archived as messages arrive. When doing a topic search, however, one sees the original
posted message and sender.
Results of Key Informant Interviews
Results of the HPC interviews
Health Promotion Contact interviewees were asked about factors that contribute to their use of
HPLINK. The following are the four main contributing factors discussed:
1.
Access
The Health Districts are very uneven in their access to and experience with electronic
communications. All the respondents identified access to Internet and e-mail at a personal
workstation as factors that contribute to the familiarity and the use of electronic communications
in general. Access and use of e-mail within the Districts is not uniform.
2.
Proficiency and comfort
The HPCs are at various stages of proficiency and comfort with electronic communications.
Most respondents use fax, and print-and-circulate to pass on information, although some are
primarily using forwarded e-mail. Most of the respondents indicated being more comfortable
using e-mail than the Internet.
3.
Training and/or working knowledge
Several respondents identified both training and working knowledge as factors that facilitate the
use of electronic communications. While training was not identified as absolutely necessary,
time for hands-on, trial and error experience was seen as important.
4.
Technical support
Technical support, or help from colleagues, made using electronic communications easier.
Factors that hindered use of HPLINK included:
HPLINK Evaluation - 9
SHHP: Building Health Promotion Capacity
1.
Time management
Available time is an issue mentioned by all respondents. Volume of messages was more
problematic for some than others, but virtually all identified that they had to be selective about
what they read or saved. Accumulated e-mail messages tended to be especially problematic for
people who worked part-time, or who spent part of their time working at other locations.
2.
Comfort or confidence level
About half of the respondents specifically identified or intimated that comfort or confidence level
limited their participation in the listserv (i.e., posting messages). Comments like I don t feel
I m qualified or, if there was a topic I really knew, may indicate lack of confidence. Several
commented on the fact that they did not know who the other subscribers were. Others noted they
would be more comfortable phoning the person who posted a message to pursue a discussion.
Regarding networking with colleagues, most respondents recognized the importance of personal
communications. The listserv was, however, seen as useful to know who was doing what, and
for the opportunity to network.
3.
Practicality or relevance of topics
At least one third of respondents felt the topics were not particularly relevant to their practice.
The issue of confidence level and relevance seem closely linked. One comment that emphasizes
this point was: I get the impression that the people [posting] don t care that I m working on a
sharps disposal program; it makes you feel a bit small in the scheme of things. While some
thought the topic matter was too broad, others appreciated the broadness. One respondent said
that limiting the number of messages one received would be preferable to limiting the broadness
of scope.
Interviewees were asked how they used HPLINK information. The following were the most
common categories of responses:
1.
General knowledge and background information
The main uses appears to be for general knowledge or background information, and for keeping
up-to-date on what s happening out there. Quick access to information is highly valued.
Several respondents commented that HPLINK helped health promotion practitioners avoid
recreating the wheel.
2.
Information sharing: I act as a gatekeeper for health promotion.
Respondents, in their daily work, often act as gatekeepers of information, channelling relevant
information to appropriate individuals and departments, both within and outside their District
organization. I feel it s part of my job to let people know about the information, even when I
don t hear back whether it s useful. Information sharing was most often identified as the chief
or most valuable function of the listserv.
3.
Networking opportunities
HPLINK Evaluation - 10
SHHP: Building Health Promotion Capacity
Most respondents recognized that HPLINK provided a good opportunity to network, and most
commented that it was an opportunity that was generally under-exploited. One respondent
indicated that HPLINK helped to augment the networking among HPCs that was taking place
through HPC meetings. Another was that it made one feel less isolated.
4.
Practical information
The respondents were evenly divided about the relevance of the information to their practice.
Half of the respondents indicated that they were receiving information that was valuable for
program planning and evaluation (one comment: concrete, non-theory based practical items ).
These were generally the people looking for best practices and research that would assist them in
practicing evidence-based health promotion. The other half felt the information was less useful;
the primary benefit was simply background information.
Interviewed subscribers were asked how satisfied they were with their participation in HPLINK.
More than half of the respondents were satisfied and the remainder were neutral to dissatisfied.
The perceived benefit for those who were satisfied was primarily the enhancement of knowledge.
About half were not able to articulate exactly how they had benefited or said they had benefited
to a minor extent. Only one respondent felt she did not benefit from subscribing.
Interviewees were asked for suggestions to improve HPLINK. The following were areas
suggested:
1.
Content
Some respondents wanted to see more local content and information on small scale initiatives.
Several respondents noted that the topics are broadly based, of national and international focus,
and that initiatives with a more local flavour would be appreciated. A focus on small scale
initiatives, they said, may encourage more HPCs to participate in discussions. Only one
respondent contradicted this, saying the HPCs got sufficient local information from other
sources.
2.
Initiating discussion
The suggestion was made to jump start discussions and experiment with topics. A question
could be used to focus discussion, for example: Use of the population health model in your
District. What works?
Other suggestions for specific kinds of information were also made:
"
information from other sectors and ministries
"
information on awareness or acknowledgement weeks
"
more information about best practices
"
more information on evaluation
"
information on granting opportunities
HPLINK Evaluation - 11
SHHP: Building Health Promotion Capacity
"
real-time conference on a predetermined topic.
Results for Non-HPC Subscribers
Results for the three non-HPC persons interviewed were similar to the above. Access and
experience contributed to using HPLINK; this group seemed to be generally more proficient with
electronic communications. Barriers were time management, and confidence and comfort, the
latter particularly identified generally, for others, not necessarily themselves. One respondent
said,
In spite of the efforts of organizing support, people go to people they like and trust,
people who are geographically closest and whom they see face-to-face on a regular basis
and with whom they have the greatest opportunity to build a personal relationship.
Keeping up-to-date was the most common use of HPLINK. These subscribers were also satisfied
to neutral.
Additional suggestions for improvement of HPLINK included:
"
discipline or program-specific pieces
"
topics that include outcome indicators, clear statistical and policy linkages
between health promotion, health expenditures and other social outcomes
"
continued development of broad population health approaches
"
SHHP could take more responsibility in posting queries and/or information
HPLINK List Manager
Comments by the list manager were similar to those already noted for subscribers. From the
perspective of the list manager, most people subscribing are using HPLINK for information.
Most users are passive recipients of information; a few subscribers are more active. HPLINK has
not been well-used as a discussion forum, and the manager was not aware if participation was
facilitating networking among practitioners.
Suggestions for improving HPLINK focussed on ways to train people in the use of the
technology, at workshops and conferences. Technical problems have been noted as a barrier.
Requests for assistance come from subscribers, including how to post messages, and how to
subscribe and unsubscribe. The archives are not generally used, not are people knowledgeable
about how to search the archives.
The list manager frequently assists list participants in posting messages, and subscribing and
unsubscribing to the list.
Non-subscriber HPCs
HPLINK Evaluation - 12
SHHP: Building Health Promotion Capacity
Two of the three non-subscribing HPCs interviewed had only recently gained access to
computers and e-mail; the third still did not have computer access. As might be expected, access
to the technology and confidence in using the technology were the two main barriers to using
HPLINK identified by this group. This group used the more traditional methods of telephone
and fax for contacting colleagues or others, and for distributing information, would print material
for circulation. They also used more traditional sources for information, e.g. reading journals,
and attending workshops. Forwarding of e-mail was beginning to be used by the two HPCs who
had computer access. Use of the technology was a low priority for this group at the time of the
interview. Time to use the technology was mentioned by all three as a limiting factor.
The kinds of information this group was interested in was on best practices and evaluation. All
three HPCs knew that HPLINK was available, but had not joined the list, either because of access
or time limitations.
Two of the interviewees made additional comments:
I feel like a real dummy. You said you wanted to ask me these questions even though I
haven t seen HPLINK. I d be interested to see how many of the HPCs in the smaller
districts have access to it.
I wear many hats in the organization I m in and I know my role when I go to a HPC
meeting is significantly different from some of the others. When I initially started to come
to those meetings, I certainly noticed people come and go. And that s when I heard about
it (HPLINK). I know they re working on an orientation package but it would have been
nice to have a very specific piece of information, this is what it is, this is how you get
hooked up. What we got was very informal....
HPLINK Electronic Survey Results
Background information on respondents
The majority of respondents (63%) were community or health promotion practitioners. Another
20% of subscribers were either middle or senior managers. The remainder were students,
researchers and private consultants.
The largest proportion of subscribers were employed in health districts (48%); universities,
NGOs and government organizations had equal representation of 15% each. The other
subscribers were privately employed, self-employed or other.
When asked about their years of experience working in health promotion, 40% had over 10 years
of experience in the field. Another 25% had between three and five years of experience.
Twenty-two percent had less than two years of experience and the remainder had between six and
ten years.
HPLINK Evaluation - 13
SHHP: Building Health Promotion Capacity
The majority of respondents (87.5%) had access to a computer at a personal work station while
the rest had access through shared work stations; 65% also had access to a home computer. All
respondents used e-mail communication in their work, 97% used the internet, and 85% used
listservs. When asked about their comfort level in using electronic communications, most (65%)
indicated that they were very comfortable using this type of communication. Fifteen percent felt
they were expert and the remainder were somewhat comfortable with this technology.
When asked where respondents had first heard about HPLINK, 42% responded that they had
been introduced to it at a Health Promotion Summer School, while 33% had been introduced by
other colleagues. Only 7% had heard about it at HPC meetings.
Using HPLINK
The information found on HPLINK is used in a variety of ways in respondents work. Keeping
informed was the most common response. See Table 5 for other uses of HPLINK information.
Table 5.
Applications of HPLINK Information
Uses
Freq uen cy*
Perce nt
Keeping informed on current issues
34
85
Gathering background information
29
72
Inform ation on c onfere nces a nd even ts
28
70
Sharing information
20
50
Requesting information
13
32
Network with others
12
30
Develop programs
9
22
Prepa re briefs a nd repo rts
9
22
Discuss issues in health promotion
8
20
Sup plem ent c ours e m ateria ls
4
10
* Frequency is in response to each use
Respondents were asked to rate their primary use of HPLINK in their work. Keeping informed
was ranked first again, followed by gathering information, sharing information and requesting
information, for example:
I am putting together collections of resources to pass on to practitioners in different
HPLINK Evaluation - 14
SHHP: Building Health Promotion Capacity
areas. (#15)
[I use the information] to expand my bookmarks reference for information I need or might
need. (#19)
Respondents were asked how and with whom they shared the information they found on
HPLINK. The most common mechanism used was to forward messages to colleagues (33%),
followed by informal conversations with colleagues (28%), printing and circulating information
to colleagues (20%), forwarding messages to Board members, informal conversations with the
community/public (12%), informal conversations with Board and management (11%), and
presentations to colleagues (10%).
Sixteen percent of respondents indicated that they had posted messages to HPLINK during the
time they had been subscribed. For those who had not posted a message, the respondents were
asked to give a reason why they hadn t. Some respondents indicated that time was a factor:
If I accessed at home I might, but at work there is not time. I might post if I was looking
for some information. (#38)
[I am] busy with other duties at work, but I like to browse HPLINK. (#01)
[It is] not a priority, lack of time, can see myself using it more when I get back to work
(currently I am on educational leave). (#07)
Others felt that they did not have the expertise, either professional or technical:
Not sure exactly who I m talking to and usually have forgotten how to post and can t find
the e-mail explaining how to post and am not sure what level people want posting methodology, programs? (#10)
I did not feel I had the expertise to respond to the questions that were being asked. (#19)
I don t feel that I m that well informed yet. Prefer to read other people s views to
familiarize myself with the issues. (#32)
Barriers to Using HPLINK
Lack of time was listed as the most common barrier to using HPLINK (45%), followed by not
enough expertise (7%) and content not relevant (7%). Twenty-eight percent of respondents
did not feel they experienced any barriers to using HPLINK.
A few respondents commented on barriers as well:
HPLINK Evaluation - 15
SHHP: Building Health Promotion Capacity
There is a lot of information, having the time to go through it all is sometimes difficult.
(#12)
We do a lot of communicating/sharing through our professional group, therefore this
isn t my primary way to communicate about health promotion relevant to my practice.
(#31)
Elements of HPLINK That Respondents Liked
Respondents were asked, in an open-ended question, what they liked about having access to
HPLINK. Many remarked on the ease with which the information was made available, through
e-mail:
I like that it is available electronically and is so simple and easy to use. (#9)
Free. (#17)
[It is an] easy way to post a question and obtain help. I like the access to information.
(#21)
Easy access. Current and relevant information. The fact that it just pops up! (#22)
Others liked the fact that work was being done for them, in finding and screening information:
Sometimes I feel like I have my own personal research assistant, gathering information
from other sites, organizations and institutes. Thank you. (#35)
Someone else does the work in finding the latest information. I don t have the time to do
that much searching. (#30)
Saves me time! And to have it welcoming me in the mornings is a great way to start the
day. I feel like I ve quickly jumped in and have relevant links - sometimes in areas that
we hadn t considered - so it, at times, helps to bridge gaps/connect the dots. (#26)
Respondents liked the kind of information that appears on HPLINK:
I like that most communications are about Canadian materials. (#19)
I like that it tells me things that I may not know about and that it reminds me about other
items that I need to follow up on. (#20)
Edited, reliable sites. (#3)
HPLINK Evaluation - 16
SHHP: Building Health Promotion Capacity
Picks up on provincial events and links that might not appear on other lists such as
CLICK4HP. (#8)
Some respondents felt that the variety of information topics, including conference and training
opportunities, is helpful to them in their work:
I have just received information about some workshops coming up in the fall which I can
pass on to people in the community. I like being able to share the information I have with
people in community groups, so they can participate in upcoming training lessons that
are being offered in our province. I also like being able to access seasoned community
developers who work in the health field about research that relates directly to issues we
are facing in the community. I have referred two colleagues to the link. (#29)
I like the links to other related sites. I like information on current issues and new reports.
I like upcoming relevant conference information. I also like the feature of posting a
message and have used it with success. I am very excited about being a subscriber and
have shared this with others. (#25)
Elements of HPLINK That Respondents Disliked
Respondents were asked, in an open-ended question, what they disliked about HPLINK as an
electronic method of communication.
The issue of relevance to an individual s area of interest was a common comment:
Irrelevant posts! But I guess that goes with the territory. (#5)
A lot of topics are not relevant to me. (#5)
Multiple messages which duplicate other sources, but I can always delete, can t I? (#2)
Some participants are concerned that they might become overwhelmed with too many e-mails.
The average number of postings per month is about 22, which is about one per working day.
Too many messages to always keep up with. Summaries of things would help at the
beginning of posts. (#16)
If there are too many messages it takes up my computer space. (#30)
There gets to be too much of it sometimes. (#40)
Again, time as a limiting factor is mentioned by a number of respondents:
HPLINK Evaluation - 17
SHHP: Building Health Promotion Capacity
Lack of time to read, learn and apply some really interesting concepts that are shared
over the HPLINK. (#37)
I don t have enough time to really read all the information and participate in any
discussions. (#22)
Feelings of inadequacy when I don t have enough time to respond to all inquiries, or
download/explore sites for information. (#3)
Finally, some respondents felt that the listserv was not being used to its full potential:
[It] doesn t really appear to be used as a discussion listserv - more just FYI stuff. (#28)
[There is] not much debate or discussion, but then I don t contribute much, a few post a
lot. (#17)
Because there is little discussion on the list, I don t get a sense of what anyone else is
doing about/with many of the issues being raised - however, it is a two-edged sword in
that is there were more posting, I might be more inclined to simply delete rather than
skim the message. (#8)
Satisfaction with HPLINK
Respondents were asked to rate their overall satisfaction with HPLINK on a scale of 1 - 10, with
1 being most dissatisfied, and 10 being most satisfied. Responses ranged from 5 to 10, with the
majority (52%) reporting a response of 8.' Thirteen percent each responded to 5, 7, and 9.
Influence of HPLINK on Personal Practice
An open-ended question was asked concerning the influence that participating in HPLINK has
had on personal practice.
A number of responses indicated that participants felt their knowledge and awareness around
health promotion had increased as a result of subscribing to HPLINK:
It has increased awareness of health promotion and what is going on to deal with issues
in health promotion. As a graduating student who is interested in entering the health
promotion field it is a very valuable resource. (#1)
[I am] more knowledgeable about what is going on in other areas. (#4)
[It has] increased my knowledge on an array of topics. (#6)
HPLINK Evaluation - 18
SHHP: Building Health Promotion Capacity
[It has] increased my knowledge about health promotion issues therefore making me a
better practitioner. (#7)
It s made me more aware of what s out there. (#15)
Several respondents liked the increased access to information as a means or networking and
stimulating thinking:
As a woman in rural Saskatchewan many publications, journals and internet links would
not come to my attention except through a vehicle like HPLINK. Because I am selfemployed I need to feel connected with other people who have similar interests. Because
I am curious about health and health promotion I want to know what others are doing.
(#19)
It helps to keep me informed and to stimulate my thinking. (#20)
I can easily obtain background information on certain topics. Great for research use.
(#21)
Information posted has caused me to read articles/research topics I previously would
have said was of little importance to me or my practice, such as poverty. (#25)
There were some respondents who were less enthusiastic about HPLINK s effect on their
practice:
Not a huge impact, but some. (#31)
No real influence other than keeping me better informed. (#28)
I don t think it has influenced my practice, but some of the information I have passed on
to others has assisted them in theirs. (#38)
In and of itself, it has not influenced my practice in any substantive way. (#8)
Participants were asked, in an open-ended question, for suggestions on improving HPLINK.
Several suggestions related to increasing its use as a discussion tool:
...could try selecting health promotion themes/topics such as capacity building and/or
moderate a discussion and sharing if there is interest. (#20)
Add access to a discussion/bulletin board where questions are constantly posted and
HPLINK Evaluation - 19
SHHP: Building Health Promotion Capacity
viewable, therefore allowing others to explore the questions and responses or to add to
them. (#26)
If the person responsible sent out a question periodically...maybe more people would be
more apt to respond. Also maybe someone could be identified to periodically provide
information on a specific topic. (# 31)
Finally, participants were given the opportunity to make any general comments. These are the
comments that were received, excluding the ones indicating thank you for the service:
I appreciate the information shared on HPLINK and can see myself making more and
more use of HPLINK for a variety of reasons as I get more familiar with it as a tool and
as I become more comfortable using it as a method of communication. (#7)
As I ve mentioned I am very new to this listserv. One thing I was questioning when
joining was what does this listserv have to offer that I can t get from CLICK4HP? I
joined to find out but it is too early to tell. I m wondering if HPLINK is supplemental
and very Saskatchewan focussed, or if it will follow the same trend as CLICK4HP and
become somewhat international. I guess I ll find out. (#18)
I like the link, but there is a terrific amount of information which comes across which I
file for reading later. I forward a lot of it to the community services staff. (#38)
I do appreciate this service. It s good. There is just so much information in the world
today and it feels like being bombarded sometimes. Not your fault. Just the state of
technology I guess. (#40)
Other Sectors Where HPLINK Might be Expanded
All respondents were asked to suggest other sectors that might be interested in participating in
HPLINK. The following is an aggregate of those suggestions:
"
"
"
"
"
"
"
"
"
"
Human Services Integration Forum
recreation
education
justice
social services
community living
environment
more HPCs
other government departments whose work impacts on health
Saskatchewan Parks and Recreation
HPLINK Evaluation - 20
SHHP: Building Health Promotion Capacity
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
mental health
public health
nutrition
RICs
housing women and children s health centres
health action centres
MHOs
public health nurses and nurse managers
directors of community services
HSURC and other research bodies like CUISR
Health Canada regional officesFSIN
CBOs
Saskatchewan Directory of Human Services
community development workers
Quint
Saskatchewan Institute for Prevention of Handicaps
College of Dental Surgeons of Saskatchewan
Dental Therapists and Hygienists Associations
Diabetes Coordinators
Saskatchewan Cancer Association
community colleges
recreation boards
Summary and Conclusions
Objectives of SHHP Interventions
HPLINK has been one of the interventions used during the dissemination phase of the SHHP, to
help build and expand individual capacity for health promotion practitioners. Those who
responded (63%) were mostly community and health promotion practitioners. Another 20%
were managers, while the remainder were a mixture of researchers, students and private
consultants. HPLINK is therefore providing access to an array of practitioners.
From the results of the survey, it appears that for those who responded, HPLINK has had some
success in increasing capacity in the areas of knowledge and resources, and in increasing the
availability of useful tools. We do not know if it has contributed to individual skill development
or to commitment. There is some indication that HPLINK has increased awareness of the
broader issues in health promotion for some of the subscribers. It also has reached more than just
HPCs, i.e. managers, community workers and researchers.
HPLINK Evaluation - 21
SHHP: Building Health Promotion Capacity
HPLINK Objectives
HPLINK has had most success in providing resources and information for practitioners, many of
whom indicated that they have gone on to share these with others in their work place or
community. It has been somewhat successful in mediating communication amongst
practitioners, as HPLINK has not been used to a great extent for discussion of issues or to ask for
input into a specific issue or problem. There has been some sharing of knowledge from
subscribers, to the general list membership.
Response Rate
With regard to the objectives of the evaluation, most of these have been met through the various
methods used to gather information. A total of 40 subscribers to HPLINK responded to the email/internet survey (35%). This is an excellent response rate for this type of survey (Itracks,
personal communication). An additional 17 people were interviewed by telephone or in person.
Some of those interviewed may also have responded to the survey, but that information is not
available, due to the anonymous nature of the survey.
Positive Aspects of HPLINK for Subscribers
In general, subscribers liked the kinds of information that HPLINK provided. Satisfaction was at
a reasonably high level (average 8/10) for the majority of respondents. The range of information,
the Canadian focus of most of the information, and the availability of information without having
to search for it themselves were appreciated by the majority. People also appreciated that much
of the information had been pre-screened and summarized.
Many people were able to use the information for themselves, or to pass it on to interested
colleagues or community groups. There was indication that some practitioners were sharing
information with community members, and with managers and Board members, as well as with
their own colleagues.
A number of respondents felt that this kind of communication saved them time, as searching for
information was done by someone else, and they could simply assess whether they thought the
information would be useful at some future time, and file it away.
Many would like to see it become more of a discussion venue. This is tempered by the fact that
many people do not feel they are able technically to participate in this manner, or that they have
anything of interest to contribute.
Negative Aspects of HPLINK for Subscribers
Participating in an electronic resource such as HPLINK requires time, access and a certain level
HPLINK Evaluation - 22
SHHP: Building Health Promotion Capacity
of skill with computers and electronic networking. Many people indicated that they did not have
adequate time at work to follow up on information, or to spend time assessing the information
they had received. Others indicated that they had various degrees of comfort using this kind of
communication.
This is linked to the apprehension expressed by a number of those surveyed or interviewed of
being overwhelmed by too much information, or too many list e-mails to deal with. The list
manager has made a conscious effort to limit the number of postings she makes, to try and keep
the number of messages to no more than one or two per day. In fact, it often happens that there
will be no messages for several days at a time. The number of messages that appeared during
2000 worked out to about two messages per person per year. This is about the same rate of
posting for CLICK4HP, and for HIF-NET, two much larger lists. However, on a percentage
basis, the list manager continued to make the majority of postings (see Table 2).
A number of respondents felt that the information that appeared on HPLINK was not relevant to
their practice. This is to be expected, given the variety of people who do subscribe to such a
service. Much of the information may be of general interest, while specific topics will appear
that may or may not be of interest to an individual. This is balanced by the comments of a few
respondents who had used information that they would not have considered previously, that is
they were able to see the information and think outside the box.
Marketing to a Greater Audience
Many suggestions were made for others to include in HPLINK. This is an area that will need to
be followed up in the near future, to expand the list. A priorization will need to be made of
which group(s) to approach first. It should be noted that with more people on a list, there will be
more activity, although the actual rate of messages per subscriber may not change.
Suggestions for Improvement of HPLINK
"
Post list operation information on a regular basis, e.g. once every three or four
months, as a reminder on how to post, reply and subscribe.
"
Find ways to seed discussion questions or requests for information, on a regular
basis. This could be done, as was tried in the past, by having members of the
SHHP pose questions, or by contacting individual practitioners, and asking them
to pose a question or share a resource.
"
Ask list members to share their favourite resource/site and where or how to access
it.
"
Continue to locate Saskatchewan and Canadian information; ask subscribers to
HPLINK Evaluation - 23
SHHP: Building Health Promotion Capacity
share their information.
"
Put out information on the archives and how to use them.
"
Keep postings to the list to an average of 1 - 2 per day, if possible.
"
Make sure that subject lines describe the content of the message as much as
possible.
"
Put out technical information on lists occasionally; i.e. what is the objective of a
list, why responses are important, how it operates.
"
Expand the list to other sectors.
Implications for SHHP
1.
Technical aspects of the listserv
Given the various stages of proficiency and comfort with electronic communications expressed
from the interviews and surveys, plus the relatively frequent enquiries sent to the network
manager about how the listserv works, and the errors in using the listserv, there is a need for
training and information on electronic communication mechanisms such as email and listservs.
It is quite likely that most people learn as they go and as a result, can have difficulty problem
solving or even following simple instructions.
Therefore, because HPLINK is an SHHP intervention, how far do we go in providing basic
technical instruction? Where and how can we provide such instruction, given that those
subscribing to the listserv come from a variety of backgrounds and institutions, both in and
outside of Saskatchewan (although the majority are within the province)?
Hands-on training sessions could be provided at events such as Summer School. Access to
computers and internet would be a prerequisite for this type of training. Short presentations
could be made at other meetings, such as HPC meetings on a regular basis, so that new HPCs are
brought up to date.
The list manager can, on a regular basis, send out information type notices on HPLINK,
describing what a listserv is, what it can or cannot do, how to use it and how to use the archives.
2.
Marketing
There were a large number of groups and sectors suggested as areas for expansion of HPLINK.
SHHP will need to priorize this list and develop a mechanism for advertising HPLINK to these
HPLINK Evaluation - 24
SHHP: Building Health Promotion Capacity
groups. Again, the issue of training and/or provision of simple technical information may need
to be considered when recruiting new subscribers.
Another factor to consider is the concern that a number of subscribers had about the volume of email that a listserv generates. If more people are added, the actual number of e-mail postings
may increase, although the number per participant may not change. SHHP may want to consider
advertising to a limited number of groups, in order to keep the listserv as a manageable size for
its subscribers.
3.
Encouraging discussion
HPLINK has acted primarily as a source of information on various aspects of health promotion.
It has served only sporadically as a discussion forum. One of the objectives of HPLINK was to
act as a place where discussion could occur. Do we still want to pursue this objective? If so,
there are two options to consider here:
"
the listserv operates on its own, that is, whatever subscribers contribute or ask is
what occurs.
"
from time to time, questions or discussion points are seeded on the listserv; who
does this, how often, what kinds of questions? This may be the option of choice if
we regard HPLINK as an ongoing, targeted intervention, i.e. we make a concerted
effort to make sure that discussion occurs and that people know how to
participate.
One respondent had suggested that a separate bulletin board be set up to discuss specific
questions. There are a number of reservations to this suggestion however. Can the University
system offer such a service? Could people who have some difficulty with a simple listserv
manage to participate in a bulletin board type of discussion? Would people have internet access
which would be necessary for this type of discussion? Discussion can occur directly through
HPLINK, particularly if people understand the mechanics of the listserv. An extra place for
discussion is not really necessary at this time.
4.
Advocacy for education and equipment
Particularly for HPCs and other district health workers, computer and its attached e-mail and
internet access, although improving, is still limited. Can SHHP advocate for equipment access
plus training and time to learn and use this kind of technology and communication within the
health districts? If so, how to go about doing this?
5.
Share results of the evaluation
HPLINK Evaluation - 25
SHHP: Building Health Promotion Capacity
We will need to determine mechanisms for sharing the results of the evaluation with HPLINK
subscribers, as well as other audiences.
6.
Plan for the future of HPLINK
We will need to develop a plan for the continuation of HPLINK once the current phase of the
SHHP comes to an end in 2003. Participation continues to grow, such that membership is about
145 in mid-February 2002. A decision whether to continue with this intervention, and who will
oversee it will need to be made in the final year of the SHHP.
HPLINK Evaluation - 26
SHHP: Building Health Promotion Capacity
Appendix 1- Interview Guide
Key Informant Interviews - HPLINK Evaluation
Introduction:
My name is Judith Wright. I m a graduate student working with the Saskatchewan Heart Health
Program. We re doing an evaluation of HPLINK, the listserv for health promotion professionals
that was launched in August 1999 by the SHHP. I d like to ask you a few questions about
electronic communications in general and about HPLINK in particular. Your responses will be
kept confidential, and won t be linked to your name. Do you mind if I tape this conversation for
accuracy?
Interview Questions:
1. We realize that everyone has varying experience with electronic communications. Let s
start by talking a little bit about how you use electronic communications in your work?
(prompt: e-mail, Internet, listservs)
2. What kinds of things make it easy/would make it easier for you to use electronic
communications?
3. Based on your experience with listservs, what would you say is the chief or most valuable
function of a listserv?
4. The next few questions are about HPLINK in particular. I d like you to think back about
your experience with HPLINK? How did you first hear of HPLINK?
5. How do you use HPLINK in your work?
6. What is your experience with searching HPLINK s archives?
7. What were you looking for?
8. How do you share the information with others (eg. forward, print and circulate)?
9. How satisfied have you been with HPLINK? (prompt: on a scale from 1 to 10 with 10
being the highest level of satisfaction)?
10. What do you like about HPLINK?
11. What don t you like about HPLINK?
12. How have you, or have you, benefited from using HPLINK?
13. One of the objectives of HPLINK is to serve as a discussion forum for health
practitioners. How well do you think HPLINK functions as a discussion forum or place
for idea exchange?
14. Another objective is to facilitate networking among health practitioners. How does
subscribing and participating in HPLINK facilitate networking between practitioners?
15. The last few questions are about future directions for HPLINK? In what ways do you
think HPLINK could be improved? What kinds of topics, activities, or approaches
would you like to see on HPLINK?
16. So far we haven t advertised HPLINK too broadly. Do you have any suggestions as to
how to reach more people or sectors, ideas on who we might like to target specifically?
17. That completes my questions. Do you have any additional comments?
Thank you for taking time to share your comments.
HPLINK Evaluation - 27
SHHP: Building Health Promotion Capacity
HPLINK Evaluation - 28
SHHP: Building Health Promotion Capacity
APPENDIX 2 - Electronic Survey for HPLINK
The survey can be seen on the web site:
http://www.usask.ca/healthsci/che/prhprc/hplinksurvey.html
1. I am a:
Community-based practitioner
Researcher
Middle manager
Senior manager
Private consultant
Other (please specify)
2. I am employed by a:
Health District
University
Non-government Organization
Government Organization
Private Consulting Firm
Other (please specify)
3. With regard to my experience in the health promotion field:
I am new (two years or less).
I am established (3-5 years).
I ve been around (6-10 years).
I can t remember (more than 10 years).
4. I have computer access at a:
Personal work station
Shared work station
Home
Other (please specify)
5.What types of electronic communications do you use in your work? (Please check all that
apply.)
E-mail
Internet
Listservs/news groups
Other (please specify)
6. How comfortable are you using electronic communications?
I am an expert!
I am very comfortable.
I am somewhat comfortable.
I am somewhat uncomfortable.
HPLINK Evaluation - 29
SHHP: Building Health Promotion Capacity
I am scared silly!
7. How did you first hear about HPLINK? (Please select one only.)
At Health Promotion Summer School
At Health Promotion Contact Meetings
From the brochure
From colleagues
Other (please specify)
8. One of the objectives of HPLINK is to serve as a discussion forum or place of idea exchange.
Have you ever posted a message to HPLINK?
Yes
No
If not, why not?
9. How do you use HPLINK in your work? (Please check all that apply.)
Gathering background information
Preparing briefs and reports
Requesting information
Networking with others
Discussing issues in the health promotion field
Sharing information
Developing programs
To supplement course materials
To find employment
To keep informed on current issues
To find out about conferences and events
Other (please specify)
10. Of those you checked off in question 9, what would you say is your primary use?
11. With whom, and how, have you shared HPLINK information? (Please complete the
following table.)
WHO
Colleagues
Management
Health District Board Members
Community Agency Representatives
General Public
Other (please specify)
HOW
Print and circulate
Forward messages
Informal conversation
HPLINK Evaluation - 30
SHHP: Building Health Promotion Capacity
Presentations
Written reports
12. What is your chief barrier to using HPLINK? (Please select one only.)
I do not have convenient access to a computer.
I do not have convenient access to E-mail.
I do not have convenient access to Internet.
I do not have time.
I am not comfortable with the technology.
The content is not relevant to my practice.
I am not sure who the subscribers are.
I do not feel I have enough expertise in the topics discussed.
I use other ways to communicate with others.
I do not face any barriers in using HPLINK.
Other (please specify)
13.What do you like about HPLINK?
14. What do you dislike about HPLINK?
15. Overall, on a scale from 1 to 10 (with 1 being low and 10 being high), how satisfied are you
with HPLINK?
16. HPLINK is one of the capacity building interventions offered through the Saskatchewan
Heart Health Program on Building Health Promotion Capacity. How would you say being a
participant in HPLINK has influenced your practice, if at all?
17. In your opinion, how could HPLINK be improved?
18. Any other comments?
Thank you for your participation in the HPLINK evaluation!
HPLINK Evaluation - 31