Scoping study related to development of a Web
portal for users of education evidence bases
Report presented to the Teacher Training Agency – 10 May 2004
Phil Sheffield
Manager: British Education Index
Sam Saunders
Project Officer: British Education Index
Scoping study related to development of a Web portal for users
of education evidence bases
A summary
General introduction: the purpose

There is a lot of interest and effort being turned to the ways in which research
evidence can be used by practitioners in education. The phrase "Evidencebased practice" sums up a strong movement in the field.
Section A. The scoping study
1.
The questions

2.
Our principal question is “how can high quality educational databases in the
UK be linked electronically so that users can automatically search the
research?”
Method

28 large, significant or representative internet resources have been surveyed.

The 28 resources were described in terms of 26 variables

Eight categories differentiate the typical contents of the 28 resources:
1.
2.
3.
4.
5.
6.
7.
8.
3.
bibliographic records;
eprint research reports;
good practice summaries;
research project descriptions;
research reviews;
research result summaries;
resource guides; and
statistical reports.
The current availability

Exponential growth of the Internet renders all description incomplete

We have surveyed many of the most notable sources of research evidence
The general picture
 Google at present indexes only a proportion (maybe a third) of what can be
found.

The time needed to find relevant material is disproportionate to the time that
will then be available to read and digest it.

Perhaps half a dozen sites contain, between them, a large proportion of the
available evidence.
1
Scale
 The 2001 Research Assessment Exercise reported 2,045 full-time equivalent
researchers in education, compared (for example) to 838 in economics and 859
in sociology.
 The British Education Index, lists approximately 5,000 journal articles per
year
 The 28 surveyed resources account for approximately 25,000 resources
 A single search of the twelve most populous quality sites in the list would
address approximately 7,000 reports of various kinds.
 Reducing the number to the six most populous would still yield a collection of
about 6,000 reports.
 Including the bibliographic records of the BEI would add 90,0001 more.
4.
Experience of using the individual sites
Database formats
 All web resources have a degree of organisation, imposed by the nature of
mark-up language.
 Most of the services we have looked at use database structures that are robust
and manageable.
 The diversity of formats being used would not support the construction of a
cross search facility as they stand.
 They would allow the relatively simple generation of new catalogue records
(metadata) to a standardised format.
Database searching
 A simple search box for users to enter "keywords" can easily lead to
inadequate results
 A little sophistication in the provision of fixed search categories or limiting
terms can help improve results.
 We found very little common ground in the variety of search options offered
by the services we have considered.
 All the sources we looked at had value and a rationale, but we have to
conclude that diversity and idiosyncracy are the principal characteristics of the
information landscape for education.
 A move towards a common interface for most of the major services would at
least obviate some of the need to learn six or seven different systems.
Transparency
 A simple first appearance avoids intimidating new users
 Confidence in the precision and completeness of research results are needed.
 Therefore richer set of fully described features should be present in a search
environment, perhaps in the background.
1
This figure excludes 44,000 pre-1986 BEI records which will also be available for searching from
mid-2005 onwards (note added October 2004)
2
Systematic Reviews
 Systematic reviews provide a critical test for the quality of individual
databases.
 They depend on known properties of specific databases in specified search
environments.
 In contrast, a simple and standard list of “keywords” cannot result in
systematic returns from a range of databases.
 Difficulties will not be consciously noticed by a novice user, but the
consequences will be present in the quality of results that they discover.
Section B. Recommendations
Introduction


5.
The focus of the recommendations is on the cross-search facility rather than
the internet environment through which the facility is exposed.
The internet environment through which the facility is exposed will grow
around it.
Existing models and options







6.
The National Electronic Library for Health emphasises evidence-related
information and presents information maintained by the site publisher together
with hyperlinked lists of resources.
The TRIP (Turning Research into Practice) database provides cross-searching
of certain resources in the National Electronic Library for Health.
TRIP is an interesting case study and has characteristics which usefully inform
the recommendations.
Commercial solutions are available to provide some cross-search functionality.
Successful cross-search solutions often use the Z39.50 standard.
Z39.50 offers a reliable option for cross-searching of compliant databases.
Z39.50 is technically complex and, therefore, an inappropriate short-term
solution.
The recommendation





The recommendation takes account of the report’s analysis of existing
resources and practices, and of issues like scalability and developments in the
general information environment.
It is recommended that eight initial information sources transform their
appropriate information according to an open source standard, that the
gathered information is collected by a central agency and is re-presented
within a specially created database.
It is recommended that the Open Archives Initiative Protocol for Metadata
Harvesting (OAI-PMH) provides the basis for information exchange.
The OAI-PMH has growing international weight, is well documented and
scalable, and puts emphasis on its ease of use.
The solution demonstrates similar characteristics to those identified in TRIP
and through Z39.50 applications.
3
Costs
 Costs will be incurred at sites providing information and at the site collecting
and presenting information.
 Some initial technical work will be necessary at all sites responsible for
maintenance of the eight information sources.
 Other costs relate to the provision of the database and to the development of
its functionality and presentation, and to the analysis of options for refined
subject searching.
7.
The longer term




Consideration must be given to the longer-term future of the cross-search
facility, and of the internet environment, or “portal” which delivers it, and
grows around it.
Greater collaboration is advocated between the originators of information
relating to educational research and the services which support them, also
between both of those parties and information seekers.
Increasingly, information should be accessible wherever it is felt to be
beneficial, but consideration should also be given to its availability from a
singular facility, responsible for the managed re-presentation of
complementary information supporting educational research, policy and
practice.
The portal is seen as an organically developing facility whose roots are in the
provision of cross-searching of the selected evidence bases as outlined in this
report.
4
Scoping study related to development of a Web portal for users
of education evidence bases
The Main Report
General introduction: the purpose
This report is written in response to a contract between the British Education Index
and the Teacher Training Agency in conjunction with the Department for Education
and Skills, the General Teaching Council for England and the National Educational
Research Forum. Its remit was to produce a survey and some recommendations about
the on-line evidence base for teaching and learning. There are three main parts:
 Firstly it reviews the range of research evidence currently available on the
internet.
 Secondly it identifies some existing approaches to the identification of
research evidence that already exist.
 Thirdly it goes on to make suggestions about steps that agencies could take to
increase and simplify the accessibility of such resources in the future.
The technical, research and publication activities that support educational practice are
complex and changing. Inevitably, all our observations and recommendations have to
be made in a tentative and conditional way. We understand, through our close
involvement in what has been called the "information landscape" of education over
many years that secure knowledge and sound recommendations can be quickly
overtaken by developments in any one of the three broad areas of activity. An
awareness of the incompleteness of all information services also gives us pause to
warn readers that we could have missed material that would have been important to
our conclusions. We have, accordingly, tried to make our recommendations
appropriate to a changeable and somewhat uncertain environment.
In order to achieve the first of our purposes, we identified a number of collections and
services that would include most of the major collections and a representative number
of sources from other broad categories of smaller resources. An initial identification
of approximately 50 websites was gradually reduced to a primary list of 28 for closer
examination and description. This final list is given in Appendix 1.
Section A. The scoping study
1.
The questions
Questions of how, why and whether school teachers (in particular) make research
evidence a part of what has been called their "craft knowledge" (McIntyre and Hagger
1992) have been explored in some depth (Hemsley-Brown and Sharp 2003; Nutley,
Solesbury and Percy-Smith 2003). It seems likely that close attention to published
research is normally associated with periods of professional study and with progress
into senior roles. (Galton, 2000; Teacher Training Agency, undated). At any one time
it might be a minority of the teaching profession who are actively considering
educational research findings but the revitalisation of their practice, and the leadership
roles they move into as part of their professional development as teachers might be
supposed to have a significant impact on colleagues and on pupils' learning. Given
that teaching is a graduate profession whose members have a vested interest in
5
learning, it is not surprising that many teachers are interested in, have conducted, or
occasionally consult evidence from research. An earlier study by Hannan, Enright and
Ballard (1998) found that a third of the teachers in their small sample had been
directly involved in research activity during their careers.
Attempts to improve or understand the relationship between research and practice are
major themes in a number of national initiatives. One significant indicator is that the
Economic and Social Research Council's current Teaching and Learning Research
Programme has "Transformation and Impact" as one of its five major themes,
impinging on all the individual projects funded by the Programme. Another sign is
that of the 340 articles most recently included in the British Education Index (at
March 2004), 37 have been assigned "Theory Practice Relationship", "Evidence
Based Policy" or "Research Utilisation" as subject terms. In addition a substantial and
practical interest in the value of research to teaching and to school leadership is
evidenced in current projects being managed by the Teacher Training Agency and the
General Teaching Council for England among others.
This survey of the current state of play is, therefore, being conducted in propitious
times. In this report we focus on one principal question, as set out in the original
invitation to tender, of “how high quality educational databases in the UK might be
linked electronically so that users can automatically search the research evidence”.
2.
Method
The selection of resources for consideration
The tender document identified nine categories that provided the starting criteria for
inclusion of specific information resources in our study. Comparing these categories
with resources already known to us through work compiling the British Education
Internet Resource Catalogue gave us an initial set of 49 resources. Of these, 28 proved
large enough, significant enough, or representative enough of general approaches to
secure reasonably definitive coverage of current availability. The 28 resources, with
brief descriptions, are in Appendix 1
The descriptive scheme
In order to describe and compare the diverse 28 resources we chose 26 significant
variables (see Appendix 2). These variables were initially informed by the fields
adopted by the British Education Internet Resource Catalogue. Additional aspects
were added in response the requirements of the tender document. Closer examination
of the resources themselves then provided a basis for refinement and further addition
to the list of variables.
A final stage of inspection led to the creation of eight categories for resource content
types which could differentiate the typical contents of the 28 resources with sufficient
clarity. These categories were: bibliographic records; eprint research reports; good
practice summaries; research project descriptions; research reviews; research result
summaries; resource guides; and statistical reports. A ninth "mixed" category was also
needed.
The direct survey of each of the resources was done by two individuals in the British
Education Index, with a number of return visits to check, clarify and adjust the
descriptions.
6
Our observations allowed us to record values or qualities for most of the 26 variables
for most of the 28 resources. We then sent the completed (but provisional) data sheets
directly to an address that we hoped would reach the managers of each resource,
asking them to check that our observations were acceptably accurate. Where we had
been unable to make observations, we asked the managers to provide details. An open
ended questionnaire was included with the data sheet so that we could elicit
information about future plans and possibilities – especially in relation to questions of
interoperability. Many of our enquiries (11/28 responded) did not yield replies.
Nevertheless, we hope that the accuracy of our observations is sufficiently robust to
support the recommendations that follow.2
3.
The current availability
To lay a claim that all resources have been discovered, or to make a statement about
the lack of particular types of resource would be unwise. The exponential growth of
the Internet renders all description incomplete, and the resources we have described
(as noted above) may have characteristics that we have missed. What we can claim is
that we have surveyed many of the most notable sources of research evidence that
might be available and useful to education professionals like teachers through the
internet. As our initial study got under way, a number of new resources appeared on
the web. This report applies to the situation at March 1st 2004.
The general picture
Google searches of web resources limited in turn to the ac.uk, org.uk and gov.uk
domains on “research evidence” and “education” yield a combined total of 13,000
results. The three sets of results can themselves be searched more precisely for such
things as “class size”, “single sex classes” or “teaching assistant”, and sets of more
manageable proportions are generated. The results are initially impressive, and very
few irrelevant urls are presented. Comparing them with dedicated collections,
however, it is clear that Google at present indexes a proportion of what can be found
by going to individual sites, but not all. In the case of Education-line it is about a third.
At the same time it also reveals some ephemeral material that, while not irrelevant,
has to be sifted out. So while the Google approach has the potential to be helpful, it
cannot match the utility of a site like TRIP Plus in the field of medicine which enables
the equivalent of a cross search of nearly all the relevant items in 80 separate
collections.
For time-pressed practitioners the consequence of the general picture in education is
that while an appropriate and fruitful set of research findings might be available, the
time needed to find them is likely to be disproportionate to the time that will then be
available to read and digest them. In our group of 28 resources, perhaps half a dozen
contain, between them, a very large proportion of the available evidence.
Approaching each one in turn requires six sessions devoted to learning how to make
best use of each resource, and six search sessions systematically conducting each
enquiry.
2
The original report contained an appendix listing characteristics of sites in tabular form. This has been
excluded from this version of the report given the confidentiality with which some information was
submitted by site managers, and the time sensitive nature of some of the information (note added
October 2004)
7
Scale
As a partner in the public provision of education, particularly (but not exclusively)
through teacher training in university education departments, educational research
accounts for a large number of individual researchers and a high volume of published
research output relative to other scholarly fields. The 2001 Research Assessment
Exercise published data on numbers of researchers from each of 60 broad subject
areas (HERO 2001) While the number of researchers included in the exercise is only a
proportion of all active researchers, education was ranked in fourth place in 2001,
with 2,045 full-time equivalent researchers, compared (for example) to 838 in
economics and 859 in sociology. This pattern has persisted over the last three
assessment exercises. The British Educational Research Association reports a
(growing) membership of 1,300. The British Education Index, concentrating mainly
on UK published journals indexes approximately 5,000 journal articles per year
(many but not all of which are research reports).
In the survey of 28 UK web sites of research evidence, a very approximate figure of
25,000 could be suggested as the current number of accessible documents describing
research outcomes. In addition, The British Education Index contains bibliographic
records of approximately 90,000 printed articles dating from 1985 to the present. In
recent years most of these have become accessible via the web, albeit on a
subscription basis. Worldwide, it can be safely assumed that these numbers are
matched in many other territories, frequently in English.
Questions about the rigour and relevance of this output have been debated at length,
and over a long period. See, for example, Stephen Gorard's brief summary (Gorard
2003). Its quality, nonetheless, can only be debated if it can be found. The efficiency
and value of more stringent selection and summarising services, such as systematic
reviews, will themselves depend on the providers of those services being able to
identify material to fit their specified purposes.
As far as the UK is concerned, if a single search made it possible to identify relevant
full-text items from the twelve most populous quality sites in the list we have
examined, something like 7,000 reports of various kinds would be available.
Reducing the number to the six most populous would still yield a collection of about
6,000 reports.
Types of resource
The number of reports is one thing. The quality, relevance and utility of each report
are more difficult questions. A first step in evaluating quality is to enumerate and
distinguish characteristics. We have already suggested that eight broad types of
content might be considered in the search for research evidence that can contribute to
professional practice in a direct way.
Bibliographic records: Short descriptions of the standard properties of a document or
other resource, usually giving details of title, authorship, subject matter and
availability. For example: The British Education Index; Educational Research
Abstracts. While such services tend not to present full-text reports, there is a growing
tendency for software to integrate such databases with full text recovery that
automatically takes account of the user's situation relative to subscriptions and
payment methods. Even with no direct internet access to full reports, bibliographic
records aim to be sufficiently rich to enable a user to decide whether to use the
bibliographic data (British Library Document Supply Centre Shelf Mark, or ISSN
number, for example) to enable accurate recovery from a document supply service.
8
The integration of such services within an evidence-focussed portal would clearly
extend its value.
Eprint research reports: Generally accessible computer files containing the full text of
articles describing and reporting the findings of individual research projects. For
example: ePrints-UK; Education-line. Since the initiation of the eLib programme in
1995, the UK has seen a steady increase in internet publication of research results in
pre-print or post-print forms, independently of the commercial and learned society
publishing services that have dominated scholarly publication in the modern period.
The most recent development in the UK has been the establishment of institutional
eprint repositories in association with the Open Archives Initiative. For the purposes
of this classification scheme, on-line journal articles would also be called eprint
research reports although we have not explicitly considered them.
Good practice summaries: Descriptions of practice which has been identified as
exemplary. For example: TeacherNet Case studies. Not strictly research evidence, but
serving a similar need among busy practitioners in providing insights from outside the
immediate context of individual practice.
Research project descriptions: Descriptions of research projects which may or may
not include a summary of findings. For example: Current Educational Research in the
UK. While descriptions of the project per se are outside the immediate scope of this
study it is clear that a well-established database of project descriptions is likely to
include links to abstracts and reports of findings.
Research reviews: Summary reports based on two or more sets of research findings.
For example: BERA Academic Reviews. This category would also include systematic
reviews such as those being undertaken by The Evidence for Policy and Practice
Information and Co-ordinating Centre (EPPI-Centre)
Research result summaries: Digests of single or multiple sets of research findings on
specific themes. For example: GTCE Policy and Research: research of the month.
These differ from research reviews only in that a review seeks "meta-analysis" which
might produce generalisation more powerful than any of the individual reports, a
result summary is more concerned to present salient points from individual research
reports.
Resource guides: Bibliographic lists which identify the locations of internet resources.
For example: SOSIG; British Education Internet Resource Catalogue. Such databases
may in turn direct users form the descriptions which are their principal contents, to
resources of other types, including individual eprints.
Statistical reports: Tables of statistics, with or without a linking commentary, but with
sufficient information to make the raw figures meaningful. For example: Statistical
First Releases. Such resources might themselves become a data source for further
research. In their organisation of data collected by third parties they can contribute to
the development of policy and practice at school or other local level in similar ways to
other sorts of research output, providing comparative data for benchmarking, or for
the extrapolation of trends for planning purposes.
Some resources combine two or more of these types, and might be referred to as
"mixed".
9
4.
Experience of using the individual sites
Considering the characteristics of individual sites together it is possible to identify a
number of factors which might be considered problematic and/or promising.3
Database formats
Even an html page presenting a list of links to half a dozen summary papers has a
degree of organisation that can be exploited by search engines or information
managers. No collection of resources, however, should entertain the long term
adoption of lists as a means of storing and presenting its contents. Beyond that
simplest of data structures (used by the BERA Academic Reviews, for example)
proprietary database structures of various kinds and degrees of sophistication enable
web site managers to add material in consistent ways and to enable their own facilities
to discriminate (for example) between an author name, a title and an abstract in search
facilities. Most of the services we have looked at use database structures that are
robust and manageable. However, the diversity of formats being used would not
support the construction of a cross search facility as they stand. They would, however
allow the relatively simple generation of new catalogue records (metadata) to a
standardised format. Such records could then form a single database that would
provide the kind of results that would be demanded of a cross search facility. That is,
records from a variety of collections could all be returned in a standard format to a
single location.
Database searching
Information seeking in a field as complex and dynamic as education can only be very
simple if expectations of precise or comprehensive retrieval are low. A demand for a
familiar-looking (often mis-described as “intuitive") and simple interface can be met.
But the simpler and more familiar it is for the average (occasional) user, the more
average and unsatisfactory the results are likely to be for a user with precise or critical
needs. A simple search box for users to enter "keywords" (the term itself does not
enjoy consistent usage) makes for an easy start, but can easily lead to a frustrated
subsequent experience (even if the user may be unaware of all the valuable records
that have been missed through injudicious choice of a particular search word or
phrase). A little sophistication in the provision of fixed search categories or limiting
terms can help improve results. The principle that good habits can be efficiently
transferred to new search contexts is frustrated, however, by the fact that we found
very little common ground in the variety of search options offered by each of the
services we have considered. A move towards a common interface for most of the
major services would at least obviate some of the need to learn six or seven different
systems. DfES research reports and Ofsted publications exemplify two of the
problems we encountered. The DfES advanced interface relies on a relatively
sophisticated database structure, but constrains the advanced searching that would be
possible to a limited set of options that frustrate precise searching while offering a
complex set of options that would intimidate naive users. The Ofsted option of one
text box and a “Go” button simply allows for instances of the search word in title or
summary to be found, while the search tips suggest that documents’ full text will be
3
As noted earlier, the original report contained an appendix listing characteristics of sites in tabular
form. This has been excluded from this version of the report given the confidentiality with which some
information was submitted by site managers, and the time sensitive nature of some of the information
(note added October 2004)
10
searched. All the sources we looked at had value and a rationale, but we have to
conclude that diversity and idiosyncracy are the principal characteristics of the
information landscape for education.
Transparency
In trying to describe the ways in which each service organised its information, we
sometimes had to resort to indirect measures and inferences. We would suggest that to
get the best results from a database search, a user needs to know as much as possible
about things like the total number of records being searched and the number and
nature of the fields into which the metadata is organised. When a search is composed,
a user is helped by knowing that plural forms will be included, or that truncated
search strings are allowable. Knowing which fields will be searched is fundamental.
Phrases need to be distinguishable from simple coincidence of two words in a text.
Knowing that some users can be intimidated by apparent complexity, such
information is probably best kept in the background, with clear links to “further help”
or “advanced searching” (Google is exemplary in this respect) but national collections
of important documents should not frustrate intelligent users' attempts to understand
and make the most of them.
Systematic Reviews
The literature searches used to inform systematic reviews provide a critical test for the
quality of individual databases. Needing both precision and comprehensiveness, they
make demands on all aspects of database design, content management and search
options. The exemplary strategies set out by the Centre for Reviews and
Dissemination (CRD undated) for example, depend in a fundamental way on the
known properties of specific databases in specified search environments. Terms,
fields, and Boolean combinations are spelt out and justified for different purposes in a
way that draws attention to the characteristics of a database that would make it
valuable. In contrast, a simple and standard list of “keywords” to use in searching a
sequence of databases cannot result in systematic returns. The same keyword aimed at
different instances of different databases will operate in different ways and yield
results that are inconsistent with the user’s requirements. Difficulties experienced in
the construction of such strategies by experts will not be consciously noticed by a
novice user, but the consequences will be present in the quality of results that they
discover.
Section B. Recommendations
Introduction
These recommendations focus on the means of achieving searching of information
originated by different agencies rather than on the environment through which the
searching will be done or displayed. The recommendations do not present a “single
portal”, as envisioned in the specification, as being a prerequisite for the satisfaction
of the specification’s main objective of facilitating access to research evidence. While
part of the process will, necessarily, be managed by a single agency, and there may be
a focus for presentation of the information on the internet, it will be possible to
display the search facility, or information generated from it, within any fitting internet
site. Indeed the recommendations see the distributed and customised presentation of
the information as central to the promotion of awareness and use of it. It is our view
11
that a “portal” will grow over time, but that a pragmatic approach to the distribution
of information will provide a useful and usable resource as well as establishing the
potential for that growth to happen. The TTA’s own projected TTRB may provide one
trial ground for selected information from the overall resource base.
This section of the report
 outlines some existing models and options for cross searching
 makes a practical recommendation for the particular specification context, and,
 speculates on the place of the particular solution in a broader information
infrastructure to support the interests of the commissioning agencies.
5.
Existing models and options
The National Electronic Library for Health (NELH, http://www.nelh.nhs.uk) exhibits
some of the characteristics required by the specification. It emphasises evidencerelated information and presents information maintained by the site publisher together
with hyperlinked lists of resources which are usually accessible to its users. Of
particular interest in our context is the TRIP (Turning Research into Practice)
database (http://www.tripdatabase.com) which is accessible within and outwith the
NELH site and, at the time of writing, offers a pilot cross-search of information
associated with the site. TRIP’s origins and principles are self-described as follows:
“From a small beginning the TRIP Database has developed and grown into one of the
world’s foremost medical internet resources […] The TRIP Database was created as a
result of the explosion of ‘evidence based’ materials being published. Unfortunately,
for health professionals, this high-quality material was being placed on the separate
publisher’s websites which made locating the material difficult. Has a busy health
professional got the time to visit 15 websites looking for the material they need? […]
The basic principle has stayed the same since the start of the site. A high-quality
resource is identified that allows unhindered access to the publication. The Title, URL
and Date of Publication is recorded and added to the database. Subsequently the URL
is ‘spidered’ and this process ‘captures’ the relevant text from the site and uses that to
allow text searching. In 2003 the decision was taken to move the TRIP Database onto
a more commercial footing. This was taken due to the need for a predictable income
stream to allow for the accommodation of increasingly sophisticated enhancements to
the site.”
(sic - About TRIP: http://www.tripdatabase.com/index.cfm?method=application.about
URL checked 28 April 2004)
For future reference we feel it is useful to make some general statements based on
TRIP’s experience:
 somebody makes decisions about the quality of information
 information is collected from identified sources (sources are presumed to be
complicit in the process)
 collected information is added to a managed database
 material referenced in the database is freely accessible (but, because of particular
arrangements within the health sector, the major medical bibliographic database is
also freely searchable, albeit with the associated problems of getting access to
commercially published journal articles, for example)
 a small resource becomes large (although it is not possible to discern the scale of
use from the site itself)
12
 free database access becomes fee-based (although five free searches a week are
available without subscription)
 income supports technical development
 searching requires sophisticated facility management (including, amongst other
facilities, online suggestion of alternative search terms to those introduced by the
users, synonyms and misspelling features, and access to “registered keywords”).
TRIP, at least publicly, exhibits no ambition to work outside its particular information
domain, medicine, but it does provide an interesting model, consistent in many ways
with the vision in the specification. It is worth noting, however, that the vocabulary of
medicine is much more precise than that of education and that considerable attention
would need to be devoted to that problem if TRIP were seriously considered a
solution to the specification, assuming its managers were interested at all (in
recommending an alternative solution, the authors have not felt it appropriate to check
on this). Also, any continuing fee-based model, a principle now apparently firmly
established by TRIP, would inhibit access by the user groups identified in the
specification.
Commercial solutions are, of course, an option for the provision of an integrated
search facility. There are numerous commercial and open source “portal” products.
The authors reluctantly concluded that serious evaluation of contenders was not
possible within the timeframe and budget presented by this study. It is also fair to
speculate that new products would have been made available even between the time
of the writing and presentation of the report. Within the timeframe of this report it
would be difficult for a company to provide a quotation, however vague, for a
solution to the diverse requirements expressed in the tender. If the commissioners of
this report decide that a commercial solution is preferable to the recommendation, we
feel that it would be desirable and necessary for them to issue a detailed specification
for a service.4
These range from large companies with custom-written and commercially protected
software to smaller companies with open source leanings, often tied to consultancy As
far as it is possible to tell from publicly available information, each of these solutions
takes account of an industry standard for cross-database searching, Z39.50.
The Z39.50 standard specifies a client/server-based protocol for searching and
retrieving information from remote databases. The National Information Standards
Organization (http://www.niso.org/z39.50/z3950.html). defines Z39.50 as “a
computer protocol that can be implemented on any platform […] a standard way for
two computers to communicate for the purpose of information retrieval. A Z39.50
implementation enables one interface to access multiple systems providing the enduser with nearly transparent access to other systems.” An example of such a facility,
established with funding from the Joint Information Systems Committee (JISC) and
subsequently from the Yorkshire and Humberside Universities Association, can be
found at http://riding.hostedbyfdi.net/riding/index.html.
Few of the information sources discussed in the scoping study comply with the
Z39.50 standard. Tools are available to enable compliance but there will be a high
technical overhead in their application. While it is the view of the authors that Z39.50
offers the best option for cross-searching of compliant databases, we do not feel that it
4
A paragraph that mentioned particular company products as examples of commercial offerings has
been excised from the original report and the following paragraph has been amended accordingly.
These omissions do not affect the matter of the report (note added October 2004)
13
is a realistic option for the identified education evidence bases in the short term. Also,
while Z39.50 would, with adequate technical input, provide access to the larger
resources of interest to the commissioning agencies, specifically those with wellstructured, “library-like” information, some resources will be available in other, less
structured forms, provided by smaller scale organisations. The medium-term
scalability of any Z39.50 option would need to be carefully considered.
6.
The recommendation
In making the recommendation, the authors have taken into account the success of
resources like the TRIP database, the desirability of according with standards (like
Z39.50), and other factors like scalability and the development of the Joint
Information Systems Committee (JISC) information environment and e-government
initiatives. The recommendation is also informed by the BEI office’s own current and
ongoing experience of developing a “portal” to facilitate cross-searching of data
which it singularly manages, and of associating that data with complementary sources
and with Web services.
The recommendation is that information, designated as appropriate by originating
authorities, is reformatted by those authorities and represented in an internationally
recognised open source format (Open Archives Initiative compliant); that the
reformatted information is collected by one agency, introduced to a simple but well
structured database and made searchable through a simple search interface. In the
project’s initial phase, information could be supplied by the eight resources identified
as particularly important by the scoping study, i.e. Education-line, British Education
Internet Resource Catalogue, TeacherNet - Best Practice Research Scholarships,
DfES Research, National Foundation for Educational Research - Research Outcomes,
The Research Informed Practice Site, Research Evidence in Education Library, and
GTCE Policy and Research: research of the month.
This recommendation presumes that resources are maintained in a structured format
and that each participating organisation can draw on the requisite internal, technical
support to make appropriate information available to the Open Archives Initiative
Protocol for Metadata Harvesting (OAI-PMH http://www.openarchives.org/OAI/openarchivesprotocol.html). We believe that this is
a reasonable presumption given that organisations identified as being prime initial
candidates for inclusion in the cross-search already provide sophisticated technical
presentation of their information. The BEI office, with its University technical support,
has recently, and successfully, experimented with making internet resource catalogue
records available for re-presentation to the Social Science Information Gateway
(SOSIG) through the OAI-PMH. Some early lack of clarity of requirements at the
destination end meant the technical work took around three days at the originator’s
end, a “one-off” act. Local technical input might be costed at five days’ technical
input per site (costs will vary).
The OAI principle has international weight, is well documented and scalable, and puts
emphasis on its ease of use. The OAI site provides instruction for sites making
information available for harvesting
(http://www.openarchives.org/OAI/2.0/guidelines-repository.htm) and for sites
amalgamating (harvesting) information
(http://www.openarchives.org/OAI/2.0/guidelines-harvester.htm). For an application
of the principle, see http://www.rdn.ac.uk/resourcefinder/. For a large-scale
implementation of the OAI-PMH see PubMed Central
14
(http://www.pubmedcentral.gov/), the U.S. National Library of Medicine's digital archive
of life sciences journal literature, specifically
http://www.pubmedcentral.gov/about/oai.html. Nearer to physical and disciplinary
home, the ESRC’s Teaching and Learning Research Programme (TLRP) has begun to
collect project and Programme outputs in a DSpace digital archive: DSpace very
deliberately supports OAI.
The collocation of information from different sources in one database affords a
number of benefits, among which:
 the possibility of examining subject indexing used by different services (with a
longer-term view to providing some form of consistent search vocabulary within
the database, or mapping between different services’ uses of words and phrases)
 reduction in duplication of effort. We have some awareness that the same
information is maintained in different forms in different places (e.g. information
about journal articles appears in several sources). The identification of such
overlaps could lead to singular maintenance of core information with a facility for
its customised presentation in different places.
Remodelling the list of apposite characteristics we noted earlier for the TRIP
Database, we can make the following generalisations about the recommended
procedures:
 somebody makes decisions about the quality of information: it is recommended
that, for the evidence base initiative at least, the originator of the information
makes this decision, in the initial phase making accessible to the harvester only
information which satisfies criteria established by the commissioning agencies. In
some respects, the quality of the new database is defined by the sources on which
it draws
 information is collected from identified sources (and sources complicit in the
process): the OAI arrangement makes sources complicit, in both regular
identification and presentation of information to the harvester. Item references in
the new database could either identify the original source of the information,
provide a link through to the original source, or both
 collected information is added to a managed database: initially the structure of
the database will be simple, based on the metadata collected from the originating
institutions, but is likely to include resource titles, for example, basic resource
descriptions, some subject classification (differing between resources), and
temporal information (dates of record generation, amendment etc.). In the early
stages of the database searching would be done on titles, descriptions and subject
classifications, for example. Subsequent enrichment is partly dependent on
continuing income
 material referenced in the database is freely accessible (but, because of particular
arrangements within the health sector, the major medical bibliographic database
is also searchable, with the associated problems of getting access to commercially
published journal articles, for example): all resources included in the
recommended initial sample sources are freely accessible, and it is in the interests
of their originators that the information is as widely visible as possible. Only
financial or licensing issues prevent the presentation of information from major
bibliographic sources like the BEI
 a small resource becomes large (although it is not possible to discern the scale of
use from the site itself): OAI is a documented and committed open source solution:
15
its processes will not disappear in the medium-term. The recommendation would
be for the main evidence sources to be added at the outset of a service, with
phased introduction to the collection of smaller scale and new services over time.
The anticipated increase in volume of information delivered will not impact
detrimentally on any level of service. It would be desirable for the new facility to
make publicly visible some indication about the level of its use
 free database access becomes fee-based…: we presume that access to the search
facility will be free so the cost of its provision must be borne. The
recommendation for the search facility is low cost, supposing that information
providers are routinely generating the requisite information, albeit in different
form. Cost is dealt with separately below
 income supports technical development: the initial offering would present a
workable but crude view of the information. As noted earlier, the presentation of
information from disparate sources within the same facility affords an opportunity
to analyse areas where improvements might be made. We anticipate that subject
searching of material will be the main area for improvement. Development of the
initial resource is dealt with in Costs, below
 searching requires sophisticated facility management (including, amongst others,
suggestion of alternative search terms to those introduced by the users, synonyms
and misspelling features, and “registered keywords”): there will need to be a
clear commitment to development of the resource as initially presented. Some
consideration of this is given in Costs, below.
Costs
Real costs can only be provided by agencies invited to develop the facility as
envisaged. The information below is intended to be helpful rather than definitive,
outlining tasks which we see as necessary and desirable.5
All institutions
 initial local programming costs at each institution (up to five days’ work at
local rates for eight resources)
Notes
The lead site will need to communicate with developers at each providing site,
working towards agreement on the schema for presentation of data; developers at each
site will need to work on programs for transformation of local information. It is quite
possible that there may be local uncertainties about the externally-directed dictation of
a new procedure and the authors flag this as a risk factor. OAI is predicated on a will
to cooperate: in some respects the recommendation “enforces” adoption. The choice
of the harvesting agency and facility location could prove significant.
Harvesting institution
 Write an XML schema which will be used in conjunction with the OAI static
repository schema and create an initial database structure to accommodate the
information
5
The original report contained estimated costs for the activities outlined. This version of the report
identifies work deemed to be necessary (against bullet points) but omits estimated costs (note added
October 2004)
16




maintain procedures for information harvesting and (e.g. monthly) database
updating, and
maintain access to the database (including hardware costs)
routinely link-check resource information recorded in the database
liaise with sites wishing to have the site accessible from their own internet
sites and customise information in such cases
develop database functionality (e.g. during 2005)
analyse subject terminology presented by services


Notes
The association of the search facility with an existing website would involve some
design work, branding, text creation, etc.
If a specific site is established to present the search facility, there will also be costs
associated with registration, website design, branding, maintenance, etc. The
commissioning agencies are likely to have access to information about such costs.
The association of the facility with an existing site or any dedicated “portal” site could
lead to an initially simpler presentation with, for example, a hyperlinked list of the
resources included in the cross-search, with listed resource names associated with a
description of the resource, an explanation of the search facility and context-sensitive
help in its use.
Development of database functionality includes search interface development and is
presumed to involve the developer of the initial system. We also feel that such a
financial commitment would allow the developer to make a virtue of the simplicity of
early public releases of the service data and have confidence in their ability to develop
the facility.
The analysis of subject terminology is included on the basis that the database will
accumulate various subject description systems, affording an opportunity to examine
mapping between terms to provide a more coherent subject approach to the database.
Promotion
 Various promotional activities
There will be various options available to draw attention to the database (and, thereby,
to the sites from which content originates). Among those worth exploring are personal
email notifications about the addition of particular content; RSS news services
running on personal or institutional Web pages and reflecting service or content news,
or customised presentation of search results on personal or institutional Web pages
(see the “BEI resource base” accessible from
http://www.escalate.ac.uk/resources/wideningparticipation/ for an example of
searches of BEI databases on the particular topic of widening participation, running
on the pages of the education subject centre of the Learning and Teaching Support
Network). Printed materials will also be desirable.
7.
The longer term
The recommendation means that:
 appropriate information continues to be presented through existing sites
 the same information can be distributed to other sites where its benefits can be
seen
17
 a single portal presents all the information alongside complementary material
from other sources
Adopting the recommendation would depend on, perhaps even provoke, trusting
relationships between organisations with particular reasons for generating and
displaying information. In examining resources for this report we have seen, again
and again, custom-written descriptions of the same resources, ubiquitous lists of links
to the same resources, and self-perpetuating referencing of the same places without
any direct association between the valuable content of the referenced sites. Web
services increasingly encourage delivery of information to appropriate places from the
one most authoritative source, rather than its virtual proliferation. In the context of
these considerations, this might mean that such a source needs to be brought into
existence and this might influence thinking about the portal and about the relationship
of facilities with one another.
In the course of this work our attention alighted on the US National Center for
Biotechnology Information (NCBI, http://www.ncbi.nlm.nih.gov/), more specifically
its own awesome search facility, Entrez (http://www.ncbi.nih.gov/Entrez/). This
works very well, and it works very well, in part, because the information to which it
gives access appears to be closely coordinated by the NCBI itself. This arrangement
clearly allows the consistent management and presentation of the information to
which it provides access. It also, visibly and invisibly, represents a huge investment of
money, time, effort and understanding, a gravitational pull for complementary
information which even appears to have subordinated publishers’ more proprietorial
interests. A different information world, certainly, but one inference is that singular
management of an information landscape relevant to a particular discipline makes
cross-searching a much simpler enterprise. One of the significant potential problems
faced in addressing the work identified in the specification has nothing to do with the
technology. Rather, it has to do with the ability of the originators of complementary
sources of information, with differing organisational principles, to work together, not
least because in doing so some of their own infrastructure may be shown to be better
dealt with by other means. One direction for such collaborative work might be
towards a unifying resource whose content is both self-generated and managed on
others’ behalf.
However things develop, and whatever the solution adopted for the cross-search and
its internet presentation, the initiation of such work should not be seen as a short-term
undertaking. It is worth remembering that harvesting depends on the declaration and
availability of the information declared. Each originator of information for inclusion
in the cross-search has an obligation to manage and maintain accessibility to its
information.
The following recommendations consider the medium- to longer-term development of
the cross-search facility and portal.
 Smaller sources of evidence-related content identified by the project and excluded
from first-phase recommendations should be included in the harvesting process
according to a timetable to be agreed by the commissioning agencies, the content
originators, and the harvester and database host
 Consideration should be given to making accessible to the harvester, by the
recommended process, information generated by the originators of the eight initial
sources but which is outside the scope of the current specification (other
appropriate information sources available on the DfES site, for example)
18
 Work should be done on agreeing terminology and practices by which different
kinds of material, material intended for different audiences, and different
information about material, are consistently described within metadata
(terminology for research approaches, methodologies and scale, for example)
 As the database content expands, such terminology begins to act as the means by
which information seekers, or automatic agents, restrict, include or exclude
information as appropriate
 Efforts should be made to ensure that fee-based services which are considered
complementary to the freely accessible sources are made freely accessible to
appropriate communities together with the evidence databases
To give this last point some context, the initiation of a facility like the one envisaged
in this report offers an opportunity to explore closer relationships between
information originators, aggregators and seekers. Partly because of the authors’
context it is possible to envisage an integrated evidence-based facility, for example,
where
 the evidence bases are searchable
 bibliographic references and texts which feed a systematic review (or any
professional development or research) process are searchable, wherever possible
using practices which facilitate the process, and information is downloadable, and
printable, in custom-written formats to accord with known and expressed user
needs
 search vocabularies are integral to the facility
 searchers can annotate references and texts with complementary information
discovered as part of their work
 authentication by the system determines the degree of visibility of such
annotations
 significant annotations of public interest become associated with references and
texts.
With the requisite support, the BEI office could make its resources open to such
investigation.
Time has not allowed us to take into account the international context in which the
developments outlined here would take place, nor, indeed, their relationship with
other evidence-based initiatives like the independent Campbell Collaboration
(http://www.campbellcollaboration.org/) and ESRC sponsored Evidence Network
(http://www.evidencenetwork.org/home.asp). ERIC is undergoing a major, hitherto
largely invisible, transformation in the United States and the Australian Council for
Educational Research develops its own facilities in ways not dissimilar to those
adopted by the BEI. Both have significant interest in evidence-based matters. In its
recommendation to use an open standard, the option to cohere with similar national
and international initiatives is left open.
19
References
Centre for Reviews and Dissemination (undated) Search Strategies to Identify
Reviews and Meta-analyses in MEDLINE and CINAHL CRD website, University of
York <http://www.york.ac.uk/inst/crd/search.htm>
Galton, Maurice (2000) Integrating theory and practice: teachers' perspectives on
educational research paper. Paper presented at the ESRC Teaching and Learning
Research Programme, First Annual Conference, University of Leicester, November
2000 <http://www.leeds.ac.uk/educol/documents/00003247.htm>
Gorard, Stephen (2004) The British Educational Research Association and the future
of educational research Educational Studies, Vol.30, no.1
Hannan, Andrew; Enright, Helen; Ballard, Paul (1998) Using research: the results of a
pilot study comparing teachers, general practitioners and surgeons Education-line
Hemsley-Brown, Jane; Sharp, Caroline (2003) The use of research to improve
professional practice: a systematic review of the literature Oxford Review of
Education, Vol.29, no.4
HERO (2001) Results of the 2001 Research Assessment Exercise: a Microsoft Excel
spreadsheet <http://195.194.167.103/Results/all/all.xls>
McIntyre, D., Hagger, H. (1992) Professional development through the Oxford
internship model British Journal of Educational Studies, Aug 1992, Vol.40, No.3
Nutley, Sandra, Solesbury, William and Percy-Smith, Janie (2003) Models of research
impact: a cross-sector review of literature and practice Learning and Skills Research,
Vol.6, no.3
Teacher Training Agency (undated) Deputy/headteachers' views on accessing and
using research and evidence - results of a pilot survey
Wishart, Jocelyn; Oades, Caroline (2003) What do teachers, learners and other
education advisors want from a web based educational portal? Paper presented at the
British Educational Research Association Annual Conference, Heriot-Watt University,
Edinburgh, 11-13 September 2003 <http://www.leeds.ac.uk/educol/00003555.htm>
Web sites
About TRIP <http://www.tripdatabase.com/index.cfm?method=application.about>
Campbell Collaboration <http://www.campbellcollaboration.org>
Convera <http://www.convera.com>
Entrez <http://www.ncbi.nih.gov/Entrez>
Escalate Widening Participation: resources from the BEI
<http://www.escalate.ac.uk/resources/wideningparticipation/>
ESRC sponsored Evidence Network <http://www.evidencenetwork.org/home.asp>
National Library of Medicine's digital archive of life sciences journal literature
<http://www.pubmedcentral.gov/about/oai.html>
NCBI, <http://www.ncbi.nlm.nih.gov>
Open Archives Initiative Guidelines for Harvester Implementers
<http://www.openarchives.org/OAI/2.0/guidelines-harvester.htm>
20
Open Archives Initiative Guidelines for Repository Implementers
<http://www.openarchives.org/OAI/2.0/guidelines-repository.htm>
Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH)
<http://www.openarchives.org/OAI/openarchivesprotocol.html>
PubMed Central <http://www.pubmedcentral.gov>
RDN ResourceFinder <http://www.rdn.ac.uk/resourcefinder>
Riding <http://riding.hostedbyfdi.net/riding/index.html>
The National Electronic Library for Health (NELH) <http://www.nelh.nhs.uk>
The National Information Standards Organization (Z39.50)
<http://www.niso.org/z39.50/z3950.html >
Trip Database Plus <http://www.tripdatabase.com>>
21
Appendix 1
The 28 web addresses identified for closer examination
URL
http://brs.leeds.ac.uk/~beiwww/beid.ht
ml
Title
Description
Education-line
Education-line is a freely accessible database of the full text of
conference papers, working papers and electronic literature which
supports educational research, policy and practice
http://www.ofsted.gov.uk/publications Ofsted Publications
This collection includes all Ofsted reports other than inspection
reports. Research, guidance and other documents are added as they
are published.
http://www.dfes.gov.uk/research
DfES Research
This site provides details of all the research the Department has
commissioned or published since 1997, on-line registration for
inclusion on the Department's database of research contractors, and
allows registered contractors to submit expressions of interest in
projects the Department is commissioning.
http://www.teachernet.gov.uk/casestudi TeacherNet Case studies
Summary descriptions of good practice case studies from across the
es
TeacherNet web site have been pulled together and made searchable
through this one page. Each case study is described under standard
headings.
http://www.teachernet.gov.uk/professio TeacherNet. Best Practice Research
This collection gives access to standardised reports produced by
naldevelopment/opportunities/bprs
Scholarships
teachers in the Best Practice Research Scholarships initiative
http://www.hmie.gov.uk/publication.asp HM Inspectorate of Education HMI reports from Oct 1990 onwards on topics other than individual
Publications
school (or other) inspection reports.
http://www.ceruk.ac.uk
Current Educational Research in the UK The database is held in the Library at the NFER and covers current
research and research completed since 2000. Users can discover
topics that are under examination but which might not have reported
findings.
22
http://www.ncsl.org.uk/index.cfm?pagei NCSL Leadership evidence base - a-z
d=kpool-foundations-evidence-a-z
http://cem.dur.ac.uk/ebeuk/research/ters
e
http://www.dfes.gov.uk/rsgateway/DB/
SFR
http://www.bera.ac.uk/publications/acre
views.php
http://www.gtce.org.uk/research/romho
me.asp
http://eppi.ioe.ac.uk/reel
http://www.lsda.org.uk/research/reviews
http://www.lsda.org.uk/pubs/
As the first step in developing an evidence base about successful
school leadership, a number of key people nationally and from
around the world were invited to write short essays drawing upon
thinking on effective school leadership. They are collected here
according to theme, listed alphabetically by author name.
Towards Experimental Research
The TERSE reports library presented summary reports that met the
Syntheses in Education - TERSE
criteria set by researchers at the Centre for Educational Management
Reports
at the University of Durham.
Statistical First Releases
A database of statistical bulletins containing a summary record and
full text in a variety of formats
BERA Academic Reviews
This series is a set of specially written reviews of academic research
written for non-specialist but interested education professionals,
covering areas of practical concern.
GTCE Policy and Research: research of A monthly series of research reviews on topics which are of
the month
practical interest to teachers. The most appropriate existing research
reports are reviewed and summarised in a standardised way to
enable rapid identification of their relevance to he reader.
Bibliographic details are given to enable access to the original texts.
Research Evidence in Education Library This is a growing collection of systematic reviews of educational
research conducted by expert groups. As well as presenting a list of
the completed reviews, a database of articles considered in each
review (with annotations) is being created. Other articles considered,
but not used, are listed in single documents.
Learning and Skills Development
The site presents a list of research reviews that are in progress,
Agency. Research Reviews
categorised by theme, with a collection of documents for each
review that is added to as the work proceeds.
LSDA Publications
Database of Agency publications including newsletters and research
reports
23
http://www.nfer.ac.uk/research/project_ National Foundation for Educational
summaries.asp
Research. Research Outcomes
http://www.scre.ac.uk/resreport/index.ht SCRE Research Reports
ml
http://www.scre.ac.uk/pubs/index.html SCRE Publications
http://www.scotland.gov.uk/publication Scottish Executive publications
s/subjects.aspx?subtreelevel=0&subtreei
d=464
http://www.sosig.ac.uk
Social Science Information Gateway
24
The website lists summaries of NFER research conducted since
1998. A drop down menu can be used to access 31 pre-selected
categories. Project summaries of findings are available to view
online or download (Microsoft Word format). Links from a left hand
menu lead to external publications and information about NFER's
other publications and activities. A site search facility is also
available.
This is a collection of research reports written between 1994 and the
present under the auspices of the Scottish Council for Research in
Education
This is the publications catalogue for the Scottish Council for
Research in Education, covering research reports, summaries,
newsletters and briefings. All editions of the now discontinued
Research in Education and Observations series are included.
This section of the Scottish Executive's website provides access to
the full-text electronic versions of publications from 1997 onwards.
A list of latest publications is available and the full list can be
searched alphabetically by the first letter of the title in date order.
From 2001 onwards all publications are available in both html and
pdf formats. Education is one of the areas fro which the Scottish
Executive have responsibility.
The SOSIG Internet Catalogue is an online database of selected
Internet resources relevant to the study of a number of Social
Science and related disciplines, including education. A wider set of
social science websites can be searched free text independently of
the main catalogue.
http://www.ase.org.uk/htm/book_store/i Association for Science Education ndex.php
Bookstore
http://www.standards.dfes.gov.uk/resear The Research Informed Practice Site
ch/
http://cogprints.ecs.soton.ac.uk/
Cogprints
http://www.tandf.co.uk/era
Educational Research Abstracts
http://www.nrdc.org.uk
National Research and Development
Centre for adult literacy and numeracy
http://eprints-uk.rdn.ac.uk/search/
ePrints-UK - search demo v0.02
http://brs.leeds.ac.uk/~beiwww/beirc.ht British Education Internet Resource
m
Catalogue
http://vtc.ngfl.gov.uk
Virtual Teachers Centre
25
This is the home page for the Association for Science Education
(ASE). Aimed at teachers and advisers, this site provides a number
of links to resources, journals, conferences as well as further details
about membership, news and regional branches. There is also an
online bookshop and an area for INSET listings courses and training
details. The Association policy statements are also accessible as pdf
files. There is a separate Members' area which requires a
membership number to access.
A collection of digests of recent papers from research journals.
Chosen for relevance to raising standards in schools and catalogued
using a controlled vocabulary of subject terms and descriptors
An electronic archive for self-archived papers in Psychology,
neuroscience, and Linguistics, and of Computer Science, Biology,
Medicine, and other disciplines relevant to the study of cognition.
Research directly relevant to educational practice is included, but
isolating it using the search facilities requires ingenuity.
A commercial database of international educational research
abstracts from over 500 journals.
The NRDC website includes a list of research reports (with
bibliographies) and a second list of two-page summaries. The
general theme is adult literacy and numeracy. Some conference
papers and other reserch related publications are available. The site
offers simple and "full search" facilites, but test searches in both
modes were unable to discover documents know to exist on the site.
A cross-searching facility for the e-prints UK project
The Catalogue provides descriptions and hyperlinks for evaluated
internet resources within an indexed database. It is designed to aid
the identification of useful internet resources by people with a
professional or scholarly interest in education or training
An access point to a variety of selected resources, activities, news
items and databases for teachers
Appendix 2
Fields used to describe the 28 resources
1
2
3
URL
Resource Title
Resource Content Type
4
5
Information Structure (simplest level)
Search Options Available
6
Fields Offered for Searching
7
Other Fields Apparent in Display
8
Controlled Vocabulary for Subject - yes/no
9
Controlled Vocabulary for Descriptors - yes/no
10
Name of Thesaurus/Scheme Used
11
Number of Records in Total
12
Number of Records Relevant to this Study
13
Document Type
14
15
Rate of increase / updates
Access Restrictions
The internet address of the resource in question
The title of the service or website
The kind of records or documents managed by the service, as
determined by their content
The database or other organisation of the separate resource items
The kinds of search facility available to a visitor for discovering items
within the collection
The named and searchable fields offered by the site for searching,
whether singly or in combination
The fields which become apparent in the display of results form the
search, in addition to those already offered for searching
Does the collection have a list of specified terms with which to
describe the subject matter of each item?
Does the collection have a list of specified terms with which to
delineate other descriptive fields, such as educational sector, language
of resource, intended audience or geographical coverage.
The name of the controlled vocabulary or thesaurus in use in the
collection.
An approximate (2 significant figures) estimate of the number of
individual documents or records at March 1st 2004
An approximate estimate of the number of documents or records of
direct relevance to education practice.
Regardless of content, the type of document in terms of it’s formal
presentation as (say) a journal article, a conference paper or a
An estimate of the number of records added per month
Any subscription or registration restriction on full use of the resource
26
16
Inclusion criteria
17
Quality assurance
18
Z39.50 compliance
19
OAI compliance
20
Dublin Core Metadata compliance
21
22
Creator
Publisher
23
Intended purpose
24
Intended audience
25
Documentation
26
Description
The factors deciding on which records or documents are chosen for
inclusion in the collection
Procedures which help to guarantee inclusion criteria and other quality
measures
The accessibility to the database from a search based on the Z39.50
protocol
The provision of an xml file which complies with the OAI standard for
metadata harvesting
The inclusion of Dublin Core metadata in individual documents or
records
The name of the individual or organisational author(s)
The name of the web publisher (or owner of the web server if no other
information is available)
The purpose as described by the creator or inferred from the creator’s
web site
The intended readership as described by the creator or inferred from
the creator’s web site
The existence of support material to describe the structure and nature
of the data and features of the search and display.
A short text summary of the resource as a whole
27
Appendix 3
Table 2 Values used under the Resource Content Type heading
1
2
3
4
5
6
7
8
bibliographic Short descriptions of the standard properties of a document or
records
other resource, usually giving details of title, authorship, subject
matter and availability.
eprint
Generally accessible computer files containing the full text of
research
articles describing and reporting the findings of individual research
reports
projects.
good
Descriptions of practice which has been identified as exemplary.
practice
summaries
research
Descriptions of research projects which may or may not include a
project
summary of findings.
descriptions
research
Summary reports based on two or more sets of research findings.
reviews
research
Digests of single or multiple sets of research findings on specific
result
themes.
summaries
resource
Bibliographic lists which identify the locations of internet
guides
resources.
statistical
Tables of statistics, with or without a linking commentary, but with
reports
sufficient information to make the raw figures meaningful.
28