Business and Economics
ABDC Journal Quality List
Review 2013
1503 & 1599
Ingrid Nielsen
Department of Management
Faculty of Business & Economics
Monash University
How were panel members chosen?
• A call for self-nomination to panels 1501, 1502, 1503, 1504, 1505, 1506
and 1507 was made on 8th April by Stephen Taylor, Chair of BARDsNet
• Applications closed on 19th April
• Applications were reviewed by the BARDsNet Executive, the overall
Review Chair, Robert Faff, and the already appointed Panel Chairs
• Due consideration was given to size, disciplinary representation, gender
balance, availability and demonstrated expertise to contribute to the
process
• I was advised on 3rd May that my application to be a panel member had
been successful
• Initially, the panel comprised Neal Ashkanasy, Bob Cavana, Gavin
Jack, Ingrid Nielsen and Vikas Kumar,
ABDC Journal Quality List Review 2013
19 June 2014
2
What happened specifically with 1503?
• For personal reasons, the appointed 1503 Panel Chair, Neal
Ashkanasy, decided to step aside from the Chair role
• I was appointed to assume the Chair role on 21st May
• Following feedback about a perceived expertise gap on the 1503 panel
in the HR/ER areas, Adrian Wilkinson was added to the panel, also on
21st May
• The 1503 panel was larger than any other panel, because of the size
and disciplinary breadth of 1503 (741 journals in 2010; 821 in 2013)
• There may have been people in the community who remained unhappy
with the panel’s gender balance, but I was not unhappy with it
• The particular mix of panel members worked really well together over
the coming months on what was an enormously complex task
ABDC Journal Quality List Review 2013
19 June 2014
3
How did the process unfold: submissions
• Calls for submissions were made on the ABDC website on May 1,
2013, with a deadline of May 31, 2013
• Submissions could be made in four (4) separate categories:
•
•
•
•
Request to add a new journal (Form A)
Request to downgrade a journal’s ranking (Form B)
Request to upgrade a journal’s ranking (Form C)
Request to transfer a journal’s FOR code (Form D)
•
All submissions were managed through the ABDC Office, where
each was tagged to reflect the panel (MAN), the Form and the
journal – eg., MAN_FC_F_096
• A ‘short form’ of each submission was also created in the ABDC
Office, tagged with an ‘S’ – eg., MAN_FC_S_096
ABDC Journal Quality List Review 2013
19 June 2013
4
How did the process unfold: submissions
• Dropbox files were created in the ABDC Office for each Panel
containing all submissions for a given Panel
• An Excel spreadsheet summarizing each Panel’s full complement of
submissions was also contained within the Dropbox and each
submission number corresponded to the Excel line number
• Links to the completed Dropbox files were sent to Panel Chairs on 24th
June
• Submissions with missing information and/or made on an incorrect form
were initially highlighted in the Excel sheet in yellow until all submitters
had been contacted and asked to update missing information and/or
use the correct form
ABDC Journal Quality List Review 2013
19 June 2013
5
How did the process unfold: framework of
the panel deliberations
• In respect of logistics, the ABDC allowed each panel to devise its own
internal processes as there was much heterogeneity in terms of numbers
and locations of panel members as well as submission numbers
• In respect of referents, panels were guided to refer to a reasonably
homogenous set of indicators, however there was appropriate leeway
given to differences in the ways certain disciplines viewed particular
indicators
• It would be reasonable to say that common referents across the panels
included recourse to other well-established rankings, SSCI impact factors
and immediacy indices, citation metrics, rejection rates, editorial board
composition and in some cases, actual peer review was undertaken and
further views were sought from the community
ABDC Journal Quality List Review 2013
19 June 2013
6
How did the process unfold: 1503 panel
deliberation principles
• A decision was taken early on that no panel member would be eligible
to deliberate on a submission for any journal for which he or she was a
past, current or incoming editor or for which he or she was currently on
the editorial board or had held editorial board membership within the
past five years
• In respect of the above, a decision was taken that in the event that less
than four panel members remained to deliberate any given submission,
the Chair would to seek the view(s) of other community members to
take the total number of decision points to at least five
• A decision was taken early on that all panel members must disclose
during deliberation on any given journal that he or she had published in
that journal during the past five years
ABDC Journal Quality List Review 2013
19 June 2013
7
How did the process unfold: submissions
to 1503
• A total of 205 submissions were made to 1503 as follows:
•
•
•
Form A – 68
Form B – 1
Form C – 136
•
The panel endorsed removal of forty journals, where the content was
deemed out of scope, the journal had insufficient English language
content or the journal was a duplicate in error on the original list
•
Panels were required to arrive at ranking distributions in the following
approximate ranges:
A*: 5 – 7%
A: 15 – 25%
B: 35 – 40%
C: remainder
•
•
•
•
ABDC Journal Quality List Review 2013
19 June 2013
8
How did the process unfold: 1503 panel
deliberation process
• A first cut was achieved via responses to a multiple choice Google doc,
eg.,
•
•
•
•
I agree that journal X should be upgraded from B to A
I disagree. Journal X should remain a B
I disagree. Journal X should be double upgraded to A*
I disagree. Journal X should be downgraded to C
• Where all members were in agreement, the submission outcome was
pencilled as resolved
• Where majority plus 1 (5) was achieved, the submission outcome was
pencilled as resolved except for cases where the dissenter was
considered the primary expert in that area
ABDC Journal Quality List Review 2013
19 June 2013
9
How did the process unfold: 1503 panel
deliberation process
• A teleconference was held to attempt to resolve the ‘expert dissenter’
cases
• Each unresolved submission was assigned to two panel members
whose tasks were to make a written case for or against the submission
• These cases were submitted in writing to the Chair and pulled together
into a second round Google doc for voting
• A face to face meeting of the panel followed, where all unresolved
submissions were discussed and resolved, with two exceptions…
ABDC Journal Quality List Review 2013
19 June 2013
10
How did the process unfold: APJHR &
JMO
• Almost as much time and consideration was given to these two decisions as to
the rest of the review task combined
• For each journal, multiple confidential views were sought in light of the ‘local
relevance’ factor
• Several people declined to provide a view because they could not arrive at a
decision themselves
• We were required by the ABDC to retain the status quo unless the case for
change was clearcut and in neither case could we achieve a majority view
supporting change
• In both cases though, the panel and its ad hoc members strongly encouraged
resubmission to the next review
ABDC Journal Quality List Review 2013
19 June 2013
11
How did the process unfold: between
panel relationships
• Where submissions existed that proposed FOR code change, these
discussions took place between the implicated Panel Chairs
• Where erroneous double coding existed, decisions as to final FOR codes took
place between implicated Panel Chairs and the higher ranking was adopted
• Discussions about any other between panel issues took place at two face-toface meetings of the Panel Chairs, held at UTS
• The major between panel issue arose as a result of the Economics panel taking
a decision to wholesale delete 236 journals from their portion of the list, without
advising any of the other Panel Chairs
• A request came to me on September 17 to review the titles (not the rankings) of
the deleted Economics journals with a view to developing a solution to
reinstating them in the list
• My review of these titles precipitated the creation of a 1599 coded section
ABDC Journal Quality List Review 2013
19 June 2013
12
How did the process unfold: Economics
delisted journals in 1503 and 1599
• Given time and availability constraints, the tasks of the 1599 panel were
undertaken only by myself and Gavin Jack
• Of the delisted Economics journals, 65 were brought into 1599 and 82
we brought into 1503
• Taken together, the journals brought into 1599 formed a fairly internally
consistent group of journals covering environmental science, political
science and international relations
• The journals taken in to 1503 were those where a disciplinary presence
of the journal type already existed in 1503 (eg., public administration)
ABDC Journal Quality List Review 2013
19 June 2013
13
How did the process unfold: feedback
• The draft list was released for feedback on 11 September, 2013 with a
deadline for responses of 30 September, 2013
• The criteria for feedback were the identification of errors in the draft list
(ranking, spelling, FoR code, duplication etc); and did not extend to
reiteration of previous arguments
• 406 responses were received, 142 of these related to 80 journals in
1503
• Responses of the following type were acted upon:
• Duplications
• Reports of predatory journals
• Errors in the title or ISSN/ISBN
• Responses of the following type were not acted upon:
• Reiteration of a previous submission with no further evidence
ABDC Journal Quality List Review 2013
19 June 2013
14
Final outcome
• The final rankings distributions for 1503 and 1599 were:
•
•
1503
• A*: 6.8% (56)
• A: 23.4% (192)
• B: 24.7% (203)
• C: 45.1% (370)
1599
• A*: 10.6% (7)
• A: 28.8% (19)
• B: 33.3% (22)
• C: 27.3% (18)
ABDC Journal Quality List Review 2013
19 June 2013
15