Guidelines
Post-Conference Thank You Letter to Moderators
for
Abstract Review
(if not a Division/Committee Chair)
Timing: Should be sent out Last Week in April.
Dear Texas Water 20XX Moderator,
On behalf of the Board of Directors for Texas Section AWWA (TAWWA) and the Water
Environment Association of Texas (WEAT), I want to thank you for volunteering your time to
moderate the Technical Sessions at Texas Water 20XX and making it an outstanding success.
The Texas Water Conferences ensure that speakers are presented with the best platform
available to present their work and guarantees that the Conference attendees receive an
educational opportunity, par excellence. Your commitment to helping keep the most relevant
and innovative technical information in the hands of Texas Water/Wastewater Industry
professionals will continue to make Texas a leader in the nation for protecting the health and
environment of its citizens.
As we close the book on this year’s Conference, we have already begun the planning for
next year’s Texas Water Conference and encourage you to continue to participate in all future
conferences.
Thank you, again, for being part of the huge success that is Texas Water.
FirstName, Last Name
TAWWA Program Chair
FirstName, Last Name
WEAT Program Chair
Attachments
None
GUIDELINES FOR REVIEWERS
ON RANKING ABSTRACTS EQUITABLY AND CONSISTENTLY
Rationale for the Abstract Selection Process
The Annual Texas Water Conference is held over 3 days in a venue with limited capacity.
Hence, both Conference space and program time dictates that only a certain number of Abstracts
can be presented at the Conference. Therefore, a selection has to be made to meet Conference
capacity limitations.
The selection process also ensures that the Texas Water Conference retains a threshold
for scientific quality; only Abstracts that are of a certain standard are admitted into the
Conference, and limiting Abstract numbers upholds the scientific quality of the Conference.
And finally, a clear selection procedure ensures that;
Submitting authors can have confidence that the process of selection is fair, and
Abstract reviewers understand that their assessment of each abstract is vital.
Reviewer opinion and scoring underpins Abstract selection and therefore directly impacts
the content of the Conference Program.
In the end, Texas Water attendees can have confidence that the Conference Program is
developed independently, based only on a quality-driven process.
Suggested Review Process
There is no right or wrong approach to reviewing and rating abstracts. The goal of the
review process is to improve the consistency of ratings among reviewers and support the
development of a technical program that
•
•
•
•
•
is of high quality
is relevant, timely, and excites conference attendees
reflects innovation and diversity in water/wastewater research, practice, and education
reflects a balance between research, practice, issues and concerns, and education
reflects a balance between the varied areas that contribute to and support the
water/wastewater industries
The following suggestions have been put forth by frequent reviewers as steps that have
made the review process efficient, fair, and balanced.
1. Review the voting criteria outlined in this document.
2. The Division/Committee reviewing the Abstracts should be comprised of at least 3 reviewers
(at a minimum).
3. Print out a copy of each abstract or cut and paste abstracts into a Microsoft Word document.
a. Read each abstract to obtain a general understanding for the overall range of abstracts
and make initial notes and comments.
(Based on past experience, several abstract reviewers have commented that it is best to
review all the abstracts in one sitting.)
4. Re-read the abstracts and make more detailed notes and comments.
a. Based on past experience, several abstract reviewers have stated they found it useful to
separate the second reading by two days. Some individuals report that they have a
strong positive or negative reaction on the initial reading of an abstract which often
changes when they have time to reflect on whether their initial response was valid.
5. Rate each Rubric Question (below) utilizing the 1 – 5 Rating system:
a. Some reviewers review the highly- and poorly-rated abstracts a second time prior to
recording their ranking to ensure fairness.
b. Some reviewers check their consistency in a variety of ways—some rate the abstracts
at two sittings independently, then compare ratings. Others rank order after applying
ratings, then compare ratings to see if they are consistent with the ranking.
6. As a Reviewer, once you have completed your rating, submit your numbers to the
Division/Committee Chair, so they can be compiled with all other reviewer ratings.
7. The Division/Committee Chair will compile all reviewer ratings and make the final
Prioritized Ranking of Abstracts, based on a cumulative score of all reviewers. The
Division/Committee Chair will forward the final Prioritized Ranking to the appropriate Program
Chair by the specified deadline.
Ranking Rubric
There are 2 different Review Rubrics; one for TECHNICAL Abstracts, and one for
NON-TECHNICAL Abstracts. Each Rubric contains a series of abstract review questions in
which the reviewer is asked to assign a rating using a scale of 1–5, with 1 being the lowest rating
and 5 the highest. To support the assignment of ratings, additional guidance questions designed
to clarify the intent of the question are provided.
The questions are in 3 different categories, with each category having its own “weight” in
the final score calculation.
Content is the highest weighted category, with 80% of the score attributed to it. It is
comprised of 5 questions.
Writing/Syntax has a 10% weight in the final scoring, with 2 questions related to the
presenter being able to convey the content of their Abstract.
Presenter has a 10% weight in the final scoring, with 2 questions related to the presenter
themselves.
Justification for Weighted Ranking
The examples below show two ends of the spectrum; an Abstract High on Writing/Syntax
and Presenter, but low in Content scores, versus an Abstract High in Content , but low in
Writing/Syntax and Presenter scores.
High Writing/Syntax and Presenter Scores/Low Content Scores
Category
Question
Score
Non-Weighted
Writing/Syntax
Title
5
5
Presentation
5
5
Content
Innovation
1
1
Correctness
1
1
Practical Impact
3
3
Completeness
1
1
Technical Depth
1
1
Presenter
Texan
5
5
Presenter Quality
5
5
TOTAL
27
Weighted
2.25
2.25
1.44
1.44
4.32
1.44
1.44
2.25
2.25
19.08
High Content Scores/Low Writing/Syntax and Presenter Scores
Category
Question
Score
Non-Weighted
Writing/Syntax
Title
1
1
Presentation
1
1
Content
Innovation
5
5
Correctness
5
5
Practical Impact
3
3
Completeness
5
5
Technical Depth
5
5
Presenter
Texan
1
1
Presenter Quality
1
1
TOTAL
27
Weighted
0.45
0.45
7.2
7.2
4.32
7.2
7.2
0.45
0.45
34.92
As you can see, using a Non-Weighted Ranking system, both Abstract Authors scored the
same (27 points) and have the same chance at presenting. However, the Weighted Ranking
system shows the Abstract with the higher Content score is able to score the Content portion
higher and therefore identify Abstracts that may be of more interest to the Conference Attendees.
High Number of Abstracts (20+ Abstracts)
For Divisions/Committees that have to review a large number of Abstracts, in a short
amount of time, it is recommended that they:
Rank using only the 5 questions within “Content”, since it is 80% of the ranking
weight.
This should help speed up the reviewing process and not diminish the integrity of the
review/ranking process.
Rubric For
1
2
3
4
5
Title:
The Title is descriptive of question
or work performed.
There is no connection
between the Title &
Abstract content, or it is
not clear.
The Title & Abstract
content are clear, but
needs refinement.
The Title & Abstract
content align.
The Title & Abstract
content perfectly align.
Presentation:
The abstract topic is clearly stated
and easy to understand.
10+ grammatical and/or
spelling errors.
Unintelligible sentences,
results/goals.
3 – 5 grammatical and/or
spelling errors.
Fairly clear writing
1 -2 grammatical and/or
spelling errors.
Good, Clear Writing
Grammar and Spelling
Correct.
Easy to Understand.
Innovation:
Does the abstract submission show
a high degree of originality?
No New Contributions
to the field.
The Title & Abstract
content connect, but
Author didn’t
successfully convey
connection.
6 – 9 grammatical and/or
spelling errors.
Awkward syntax,
difficulty in
understanding goals.
Good story but no
meaningful contributions
or findings.
The conclusions are not
effective nor supported
by any data.
Great validation of
previous findings/studies
with a good number of
insights.
The conclusions are well
formulated and are
supported by data.
Potentially breakthrough
findings.
Correctness:
The abstract content is technically
correct.
Practical Impact:
Information in Abstract has a
practical impact for attendees.
Completeness:
The work is completed, or shows
strong promise of being completed.
Would be of interest to
<50% of the attendees.
Good validation of
previous findings/studies
with a new insight or
two.
The conclusions are
moderately effective and
are partially supported
by data.
Would be of interest to
60% - 70% of the
attendees.
The work shows promise
of being complete by the
time of the Conference
The Abstract shows
signs of having Vendor
Advertising, with little
comparison to Piloted
data from other Vendors.
The Abstract has
minimal Vendor
reference, & only as a
comparison of Pilot
results of numerous
Vendors.
Presenter
10%
Content
80%
Writing/Syntax
10%
Technical Abstracts
The work will not be
complete by the time of
the Conference.
Technical Depth:
The abstract demonstrates
technical depth and is free of
Vendor advertising.
The Abstract is only
promoting a single
Vendor, with no
supporting Technical
Depth.
Texan
The Author works for a
municipality or firm that is located
in Texas?
The Author does not
work for a municipality
or firm located in Texas.
Presenter Quality
If the Author has presented before
(anywhere), was the quality of
their presentation High?
The Author is not
capable of High Quality
presentations & the
Reviewer would not
attend nor recommend
attendance of the
presentation.
The conclusions are
minimally effective and
only partially supported
by data.
Would be of interest to
50% - 60% of the
attendees.
N/A
The Abstract is riddled
with Vendor related
advertising.
N/A
N/A
The Author does not
work for a municipality
or firm located in Texas,
but performed the work
for a municipality in
Texas.
The Author is unknown
in the Quality of
previous presentations.
Would be of interest to
70% - 80% of the
attendees.
The conclusions are very
well formulated and are
strongly supported by
data.
Would be of interest to
80% - 100% of the
attendees.
The work is complete.
N/A
N/A
N/A
The Abstract is
completely free of
vendor advertising.
The Author works for a
municipality or firm
located in Texas.
The Author is capable of
High Quality
presentations & the
Reviewer would either
attend or recommend
attendance of the
presentation.
Rubric For
1
2
3
4
N/A
The Title of the Abstract
is adequate.
N/A
3 – 5 grammatical and/or
spelling errors.
Fairly clear writing
1 -2 grammatical and/or
spelling errors.
Good, Clear Writing
Good story with a new
insight or two.
Great story with a good
number of insights.
The Lessons Learned are
moderately effective and
are partially supported
by presentation content.
The Lessons Learned are
well formulated and are
supported by
presentation content.
Would be of interest to
60% - 70% of the
attendees.
The work highlights an
issue within Texas that
has been a problem
within the last 2 to 3
years, or the near future.
The Abstract shows both
research and practical
approach, but is heavily
tilted towards one or the
other. Not well
Balanced.
The Author does not
work for a municipality
or firm located in Texas,
but performed the work
for a municipality in
Texas.
The Author is unknown
in the Quality of
previous presentations.
Would be of interest to
70% - 80% of the
attendees.
5
Presenter
10%
Content
80%
Writing/Syntax
10%
Non-Technical Abstracts
Title:
Is the Title Creative and EyeCatching, but descriptive enough
to convey the content of the
presentation.
Presentation:
The abstract topic is clearly stated
and easy to understand.
The Title of the Abstract
generates confusion
and/or does not illicit
any interest in the
Presentation.
10+ grammatical and/or
spelling errors.
Unintelligible sentences,
results/goals.
Innovation:
Does the abstract submission show
a high degree of originality?
Lessons Learned:
The abstract contains Lessons that
can be applied to situations others
may encounter.
Rehash of the same story
told over and over in the
industry.
The Lessons Learned are
not effective nor
supported by any
presentation content.
Practical Impact:
Information in Abstract has a
practical impact for attendees.
Timeliness:
The work supports an issue that is
of current concern to the Industry
within Texas.
Would be of interest to
<50% of the attendees.
The work does not
highlight an issue that is
a current or ongoing
issue within Texas.
Balance:
The abstract demonstrates a
Balance between research and
practical application.
The Abstract only
presents Research or a
Practical Application,
but not how both are
intertwined.
Texan
The Author works for a
municipality or firm that is located
in Texas?
The Author does not
work for a municipality
or firm located in Texas.
Presenter Quality
If the Author has presented before
(anywhere), was the quality of
their presentation High?
The Author is not
capable of High Quality
presentations & the
Reviewer would not
attend nor recommend
attendance of the
presentation.
6 – 9 grammatical and/or
spelling errors.
Awkward syntax,
difficulty in
understanding goals.
Good story but no new
meaningful
contributions.
The Lessons Learned are
minimally effective
and/or are only partially
supported by
presentation content.
Would be of interest to
50% - 60% of the
attendees.
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
The Title of the Abstract
is Creative and generates
a large amount of
interest in the
Presentation.
Grammar and Spelling
Correct.
Easy to Understand.
Story exhibits a high
level of breakthrough
findings.
The story presents
Lessons Learned that are
very well formulated and
are strongly supported
by presentation content.
Would be of interest to
80% - 100% of the
attendees.
The work highlights an
issue within Texas that
is currently causing
major problems within
the Industry.
The Abstract easily
shows the presenter
researched the issue and
implemented a practical
approach that was very
effective.
The Author works for a
municipality or firm
located in Texas.
The Author is capable of
High Quality
presentations & the
Reviewer would either
attend or recommend
attendance of the
presentation.
Prioritization
As a committee, prioritize your ranked abstracts, based on the cumulative scores of all
reviewers using the Ranking Rubric (above). With your committee, discuss the strongest
abstracts by score and rank them in numeric order from highest (#1, best) to lowest.
There are several joint TAWWA / WEAT sessions (reference table below). If you are a
chair of one of these joint topics, please work with your TAWWA or WEAT counterpart to
finalize your rankings. The final rankings for these joint categories will be decided by teamwork
from the TAWWA and WEAT committee chairs instead of by the Program Chairs as in years
past.
Notes:
It is okay to show preference to local topics. Please notify us if there are high ranking
abstracts that your committee reviewed that relate to specific local topics as there may be a
separate “local topics” session if there are enough.
It is okay for committee members to rank their own abstracts as long as they are
reminded to be honest with themselves on how their work stacks up against others. In addition,
your committee members will be ranking your abstract as well.
Do not worry about the abstract being submitted under more than one subject area. We
will sort through that at a later date.
If an abstract does not fit your subject area, let your Program Chair know and suggest
where you think it belongs. The Program Chair will contact the applicable committee chair so he
/ she can consider it.
Keep in mind that those abstracts that are not selected for a Texas Water presentation slot
will still have a chance to be selected under the Poster session.