Running a Study from A to Z
Panelists:
Katie (Snead) Boyd: 5th year I/O, working on Qualtrics Survey Building Software in
the Provosts’ Office, Available to all VT students to create a dynamic survey with 20
types of questions possible (multiple choice, matrix/table questions, graphic sliders
or forced choice), randomizing questions or blocks or recipients. Qualtrics also has
an email function to send your survey as a url, reminders or thank you emails, even
confidentially. It tracks their progress (opening, progress of survey per participant).
Also makes data export easy to create reports (pie graphs, charts) as long as you
delineate graph type. Qualtrics is Excel, SPSS and xml compatible.
Derek Spangler: manages the SONA System for psychology. SONA is an online tool
to recruit mainly psychology students for your study and incentivize participation
for extra credit for instructors, experimenters, and researchers. Instructors can
choose to offer extra credit for classes. It is possible to post survey urls on SONA.
Reminders will be sent to award participation and reminders are sent to
participants to attend your study. The flexibility offered by SONA allows your study
to be in lab, online or both. There is a manual to be made available via the list-serv.
The first step is to request an account and post studies as long as these studies are
IRB-approved. Currently, you must offer a cover sheet to specify number of
participants you need, credits you will offer and how you arrived at these numbers
to sona.vt.edu and receive feedback prior to posting a study. Consider the common
survey that serves as a pre-screener so some students will be able to see studies for
which they qualify. You can send items for the pre-screener to either Derek or Neil
for next semester. This uses credits more wisely so look forward to emails about
the updated pre-screener. Feel free to email with questions!
Shane McCarty: 4th year I-O Psychology. Shane will discuss how to select the
correct individuals and balance their development versus project goals as well as
giving good feedback. Independent studies take a 3-credit course. Undergraduate
researchers’ roles depend on the lab and goals of the lab, particularly depending on
fellowships or scholarships. Shane’s URs have spanned applied research interests
within 12 schools and collecting levels of data. The needs of the project dictate how
many undergraduate researchers. Diverse experiences among undergraduate
researchers they can apply in their next role aids in development. Content can be
taught but matching individuals to the vision and roles available matters most. Each
individual was observed prior, considering meeting attendance and individual
interviews were weighted heavily. Selecting from previous field studies allows
seeing their experiences prior to their selection. Training occurs most for those who
least fit the project. Perhaps working on multiple projects would better serve those
in hindsight. Consider ascertaining skills from peer reports.
Managing occurs best with testing different software. Asana is highly
recommended for organization since you can use it to assign tasks, take notes, and
supplement this tool for meetings to keep everyone on the same page. The nicest
feature is you can have collaborators for a task and be notified when a task is
complete. You can customize accessibility and grant timeline charts in a visual map
to completion. This allows easy customization of timeline depending on lateness of
one task being completed. Recommendations depend on student goals and the
types of their connections they want to make (meeting school psychologists, or
social workers). Set personal and project goals as equal so their personal
development hinges on project goals and the career they’re working towards.
Matching the right student for the right role is useful.
Open up to questions:
Issues you wished you had known with running your own study?
Katie: use the full capabilities of the software that has been designed for research
(e.g. use Qualtics to email students, allows it to track students). Use the pre-screen
to be able to look at, even if you have certain cutoff scores. Use free tutorials and
webinars for how to set up a panel and embed data into your panel with specific
qualities. For example, if you know the gender ahead of time, you could embed this
rather than asking this as a question to your participants.
Derek: You can’t run certain participants (e.g. smokers or irregular heartbeat).
Discovering this when you are analyzing your data. This could save you hours with
a proper pre-screener.
Shane: Expectations of managing multiple research projects. This takes a lot of time
and practice to handle these simultaneously. It would be great to have advice on the
process of keeping track of multiple studies.
How do you gauge the degree of involvement of faculty in your study?
Katie: It depends on how comfortable in your research topic. For example, ensuring
your study follows the correct theories, development and software with Rachel
Diana, even though cognitive is not my primary area. By dissertation occurs, the
involvement will be less and less with more decision-making on your part.
Derek: without outside influence from his/her advisors, it was disadvantageous. Be
sure to have a conversation with your advisor, particularly in outside lab papers and
projects.
Shane: our setting is primarily hierarchical and part of the challenge is negotiating
research etiquette, informing your advisor, and maintaining good relationships with
your advisor and collaborators.
What do I do if a faculty member does not support the research project I
would like to engage in?
Katie: there are multiple reasons for lack of support. They may be overloaded, it
might be outside of their expertise domain or it may have been researched or not a
substantial contribution. Be sure to dialogue about the rationale.
What do you do if you have a target sample size and recruitment fails by deadlines?
Derek: check out other departments’ SONA systems (communications, and
marketing). Be sure to amend your IRB to include students from other departments.
Katie: Management department, particularly at the beginning of the semester. This
data is really impressive with high reliabilities. Recruiting Corps members is also
helpful.