Best Practices:
Independent Third-Party
Verification of Placement
Christopher Lambert
Associate Executive Director
ACCSC
Webinar Agenda
•
Foundational Thoughts
•
Guidelines for Employment Classification: A Documentation
Framework
•
The Independent Verification Process
•
First Look: Independent Verification Results
•
Best Practices: Lessons Learned
•
Frequently Asked Questions
Foundational Thoughts
Foundational Thoughts for Today’s Presentation
Meaningful things are happening every day at
ACCSC-accredited institutions.
In 2014-2015, ACCSC conducted over 500 on-site evaluations
Every single day students are engaged in the classroom/labs, going on externships, taking
licensure exams, graduating with skills needed to gain entry into a vocational field, and
ultimately starting on their career path.
•
The average graduate placement rate at ACCSC-accredited institutions from 2012
to 2014 was 76%.
Foundational Thoughts for Today’s Presentation
Building / Reestablishing Confidence
ACCSC has always required schools to maintain verifiable records of initial
employment and believes that initiatives like the independent verification of
graduate employment, unannounced visits, and regular student achievement
reporting is building / reestablishing confidence around the success of ACCSCaccredited schools and the students they serve.
Foundational Thoughts for Today’s Presentation
Challenges Remain
Despite the positive trending in student achievement, demonstrating acceptable
rates of student graduation and graduate employment in the career field for
which the school provided education remains a significant challenge
155 Total Team Findings in 51 Team Summary Reports
Considered at the February 2015 Commission Meeting
25
20
15
10
5
0
Student Achievement
Admissions
Adverstising
Management
PAC
Avg. – Slightly More than 3 Team Findings Per TSR
Student Achievement
Admissions
Adverstising
Management
PAC
237 Total Team Findings in 67 Team Summary Reports
Considered at the November 2014 Commission Meeting
30
25
20
15
10
5
0
Student Achievement
Management
Admissions
IAIP
PAC
Avg. – Slightly More than 3 Team Findings Per TSR
Student Achievement
Management
Admissions
IAIP
PAC
July 1, 2013 – June 30, 2014:
ACCSC Accredited Institutions on Reporting
60
50
40
30
20
10
0
Refund Reporting
Program Advisory
Committee
Reporting
Heightened
Monitoring
Outcomes
Outcomes Warning
Changing the Dialogue
From Skepticism to Confidence
Increased Federalization of Higher Education
The expectations of accreditors are
changing significantly
Accreditation as we know it today has been subject to increased scrutiny and criticism by
a variety of organizations and policy makers who have valid questions about whether or
not accreditors are fulfilling their promise. Fundamentally, does accreditation help ensure:
1) the quality of education provided by its member institutions for students; and
1) the continued integrity of the Title IV federal student financial aid program.
Changing the Dialogue:
From Skepticism to Confidence
The enterprise of accreditation is evolving and improving.
• A key example of how national accreditors have embraced the challenge of
demonstrating that accreditation is both reliable and dependable without
increased involvement by the federal government is in the area of student
achievement assessment.
• In 2011, ACCSC initiated a new process whereas the Commission subjected the
reported rates of student achievement to independent verification
• 10% random sample of Annual Report Data.
Changing the Dialogue:
From Skepticism to Confidence
• In 2014, ACCSC adopted a policy requiring institutions seeking accreditation to
engage an independent third-party auditor to verify the most recently reported
graduate employment data as a means:
• to bolster confidence in accreditation and
• to reinforce a long standing requirement that institutions maintain “verifiable
records of initial employment.”
• ACCSC believes that the third party independent audit requirement significantly
strengthens accreditation and helps to change the dialogue from skepticism
to confidence.
Guidelines for Employment
Classification
A Documentation Framework
POLICIES AND PROCEDURES –
Section I (A)(1)(d), Substantive Standards, Standards of Accreditation
The school has and adheres to policies or protocols in regard to
employment classification (reasonable time period, sustainability,
relatedness, etc.).
Action Items:
• Are the school’s policies in line with ACCSC’s Guidelines for Employment
Classification?
• Appendix VII of the Standards of Accreditation
• Is the school working to update its policies regarding how it ensures that it maintains
verifiable records of initial employment?
• If the school is working with a third-party, is there a signed contract?
Verifiable Employment Record
A verifiable employment record includes, at a minimum the following
information:
• Graduate Name and Contact Information;
• Date of Initial Employment;
• Place of Employment;
• Employer Address and Phone Number;
• Employer Contact Person/Supervisor; and
• Descriptive
Job Title and Duties.
Action Item:
Does the record include all areas as required?
Justification for Employment Classification –
Appendix VII – Guidelines for Employment Classification
The employment classification is appropriate and reasonable
based on the educational objectives of the program.
Action Items:
Does the placement seem sensible based on the program objectives that
the graduate completed?
If the Third-Party was unable to verify the placement, can the school
demonstrate a reason for listing the graduate as employed?
Justification for Employment Classification –
Appendix VII – Guidelines for Employment Classification
Employed in Field:
I. is for a reasonable period of time,
II. is based on program objectives, and
III. can be considered sustainable (e.g., not a single day of employment).
Action Items:
• Does the school follow its internal policy regarding employment for a reasonable period of
time?
• When is the school capturing the information, is it the first day the student works?
• Is there any additional follow-up after a period of time?
• Is the policy consistently applied to all graduates?
Justification for Employment Classification –
Appendix VII – Guidelines for Employment Classification
Regular Employment
I. is directly related to the program from which the individual graduated;
II. aligns with a majority of the educational and training objectives; and
III. is a paid position.
Action Item:
• Does the documentation reference the skills required for the position which match the
educational and training objectives of the program?
• Verifiable Record + Job Description / Announcement
Justification for Employment Classification –
Appendix VII – Guidelines for Employment Classification
Self-Employment
The school secures written documentation from the self-employed graduate
verifying that the employment is valid including, at a minimum, a statement signed
by the graduate which includes the following:
• The graduate’s name and contact information;
• An attestation that the self-employment is aligned with the individual’s employment goals, is
vocational, and is based on and related to the education and training received;
• An attestation that the graduate is earning training-related income; and
• In cases where a licensure is required for employment, an attestation that such licensure has
been achieved.
Action Items:
• Does the self-employment statement or form include all areas as required?
• Is the license current/active?
• Did the student take the exam but not yet receive a result?
Justification for Employment Classification –
Appendix VII – Guidelines for Employment Classification
Career Advancement
Students that are already employed at the time of graduation can be considered
employed when completing the program of study as follows:
The school shows with written documentation from the employer or the graduate that the
training allowed the graduate to maintain the employment position due to the training provided
by the school; or
The school shows with written documentation from the employer or the graduate that the
training supported the graduate’s ability to be eligible or qualified for advancement due to the
training provided by the school.
Action Items:
• Is the hire date prior to the graduation date?
• If so, did the school provide Career Advancement documentation?
Independent Third-Party Verification
of Employment:
The Process
Be Sure to visit the Resources / Independent
Verification Section of www.accsc.org for more
information on The Independent Third-Party
Verification of Employment Process
Independent Third-Party Verification of
Employment Process: Selecting a Third-Party
• The auditor does not go through any type of approval process with ACCSC
prior to the project
• To date 21 different third-parties have submitted an independent audit.
• The auditor is required to attest to the following to verify its independence and
third‐party nature:
• Not affiliated with the school
• Does not provide any other service to the school
• Exception: Certified Public Accountant
• The auditor must verify employment data in a manner independent from the school
Independent Third-Party Verification of
Employment Process: Selecting a Third-Party
• The auditor must select a minimum of a 50% sample of employed graduates from each
program offered
• The auditor must secure written or verbal verification from the graduate or employer
• Not just an audit of “back-up” documentation
• The auditor must provide a summary of results and a description of the methodology used
• The auditor must sign a signed certification attesting to the accuracy of the findings and the
auditor’s independence
Independent Third-Party Verification of
Employment Process
• The school must engage an independent third-party to verify the employment data in the
Graduation and Employment (“G&E”) Chart(s) submitted in the most recently submitted Annual
Report at the time of the due date of the SER.
• 2015 ACCSC Annual Report
• The independent third-party must select minimally a 50% sample of employed graduates
(classified as “Graduates - Employed in the Field”) and report the results from that sample.
• 50% sample from each Graduation and Employment Chart filed in the ACCSC Annual
Report.
• Program by Program
• Once the 50% sample has been selected by the independent third-party, for the purposes of this
application, the independent third-party may not alter the sample in an effort to get better
results.
Independent Third-Party Verification of Employment
Process
The report must include a summary of results for each G&E Chart that includes the following
(at a minimum):
a.
b.
The total number of students in the sample; and
The number of students in each of the categories below and the percentage in each
category:
• Verified as Correct (graduate’s employment is verified as listed in the school’s record);
• Not verified (unable to verify after all attempts);
• Verified as Not Correct (e.g., graduate not found in employer records, position is
unpaid/externship, other discrepancies); and
• Verified but Different (e.g., different start date, job title, employer, etc.).
Any additional information or response the school deems appropriate regarding the above results.
Independent Third-Party Verification of
Employment Process
• An on-site evaluation team reviews the verification results from the independent third-party
• A school may provide the updated information to the on-site evaluation team as long as the
updated information was subject to the same methodology identified in the school’s SER.
• If an on-site evaluation team does find that a school has not demonstrated compliance with
accrediting standards due to the accuracy or validity of the data, the team will likely include a
team finding / request for additional information in the Team Summary Report
• The school will be required to respond to the Commission with supporting documentation
to demonstrate that the student achievement data is accurate and verifiable.
• The Commission, not the on-site evaluation team, makes the final determination regarding a
school’s compliance with accrediting standards.
First Look:
Independent Verification Results
•
2011 – 2014 ACCSC Annual Reports
4 Year Averages – 2011 to 2014 Random Sample
303 Schools Randomly Selected for Independent Verification
12,894 Employment Records
•
•
The percentage of records classified as “Verified as Correct”
•
•
The percentage of records classified as “Unable to Verify”
•
•
73% (9,348 of 12,894)
21% (2,706 of 12,894)
The percentage of records classified as “Verified as Not Correct”
•
4.4% (580 of 12,894)
Random Sampling Results 2011 - 2014
80
70
60
50
40
30
20
10
0
Verified
Unable
Not Correct
First Look:
Independent Verification Results
• 2014 / 2015 – Accreditation Cycle
Independent Verification:
2014/15 Accreditation Cycle
131 Schools with Results
•
21 Different Independent Third Parties
•
10,010 records
99% (130 of 131) schools met the required 50% sample.
• 10% (13 of 131) submitted 100% samples.
Independent Verification:
2014/15 Accreditation Cycle
Verified as
Correct – Mean
Verified as Correct
– Median
Not
Verified
Verified but
Different
Verified as
Not
Correct
January 2014
76%
83%
18%
4%
2%
March 2014
84%
94%
11%
4%
1%
July 2014
78%
83%
14%
6%
2%
September 2014
63%
67%
25%
8%
3%
Overall
75%
79%
17%
5%
2%
Independent Verification:
2014/15 Accreditation Cycle
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Jan-14
Mar-14
Verified as Correct - Mean
Verified as Correct - Median
Jul-14
Unverified
Verified as Not Correct
Sep-14
Verified but Different
Best Practices /
Lessons Learned
Best Practices / Lessons Learned
Program Viability and Assessment
Accreditors expect schools to conduct a realistic assessment of the viability of program
offerings and the factors impacting student achievement.
In all cases, accreditors expect that a school will take appropriate and responsible action
regarding any program offering that is not meeting expectations.
• Schools should makes the decision about a program’s continued viability, and should not
put this decision into the hands of a regulatory body (state or accreditor).
Some guiding questions include:
• At what rate are students graduating?
• At what rate are graduates passing licensure exams?
• At what rate are graduates getting a job in a training related field?
Best Practices / Lessons Learned
Program Viability and Assessment
• Are graduation and employment rates getting better?
• Getting worse?
• Each year, accredited institutions are required to submit student achievement data to their
accreditor via an annual report
• How is the data trending year over year?
• Are there any mitigating factors impacting the school’s ability to demonstrate successful student
achievement?
• Economic Conditions, Unemployment, Student Population Served, etc.
• Did the school examine the rates of student graduation in relation to admissions standards?
• Is the school “open enrollment?”
• What changes has the school made with respect to the program curriculum or career service
initiatives?
Best Practices / Lessons Learned
Working with an Independent Third-Party
Keep in mind that the results that are provided by the independent third party should be:
• Useful – The report should provide specific details in each of the categories identified that
presents the results in an easy to understand format for the school.
• Usable – Does the report provide the necessary information, in an organized, detailed
fashion to afford the school an opportunity to explore any “unable to verify” or “verified as
not correct”?
• For any graduate found to be “verified as not correct” the report should be specific as
to why.
Best Practices / Lessons Learned
• Sortable – the school should be able to easily sort through the data, for example
by category or by employer, to gain additional information on any trends in the
results of the audit.
• The school is responsible for making sure third-party results are submitted ontime and organized in accordance with the guidelines.
• The third-party should be able to meet the institution’s timelines and needs.
• Be specific with the due date of results as part of the contract with a thirdparty.
Best Practices / Lessons Learned
Start Early.
• Real Time Verifications – Do not wait fox six months to begin the verification
process.
• Older employment data proves to be more difficult to verify.
• Batching Verifications - Weekly? Monthly? Quarterly?
• Verify the employment AFTER the graduate started work.
• This is the most common reason for an invalid placement
• Many times a graduate is offered a position (after externship) and the school
counts that student as employed in field; however, the school did not verify that
the person actually started.
• Verify 5, 7, or 14 days after the graduate started, began working, and received
pay.
Best Practices / Lessons Learned
Incorporate Third-Party Verification into Regular On-going Processes
This requires budgetary / financial support
• For programs that have stable/reliable/consistent employment results, consider having an
independent third-party conduct a 10% / 25% sampling each year.
Consider having an independent third-party conduct a 50% to 100% sampling every year for
programs with:
1.
A significant number of self-employed graduates (prove more difficult to verify),
2.
Programs that have historically had more challenges with employment, or
3.
Programs that have not faired well during a previous verification process (i.e. high number of
unable to verify / verified as incorrect)
Best Practices / Lessons Learned
Set Expectations with Students
Begin on the First Day of Orientation and continue throughout the program
• Help Students to Understand Their Role in the Accountability Framework
• Accreditation Resources: New Student Letter / Graduation Letter
Establish protocols for post-graduation activities that can be managed and tracked.
• Admissions Protocols as Template
Have students sign a release form that permits the school to obtain employment
information
• Signed and dated upon graduation, not start, as frequently employers that
require a signed release want it to be signed within 12 months of the request.
Best Practices / Lessons Learned
Set expectations with Career Services
• Random, Unannounced, In-House Review of Graduate Employment BEFORE subjected to ThirdParty Audit.
• Ensure Complete, Descriptive, Clear, Legible, Graduate Employment Records
• Maintain up-to-date employment records in the school’s database or in another centralized
manner - Not in individual files or post-it notes
• “Sloppy” record keeping is another common reason why a record is not verifiable
Include:
Best Practices / Lessons Learned
Set expectations with Employers
• Work proactively with Employers
• Set expectations, Get feedback
• Batching Verification
• When we contact employers who hire multiple graduates, have the third-party
do it as one call/email/fax with the request for multiple graduates.
• Avoid calling employers to “scrub” data in advance of the third-party verification
process, only to have your 3rd party call them.
• Employers and Graduates get frustrated that “they just gave that information”
Best Practices / Lessons Learned
Maximize Your Relationship with Your Designated Third-Party
• Get weekly reports while the project is underway
• Ask for the option to update incorrect emails or wrong telephone numbers while
the project is underway
• Require the third-party to use the revised contact information
• Ask the third-party about the number of calls/emails and the pacing of outreach as
to not frustrate your employers/graduates
Frequently
Asked
Questions
Frequently Asked Questions
Q:
Does my school have to have an independent third-party verification of
employment data every year?
A:
No.
ACCSC requires independent third-party verification only at the time of the Application for
Renewal or Initial Accreditation and results are due at the same time as the Self-Evaluation
Report (“SER”).
• A school may opt to have employment information verified annually or on-going if it
chooses to.
Frequently Asked Questions
Q:
Which Annual Report will my school need to have verified by the
independent third-party as part of my application for accreditation
and submission of my SER?
A:
Schools must have independently verified the employment data from the
Graduation and Employment Chart(s) from the most recently submitted
Annual Report on file with the Commission at the time of the due date of the
SER.
Frequently Asked Questions
Q:
In order to be considered independent, the third-party cannot be one that
provides “other services” to the school (with the exception of a Certified Public
Accountant). What does “other services” mean?
A:
“Other services” means that the school must choose a third-party that the school does not
have any other contract or arrangement with to provide services or support to the school.
Other services are those such as, but not limited to:
•
•
•
•
•
•
•
•
•
•
Recruiting, advertising, and admissions;
Admissions documentation authentication or translation;
Education Credit Assessment;
Background checks and drug screening services;
Administrative services (e.g., payroll);
Student support or help desk services;
Library resources;
Information technology support;
Employment placement services or research to find where graduates are working;
Student loan repayment support or management;
Frequently Asked Questions
Q:
What if my school’s independent third-party is able to secure
additional verifications after the school submits the SER and prior
to the on-site evaluation?
A:
A school may provide the updated information to the on-site evaluation team
and may have the independent third-party update the summary of results as
long as the updated information was subject to the same methodology
identified in the school’s SER.
In this case, the school would also be required to submit the updated
information to the ACCSC office to ensure that it is updated in the school’s
record with the Commission.
Frequently Asked Questions
Q:
Can my school provide the on-site evaluation team with G&E
Charts using a more recent report date than that which was
submitted in the annual report?
A:
If a school wishes to present a G&E Chart using a more recent Report
Date than that which was submitted in the school’s most recently submitted
Annual Report, that G&E Chart must be verified in addition to the
G&E
Chart submitted in the Annual Report, and must be verified using the
same
methodology and reported on separately by the independent third-party.
Questions?
2015 PDC
September 9-11, 2015
Renaissance Arlington Capital View
Arlington, Virginia