VIRGINIA COMMONWEALTH UNIVERSITY VCU AUTISM CENTER FOR EXCELLENCE WEBCAST CAPTIONING PROVIDED BY: CAPTION FIRST, INC. >> MARIANNE MOORE: Hello. My name is Marianne Moore. I am with the Virginia Department of Education. >> ELIZABETH EVANS GETZEL: My name is Elizabeth Evans Getzel. I am at the Virginia Commonwealth University Rehabilitation Research and Training Center. And we have contract with the VDOE to create the database for Indicator 14 data collection. >> MARIANNE MOORE: For some time now we have really focused on writing good transition IEPs and the technical assistance provided by the training technical assistance. IEP writing has been the focus and we have seen steady improvement. Statewide Indicator 14 is consistent over 90%. An IEP is important in implementing transition service and results are also important. As we consider methods and practices that get good outcomes we should focus on our Indicator 14 data. Your transition specialists at the training and technical and assistance centers have received professional development that will help tie your efforts to results. This Webcast is designed to get you started looking at your Indicator 14 data. We want to talk today about several things during our time together. I think that it is important that we review how to access your Indicator 14 reports. I know that I get a number of questions about that from school divisions. We want to make sure that you understand how to access those on your school division website for better use of the data that's being collected. We want to discuss what types of data are available in terms of the reports that are on your website and we will be looking at a few samples of that today to give you an idea of how the data can be used. It is really important and this will be talked about throughout this Webcast is about how your indicator is calculated. I know there has been some confusion about the actual Indicator 14 data that is submitted to the Federal Government February 1st of each year and what is the data on your database. For example, I will talk about that when we talk about employment in particular. So that you understand how Indicator 14 is really pulled from the data that you are collecting, it is like almost a subset of the overall information that you are providing. And most importantly we want to discuss strategies on data that can be used for local programming decisions. One piece of information to really bear in mind that for small school divisions how you determine you are going to report out your data is very critical because you are dealing with smaller numbers and you want to make sure that the data that you are presenting to the public using for the program decision is something that makes it -- makes it transparent as to who these individuals might be. So when we talk about a self-determining or a response, if you will, to a particular question, that is not -- that should not be reported out in the sense that there could be identifiers as to who is in higher ed or who is in employment. So we want to make sure that for the smaller school divisions that you look very carefully at what your data is and make sure when you are reporting out that it is done in such a way that it does not identify individuals who may be in your database. Let's review how you access your Indicator 14 data so that everyone understands the process to obtain the reports that are on your website that you can easily access. I am sure many of you are familiar with our worksupport.com address because that's how you -- this is the same address that you use when you input your data. And you will come to the log-in page and then you will type in your division's administrator code. And the administrator code is sent to those individuals identified as the contacts for receiving information from us during the Indicator 14 process. And then once on the site you will click on see frequency response report. Let's look at what that is on the website. So we have typed in the worksupport.com Indicator 14 web address. And you are now at the point where you are typing in, this is the log-in page and you are typing in the information which is your passcode, your administrator passcode so that you will see the reports. So as you can see when you are first -- when you are first entering data and you are not using the administrator passcode you really see add survey data for new student or see survey completion report. So those are the reports that you are entering as you are collecting the data. Once you type in the administrator passcode you now see two other areas. See survey response frequency report and this report gives in realtime, meaning as you are entering the data it gives you in realtime what your Indicator 14 data looks like. So that will always -- if you want to see how things are going with your data entry you can always click on your survey response frequency report and see that. Remember that Indicator 14 has very -- that is submitted to the Federal Government has very specific definitions as to how we then oralize the data after it is all completed. It is a hierarchy in terms of how data is organized. So it is the percent of youths who are no longer in secondary schools and had IEPs at the time they left the school. When we talk about students who are no longer in secondary school that can include dropouts who drop out in the 9th grade. You are looking at individuals from 9th grade or 12th or beyond in terms of once they leave and at the time that they leave they are on an IEP. First definition for Indicator 14 is enrolled in higher education within one year of leaving high school. Anyone who checks yes, that they are in higher ed, they have completed a semester, they will be calculated enrolled in higher ed. Second one is enrolled in higher ed or competitive employment and there are definitions of competitive employment. Those two numbers are added together for that second part of the indicator. And the third part really takes higher ed, your competitive employment number, if they have been involved in post secondary education which is other post secondary education and training which also has a specific definition or other employment once you are leaving high school. And what's important to understand is that Indicator 14, the information that you are entering on to the database is really a subset of the total information that you are collecting. So you will be able to look at your Indicator 14 data and on your frequency report that we just looked at in terms of the arrow pointing to clicking on that, we give you a nonduplicated count so you can see how many specifically are in higher ed, how many are specifically meeting the competitive employment and then how many are in post secondary education or training or other employment. Then when we calculate the indicator if you remember the table -- I sent out tables to everyone that looked at what your Indicator 14 data was, compared your LEA indicator percentages compared to the state target. So I think that it is just really important to bear in mind that there is a lot of data on your Indicator 14 site that goes really beyond the Indicator 14 numbers that are being submitted. I also want to point out that you have an option on your database for legacy data, and that data is for the last four years, 2007 to 2010. In 2010 the definitions changed for Indicator 14. Your current 2010 and 2011 data that you just completed are the only two years that you can compare your information because they are based on the definition. 2007 and 2009 are under different definitions. If you look at your indicator numbers prior to 2010 it is really comparing apples with oranges because they do function under a different definition. You do have two years of comparison data and that can be helpful to look at any -- that might be happening in your school division. And remember that the data is always submitted on February 1st of the year we -- the Virginia Department of Education just submitted in the processing submitting the data for this year's collection. >> ELIZABETH EVANS GETZEL: I have one question. Because we were talking about comparing data that DOE collects year to year. What about a school division that wants to compare data with another agency's data. What cautions should they be aware of? >> MARIANNE MOORE: What's important and why we are spending the time in the short amount of time, to understand how to look at your data and what are the parameters on your dataset for the school division and you need to have that same idea for other agencies. What are their definitions for employment or what are their definitions of the individual. Because databases really function and are driven by what the definitions are. So you really need to get an idea and make sure that when you are comparing that you are talking about similar individuals, if you will, by definition and similar ways that employment is counted. >> ELIZABETH EVANS GETZEL: Okay. So we are looking at not -- we are looking at cumulative dates for employment whereas someone else might be looking at consecutive days of employment. >> MARIANNE MOORE: Exactly. Some surveys look at enrollment whereas we look at enrolled for a term. Those could really make data stuff look very differently. >> ELIZABETH EVANS GETZEL: Yes. Exactly. >> MARIANNE MOORE: Thank you for bringing that up. Again we are back on to our website. And the arrow is pointing to see legacy data. When you click on that you will get the years, 2007, 2008, 2009, 2010 and after now the data is going in 2011 will be put on that as we begin to collect the 2012 data. Once you log in and you look at the reports that are available to you, you might determine that you need some additional -- very specific reports. For example, you might be interested in what the post school outcomes look like for a specific disability category or interested in outcomes for a group that exited school in a specific way and those requests you would need to make through Paul Raskopf. He is the director of data and finance in the special ed -- division of special education and student services. But when you are making that request through Paul Raskopf it would be helpful if you copy in Elizabeth Evans Getzel and myself. You see all the contact information that you need on the slide. This is just one of the datasets that we can look at through our reports. But before we look more closely at this I just want to again caution that small n is always important because you pool a size of your respondents may skew what you -- what you are thinking. So that's always important to take that in to consideration. Remember percentages don't tell the whole story, particularly when you are working with small -- that small n. It might be helpful when looking at data to be asking yourself what is missing here and does this represent our population that we want to look at. You also need to think about how many years worth of data do I need to look at before I can determine I see a trend. It is helpful to know also what's going on in your community. For example, when you look at employment, one year the employment rates could be pretty depressed and you might look back in your community and realize that one of your major employers has -- had cutbacks or has even left your area. You also need to consider things like budget and staffing challenges. That not only you are facing but as school divisions face those organizations in your community that provide support and services, other agencies that provide supports and services may also be looking at reduction and resources. So it is important to look at -- as you are analyzing your data and looking at it to really consider more than just the numbers that you see on the paper. >> ELIZABETH EVANS GETZEL: If I could add something, another important thing to remember, and we talk about this, talk to a number of school divisions about this is your response rate. You need to look at how many people you were able to contact and get information and in looking at that number it is almost like how many are you missing, if you will, and can you speak to the whole population based on the number that you have. Now for Virginia total, the total state total with the completed survey is a representative sample. So you just want to look at it to make sure that you are looking at numbers that are -- that give -- that do reflect your school population of leavers. So lots of things to think about as you analyze and look at your data. >> MARIANNE MOORE: We pulled this data from our favorite county, Jefferson County and, of course, you all know your geography of Virginia and know that Jefferson County really doesn't exist. So fictitious county and this is -- this data is pulled from a student record review. So someone is looking at the student's IEP record and they are determining whether or not the student was referred to an outside agency. And you can see that 46% of the youths were referred to some agency or organization that could be helpful as the student exits school. Excuse me. 48% were not referred or there is no record of any referrals. Since agency linkage is an important part of transition Jefferson County might want to consider looking at the nonrefered or the no record of referral. That number which is what -- 48%. You might want to look at that population and determine are they unengaged. I mean this is the population of youngsters that are not employed and not going on in school? You also might want to consider even though that the student -- maybe this was discussed with the student and family during a meeting that they determined that they didn't need any referrals or they didn't need to use an additional agency. Maybe the student was going to be employed in a family business and they felt they didn't need assistance or after moving in to education or a training program they are doing so well in school that they feel they don't require any supports at that time. It is helpful to look at the group that wasn't referred. And then another thing to look at, too, when you look at some of the other referrals, like the employment commission or to Social Security Administration or community service board, is to consider, you know, are you using all of the resources that are available to you in your community. So you -- maybe you would be thinking are we fully utilizing the services that are available to you through our workforce center. Lots of things to consider when you look at this -- just this information that comes from a record review. >> ELIZABETH EVANS GETZEL: This is at the front part of the survey when the individuals conducting the survey is filling in the demographic information of students of potential contacts that you will be making and does a record review of the agency. >> MARIANNE MOORE: Now this report is actually from survey respondents and it again is talking about what agencies were you referred to. And you can see that -- I mean if you add up some of the numbers 35% identified they were referred to an agency or organization. And then look at the number that were not referred or didn't respond and that's 65%. So you might ask yourself just looking at those numbers do the young people that we are working with know who is sitting at the table and helping them make some decisions. They may consider anybody who is at school which is where most IEP meetings happen. That if you are at the school you must be part of the school division. They might not recognize that someone is with another agency and that other person has some services that they can offer. So maybe Jefferson County, if they determine that young people really don't understand that whole linking process, they might want to consider how can they help students be more self-determined so they take an even more active role in that whole process. This question and data that we get from the question just asks youngsters to tell us what classes they felt were most helpful to them. And you can see that they identified math, career and technical education courses and English clearly as the leaders in this area. And that might be very good news to know that in Jefferson County the young people are seeing a really close connection between the classes they take and having good post school outcomes. Now this question deals with what classes do you wish you had. So it is kind of now that I am out of school and I have been out for a year and I am doing whatever it is, college, training, employment, what things looking back do I wish I had taken. And again we see that career and tech or computer related courses account for 36% of the responses. Now in Jefferson County as they look at that data they may think that maybe we need some for career and tech ed offerings for our students. It could be they need to revisit a process for application and how they schedule and when they schedule students to make sure that students with IEPs have the same access to the application process as other students. They may also want to consider should we have more collaboration with career and tech ed classes. It is quite common now in Virginia for the English class to have a collaborating teacher or a mathematics class to have a collaborating teacher, that it might not be a common practice to have collaborating teachers in our career tech ed courses. As you look at the data what happens is it prompts you to ask for questions and to mine your data a little deeper to come up with some conclusions and also helpful as you look at this data what are our course offerings going to look like in future years. And it may prompt a special ed director to request some more funding or to request that potential cuts not happen knowing that students need more of a particular course offering so that they can have more successful outcomes. As you can see on this table students with specific learning disabilities is the larger group that's in college. We do have some students with intellectual disabilities that are counted. So Jefferson County may want to look at this to say are there particular programs. Are we ensuring individuals who say that they are interested in college and want to look at those options are facilitating that in their IEP meetings and working with students and families to make sure that that is happening. And to determine what are those higher education options and are there -academic preparation in place so that the students can then transition to these programs. >> ELIZABETH EVANS GETZEL: Another question on the survey as you move through the survey and are getting responses, if a student indicates or a respondent indicates that currently they are not participating in two or four-year programs then it moves to post secondary education or training programs and there are specific definitions around this as well. There needs to be -- there needs to be a formal application process for considering post secondary education or training programs and they also need to have completed a semester. Now a semester for some of the programs that may be short-term education or employment such as ten weeks or that kind of thing, can be counted in this category. So we need to look at in a sense we have quite a high number that there is no participation. And we want to reflect on some of the information that Marianne was talking about in terms of needs, they wish they had taken more vocational technical courses. We need to see if there is any connection at all in terms of what is happening to students who may want to go in to more vocational technical coursework that could be going in to vocational technical school or some other opportunity or where they are in terms of completion of their diploma so that we can see what's happening with individuals participating in post secondary ed or training if the two or four-year option is something that they did not want to pursue. >> MARIANNE MOORE: The next is employment and I just want to spend a little bit of time about employment. You will have on your database a table that is employment: All settings and this really helps to give you a picture of your respondent pool who aren't employed. And this includes individuals who may be in higher education and employed, which is really very much the trend now, most students cannot afford college unless they are working and it includes everyone by setting and provides probably a better global picture of employment. Remember your Indicator 14 employment definition is competitive employment that you are -- that the student is earning at or above minimum wage, is working 20 hours a week or an average of 20 hours a week and for 90 cumulative days since they have exited high school. And that can count as one job or more than one job but we are looking at that. This particular table shows you everyone. So your Indicator 14, when you look at the nonduplicated count you will see a number that says competitive employment. When you look at that number that number is the one that meets the indicator. And I have had questions from some school divisions about these employment numbers because when you come to this table it looks different. So Jefferson County would be looking at this table saying how is our overall employment and where are individuals going overall in to employment whether they are in college or not. And so it is important to look at the overall so that you do get that sense of what is happening in your school division employment wise. And then you can look at your indicator number to look at those individuals that meet that definition which means that they are earning minimum wage. They are in inclusive employment settings that they are working a certain amount of hours, 20 hours a week and that it has been 90 cumulative days. So just bear in mind when you look at this employment that this will help you look at your overall global employment but you may want to look at your Indicator 14 number because that is competitive employment and earning that minimum wage or above. In this table we are looking at who the student reports or graduate leavers reports as having helped them find employment. And I would think -- this is an interesting table because you can see clearly in Jefferson County the majority of youngsters, I think it was 83% of them really are seeing -they find their own jobs. A parent or relative helps them find the job or a friend. So they are looking at some really close resources to them helping them find work. And Jefferson County might want to think of this as this is good news, that our youth are able to utilize those close resources that don't cost any money. And that is what is helping them find jobs. It could also indicate that we have some very self-reliant students and, of course, those are all good things. But Jefferson County might also want to look at this and see how does this play in to the whole idea of again the agency linkage because you do have a certain group here, that 6% that they say something -- someone else other than an agency or a friend or themselves helped them find a job or if they don't know who is helping them find a job. So they might want to again consider how we are providing information to youngsters during that IEP process when we are linking them to agencies that can assist them if they leave school. And I think it is also interesting and very important that we look at what are the barriers to employment. Just as Liz will talk about barriers to post secondary education and training. >> ELIZABETH EVANS GETZEL: So you can see from this this table or this dataset that the respondents are clearly identifying that there is a lack of jobs right now and that makes it difficult and we would expect somebody at an entry level who doesn't have a resume with lots of experience that they might experience more difficulty finding work. That we would suspect that with a sluggish economy that adults with a rich work history can have difficulty finding employment. It is important for Jefferson County to look around in their community and recognize if they have a major employer that has left the area, that would be reflective in their employment rates. Another barrier that's identified by the respondents is transportation. So if you -- if Jefferson County doesn't have public transportation, and they have some limited resources in that area what might behoove school folks is to look at that and start working with youngsters even earlier and families to address the issue of transportation. And that involved a lot of problem solving or helping people who may not be used to looking around at a variety of resources. How they look and gather resources to help them get to work. I talked to a young man the other day on the phone and he just got his driver's license. He can't afford a car because he hasn't been working too long. But he does have employment. I am sure it feels like he is scrambling but his mom is going off to work and it meets his work schedule and she drops him off. Maybe a fellow employee would bring him back home. Sometimes because he works shift work he makes arrangements to have -- to meet somebody who works that same shift to get him to work and then also to get him home. So he is doing a lot of problem solving and thinking on his own which is a good thing, but some youngsters are going to need some help with that and some families are going to need some help with recognizing that there are a variety of ways to solve that transportation problem if you don't have public transportation. Then we also see that on this 11% have health issues that are preventing them from going to work. We have a certain number that said that there were just a lack of programs that they were interested in. So a youngster who is interested in a culinary program or getting in to video gaming, there might not be a program in their area. So again they have to start to look at other resources or other options. And again like I said this again might link back to the school division and how they work with their -- the agencies, adult service agencies, how they do the linking and are they doing it early enough and what are the community resources and how they are looking at all the community resources to explore how they can be helpful. Barriers to post secondary education, barriers to employment and barriers to post secondary education, these are individuals during the survey process when you have called and talked with them at that point in time they are not in higher ed. They are not engaged in employment and they are not involved in any post secondary -- other post secondary education, training or employment. So these individuals are the ones that when Jefferson County is looking at where some of our issues or barriers that still exist in our community, I know that many of you in school divisions that these are very familiar issues that you work on all the time. And when we look at barriers to post secondary ed the highest is lack of financial support. And I think that what we are finding and the work that we are doing in post secondary ed is that it is really important that during the IEP process students and their families have an idea of what financial opportunities are out there. And that is very difficult to track down all those options. But it is important at least to look at some primary resources, the Federal Government and their FAFSA forms and some of the other opportunities in terms of scholarships that are geared to support students with disabilities and other means for assisting students, either Social Security. So looking at sort of that range of potential financial supports and making sure that in the planning process early on that that doesn't become a primary barrier for students to be able to transition in to some form of post secondary education. Again transportation comes up as an issue. And it is something that I know personally for some students that are attending VCU, that the family works out their schedule and tries to work it so that students are able to be on campus when there is not transportation from the particular county they live in to come in to VCU. Again we are looking at some similar situations and these individuals can select more than one of the barriers. So you may have an individual that has multiple barriers that is reflected in these numbers because individuals can check all that apply to them in terms of their barriers. Again lack of programs or health issues becomes a problem, lack of support services. Personal assistant, I know that's a big issue and finding personal attendants to come on campus and work with the student and can access post secondary education. There's a number of support services that can be an issue. And lack of affordable childcare, that can be a barrier for employment and post secondary ed. >> MARIANNE MOORE: For those of you who have been collecting the data with us for a couple of years, you know that we collect more information in our post school survey than is required for the reporting purposes to the office of special education programs. But it is important that the data that we collect have some meaning of not just for reporting purposes but that it is very useful for school divisions to use to improve practices when providing transition services so we see improved outcome. If your division had an initiative to improve let's say the SOL scores, you naturally would look at instruction, who is not doing well and who is doing well. And this would help you begin to look at some of the practices that you might need to add more of or cease those practices. And add something else that is evidence based. So say Jefferson County decided that they wanted to increase the employment rate for students with intellectual disabilities. It might be important for them to see how leavers are accessing community support or whether they are accessing community support and resources. What courses are they taking while in high school that might lead to entry level employment or the ability to be successful in a training program. It could also mean that they would want to add some more community based programs like internship for students that are -- specifically designed for students with disabilities. So looking at having a lot of data that is helpful so you can look deeper or mine your data deeper to look at specific population, specific initiatives that you might want to add that should improve anyway your employment outcomes and that would be the same with the post secondary education and training programs. >> ELIZABETH EVANS GETZEL: Exactly. Again we were talking about participation in higher ed by disability category and Jefferson County could look at that and determine they want to make sure that post secondary education is an outcome that the student and family would like to see happen, that during this IEP process that options are being looked at and information shared and ways to enhance that for any particular disability category that Jefferson County could be looking at. The overall participation post secondary, other post secondary education and training, for Jefferson County they see a large portion not participating. They need to look a little deeper in terms of what does that mean and are there other options or initiatives that need to be discussed about employer based training and providing further training beyond high school. Because as we know more and more in this economy and the global market that some post secondary education further training beyond high school will be needed by individuals who are exiting high school. And that's overall and that includes all students who are exiting. So becomes particularly important students that we serve who have disabilities and nonengaged number as Marianne pointed out we do want to know what's happening to those individuals who are not employed at the time that you are contacting them or not in higher ed and post secondary education training. So what's happening with them. What's going on in their communities. And as we said that many of those themes are familiar to school divisions in terms of these issues that are really looking at who was -- who was impacted by these particular barriers and what perhaps planning wise when we look at numbers of agency referrals and what that means, it all really ties together. Your data -- data tells a story and it is a matter of understanding first and foremost what the data is you are looking at so that you can then look at what that data story is and how to implement that in to practice. When we talk about students who identify that they are lacking support services in two and four year colleges or even in employment settings, I wonder if a piece of that is our students maybe are not comfortable identifying that they have a disability so they would not receive supports if they are not asking for some additional assistance through the disability support services that are available. And that we know that those are some things that we can help students and school divisions and even with agency referrals, people in agencies can be helpful in terms of identifying there is -- the person has a disability and they need some additional assistance. So that can come through some self-determination work while they are still in high school. And I know that frequently agencies are very helpful with helping a young person learn to identify a disability in a very appropriate way. They do receive some support services. We probably could with some additional work. Jefferson County would reduce that percentage saying I can't get any support services. >> MARIANNE MOORE: And another thing I would like to say and that is that very much we thank teachers who have taken this collection very seriously and have done an outstanding job. Frequently I get calls and I know that Liz gets calls from teachers who have called a youngster who was at home. They have identified some barriers, why they are not doing -- they are not employed or they haven't gone on to school. And the teacher is able to reconnect that student so frequently with another agency that they were referred in high school or maybe didn't follow through with meeting that agency personnel or they said they didn't want any referrals while they are in high school. They didn't want to do that. And the teachers have been able to really connect, reconnect youngsters and in some cases a youngster has even come back to school because they are still eligible for special education services. So I think it is important to -- to emphasize the personal connections that teachers are making with past students and that's really I think what makes a difference to so many students. >> ELIZABETH EVANS GETZEL: Absolutely. >> MARIANNE MOORE: This is a contact information slide. Paul Raskopf, Elizabeth Evans Getzel and myself e-mail and contact numbers. If you have any additional comments or need additional reports, you need help analyzing or looking at data please feel free to call any of us with concerns or questions that you have. I want to remind you that we do have transition specialists at the VDOE training and technical assistance centers at Radford University, James Madison University, George Mason University, William & Mary and Virginia Commonwealth University. So all of our divisions have access to a lot of technical assistance to improve outcomes for children and, of course, to improve the services that we are able to provide youngsters who have disabilities. >> ELIZABETH EVANS GETZEL: We would like to close by thanking you for your participation today. Please feel free to post any questions you have on the web board and Marianne and I will respond to them. It might be 24 hours before we respond. We will respond to any questions posted on the web board. So thank you again. *****