Newsletter > Volume3 > Issue4

Evaluating Time Saved as an Index of Cost Effectiveness in PBIS Schools

September, 2006

Susan Barrett, Sheppard Pratt Health System

Becky's disruptive behaviors in the hallway have been a constant problem for her homeroom teacher who repeatedly is forced to interrupt the class lesson and respond to these issues ? frequently referring Becky to the office. At the office, Becky waits in line to see the Assistant Principal who is busy with similar problems from across the school. This scenario is typical of the manner in which many schools address challenging and disruptive students. Such scenarios can be described as reactive, time consuming, and repetitive (i.e., ineffective). There is strong evidence that time engaged in instruction is important, therefore, it is relevant to consider the amount of academic learning time that is lost for Becky using this reactive approach including the amount of classroom learning time that is lost for all students as teachers respond to problems, and the amount of time that school personnel spend responding to problems rather than teaching. Using reactive strategies at the expense of instructional time can be conceived of in terms of opportunity cost ? engaging in one activity eliminates the opportunity to engage in another. When students and school personnel spend time engaged in reactive discipline, it is at the cost of instructional time for all students. Adults could be spending more time engaging in instructional activities that facilitate greater success among the students instead of managing problem behaviors.

Schools that rely exclusively on punishment or other coercive means will find it difficult to increase academic and instructional time. However, more systemic efforts that include developing proactive rules, routines, and making physical changes in schools to prevent predictable problems have resulted in decreases in student misbehaviors, thereby decreasing costly disruptions and time consuming reactions. From this perspective, the time dedicated to prevention can help schools decrease problem behavior and save time making it possible to reinvest time in even greater prevention efforts. Schools implementing systems for preventing problem behavior need a simple and meaningful method for measuring the amount of realized time savings. This measure can then serve as both an evaluation of the effect of systems change efforts and as an index of lifestyle change within the school itself.

Cost Effectiveness Worksheet

Schools need a system for documenting time spent on disciplinary action so, to evaluate the cost effectiveness of prevention efforts. The cost effectiveness worksheet provides a template for such analyses. This simple spreadsheet allows schools to track time spent attending to reactive discipline each year. This provides a way to make comparisons across years and evaluate the effect of prevention strategies over periods of time. Below is a step-by-step description from a sample school in Maryland. Step 1: Track time spent on each referral. As a first step, the school must determine the average amount of time administrators or other disciplinary staff spend as well as the amount of class time students miss as a result of each disciplinary referral. This can be calculated by simply (1) adding two fields on the office referral form so that time is a regularly collected piece of data (i.e., number of minutes administrator spends processing a referral and number of minutes a student is out of class due to the referral), or (2) for schools using SWIS, assign one of the three extra information fields to time. In this Sample School the referrals were calculated to average 20 minutes of student time and 25 minutes of administrator time.

Step 2: Track total number of referrals. Enter the total number of referrals into to appropriate box for given years in Column 1 on the worksheet. Schools may choose to look at monthly totals in addition to annual totals. Sample School determines that they processed 800 office referrals during the 2003-04 school year and 650 during the 2004-05 school year.

Step 3: Calculate average. Enter minutes for both administrator and student into the appropriate boxes in column 1 on the worksheet. The worksheet will automatically calculate the total minutes, hours and days spent. Sample School can see that they have saved 5 days of administrator time and 6 days of student instruction as a result of a decrease in disciplinary referrals. Graphs in Column 3 will automatically be generated.

Step 4: Share the information. Graphs can easily be copied in word documents and PowerPoint presentations. Share the graphs with staff and other key stakeholders and use this information while making decisions about future efforts. Sample School can easily show faculty and staff the time savings realized as a result of PBIS efforts. This serves as a reinforcer for both the PBIS strategies in place and the collection of data. Those in the system whose responsibility it is to continue engaging in PBIS need to participate in evaluations of the cost effectiveness of their actions. 

District Evaluations of Cost Effectiveness

This same process can be used to develop an analysis of cost effectiveness across schools in a district, region, or state. For example, 12 PBIS schools in Maryland tracked their office referral data over a 2 year period (pre- and post-PBIS). Across all 12 schools it was found that there were 5,606 fewer office referrals after the implementation of PBIS. In this case if one office referral is equal to 15 minutes of administrator time and 20 minutes of student instructional student time lost, then the following calculations can be made using the worksheet: 5,606 x 15 = 84,090 minutes, (divide by 60 minutes) 1401 hours, or a total of (divide by 8 school-day hours) 175 days of administrator time. Similarly, 5606 x 20 = 112,120 minutes, (divide by 60 minutes) 1868 hours, or a total of (divide by 8 school-day hours) 234 student instructional days. Individually, the district can calculate that the average PBIS school saved (176 divided by 12 schools) 14.6 administrator days and (234 divided by 12 schools) 19.5 days student instructional days.

Time Engaged in Instruction and Impact on Academic Achievement

Research is quite clear that academic achievement in schools is highly related to the amount of time a student spend engaging in instruction. Because students who are often referred are missing academic instruction, it is reasonable to assume that academic achievement would be lower among those students. Considered another way, it is reasonable to assume that if students spend more time in the classroom engaged in instruction we would see a corresponding increase in academic achievement. Using the cost effectiveness worksheet each school can examine the relationship between instructional time and academic achievement. For example, one elementary school in Kentucky was able to calculate a savings of 97 school days for both administrator and students. At the same time, academic testing results demonstrated increases in Reading, Language, and Math on a national achievement test (CTBS).

The calculations below demonstrate how this school estimated the cost effectiveness of their efforts based on both time savings and academic increases. They chose to look at what increased academic engaged time might mean in terms of academic achievement with their students. Of course, not all of the 400+ students in this school received additional time ? but the students who did experience more time in the classroom probably needed the time more than others and may be the most likely to benefit from that additional instruction. Thus, the benefit here is not to the academic scores of the entire school but to a smaller group of students that previously had been failing at a higher rate.

It must be cautioned here that this is does not constitute a scientific analysis of the cause of student achievement. However, this is an example of how schools may choose to examine their data as a means of evaluating the cost effectiveness.

All content of this newsletter and the website are the property of the Technical Assistance Center on Positive Behavior Support. Duplication without express written consent by authors is strictly prohibited. For any questions please contact: Laura Riffel