TIPS (Team-Initiated Probelm Solving):

Enhancing Data-Based Decision Making

Abstract:

 This project is a four year research program focused on improving the collection and use of data for decision-making in schools. The basic thesis is that one of the most efficient strategies for promoting adoption of evidence-based practices in schools is providing school personnel with (a) appropriate data management systems, and (b) practical procedures for using data in daily decision-making. There are an increasing number of educational data-management systems being adopted by districts and schools across the country (e.g., SASI, Pentamation, PowerSchool, eSIS). Little information is available, however, about which features of these systems will be of greatest value for improving the adoption of evidence-based practices, and increasing outcomes for students. The project is designed to build the research tools needed to establish this area of study and provide the preliminary data needed to guide larger research efforts.

The project was initially planned to be conducted and the University of Oregon. However, to take advantage of the fact that Dr. Newton relocated to North Carolina, we redesigned the project to (a) include subcontracts with Dr. Bob Algozzine and Kate Algozzine at UNCC, (b) recast the tasks to distribute activities between Oregon and North Carolina, and (c) establish a regular communication system (using poly-com video conferencing) to allow a single workgroup composed of Dr. Rob Horner, Anne Todd, Dr. Steve Newton, Dr. Bob Algozzine, and Kate Algozzine.

A concept paper has been completed that summarizes the current literature on data-based decision-making and proposes the frame work for using this information within schools.

The Team Initiated Problem Solving (TIPS) training content has been developed and used for training four Field-Test schools (two in North Carolina and two in Oregon). Revised based on the initial field test, and used with two additional field test efforts, the training materials for TIPS are now ready for dissemination. An added product for the project has been the recent development of a TIPS Trainer of Trainers' Manual. This manual can be used by state trainers to build regional capacity in TIPS procedures.

The Decision Observation Recording and Analysis (DORA) instrument has been developed, field-tested and revised. This instrument allows observers to record if a school team is conducting active problem solving, using relevant data, and following problem-solving procedures recommended in the literature (and the concept paper). We have developed the technical manual for using DORA, and a training manual that researchers could use to train observers in the use of DORA.

The examine the technical adequacy of DORA we have (a) examined inter-observer agreement, (b) content validity by comparing DORA to the current literature, and (c) construct validity by comparing user performance on DORA and the standardized SPSI-RL index of problem solving. These data are currently being crafted into a paper that will provide the research community with the psychometric features of DORA needed to encourage broader application of the measure. A draft of the Technical Adequacy Paper has been developed and awaits final analysis of SPSI-RL data from our Randomized Control Trial to allow final completion.

During the Year 1 of the project, four schools were selected for initial field testing of the TIPS and DORA materials. Two schools in North Carolina and two schools in Oregon participated in team training, and on-going implementation.

The results of this field test document (a) schools are able to acquire the skills defined in TIPS procedures, (b) the DORA instrument can be used reliably with schools implementing TIPS, and (c) team members report the TIPS process to be an improvement in efficiency and effectiveness.

If the TIPS procedures are to be used at a broad scale, they must be able to be delivered by typical district trainers. We conducted this descriptive study as an additional activity with the project. Four schools in North Carolina received training from typical district trainers who went through our TIPS Trainer of Trainers material. The resulting professional paper is currently under editorial review.

A major objective of the project was to design the experimental, single-case study that could validate both DORA and TIPS. This study was completed in Year 2 with four schools in Oregon. Data document that TIPS training was functionally related improved use of data by school teams. A professional paper describing the methods and results from this analysis is currently under editorial review.

The project was funded to conduct a quasi-experimental design testing the TIPS content using the DORA observation tool. Based on the success of the first two years of the project, we have modified the group design to be a formal randomized controlled trial. The methods for the study have been revised, and 34 schools have participated in the formal analysis. Results confirm that TIPS training is functional related to improved team meeting foundations, and improved use of data for defining and solving problems related to the behavior and/or academic performance of students. The final analysis of these data, and development of the professional paper from this analysis, are currently in progress.

An important feature of this project is that progress for each objects and task is updated every three months on a web-based management site that allows all project faculty, and the IES project officer continuous access to (a) progress related to project objectives, (b) products completed and in progress, and (c) summary information related to data-sets, participating schools, and deliverables.

A summary of the expected and current status of all project Objectives and Tasks is provided online. Summary of the fiscal status of the project is provided by the University of Oregon's Office of Research Services Administration. All project objectives/tasks proposed to-date have been met or exceeded.

Funding:

U.S. Department of Education                                                                                     

Duration:

 4 Years: 07/01/2008 to 06/30/2011

Staff:

• Anne Todd
• Rob Horner

Contact: