Plan and Execution

The evaluation plan focuses on providing information about the implementation of the program during the grant period and the outcomes of project participants. This information will be used to inform ongoing project development and to determine whether the project is meeting its stated objectives. Through a combination of qualitative and quantitative approaches, the evaluation team addresses the extent to which the preceding objectives have been attained. 

Products to be generated by the evaluation team include: reports to project management staff regarding program progress and data findings; a final evaluation report to the program describing the findings. Based on information concerning the development of the project, the evaluation team will: describe the process of implementing the project; identify critical issues during implementation; describe in detail the critical features of the program; and discuss what was learned during the program. 

The evaluation has three components:

Performance Monitoring System -  The performance monitoring system allows
a assessment of the extent to which the project has been implemented as planned and whether staff, service provider and participant activities are in line with those intended. During this phase, the evaluator and key staff met to refine the plan and define performance standards for each primary objective. Data supporting this phase comes primarily from a client tracking system and process evaluation to detect the extent to which the program is functioning as planned. Based on this plan, brief management reports can be generated quarterly during the life of the project. These reports are distributed to key project staff and in quarterly meetings to discuss implications and recommendations based on the findings. 

Qualitative and Quantitative Analysis - We will gather descriptive information 
on the development of the project. Using interviews, observations and surveys we produce descriptive information on proposed versus actual implementation of the project, critical issues encountered by service providers as they attempt to implement the project, staff activities, descriptions of participants, and lessons learned from the project. For example, qualitative data may point to certain hypotheses about the extent to which program participants' attitudes and behavior in the classroom improves. In addition, if observational and interview data point to problems with any specific aspects of the project's implementation, then staff can make any necessary adjustments and continue to observe the results of the adjustment. Furthermore, documenting project implementation, adjustments along the way, and lessons learned provides a rich resource of information for others who may want to replicate this project. Data collection formats may include (1) a review of narrative accounts of randomly selected teachers, student and staffs' involvement with the project, (2) standardized open-ended interviews or surveys of program participants, administrators and service providers and (3) field observations and (4) surveys for all program participants. 

Outcome Evaluation - The outcome evaluation utilizes existing data sources 
to obtain a great deal of participant information. Thus, participant demographics, services received while in the program, length of time in program, participant test scores, etc. will be collected. In addition, we will collect supplementary data regarding participant perceptions of the program. During the initial and follow-up evaluation interviews, a battery of instruments is and will be administered by the evaluation team to each program participant.

Evaluating the Success of the Project in Meeting Its Goals

The evaluation attempts to directly address the extent to which each of the Program's goals and objectives has been met. Some of these goals and objectives fall under the scope of process evaluation; that is, determining if, in fact, the program is successfully reaching and recruiting current or potential program participants and providing the type and quantity of services proposed. Goals and objectives that are concerned with the impact or outcome providing enhanced service delivery will require the collection of a fair amount of additional data, including baseline measurements of participants and follow-up measures of each participant to assess changes attributable to the program.

The Evaluation Team