KEYWORDS: objectives, evaluation models, measurement, and comparison

The planning process remains incomplete until results or impacts of our efforts are evaluated. Unfortunately, the importance of this particular procedure is often underestimated. By evaluating the results of past goals or activities, you will be in a more favorable position to gain and share invaluable experiences. Plus you will help improve not only the planning process, but ultimately your personal skills as an evaluator and program planner.

The Evaluation Process in Brief Overview

Because we, as humans, are already unconsciously involved in the process of evaluation, the more formal process does not have to be one of intimidation, but rather, one of encouragement. As we walk the cafeteria line for example, we actually evaluate the service, as we decide in our minds whether or not the salad measures up to our expectation, or a personal standard or taste bud. The color of the lettuce, the number of tomatoes, and other items to add will enter our mind. Based on our observation and on what it is that we have in mind as a criterion we then make a judgment on its quality, on its worth. Similarly, you may decide whether or not to purchase a pair of dress shoes based on a personal preference or perhaps on information that you may have had the good fortune of gaining relative to this type of shoe. All are examples of evaluation. In concept, a more formal type of value-added evaluation plan for example, may be very similar.

From a rather simplistic point of view, we can look at a more formal type of evaluation process utilizing two primary elements, that of 1) Measurement-Description, and 2) Comparison-Judgment. Graphically, this simple but effective conceptual framework may be depicted as follows:

In a nut shell, measurement is the process of determining or describing status and amount. Numerically based, it is a basis for determining whether or not there has been a change in those variables that you may have manipulated or in the education that you have provided. In more practical terms, you may decide whether or not the educational provided activities caused the change that was expected, that was desired, in the form of perhaps an objective or standard. Exemplifying the point, an appropriate question might be, did we improve the standard of living (in terms of a measurable standard, e.g. income) of a limited resource family by facilitating the adoption of a certain innovative idea. Once again, in the measuring process, we describe what has actually happened in quantifiable terms.

Comparison is an important component as it allows us to determine what is (at the end of your program or some defined time line) with what was (at the beginning of the program, before instruction or some form of remedy was provided). Similarly, the idea of an evaluation model may be to determine what is (at the end of the program) with what should be as established by our plans, by our expectations, by our original objective. The key to the comparison state is that one compares progress (what actually happened) to a standard that you as an agent of change had originally established. Utilizing our limited resource farmer example, did we improve their income by ten percent as we set out to do? To what extent were the objectives met? It is judgment by the evaluator that becomes very important at this point. It involves a decision(s) relative to the extent that the differences between the comparisons are important. Judgment may involve our own experiences, knowledge that we accumulated about our subject, and the ability to see relationships objectively. Ultimately, we decide whether a course of action led to an outcome that was useful to an individual, a group of people, or those that we originally established to help out, perhaps even a community.

Evaluation Models

Although there are various models utilized to evaluate educational programs, a basic model that has been extensively used by Extension practitioners revolves around Tyler's (1949) model. Realizing that all models for evaluation may not take the same approach, Tyler's model defines the process of evaluation as the extent that your objectives have been realized. The sequence of steps for his "Objectives-based Evaluation" model may be portrayed as follows:

In this particular model, establishing objectives (e.g., getting five limited resource farmers to adopt an innovative idea or increasing the income level of limited resource farmers) becomes a priority. Next, the gauge, standards, or criteria that are used in making judgments are established. The evaluator subsequently decides on the specific evidence that expectations have been met. This data is then used in making comparisons and consequently, final judgments.

Useful Tips

Among the various clues to bear in mind as you evaluate the impact of a value-added effort are:

* TIMETABLE—A timetable helps determine whether or not implementation is on schedule. It also helps identify what tasks are on schedule in comparison to those that are not.

* RECORDS—A good record keeping system enables you to monitor the impact of the plan and to make necessary adjustments to ensure success. As you study past results and compare them with pre-selected guidelines, any changes from expected results can provide an early warning system for potential problems. In order for records to be useful, they must be properly summarized and analyzed and thoroughly studied.

* EXTERNAL TRENDS AND EVENTS—You should also monitor trends and events that may provide additional opportunities or pose threats to your plan.

* HAVE REGULAR CHECKUPS—In most cases, it is not possible, at the onset, to completely design an evaluation plan that will do exactly what you want it to do. More realistically, such a plan unfolds as succeeding phases of the evaluation process is carried out. Continuous revision of plans is normally done as you get into each step. This need however, is not meant to underestimate the importance of a master plan that will provide initial and overall guidance during the process.

While evaluating and monitoring the entire project does not guarantee success, potential problems may be averted by using a formalized method of evaluation. Additionally, such a practice provides an opportunity to make adjustments before major problems arise.

Internet Resources