 |
 |
 |
 |
 |
 |
 |
|
Program
Outcomes for Children
SATISFACTION
OUTCOMES
Introduction
Satisfaction surveys are a deceptively simple form of program evaluation. In the past, participant satisfaction surveys were sometimes the only form of evaluation conducted by small programs. While it is not sufficient to measure satisfaction as the only outcome of a program, satisfaction is an important starting point, because if participants are unhappy with the service provided, they will not continue to participate, and therefore they will not be able to benefit from what the program has to offer. For programs serving children, the satisfaction of parents or guardians, the morale of staff members, and the perceptions of members of the community who fund and support the program can be as important as the satisfaction of the children themselves (Ysseldyke & Thurlow, 1993).
However, it is not enough for an evaluation to simply ask participants whether they like the program. People may like or dislike a program, service, or product for very subjective reasons that have nothing to do with how effective it is in achieving its desired outcomes. For example, children in an educational program that offers little challenge might say that they like it because it makes few demands on them and serves good snacks, but we would not necessarily expect such a program to result in better academic skills for the children.
Researchers in the fields of marketing and consumer science increasingly realize that satisfaction is a complex interaction of present and past experiences, expectations, and perceptions. Besides the quality of the product or service, perceptions of satisfaction can be influenced by cost, participant needs, expectations, availability of competing services, and recommendations of family and friends (Oliver, 1997; Anton & Perkins, 1997). In the business world, customer satisfaction is “a state of mind that a customer has about a company when their expectations have been met or exceeded over the lifetime of the product or service” (Anton & Perkins, 1997, p. 3). Although the nature of services or “products” is different in community-based programs for children, youth and families, in both cases it is important to ask the right questions, and find out what particular aspects of the program are meeting expectations or need improvement. Global satisfaction questions (“Do you like this program?”) are usually less helpful than those that are more specific (“Are staff members available to answer your questions? Do they seem to have the skills and training to help your child? Do the hours of operation match the times you need services?”).
In the five-tiered model of program evaluation
which underlies the State Strengthening Evaluation Guide (Jacobs, 1988;
Callor, Betts, Carter, & Marczak, 1997), participant satisfaction
is most relevant to the activities of Tier 3, Understanding and Refining
Programs. Assessing participant satisfaction at this stage, through
surveys or interviews, can help to determine whether the program is being
implemented in the ways it was intended, and whether adjustments and fine-tuning
are needed which would improve the effectiveness of the program.
Often fairly
simple changes can improve the fit between
the program services and the needs of participants.
Outcome Components
The NCEO model, as adapted for community-based
settings, suggests three desirable outcomes in the domain of satisfaction
with the program:
- Parent/guardian satisfaction with the program services that children receive
- Community satisfaction with the program services that children receive
- Child satisfaction with the program experience
The following are some appropriate indicators
of positive outcomes for children in the area of Satisfaction, based on
the NCEO model (Ysseldyke & Thurlow, 1993), as adapted for community-based
programs by the Children’s Outcome Workgroup. The appropriateness
of any given indicator for your program evaluation depends on the ages
of the children you serve, the setting, and the goals and activities of
your particular program.
- Percent of parents/guardians who understand program services and rate them as effective, efficient, coordinated, and responsive in meeting their child’s needs
- Percent of parents/guardians who understand program services and rate them as effective, efficient, coordinated, and responsive in meeting their family’s needs
- Percent of parents/guardians who are satisfied with their own level of involvement in program decision making
- Percent of program staff who understand program and rate it as effective, efficient, coordinated, and responsive in meeting child and family needs
- Percent of program staff who are satisfied with their own level of involvement with program decision making and delivery of services
- Percent of community stakeholders who understand the program and rate it as effective, efficient, coordinated, and responsive in meeting child and family needs
- Percent of community stakeholders who are satisfied with their own level of involvement with program decision making and delivery of services
- Percent of children who enjoy their participation in the program setting
Assessing participant satisfaction is a
necessary but not sufficient part of evaluating community-based programs
such as State Strengthening projects. Surveys or interviews with
parents, guardians, staff, children, and other stakeholders are most useful
if they are brief, tailored to your specific program, and focused on aspects
of the program that are in your power to change or improve.
References
Anton, J., & Perkins, D. (1997). Listening to the voice of the customer: 16 steps to a successful customer satisfaction measurement program. New York: The Customer Service Group.
Callor, S., Betts, S., Carter, R., & Marczak, M. (1997). State Strengthening Evaluation Guide. Tucson, AZ: University of Arizona, Institute for Children, Youth & Families and USDA/CSREES.
Jacobs, F. H. (1988). The five-tiered approach to evaluation: Context and implementation. In. H. B. Weiss & F. H. Jacobs (Eds.), Evaluating family programs (pp. 37-71). New York: Aldine DeGruyter.
Oliver, R. L. (1997). Satisfaction: A behavioral perspective on the consumer. New York: McGraw-Hill.
Ysseldyke, J. E., & Thurlow, M.
(1993, October). Developing a model of educational outcomes (NCEO
Report No. 1). Minneapolis, MN: University of Minnesota, College
of Education, National Center on Educational Outcomes.
Measures |
|
| Program Outcomes for Children | |
|