7. Consistently Evaluating Outcomes |
|
Benchmark #7: Consistently Evaluating Outcomes What Is—And Isn't—Evaluation?  Evaluation isn’t collecting data—although you certainly need data to do it.Evaluation isn’t promoting your program to management—although you may be able to use your evaluation results to do so. Evaluation is determining the value of what you’ve done. It’s figuring out what the data means in relation to your objectives. Did you get the desired outcome? Was the result worth the expense? It takes thick skin to evaluate honestly. You have to be prepared to reach the conclusion that some interventions didn’t work well. You may find that a popular program costs too much and didn’t really affect employees’ health. But what’s the alternative? Without that information, you might continue ineffective interventions. With it, you can look for better solutions. And when your results are good, it’s glorious. You can spread the word to employees and management that your program is achieving its goals. What are the benefits of evaluating outcomes? 1. To see if your intervention worked Did you achieve your objective—whatever it was? Did employees learn about their health risks, get stronger or more flexible, or use the stairs more often? 2. To demonstrate the cost benefit of the intervention To get the money to repeat a program, you need to be able to show that it was effective enough to justify its expense. That’s what cost benefit means — not that the intervention saved money, but that the benefit was worth the cost. 3. To compare different types of interventions You may have tried three different approaches to increasing employees’ activity levels. With evaluation of your data, you can see which approach was most effective. You can also compare your outcome with industry standards. For instance, workplace smoking cessation programs are considered very successful if 35% to 50% of participants quit. How did your program compare? 4. To provide information about the program You will produce valuable information about your program through evaluation that you can use in reports and presentations to management, press releases, stockholder's meetings, company newsletters and meetings. 5. To give feedback to participants (and inspire others) We’ve all learned that the first rule in setting goals is to have measurable objectives. By publicizing results of evaluation, you can boost participation and show your interventions work. What are you looking for? There are endless variables that you could measure and evaluate. Narrow the search by answering a few basic questions: What results do senior managers want from the health promotion program? Do they see the primary mission of the program as health risk reduction — or presenting a caring image to employees? Are they looking for decreased health care utilization—or a recruitment tool in today’s competitive labor market? It's essential to evaluate what management wants—if you don’t know, ask them in interviews or by survey. What do the employees want? Look back at employee interest surveys and evaluate whether you gave them what they asked for, and how satisfied they were. What did the Wellness Committee or coordinator want to see? Your program goals and objectives are the true North on your evaluation compass. Did you achieve what you intended? Prepare to Evaluate Health Promotion Market Testing Before you begin, consider running a pilot program and evaluating it first. You might offer it to just one department and analyze the results, or do a smaller intervention to see how many participants you get. Before you begin evaluation, you want to have your ducks lined up. You’ll need: 1. Clear and measurable goals and objectives What did you want to achieve? Increased knowledge, behavior change, decreased cholesterol levels, higher morale? You’ve got to know where your target is before you can count how many arrows hit it. 2. Baseline data You’ve got to have “before” data to compare to after your intervention. 3. Staff who are trained to evaluate appropriately Evaluators must understand the evaluation tool and how to use it. From weigh-ins to statistical analysis, there are correct methods that should be used consistently. 4. Incentive programs and rules are understood and fairly administered Oh boy—the last thing you need is employees claiming that incentives were given out unfairly, or that they were misled about what they needed to do to qualify. Be sure that incentive programs are crystal clear and evaluated according to the rules. 5. Programs are well-planned with evaluation tools in place Everyone involved should understand how programs will be implemented and evaluated. For instance, when will data be collected? Besides baseline (before intervention) data, will you measure progress at the mid-point? Of course you’ll collect data at the end of the intervention, but how about three months later to see if the benefit lasts? 6. Budget for intervention is set, actual cost data is collected To evaluate cost benefit, you have to know the budget allocated for the intervention and the actual costs. 7. A communications plan is established Who do you plan to share your evaluation results with—and how? It’s always important to have the recipients in mind so you can tailor your evaluation to them. What to Evaluate Here’s the heart of the matter. What exactly do you measure and evaluate? Your choices will depend on the goals and objectives of management, the employees, and your wellness team. Here are some of the common targets of evaluation: The Intervention Knowledge and skills Before and after data can demonstrate that participants learned or developed skills as a result of your intervention. Reduced risk factors If you implement a comprehensive program aimed at reducing specific risk factors among employees, an HRA (or a specific questionnaire) could show that you succeeded. Participant satisfaction Participant evaluations indicate how likely employees are to spread the word and come back for more. Participation rates The number and percentage of employees who participate can demonstrate employee interest and how well your program is being promoted and supported. You can compare participation rates of different interventions as well. Effects on Corporate Costs Broader goals of health promotion programs may include helping to contain or even reduce costs. These goals usually take a longer time to impact. Health care claims It can be challenging to get and analyze health care claims information, but by working with providers you may be able to detect significant trends. Lost work days/ Workers compensation claims Safety is an important component of health promotion, and successful interventions can reduce injuries and disability claims significantly. Absenteeism rates Happy, healthy workers who are informed about self-care are absent less often. Programs that help provide emergency child and elder care can be evaluated by their effect on absenteeism, too. Productivity Many health promotion interventions address causes of “presenteeism” (employees who are present in body but aren’t productive). Depression, stress, financial and health problems are all known to decrease productivity. Effective programs may positively impact productivity. Was Corporate Culture Improved? Workplace environment An environmental audit can be repeated after making improvements in the physical environment, cafeteria choices, exercise opportunities, or safety hazards. Policy changes Changed policies that impact mental or physical health, career development, job fit and satisfaction, and work/life issues are appropriate subjects of evaluation. Perception of health promotion program Surveys that show how management and/or employees see the wellness program are an important indicator of the health and future of your overall program. Turnover and ability to attract new employees In this tight labor market, employees vote with their feet. A comprehensive wellness program can make an organization a more desirable place to work. AND NOW... Take a Deep Breath! Evaluation isn’t as difficult as it seems—if you’ve done the work up front. The success of this last of the 7 C’s of successful worksite wellness programs depends in large measure on the work you’ve done before. If you’ve got management support and a good wellness team, collected baseline data, established measurable goals and objectives, and implemented your interventions according to plan—then evaluating your program will be a natural conclusion to the cycle. Best of all, what you learn through evaluation will make your next trip through the 7 C’s much more efficient and effective. As you go around again, you'll be more adept at anticipating future needs, and you’ll pave your own road to wellness. References Chapman, Larry S. Program Evaluation: A Key to Wellness Program Survival, 1996 McKenzie J. and Smeltzer J. Planning, Implementing, and Evaluating Health Promotion Programs: A Primer, 2nd Edition Allyn and Bacon, 1997 Schaloc, R. Outcome-Based Evaluation Plenum Press, 1995 Health Promotion: Sourcebook for Small Businesses, published by the Wellness Councils of America and Canada. Call (402) 827-3590 to order. |
