Healthy food access initiatives can impact community health by increasing fruit and vegetable consumption and improving nutrition, promoting healthy shopping and cooking, promoting healthy body weight, and preventing or managing diabetes and other diet-sensitive health conditions. Healthy food access initiatives can also have valuable co-benefits of supporting local food systems and promoting local economic development, vocational skills, and job creation in vulnerable communities.

Evaluating community health interventions is vital in order to: 

• refine and improve program implementation
• identify the most effective interventions for expansion or replication
• provide evidence for funders to maintain financial support of programs 
• communicate the impact of community benefit initiatives to stakeholders and the wider community
• complete community benefit reporting requirements¹ 

This guidance brief provides recommendations and resources for evaluating healthy food access interventions that hospitals support as part of their community benefit implementation strategy to address health needs in their communities. Evaluation is a vital component of community health improvement programs, requires a significant commitment of budget and program resources, and should be planned at the same time as programs are developed.

Engaging community partners and stakeholders

There are many community organizations and stakeholders that bring unique value, skills, and experience to the evaluation process. Community partnerships can be formed across all stages of evaluation including evaluation design, data collection, data analysis, report writing, and the dissemination of findings. Relevant stakeholders may be diverse and can include community-based organizations, local public health and other government agencies, community health and development coalitions, universities, farmers, local businesses, and external contractors. 

• Public health departments are interested in initiatives that contribute to positive community health outcomes, and it is likely that health department staff have received some training in research methodology. Health department staff may be able to assist with evaluation design and data collection and analysis. Public health departments also have access to state and local data that can serve as the basis for selected evaluation measures.
• Public health, medical, and nursing school students and faculty may also be able to assist with evaluation design and data collection and analysis as an extension of their learning or part of their professional development. 
• The U.S. Department of Agriculture (USDA) National Institute of Food and Agriculture (NIFA) administers the Food Insecurity Nutrition Incentive (FINI) grant program and develops the criteria awardees must adhere to when implementing their programs. The institute requires grant recipients to conduct a program self-assessment and also participate in the evaluation activities conducted by Westat, the external evaluator for the FINI grant program. Each year NIFA organizes and records a webinar outlining the evaluation expectations of FINI grant recipients. 
• Fair Food Network and Wholesome Wave provide resources, tools, and offer technical assistance for evaluating fruit and vegetable prescription incentive programs.

Find your baseline

Determine the baseline for program participants to better understand your target population and assess the impact of your intervention. Biophysical health indicators, such as blood glucose (hemoglobin A1c) levels, blood cholesterol levels, blood pressure, and body mass index (BMI), may be collected from participants before the program begins. Participants may be asked about their medical history to identify previous diagnoses of diet-related health conditions. (Note that evaluation and data collection plans must address privacy concerns and protected health information requirements, including obtaining consent from program participants.) Pre-intervention surveys may collect information regarding nutrition knowledge, food attitudes and beliefs, cooking and shopping skills, and food behaviors. Community health information from public health departments or local organizations can also help establish reference data.

Determine your evaluation objectives and questions

The evaluation questions that will be asked and the indicators that will be measured should be tailored to the specific initiative. In determining these questions and indicators, it is valuable to first clarify the program goals. It is critically important that the evaluation questions and indicators align with organizational strategies and program goals in order to highlight successes and identify areas for improvement. Having a few well agreed upon measures to evaluate may be better than a long list. It is also valuable to align your evaluation framework and indicators with those that are widely used to evaluate similar programs in order to be able to compare results.

Start by describing your initiative, including both basic program elements and activities and the goals and outcomes you expect to achieve. 

Create a logic model to clarify and illustrate how your intervention works: the resources utilized (inputs), the activities you will undertake, and the expected short-, medium-, and longer-term outcomes (impacts) of your program activities. 

The evaluation should assess how well or to what degree the program’s activities contribute to meeting program goals and producing desired outcomes. This can be measured by developing evaluation objectives and questions relating to both program implementation (process) and program impact (outcomes). Your logic model can capture the primary and additional goals targeted, relevant program activities, and the objectives that will be assessed in the evaluation process.

Implementation questions and objectives for healthy food access programs

Consider the ‘who,” “what,” “when,” and “how” of program activities and participation in order to improve program design and implementation.²

Example implementation questions and objectives

Impact/outcome questions and objectives

Obtain information on the impact of the program on the targeted audiences. Questions should consider shorter- and longer-term outcomes of the initiative. 
 

Examples of short- and medium-term outcomes

Examples of longer-term outcomes

Recommended indicators and data collection tools for healthy food access initiatives

Evaluation objectives and questions should inform which indicators and metrics will be collected in the evaluation process. Obtaining relevant data is critical for drawing valid conclusions and building continued support for the intervention. It is recommended that program staff collect biometric health data, knowledge and behavior change data, and, if relevant, data on policy, systems, and environmental change. 

Knowledge and behavior metrics are valuable for demonstrating short- and medium-term impacts, and knowledge and behavior change outcomes can be clearly linked to program activities. Health outcomes often require longer than the intervention time period to achieve and are likely to be influenced by variables outside the scope of the intervention.

Ideally, data should be collected pre-intervention, mid-intervention, and post-intervention. 

• Pre-intervention information will allow program staff to collect baseline data against which progress can be measured. 
• Collection of mid-stream data enables assessment of activities and individual aspects of the program that may be refined to maximize objectives. 
• Post-intervention data, including 6+ months after program participation, demonstrates longer-term program impacts and sustained change. 

While surveys may be tailored to the specific goals of the initiative, employing widely used and validated data collection instruments facilitates data comparison. See the following evaluation tools and instruments for examples.

Communicate results

It is important to communicate and share evaluation results with various stakeholders to demonstrate success and build support for program continuation and replication. 

Stakeholders may include program partners, funders, hospital staff, public health agencies, nonprofit organizations working towards similar objectives, program participants, and the community at large. 

Communicating results to hospital staff may help individuals feel more pride in their organization and can help improve employee satisfaction and retention.

It is also very useful to share lessons learned from unexpected or disappointing evaluation results. A key value of program evaluation is learning from mistakes or ineffective strategies in order to adapt and improve programs.

Ideally, data should be collected continuously in order to strengthen the support and evidence of effectiveness. This is particularly useful for current and future funders who look for consistent and sustained results when determining which projects they want to support. Evaluation reports may be posted to the public online, shared at community meetings and relevant conferences, and delivered to key stakeholders. 

Example healthy food access program evaluation strategies

Oak Grove Community Fruit and Vegetable Prescription Program

• Program inputs/activities: 
  • Oak Grove Hospital contributes to funding the fruit and vegetable incentive (FVRx) vouchers and partners with several community organizations to implement the initiative. 
  • Hospital clinicians conduct food insecurity screenings at every patient visit in the emergency department and enroll patients experiencing food insecurity into the fruit and vegetable incentive program. Enrollment includes prescribing fruit and vegetable vouchers at a value of $7 per family member per week (up to $28 per week). 
  • Participants’ coupons are replenished biweekly at nutrition education/healthy eating classes. The classes begin with instruction from a dietetics (RD) student and continue with group cohort activities and discussions. 
  • Oak Grove Hospital partners with a community center to host the six-session nutrition education and healthy eating course. The community center is located in a low-income neighborhood where some program participants live and is also in close proximity to public transportation for those who do not live nearby. 
  • Ten local farmers markets and retail food outlets agree to accept both FVRx coupons administered by the program and federal Supplemental Nutrition Assistance Program (SNAP)/Women, Infants, and Children Food and Nutrition Services (WIC) benefits. 
• Length of program:
  • The program runs for three months spanning the growing season from June to August. 
• Primary goals and anticipated outcomes:
  • Reduce economic barriers to healthy foods 
  • Improve nutrition knowledge and dietary patterns of participants
  • Reduce prevalence of obesity and diet-related health conditions
  • Increase the number of community food outlets that accept SNAP/WIC
  • Increase local food producers’ and vendors’ sales to strengthen the local economy
• Data Collection: 
  • Attendance is taken at each food and nutrition class to track active participation
  • Program staff modified a widely-used survey that included questions on diet and nutrition knowledge, fruit and vegetable consumption, attitudes towards trying new fruits and vegetables, satisfaction with various aspects of the program design, and health care utilization related to diet-related health conditions. This survey was distributed at the beginning of the first healthy eating class, again at the fifth session, and once again at the last session. 
  • A clinician from Oak Grove Hospital was present at the first and last sessions to administer surveys and collect biophysical health indicators including BMI, blood cholesterol levels, and blood glucose (A1c) levels. 
  • Participating vendors, including farmers market vendors and grocery store managers, collected information on FVRx coupon redemption and increased sales. 

Washington County Food Pantry Program

• Program inputs/activities: Washington Community Hospital collaborates with a group of local organizations in Washington County to help four local food pantries improve their offerings of fresh, healthy, and local food.
  • Washington Community Hospital 
    • Plants extra garden beds in its large hospital garden to donate fresh produce to the food pantries.
    • Commits staff time each month to participate in gleaning efforts at local farms.
    • Provided a community benefit grant to purchase refrigerators to store fresh food.
    • Partners with the local public health department to conduct program evaluation.
  • Three local farms allow volunteers to collect surplus and unmarketable crops after the initial harvest.
  • A local food rescue organization collects surplus/unsold foods from participating grocers and restaurants and delivers to local food pantries. 
  • Faculty and students at a culinary school offer nutrition education and healthy cooking classes at each food pantry location to complement food pantry activities on Sundays twice a month to increase knowledge and encourage sustained behavior change as new, healthy foods are introduced. 
• Length of program:
  • The program runs year round. The food rescue organization has a larger role in the winter months when farmers are unable to grow crops outdoors. 
• Primary goals and anticipated outcomes:
  • Increase healthy food access for vulnerable populations by improving healthy food procurement in local food pantries
  • Improve dietary patterns of food pantry clients
  • Reduce food waste 
• Data collection: 
  • The quantity of produce harvested, rescued, and donated is weighed and recorded.
  • Each food pantry records the quantity and types of fresh produce clients receive. 
  • Washington Community Hospital partners with the local public health department to evaluate the initiative’s impacts.
    • A hospital clinician attends each nutrition education and healthy cooking class and recruits food pantry clients to participate in an assessment study that will take place over the course of one year. 
    • Participants agree to attend three nutrition education and healthy cooking classes over the course of three months, in conjunction with utilizing the pantry’s food resources, and to share biophysical health indicators collected by the hospital clinician (BMI, A1c, blood pressure, and blood cholesterol), and complete a pre- and post-participation survey. 
    • Local public health evaluators modified a widely-used survey that included questions on diet and nutrition knowledge, fruit and vegetable consumption, attitudes towards trying new fruits and vegetables, utilization of food pantry resources, obstacles to healthy eating, and health care utilization related to diet-related health conditions

Learn more

Identifying measurable goals and creating logic models

• Pell Institute developed an evaluation toolkit with sections that include step-by-step guidance for creating logic models and recommendations for how to best use logic models. 
• Minnesota Department of Public Health offers guidance for developing SMART objectives and also provides links to additional resources, course, and online training. 
• The Centers for Disease Control and Prevention (CDC) developed a resource for Developing Program Goals and Measurable Objectives. 
• Health COMpass, a project of the Health Communication Capacity Collaborative, developed a how-to guide for developing logic models including templates, samples, and links to additional resources. 

Additional resources for determining evaluation questions and objectives

• The Catholic Health Association (CHA) developed a resource for Evaluating Your Community Benefit Impact, which is free for members and available for purchase for nonmembers.
• Kaiser Permanente, the Center for Community Health and Evaluation (CCHE), and the Nutrition Policy Institute at the University of California developed a toolkit for utilizing the “dose” framework when planning and evaluating community health promotion strategies. Dose is a measure of both the reach and strength of an initiative on its target population. 
• Cottage Health developed an evaluation toolkit, adapted from CCHE’s resource, which includes blank worksheets for each step and relevant case studies. 
• The University of Kansas Center for Community Health and Development published an online Community Tool Box for professionals working to improve community health and initiate social change. The resource features multiple toolkits including one on “evaluating the initiative.” 
• The RE-AIM framework includes 5 different dimensions (Reach, Effectiveness, Adoption, Implementation, & Maintenance) from which program questions and indicators can be developed. RE-AIM has a variety of digital resources including a dimensions items checklist and an example scoring of a farmers market program. 
• The Supplemental Nutrition Assistance Program Education (SNAP-ED) Evaluation Framework and Interpretative Guide provides recommendations and general guidance for various indicators, outcome measures, and evaluation methodologies relating to nutrition, physical activity, and obesity. 
• The Center for Training and Research Translation (Center TRT) developed an evaluation framework for nutrition and obesity initiatives and offers recommendations for developing evaluation questions, sample evaluation plans, logic models, and additional resources. 
• The National Collaborative on Childhood Obesity Research (NCCOR) developed a Measures Registry User Guide: Food Environment to assist users in selecting and utilizing measurement tools specific to the food environment. 

Additional resources for general and food-related evaluation design and guidance

• Project Star, the training and technical assistance provider for the Corporation for National and Community Service, offers guidance for program and evaluation design. 
• The Economic Development Research Group and a member of the Economics Department at Northwestern University developed a report outlining best practices when assessing economic impacts of an initiative. 
• The National Farm to School Network developed a cross-sectoral evaluation framework for farm-to-school programs including recommendations of common outcomes to be measured. Additionally, the Center for Food & Justice Urban & Environmental Policy Institute at Occidental college offers guidance on evaluating farm-to-school programs. 
• The National Science Foundation offers recommendations for addressing cultural considerations when evaluating program activities and outcomes.