LESSON 6: Using MEAL Data

LESSON 6: Using MEAL Data

6.1 Introduction

You have now reached what many consider to be the most important phase of the MEAL cycle, where you have the opportunity—and obligation—to use your MEAL information to inform decisions. Indeed, a disciplined focus on the end use of the information will have informed your decisions throughout the processes described in chapters 1 to 5. MEAL data are critical for project and MEAL management and for the communication of project results to stakeholders.

In this chapter, we explore the purpose and practice of using MEAL data to inform project management and direction through a discussion of adaptive management. This chapter also includes guidance for meeting key stakeholder information needs, particularly in the areas of progress and evaluation reporting.

6.2 Adaptive management

In order to contribute to project improvements, MEAL information should be used as part of ongoing project decision-making. Adaptive management, as discussed in Chapter 4, encourages and supports this process. Effective adaptive management collects and analyzes project monitoring and feedback data to help project staff make collaborative, timely and informed decisions to ensure that project activities deliver intended impact to participants within the approved time, scope and budget.

Project managers need accurate, relevant and timely information to:

● Assess project progress.                                                                                                                                                                    ● Inform ongoing problem-solving and good management decisions.                                                                                      ● Understand the perspectives of participants and their levels of satisfaction with the project.                                          ● Address feedback raised by community members, both participants and non-participants.

Adaptive management also contributes to internal and external learning. In a project that embraces adaptive management, learning is not a parallel or standalone activity, it is a core activity that is part of project implementation, helping you make changes so that you are doing “the right things” in “the right way” for “the right people.” By embracing adaptive management, individuals and teams not only learn, but are also more accountable to stakeholders as they respond to project data and feedback.

In order to demonstrate the value of adaptive management, Figure 61 contrasts it with more traditional management approaches.

Figure 61: Traditional versus adaptive project management

A culture of adaptive management results from a series of intentional investments related to project design, staffing, budgeting, decision-making and more. A project that embraces adaptive management will respond affirmatively to the questions in Figure 62.

Figure 62: Is your project designed to promote adaptive management?

6.3 Progress reporting

You have already seen that using data is about much more than generating required reports. But high-quality, transparent reports in line with your donor or other internal and external stakeholder requirements are vital. Good reporting captures and explains both the successes and the challenges facing the project and offers evidence of robust evaluative thinking in the search for solutions.

Reporting and communication can be seen as the culmination of your data analysis process, recognizing that the ways you choose to include information in your reports is the final stage of interpretation.

The guidance below is critical to creating reports that resonate with your stakeholders and are useful.

● Consult your project communications plan and data flow map

This review will remind you of your communications audience, purpose and timing.

● Identify or develop report templates

Don’t make unnecessary effort when it comes to reporting. First find out whether your organization or project already has a report template that you can use. If you need to create a new one, ask your colleagues and stakeholders for examples they find useful that you can adapt for your purposes. Keep in mind that your organization may have internal reporting procedures that you must follow. And donors often provide a report template and schedule that you must follow.

● Identify donor reporting requirements

Donors frequently specify their required reporting template and schedule. Ensure that any templates you adapt or create also comply with these requirements. Given the importance of reports, many donors and organizations have created detailed guidance on how to create them. Check with your donor for their guidance on evaluation reporting. For example, USAID gives extensive guidance on how to prepare an evaluation report.31

6.4 Using data from the Delta River IDP Project

At the end of the second year of the Delta River IDP Project, the UNITAS team held an annual review meeting, during which they analyzed the monitoring data collected to date and reviewed the findings from the midterm evaluation conducted earlier in the year.

The data indicated that the project was underperforming in its efforts to achieve its strategic objective: “There is a reduced incidence of waterborne disease among IDPs.” While the strategic objective indicator stated that “The incidence of waterborne disease among IDPs is reduced by 30 percent by the end of Year 3,” data from the evaluation indicated that the incidence of waterborne disease had only been reduced by 5 percent.

However, while the incidence of waterborne disease was not declining as significantly as expected, several other data points painted a more positive picture of the project:

● Qualitative data collected through focus group discussions indicated that community members and other local stakeholders described the project as very valuable and resulting in positive changes for communities.                        ● Data relating to intermediate result 2 (IDPs improve their handwashing practices) indicated that while rates of adoption varied by community, some communities had made significant progress toward meeting their handwashing adoption targets.

The project donor’s annual reporting deadline was a month away and the UNITAS team was not sure how to interpret the data. Some members worried that the project was fundamentally flawed. They were concerned that either the theory of change was incorrect, or that they had missed a critical assumption. Others felt there were conflicting messages in the data, and it was too soon to assume that the project was flawed.

The UNITAS MEAL officer requested more time to further analyze the data sets.

First she analyzed the data related to reducing the incidence of disease by disaggregating the results by gender. She tried to determine whether the rates of decline differed between male- and female-headed households. But, the disaggregated data did not provide additional insights. After conducting several additional calculations, the MEAL advisor used the existing data to explore another question she wanted to test: “Did communities who adopted improved handwashing practices reduce their incidence of waterborne disease more significantly than communities that did not?”

She then disaggregated the data on the incidence of waterborne disease into two groups: “high-adoption communities” that had improved their handwashing practices, and “low-adoption communities” that had not changed their handwashing practices. Looking at the data through the perspectives of these two groups conveyed a completely new story. While the general progress in reducing waterborne disease was underperforming; reductions in waterborne disease among high-adoption communities was remarkable.

With this new data interpretation, the UNITAS team could confidently communicate two findings to the donor:

1. The ToC was not flawed, and remained relevant to the project context and problem.

2. Investment in improved handwashing practices was key to reducing the incidence of waterborne disease.

When sharing these results in the annual donor report, the UNITAS team visualized the data by creating a heat map that illustrated improved handwashing practices by village and how they related to reduced waterborne disease. The report narrative also recommended activities to improve adoption of improved handwashing practices in non-adopter communities. Suggested strategies included:

● Scheduling trainings at different times so older children and young adults could attend.                                                  ● Ensuring that local markets had enough soap available for purchase.                                                                                  ● Confirming that there was reliable access to water and soap in all latrines.                                                                          ● Conducting refresher trainings for communities with low adoption of handwashing.

In response to the report, the donor approved changes to the project plans. These included:

● Inserting new activities into the project Logframe, the budget and Gantt chart. These activities included providing refresher trainings in low-adoption communities and monitoring soap availability through market checks.                  ● Updating the PMP and the IPTT to include new analysis subgroups (high-adoption communities and low-adoption communities).

The updates to the project plan were then shared with community leaders, who were invited to provide ideas on how to make the refresher training more engaging and effective.

By carefully and diligently using the data from its MEAL system, the UNITAS team was able to make significant improvements to the project. Through a combination of insightful interpretation, data visualization and adaptive management, over the next 12 months, the project was able to increase the adoption of improved handwashing behaviors in all the IDP communities. This, in turn, resulted in the project meeting its strategic objective of reducing the incidence of waterborne disease.

On the completion of the 3-year project, the learning from the Delta River IDP Project was codified into a case study that was shared internally and published on the UNITAS blog. The UNITAS project manager also made several presentations at conferences on the relationship between handwashing practices and reduced incidence of waterborne disease.