In August 2017, FSD Africa launched a fresh drive to build measurement know-how across the FSD Network – a family of ten like-minded financial sector development programmes across sub-Saharan Africa funded by UK Aid. These projects – the development of a Value for Money (VfM) framework & the development of a compendium of indicators for financial sector development interventions – are the latest departure in a journey that began just over three years ago. This blog tells its story…
Over the last two decades, the role of Monitoring and Results Measurement[1] (MRM) in the field of international development has become more and more critical. The presence of MRM systems within organisations implementing development projects, irrespective of the strength of such systems, is now more the norm than the exception. Though MRM remains an important accountability tool, what’s more striking is the increasing role it plays in the ways in which interventions are planned, designed and implemented by development practitioners. Thus, MRM systems have many stakeholders – from funders and board members to project managers and delivery partners – and they need to be robust enough to meet their various needs.
While it’s critical to make MRM systems robust and efficient, achieving this is not an easy undertaking. It requires financial investment, commitment at CEO-level and the wider team, and continuous learning and improvement. This has very much been the case for the FSD Network.
Investing in stronger MRM systems began in earnest in July 2014, when FSD Africa began an FSD network-wide consultative process aimed at strengthening MRM processes within individual FSDs, both at project and programme level. The initiative also sought to achieve a more consistent approach to MRM across the FSD Network, one that would spur seamless cross-learning amongst its members. The result of this extensive consultation was publication of the Impact Orientated Measurement guidance paper, or IOM, which was launched in December 2015.
The launch of IOM, was an important milestone; one that was received with mixed feelings – of both excitement and anxiety. A vision of what an effective MRM system looked like was clearly established in theory, but the delivery of one in practice – across the FSD Network – quickly became FSD Africa’s challenge. This has required systematic effort over recent months since its launch.
Here’s what we did…
First, MRM diagnostic clinics were undertaken between April and July 2016, in partnership with Genesis Analytics. The objective of this exercise was to determine the readiness of FSDs to implement IOM. It involved evaluating the status of the different FSD MRM systems, as well as establishing the knowledge and attitudes of staff, senior management and governing bodies towards MRM, and more specifically IOM. Light-touch tailor-made support was also offered during these clinics, benefiting 47 FSD staff. Through this exercise, FSD Africa gained a better understanding of the unique needs of each FSD and their perceptions on how to address these challenges. By and large, it also provided an opportunity for individual FSDs to decipher IOM and define how they could start applying it.
Second, based on findings[2] from the MRM diagnostic clinics, FSD Africa worked with Adam Smith International to develop and deliver a 2.5-day training course christened Results Measurement for Systems Approaches in Financial Sector Deepening. The training, which took place in November 2016, focused on the five principles of IOM[3]. In total, 20 FSD staff drawn from FSD MRM units, intervention teams and programme management benefited.
One unexpected, but positive outcome of the MRM training was an increased appetite for FSD-specific MRM support, delivered through in-country support to staff. Two FSDs (FSD Uganda and FSD Mozambique) have so far benefited from this initiative. The results are evident – FSD Uganda now has an IOM-enabled MRM manual, and staff with a greater understanding of how to plan and execute results measurement initiatives. Its Board of Directors has a better appreciation of the intricacies of implementing and measuring results of projects that utilise the M4P approach, and are willing to support the course. FSD Mozambique has also just embarked on a process of reviewing its own results measurement guidelines and tools.
FSD Africa appreciates that, though IOM is a useful conceptual cornerstone, it is no panacea for measuring the results of complex financial sector development programmes. There is need to augment it with practical tools that can be easily deployed across the FSD Network.
In response, in August 2017, FSD Africa commissioned two other initiatives, which are being delivered in partnership with Oxford Policy Management:
- the development of a Value for Money (VfM) framework that seeks to strengthen the internal technical and operational capacity of FSDs to assess and manage their programme/projects’ VfM, and
- the development of a compendium of indicators that enables FSDs to better measure market system changes and how FSD programmes have contributed to these.
Amidst all these developments, is a thriving MRM Working group – an experience-sharing platform for MRM staff across the FSD Network. FSD Africa acts as its secretariat. The Chair rotates between individual FSD MRM leads.
Has this been an easy-to-execute task? Not at all. But, substantial progress has been made. In my next article, I will reflect on the lessons FSD Africa has learned during our work towards a harmonised MRM approach across the FSD Network.
——
[1] Also commonly referred to as Monitoring and Evaluation (M&E)
[2] A generally poor comprehension of the value of Theories of Change in effective results measurement, as well as insufficient skills in applying the same; widespread desire to understand how to effectively measure systemic change; weakness in assessing causality and building credible ‘contribution’ narratives; infrequent and meaningful engagement between some MRM units, intervention teams, and programme/organizational management.
[3] Aligning monitoring with measuring impact (the ‘sweet spot’); using the ToC as a strategic framework for planning and impact evaluations; focusing on the FSD programmes primary interest (as regards measurement), which is assessing changes in inclusive financial markets; identifying and measuring systemic change; measuring impact from the perspective of both the FSD programme (‘bottom-up’) and the sector/market system (‘top-down’)
