Even though the ANDE 2022 Metrics from the Ground Up Conference happened in November last year, our knowledge-sharing does not stop there. As we reflect on what we learned and reminisce on the memories that we made during our time in Bangkok, the ANDE Team has put together a list of three key takeaways from each plenary panel session, workshop and break-out session. We thoroughly enjoyed discussing all things Impact Measurement and Management (IMM) and have appreciated your great contributions of relevant knowledge and the willingness to share your professional experience.

The slide-decks have been sent out to all conference participants via email and all ANDE members will have access to them at the bottom of the “Looking Back on the 2022 Metrics from the Ground Up Conference Page” when logged in to our website. If you have any questions regarding event resources, please contact Jenny.

Opening Plenary Panel Session: Diversity, Equity and Inclusion (DEI) in Impact Measurement

Panelists: Mallory, St. Claire (ANDE), Afua Sarkodie (Dalberg), Nivisha Shah (New Energy Nexus), Rowena Reyes (Sweef Capital), Jenaan Lilani (Villgro Foundation) 

3 Key Takeaways:

• A challenge of implementing a gender lens and DEI approach into an organization is showing entrepreneurs the value of doing so when measurement is very costly. 
• In terms of using data on women entrepreneurs to diagnose gaps or inequalities, qualitative and quantitative data go hand in hand and serve different purposes. Qualitative data allows us to tell stories. The quantitative data helps with creating informed decision making. Using both data types is critical in an effective IMM system.
• With regards to the shift of IMM to remote data collection during the COVID pandemic, the panelists agree that digital platforms are not built to create inclusive systems. These platforms must be accessible to underrepresented communities, otherwise they may exacerbate existing inequalities. 

Workshops

Farmer-First Climate Metrics

Speakers: Yuan Zhou (The Syngenta Foundation for Sustainable Agriculture) and Niranjan Suresh (60 Decibels)

3 Key Takeaways:

• Climate-resilient agriculture aims to reduce the impact of agriculture on climate change while also increasing adaptive capacity and resilience of farmers to adapt to a changing climate.
• Key indicators for measuring climate smart resilient agriculture can be sorted into categories of resilience, mitigation, and profitability. They include measuring the annual crop yield stability, input use efficiency, and net farm profitability per hectacre.
• IMM practitioners can also use measures drawn from their farmers’ lived experiences, such as a farmers’ access to emergency funds, products and services, or knowledge and application of improved climate risk practices.

Storytelling as a Key Component of Impact Measurement and Management Framework

Speaker: Khutjo Langa (Entrepreneurship to the Point) 

3 Key Takeaways:

• Impact storytelling is a useful way to systematically document anecdotal evidence that expected activities occurred, and the perceived results thereof. 
• The impact storytelling framework has eight steps, including selecting the metrics you want to communicate to your stakeholder audience and sharing your “breakthrough” story–i.e., how you have impacted and created change. 
• Impact stories should unpack the change that has occurred – describe the status quo before the program intervention, highlight the first step of change, and discuss the “breakthrough,” which is where an intervention has helped the beneficiary overcome obstacles.  

Participatory Evaluation Design

Speaker: Erin Mercado (iDE)

3 Key Takeaways:

• Participatory evaluation is an approach to evaluation design that aims to redefine the process of who initiates and implements evaluation.  This evaluation style makes  local knowledge the foundation of the evaluation design. 
• IMM practitioners should set goals and build a culture of learning within their organizations by creating a community of practice to learn from each other – this can also help foster the uptake of participatory evaluation. 
• Power dynamics should be recognized when implementing participatory evaluation – power is everywhere at different levels and will be present even in a participatory session. Power is not good or bad, but it is dynamic and multifaceted. 

Outcome Mapping for Ecosystem Builders

Speaker: Abigail Perriman (Moonshot Global)

3 Key Takeaways:

• Outcome mapping measures change in systems, recognizes the power and limitations of influence and supports collaboration rather than isolation. It is a useful tool for measuring systemic or complex change. 
• Outcome mapping is focused on contribution rather than attribution.  Outcomes are highlighted rather than overall impacts, and these outcomes are defined specifically as changes in behavior from relevant system actors.  
• Using an international design process can help us implement outcome mapping. To utilize this process, the first steps include identifying boundary partners, defining the desired behavior change, and mapping strategies and practices to achieve this behavior change.

Measuring Diversity Beyond Gender

Speakers: Aneri Pradhan and Nivisha Shah (New Energy Nexus) 

3 Key Takeaways:

• Diversity is focusing on the representation of communities. Inclusion focuses on fostering a sense of belonging for communities.
• Measuring DEI beyond gender requires contextual and localized information.  There is no “second” global DEI indicator; rather, diversity should be defined in the local context.
• To measure the success of interventions in supporting underrepresented communities,  IMM specialists will need to consider the short-, medium- and long-term impact of their programs.

Culturally Responsive Evaluation

Speakers: Kadambari Anantram and Akashi Kaul (Sambodhi)

3 Key Takeaways:

• Culturally responsive evaluation (CRE) centers local culture in evaluation to align the evaluation practice with lived experiences of stakeholders with diverse backgrounds
• CRE emphasizes the importance of shared lived experience, names and pays attention to power differentials, and includes a comprehensive contextual analysis that includes social capital and civic capacity as considerations.
• Evaluators should be clear about their stance or positionality when evaluating.  Evaluators themselves bring cultural considerations into the process – therefore nothing is truly neutral or objective in evaluation

Break-out Sessions

The 4P Impact Framework

Speaker: Rohitt Malhotra (Aspire Impact)

3 Key Takeaways:

• There are many IMM frameworks on the market today, which causes issues of standardization, consistency and comparability issues.
• ESG rating frameworks are also misleading – comparative results between leading ESG rating firms clearly disclose lack of consistency between different rating firms.  
• Aspire Impact has built a proprietary 4P methodology to assist corporations, funds and non-profits in evaluating their impact performance and chart their future impact journey. The 4Ps are: Product, People, Planet, and Policy and Governance. The standards are mapped to existing IMM frameworks, including the Sustainable Development Goals (SDGs)  

Challenges in Impact Measurement for Start-ups

Speakers: Meera Siva and Ashwin Rao (Habitat for Humanity International)

3 Key Takeaways:

• As startups evolve their businesses, growth often impacts the metrics being captured. It becomes difficult to analyze trends or patterns in information because of these breakdowns in measurement. 
• Startups are business-oriented and profit-driven. They often fail to capture the effect of external factors on impact measurement.  Due to this, sometimes, results measurement is not built up or altered to absorb the effects associated with these changes.

• While startups chase valuations and profits, it is also imperative to tell the story of the organization in a compelling manner that is rooted in evidence and measurement. Investors  too should build narratives based on metrics from their portfolio.

Lean Experimentation, Replications, and Spin-offs

Speakers: Toshihiro Nakamura, Nanda Riska (Kopernik) and Reisa Putri (Perfect Fit)

3 Key Takeaways:

• Lean experimentation requires a short timeframe and a smaller sample size than a rigorous experiment. 
• The benefits of lean experimentation include learning what solutions may be promising before investing time and funding into expanding a program.
• Following successful lean experimentation, solutions should be implemented at a larger scale to collect more data. This can include comparing different prototypes or introducing the solution to different contexts.  

Bottom-up IMM for SGBs

Speakers: Morchan Karthick and Abishek Gupta (LEAD at Krea University)

3 Key Takeaways:

• Bottom-up impact measurement centers the SGB and business owner(s).  Utilizing measurement that centers the business itself allows for measurement that more accurately captures the impact of business success on household finances.  
• Ground-up metrics are characterized by more categorical variables and with quantitative numbers complemented and validated through qualitative insights.
• Examples of these metrics include: looking at the nature of digital usage vs. rate of digital adoption; number of customers vs. type of customer; or contextually-specific indicators around women-led businesses

Mainstreaming Gender in the ESG Investment Standards

Speakers: Reine Kathryn Taya Rala and Rosario Panganiban (Mennonite Economic Development Associates)

3 Key Takeaways:

• The Gender Equality Mainstreaming (GEM) Framework is a practical guide and toolkit for assessing gender equality, and for identifying, implementing and measuring gender equality mainstreaming strategies with companies.
• GEM assesses companies and rates them on a gender spectrum, with gender blind/unaware at the lowest end and gender transformative at the highest end.
• GEM can be administered through a rapid and full assessment, with the full assessment providing comprehensive data across social, environmental, governance, and gender performance.

Personalizing MSME Coaching at Scale

Speakers: Ei Shwe Sin Phyo and Su Wai Myo (ONOW Myanmar)

3 Key Takeaways:

• Personalized coaching for small and growing businesses can be enhanced through the use of data segmentation – use different lenses of stakeholder segmentation and use that data to support personalized program design.  
• Use multi-variable regression analysis to identify factors for stakeholder groups that influence different outcomes, which creates differentiated impact measurement for these groups.  
• When analyzing different factors for stakeholder groups, beware of confusing correlation versus causation.  Causal relationships are much harder to establish as compared to correlated trends.

Impact Measurement as a Two-way Data Street 

Speaker: Thai Nguyen (Oxfam in Vietnam)

3 Key Takeaways:

• Oxfam uses an inclusive metrics framework for surveying the SGBs in their iSME Development Program.  Specifically, data collection is a two-way street, wherein the SGBs that contribute data are given data visualizations with analysis and impact measurement conversations with Oxfam staff.  
• Oxfam uses rapid surveying to collect data online, called a “flash poll.”  The survey data is automatically populated in a data visualization dashboard. 
• A key data collection challenge is that SGBs are reluctant to share data, specifically about financial performance.  Data collection with these types of businesses can be labor intensive.

Gender ROI™: A Sneak Peek at the Revolutionary New Way to approach Gender in your Organization

Speakers: Rowena Reyes (Sweef Capital) and Jessica Menon (Equilo) 

3 Key Takeaways:

• Sweef Capital uses the Equilo platform for their Gender ROI tool, which is available open-source.  
• The dimensions of Sweef’s Gender ROI tool look at embedding gender equality through the organization and looking at resulting outcomes on women’s empowerment, therefore examining both the process and impacts of gender mainstreaming.  
• The Gender ROI tool looks at the extent to which an organization is impacting women’s resilience, opportunity, and inclusion through changes in organizational leadership, workforce gains, gender inclusive value chains, and overall social trickle-down effects.  

Measuring Dignified Jobs and Decent Work

Speakers: Shruti Goel (Upaya Social Ventures)

3 Key Takeaways:

• Decent jobs are measured not only by the value of money, but also the reliability of income, physical safety at work, and job security. 
• Upaya’s Dignified Jobs Index uses mixed methodologies to collect performance data across five dimensions: access to jobs for the extremely poor, quality of income, work experience, support systems at work, and financial resilience of the employee.   
• There is a gap in measurement for impact investing, as most impact investors only count the number of jobs created, rather than the quality of those jobs.  Of those that do measure job quality, most measurement systems do not get direct validation from the job-holders themselves.

Grantee Centric MEL Practices

Speakers: Madhu Jagsheesan (Women’s Fund Asia), Sheila Chanani (Global Fund to End Modern Slavery) and Shyama Sinha (Vera Solutions) 

Key Takeaways:

• • Key IMM challenges from a funder perspective center around the need to find a balance between specificity and generalizable indicators, while keeping the monitoring and evaluation components a manageable size. 
  • The Global Fund to End Modern Slavery (GFEMS) developed their indicator catalog using a bottom up approach based on their grant-funded projects’ priorities, which helped develop a grantee-centric MEL system. 

Gender-Smart MEL in the WE Rise Accelerator Programs

Speakers: Priya Thachadi, Denise Dalusong and Alyanna Supetran (Villgro Philippines)

3 Key Takeaways:

• The journey to becoming gender-smart includes conducting a landscape assessment, identifying key gaps in current practices, and program redesign according to data collected from women entrepreneurs. 
• Villgro Philippines conducts gender smart MEL using a gender scorecard, assessing across dimensions including organizational gender intentionality and strategy, supply and distribution, and human resources.
• To properly evaluate gender impact in finance, begin with mapping the local investing ecosystem (i.e. what are the types of financing available for women entrepreneurs, who are the funders explicitly investing in women, etc.)

Closing Fireside Chat Panel: Moving Forward – The Future of IMM

Panelists: Mallory St. Claire (ANDE), Bobbymon George (Sattva Consulting), Jana Svedona (FMO), Minhaz Anwar (BetterStories Limited)

3 Key Takeaways:

• Key IMM innovations to look out for include impact forecasting and shifting out of impact measurement to a more inclusive impact management and performance practice that can be used as a tool to drive capital to the sector.  
• A key area that needs more attention is how practitioners can communicate the importance of IMM in order to generate buy-in, particularly from government and private sector actors.  
• Looking five years down the road, the IMM sector would benefit from greater acceptance and utilization of both qualitative and quantitative methods.  Data triangulation is a best practice that can present a more comprehensive impact picture.