What’s good enough in SEL and Soft Skills Measurement? 4 experiences in developing assessment tools in diverse contexts

Group Submission Type: Formal Panel Session

Proposal

Programs focusing on social responsibility, psychosocial support, civic engagement, workforce participation, and other positive outcomes for children and youth often struggle to measure and communicate program impact on key social and emotional learning dimensions--particularly in challenging contexts where developing assessment instruments is not the first priority. This panel session focuses on the key factors in developing social emotional learning (SEL) and soft skills measurement, monitoring, and evaluation tools in diverse contexts. It starts with brief overviews of the experiences of four organizations in developing, testing and piloting measurement tools, sharing learning from the process within child-focused and youth initiatives in diverse development and humanitarian contexts. Each panelist will discuss their tool’s capacity to measure skills, values, and behaviors related to social responsibility, in line with the conference theme. It concludes with debates and discussions on determining what’s good enough in validity and reliability when developing or choosing a tool in different contexts, how to make decisions about “cut-off” and meaningful change scores of relevant tools, and how to determine what tool and evidence to use for a project. This session will highlight the steps recommended for choosing and/or developing rigorous measurement tools for building a strong evidence base, in order to inform important decisions around effective SEL and soft skills programming.

Evidence has demonstrated the importance of these skills for improving success in education and the workforce, and wellbeing in constraining social and political settings. Only by measuring these skills, can programs improve their ability to develop them. While a large literature is devoted to measuring these skills in Western contexts, the use of the same tools in developing-country contexts is often infeasible and irrelevant. Through sharing successes and challenges, this session contributes towards building knowledge and new approaches in measuring and evaluating these skills within humanitarian and development contexts, including the measurement of skills, values, and behaviors that contribute to social responsibility. It adds important insights to the sector-wide debate on what counts as “rigorous enough”, by presenting solutions that international organizations and other development entities can adopt and adapt when faced with similar dilemmas as the presenters. The session concludes with practical advice on what matters when selecting an SEL measurement tool, how to go about testing for its rigor, and deciding its application and use given restrictive contexts where we work.
SEL is known by its multiple names including 21st century skills, life skills, soft skills, positive youth development skills, among others. Building SEL among children, youth and young adults can contribute to outcomes in health, education, employment, and civic engagement, including broader socially responsible behavior. In the face of changing political and cultural contexts, skills such as self awareness, perseverance, expressiveness, initiative, and relationship building grow in importance for adaptation, mental wellbeing, and good governance. Organizations implementing SEL come together periodically in different forums; however opportunities for debate and collaboration around what works in SEL measurement are still limited. Building a rigorous evidence base around current practices for SEL is crucial to inform not only practice, but also policy and adequate financing for SEL.

World Learning, Save the Children, the World Bank and Innovations for Poverty Action, and the International Youth Foundation have developed, tested and piloted SEL measurement tools over the course of the past five years. Each organization had a slightly different process in developing and testing their tools, with similar shared challenges and learning. Save the Children developed its tool, ISELA, in 2015 and tested in Colombia, Democratic Republic of the Congo, Egypt, Haiti, Iraq, Jordan, Kenya, Mexico, South Sudan, Tanzania, Syria, Thailand, and Uganda. IYF’s Life Skills Survey Tool (LiSST) was developed in 2019 and tested with youth across the United States, Mexico, and Tanzania; the World Bank and IPA’s tool was developed in 2019 and tested in Nigeria and Cote D’Ivoire; and World Learning’s “WorkLinks Skills and Values Assessment (WLSVA)”, was developed in 2019 tested in Algeria during the 2020 COVID-19 pandemic, with additional small-scale testing in Iraq and the United States.

This panel will present the journey of the four organizations, engage the audience in discussion around what matters when measuring SEL, and share learning as well as recommendations on how to make informed choices in SEL measurement, when it comes to relevance, rigor, and usability of these kinds of initiatives.

The participants of this session will leave with in-depth understanding of SEL measurement steps, practical implications, and a handout with step-by-step process on developing, piloting and testing SEL measurement tools.

Sub Unit

• Monitoring and Evaluation SIG

Chair

• Rebecca Pagel, USAID

Individual Presentations

• Developing the WorkLinks Skills & Values Assessment in Algeria - Catherine Honeyman, World Learning

• Social and Emotional Learning (SEL) Measurement Adaptation, Contextualization and Feasibility: Implementation of the International Social and Emotional Learning Assessment (ISELA) - Nikhit D'Sa, University of Notre Dame; Allyson Krupar, Save the Children / American University

• Illuminating the development and testing of a life skills survey with U.S. youth - Elizabeth Kim, International Youth Foundation

• Measuring Socio-Emotional Skills for the Workforce in Africa - Smita Das, World Bank Africa Gender Innovation Lab; Clara Delavallade, World Bank