Our story
The Scenario-Based Language Assessment (SBLA) Lab was founded in 2016. This project is an international collaboration based out of the Applied Linguistics and TESOL Program at Teachers College, Columbia University in NYC. Currently, we are partnered with Columbia's Korean Language Program, the University of Maryland's Persian Language Program, the CILS Center at the University for Foreigners of Siena (Italy), and the American University in Cairo (Egypt).
We, in the SBLA Lab, believe that second and foreign language (SFL) assessment (and pedagogy for that matter) should endeavor to mirror the kinds of meaningful contextualized activities we engage in in real life--that is, the kinds of activities we need to be successful in in home, school, and work situations. For example, we, as SFL learners, might need to "read an online forum and contribute to the discussion" or "work with peers to generate and deliver an online presentation" or just "maintain and nourish relationships with friends over a meal." These language use contexts, for SFL learners, require more than being passive participants; they require the ability to marshal a complex range of resources in order to be fully contributing members of the group. For example, they involve the ability to use language accurately, meaningfully, and appropriately to communicate ideas, and to achieve overall communicative goals. They also require the ability to process what others have to say, even when some words are new or unfamiliar, so that we can respond and share our own ideas. And they require the ability to be courageous by asking others to repeat or explain.
Given this backdrop, the SBLA lab is interested in designing assessments that will measure students' ability use a range of linguistic and non-linguistic resources in order collaboratively solve problems or make decisions with others as a fully engaged member of the group. These assessments are situated within "scenarios," involving an internally consistent set of naturally-occurring, imagined scenes or events in which characters have to engage a full range of mental, social, and dispositional resources to carry out actions and interact with each other until they bring the overarching scenario goal to conclusion. Scenarios are flexible in that, similar to real life, just about anything can be engineered into the assessment design.
Given that scenarios provide a context in which examinees need to engage in, display, develop, and coordinate a range of topical, linguistic, sociocultural, socio-cognitive, dispositional, and other resources while moving through the scenario narrative with peers, we assume that they will come away with new understandings and insights as a result of assessment protocol.
We believe that SFL assessment in this context provides all the design elements to move SFL assessment beyond measures of SFL proficiency to measures of "Situated SFL Proficiency," and that the assessments are intrinsically "Learning-Oriented."
We warmly invite you to learn more about our projects and we hope you'll join us in our mission.
Our Mission: To contribute to improved and innovative ways of
conceptualizing, engineering, delivering, and scoring second and foreign language assessments;
reporting SFL assessment results and insights for summative and learning-oriented purposes
justifying the claims we make on the basis of SFL assessment
reimagining how SFL assessments and assessment information can be used can be used to indicate and moderate performance in classroom and standardized assessment contexts.
Our Appreciation: We would like to express our sincere gratitude to Teachers College for supporting this project over the years. We are also enormously grateful to an anonymous donor, who believed in this project from the beginning. Thanks to their generous support and encouragement, we have been able to make this project a reality.
NEWS
The SBLA Lab members and our partners are the invited to present their work at the 2022 Joint AAAL/ILTA Symposium in Pittsburgh. The title of the symposia is: Exploring the cross-linguistic insights of using scenario-based assessment across four typologically different languages. The abstracts are available here.
We will also be presenting our work in a symposium (same title) at the 2022 Language Testing Research Colloquium in Tokyo, Japan. Information about LTRC 2022 can be found here.
Latest publication
A Rationale for Using Scenario-Based Assessment to Measure Competency-Based Situated Second and Foreign Language Proficiency
James E. Purpura
Citation: Purpura, J. E. (2021). A Rationale for using a scenario-based assessment to measure competency-based, situated second and foreign language proficiency. In M. Masperi, C. Cervini, & Y. Bardière (Eds.), Évaluation des acquisitions langagières : Du formatif au certificatif. MediAzioni 32: A54-A96, http://www.mediazioni.sitlec.unibo.it. ISSN 1974-4382.
Abstract: In an effort to provide a theoretical rationale for the use of scenario-based assessment to measure competency-based, situated second and foreign (S/FL) proficiency, this paper traces conceptualizations of S/FL proficiency since the 1960s along with the major approaches to measuring these conceptualizations. The distinguishing characteristic of this evolution is that, as the construct broadened, the more complex the assessment methods became and the greater the potential for meaningful interpretation. This paper argues that while language-based, independent and integrated skill-based, and task-based approaches to S/FL proficiency assessment can be useful in certain assessment contexts, they are not engineered to measure S/FL proficiency in contexts of situated language use – that is, where goal-oriented task accomplishment is located within a sociocultural context, and where the ability to achieve complex tasks is embedded within the mediated engagements and social practices of a particular community. For this reason, some researchers have turned to scenario-based assessment (SBA). This paper describes how SBA has been used in the mainstream and S/FL assessment contexts, highlighting the affordances of this approach. Finally, the paper illustrates how a learning-oriented approach to assessment (LOA) (Purpura & Turner, 2018) can serve as a comprehensive conceptual assessment framework for engineering and validating SBAs.
Open access information can be found here.
Highlights
Arts & Humanities Distinguished Speaker Series
Questioning the Currency of Second and Foreign Language Certification Exams
In his Distinguished Speaker Series, Professor James E. Purpura, Director of the SBLA Lab, questions the currency of traditional approaches to L2 assessment, arguing that present designs limit their capacity to provide examinee information that is needed and valued in the real world. He argues for a learning-oriented approach to assessment as a design framework for constructing scenario-based assessments capable of measuring a broadened array of constructs, and providing better interpretative bases for educational decisions.
