Research & Evaluation

Blended Learning Evidence of Learning Pilot

I led a mixed-methods pilot designed to understand how evidence collection, user experience improvements, and feedback mechanisms could strengthen engagement, accountability, and application in professional learning.

This case study highlights how research and evaluation can inform learning design, strengthen implementation, and support continuous improvement.

Executive Summary

The Blended Learning Evidence of Learning Pilot explored how teachers experienced and applied blended learning strategies during professional learning. The pilot involved seven schools across four states and focused on increasing learning artifact submissions, reducing cognitive load, improving feedback mechanisms, and creating clearer ways to understand participant engagement and application.

The project combined UX research, LMS integration, surveys, focus groups, user interviews, Padlet, and Qualtrics to gather both qualitative and quantitative evidence. Findings from the pilot informed improvements to learner experience, accountability structures, feedback loops, and future professional learning design.

Project Snapshot

K–12 educators participating in blended learning professional learning

Project Manager, Lead Facilitator, Lead Focus Group Researcher, Data Analysis, Co-Designer

Google Workspace, Qualtrics Surveys and Dashboards, LMS, and various other edtech tools

15 months

Data analysis, research, evaluation, UX design, project management, learning design, and facilitation.

Defining the Problem

Previous versions of the product offered limited opportunities for tracking learner data and engagement.
Qualitative data suggested opportunities to improve the course’s user experience (UX) and increase learner accountability through increasing rigor and incorporating evidence of learning.
As we reflected on past learners’ experience, we wondered: how might we increase learner engagement, accountability, and rigor?

Project Objectives

Research Question

How and to what extent does collecting evidence of learning influence skill acquisition, intent to apply learning, and classroom integration?

Increase engagement/accountability for learners to demonstrate their learning by uploading a learning artifact to the LMS.

Increase feedback for learners by incorporating a self-assessment tool to accompany the learning artifact.

Reduce cognitive overload of self-guided course through UX design improvements.

Create the opportunity for internal and external stakeholders to analyze additional learner data.

Methods & Evidence Sources

Introduced self-assessment tools to provide participants with immediate feedback on progress and accountability.
Embedded Padlet for real-time collaboration, fostering interaction among educators.
Simplified the LMS interface by reducing visual distractions, improving usability, and supporting smoother navigation.
Integrated data collection forms via Qualtrics to capture learning insights and improve reporting mechanisms.
Conducted iterative testing with users to ensure the effectiveness of new tools and interface updates.
Provided workshops and instructional materials to guide educators on effectively using the updated system.
In person focus groups to speak directly to educators to:
- Gain insights on implementation challenges and successes
- Evaluate the alignment with Powerful Learning and VILS initiatives
- Identify professional learning needs and preferences

Evidence Source	Purpose
Surveys	Captured participant feedback, self-assessment data, and reflections on learning.
Focus Groups	Provided qualitative insight into educator experiences, implementation barriers, learner preferences, and support needs.
User Interviews	Helped identify gaps and opportunities in the learner experience.
LMS Data	Supported analysis of engagement, submissions, completion, and participant activity.
Learning Artifacts	How participants demonstrated application through evidence of learning submissions.
Feedback Dashboards	Helped identify trends across participants and schools.

Results & Key Takeaways

Quantitative

Teachers who completed the program demonstrated a 0.86 increase in extensity scores, indicating higher technology integration in classrooms and exceeding the project’s key result target by 15%.

Qualitative

Positive feedback from educators on course engagement and the intent to apply new strategies in teaching.

Key Takeaways

Increased Accountability: Evidence of learning opportunities and self-assessment tools improve participant accountability and motivation.

Improved User Experience: Simplified UX design reduces cognitive load and enhances engagement.

Increased Collaboration: Real-time collaboration tools increase interactivity and knowledge sharing.

Artifacts

Focus Group Findings