Taking Measure of Experiential Learning

Business Education, GBSN Member Schools

GBSN’s 2018 Learning by Doing: The Power of Experiential Learning in Management Education summit coming up this March 15-16 in Lima, Peru will provide an in depth examination on how schools use real-world projects to give their students an impactful learning experience. Business school deans, directors and faculty will explore innovative approaches to applying action based learning to curricula.

“Taking Measure of Experiential Learning” originally published by AACSB International BizEd Magazine, sheds light on how schools can measure experiential program and course learning outcomes by identifying six assurance of learning  (AoL) standards that are common to successful project-based courses and provides examples of how some business schools are applying these standards.

Two contributing authors of this article are members of GBSN’s Experiential Learning Advisory Board, Kerry Laufer, Tuck School of Business at Dartmouth and Michellana Jester, MIT Sloan School of Management. They both play an active role in shaping the agenda of the Learning by Doing summit and will be participating as speakers.


EXPERIENTIAL LEARNING IS A FIXTURE IN MOST B-SCHOOL CURRICULA, YET MANY SCHOOLS STRUGGLE TO MEASURE OUTCOMES IN PROJECT-BASED COURSES. A RECENT SURVEY HIGHLIGHTS STRATEGIES THAT CAN HELP STUDENTS REACH NEW LEVELS OF LEARNING.

WHETHER THROUGH CONSULTING projects or action learning courses, business schools around the world are showing dramatic growth in the project-based learning initiatives they offer students. Such programs allow students not only to apply their knowledge through direct experience, but also to gain critical management competencies in “soft skills,” such as managing ambiguity, managing clients, collaborating, and communicating. Some schools are going even further, by setting experiential learning requirements that students can fulfill by choosing from a menu of options.

But while business educators have gotten good at assessing compliance and completion in experiential courses, they have found that assessing students’ learning and mastery of the material is a trickier proposition. With AACSB’s assurance of learning (AoL) mandate, assessing learning in project-based courses becomes an even bigger practical challenge for business schools.

To meet this challenge, the three of us founded a subcommittee within MBA Leaders in Project-Based Education (LEPE), a professional networking group of experiential learning scholars and practitioners from U.S. business schools. Since its founding in 2014, the subcommittee has been studying trends, approaches, and challenges in project-based courses. In April 2016, our group surveyed more than 100 U.S. business schools offering project-based experiential learning courses for degree credit. Then, in June 2016, at LEPE’s annual conference, representatives from about 30 business schools discussed the survey results, sharing their experiences and insights.

NINETY-TWO PERCENT OF FULL-TIME MBA PROGRAMS SURVEYED BY THE MBA ROUNDTABLE IN 2016 OFFER AT LEAST ONE CLIENT-BASED EXPERIENTIAL LEARNING COURSE.

Most recently, the subcommittee conducted a follow-up survey and reviewed existing standards and guidelines put forth by organizations such as the National Society for Experiential Education. We want to share six AoL standards that our survey has identified as common to successful project-based courses, as well as provide real-world examples of ways some business schools are putting these recommendations to work. We hope that business schools can use these standards as a guide to evaluate their own project-based experiential learning courses.

ACTIVITIES OF EFFECTIVE AOL 
What strategies do business schools adopt to deliver project-based, experiential courses with the best learning outcomes? According to our survey, educators at responding schools do one or more of the following:

EIGHTY PERCENT OF PROGRAMS SURVEYED BY LEPE SPECIFY LEARNING OBJECTIVES IN THE DESCRIPTIONS OF THEIR PROJECT-BASED COURSES. BUT FEWER THAN 25% REQUIRE STUDENTS TO PRIORITIZE AND COMMIT TO PERSONAL OBJECTIVES.

Tailor learning outcomes to the individual.  To determine what a student should learn from a project-based course, business schools should follow a two-part process. First, a program should identify a clear set of learning objectives for all participants; second, each individual student should set at least two personal goals, which he or she communicates to program coordinators and the rest of the project team at the outset.

Most programs listed more than ten learning objectives for their project-based courses. However, many of the business educators we talked to believed that if students are not intentional about setting personal objectives, their learning will be suboptimal and accidental at best.

The Student Team Achieving Results (STAR) program at the Kenan-Flagler School of Business at the University of North Carolina in Chapel Hill requires each student to complete a personal development plan (PDP) in conjunction with a team charter. As part of the PDP, each team holds a session early in the project, where its members explicitly do not discuss client and project scope topics. Instead, they focus on their own strengths and goals. 
Prior to the session, team members use an assessment tool – usually the Myers-Briggs Type Indicator – to identify their personality types. Then, at the session, they share what they learned about their personalities and working preferences, engaging in open discussions and default tendencies.
They write their PDPs after the session – here, they must include no more than two primary individual learning objectives for the project. Those objectives can be selected from a list of formal objectives for the course, or they can be created based on conclusions students have reached during the session. For each objective, team members identify a method of evaluation and a “coach” among them who will commit to holding the team member accountable for the objectives.
In the future, UNC plans to incorporate what students achieve in relation to their PDP objectives in the reflection and evaluation process.
EIGHTY-FOUR PERCENT OF COURSES SURVEYED PROVIDE MIDCOURSE FEEDBACK, BUT ONLY 64% NOTE THAT THEY PROVIDE INDIVIDUAL FEEDBACK. JUST 48% TIE INDIVIDUAL FEEDBACK TO INDIVIDUAL GOALS.
The Tuck School of Business at Dartmouth College in Hanover, New Hampshire, provides feedback at times when teams need it most. For that reason, the school has committed to a 1:1 ratio of team advisors to student teams for its 12-week OnSight Global Consulting course. It also assigns advisors to accompany project teams as they work full-time on-sight at their clients’ locations for three weeks during the project.
Other design elements in both the required First-Year project course and elective second-year OnSight Global Consulting course are designed to facilitate feedback from both clients and advisors at an early stage in the process. These include requiring each project team to provide a set of interim client deliverables in the first four weeks. This requirement helps accelerate client-relationship development and offers built-in opportunities for the client and team mentors to provide feedback to students early enough for them to learn from mistakes, make adjustments to the team process and ultimately improve outcomes.
In the future Tuck plans to create stronger links between identifying students’ individual learning goals and providing individual feedback. 

Provide feedback early and often to teams and individuals. Timely and specific feedback allows a team and its members to self-assess and change course as necessary to achieve a better final outcome.

More than half of the courses surveyed by LEPE span a full semester, providing ample time for instructors to give feedback along the way, not just at the end. All courses should aim to conduct at least one formal mid-course assessment, with individual and team feedback tied to both course-level and individual learning goals.

SEVENTY PERCENT OF COURSES SURVEYED REPORT ASSIGNING AN INDIVIDUAL REFLECTION PROJECT.

Create opportunities for reflection.  Education scholars emphasize that learning cannot happen without reflection, particularly in project-based learning. Common examples include final reflection papers, a 360-degree instrument for peer assessment, a self-assessment, team debriefing sessions, individual debriefing sessions, or some combination of these approaches.

While any exercise in which students are asked to reflect on their experiences will improve learning outcomes, the nature and quality of the reflection also matters. Too frequently reflection in project-based courses tends to focus on team results and client satisfaction at the expense of individual learning. For that reason, assessing course learning outcomes should follow the two-step process described above. That is, a course can use a 360-degree instrument to measure learning outcomes that are the same for all students. Then, students can each complete a reflection assignment focused on individual learning outcomes, which can be assessed critically by a faculty coach or mentor. All course directors, faculty coaches, and mentors must be trained to design and facilitate reflection exercises.

While students often find reflection difficult and alien, many educators believe it is essential for students to critically examine their personal beliefs, especially within an entrepreneurial setting. Thats why the Global Entrepreneurship Lab at the MIT Sloan School of Management in Cambridge, Massachusetts has expanded the scope of its two-and-a-half-hour student debriefing sessions held at the end of each on-sight immersion. 
These sessions now include four reflective papers, two peer review surveys, on-site blogging and vlogging, two plenary debriefing sessions, a small group debriefing, and a poster presentation session. By incorporating these reflection opportunities throughout the 13-week course – as well as the three weeks that students spend on-sight working with entrepreneurs in emerging markets – the school has generated better outcomes for students and higher quality deliverables for host companies.
In 1990, the Wearherhead School of Management at Case Western Reserve University in Cleveland, Ohio, began a 50-year longitudinal study of multiple cohorts, focusing on the impact of the MBA program on the development of the students’ cognitive and emotional intelligence. The school uses the results of this on-going study to assess, support, and advance student and institutional learning. 
The data is collected during Leadership Assessment and Development, a required course where students are led through assessments and activities about their dreams, aspirations, current behaviors, strengths, and gaps in managerial and leadership skills. Students use their assessments to develop personalized learning plans for their MBA program and beyond.
Weatherhead has used data regarding the cognitive, affective and behavioral development of its students to modify its experiential learning course offerings. For example, in the late 1990s, Weatherhead added an action learning elective course to the MBA curriculum, where students completed projects with sponsoring organizations. Noting an uptick in learning skill development in the evaluation cohorts, Weatherhead revised the MBA program to require action learning for all students.

Evaluate student learning in three domains: cognitive, affective, and behavioral.  Experiential learning is an iterative process with multiple feedback loops for learning. The multidisciplinary aspect of this type of learning engagement requires the evaluation of student performance across the cognitive domain, which takes into account learning and adaptation over time; the affective domain, which encompasses areas such as student satisfaction and commitment; and the behavioral domain, which refers to the quality and quantity of performance on tasks as well as contextual performance. Each evaluation also should be delivered individually and qualitatively, if possible.

Acknowledge and incorporate the role of emotions. Emotions can facilitate or impede learning. Unlike those in traditional course lectures, student experiences in project-based courses are, to a large extent, rooted in emotions. When students have to function in new and different environments, they must move outside of their comfort zones.

For example, when team members hold different views on how to proceed with a project, it frequently leads to conflict and results in frustration, anger, and resentment. Left unaddressed, negative emotions can become barriers to, and distortions of, individual learning, narrowing the field of further experience and becoming obstacles to team success.

However, when these same emotions are acknowledged and incorporated into the team’s process, they can catalyze learning, leading to a shift in mindsets and beliefs. Effective feedback and coaching that recognizes emotions and encourages teams to verbalize them can help de-escalate conflict, minimize future occurrences, and enhance understanding of team dynamics and each student’s role in shaping them.

The Multidisciplinary Action Project (MAP) at the University of Michigan Ross School of Business in Ann Arbor, Michigan, in place for 25 years, incorporates a student-led support system for MAP students. Within the system, experience second-year MBAs complete a leadership coaching practicum to learn how to help individuals and teams confront and manage tensions and other emotional issues that emerge during their team project work. 
The practicum prepares second-year MBAs to coach the first-year MBA MAP teams, helping them manage their team processes more effectively to achieve better results and greater learning. As an added benefit, the MBA leadership coaches emerge from the experience with concrete knowledge and insight on how to coach teams toward success.
EIGHTY-FOUR PERCENT OF PROGRAMS SURVEYED REPORT THAT THEY REGULARLY COLLECT ASSESSMENT DATA ON HOW TO IMPROVE TEACHING AND LEARNING, BUT ONLY 56% REPORT USING ASSESSMENT DATA TO MAKE CHANGES TO THEIR PROGRAMS.
The FIELD (Field Immersion Experiences for Leadership Development) program at Harvard Business School in Boston, Massachusetts, employs a rigorous course review and improvement process. As part of this process the school collects data regarding key elements of the learning experience for its more than 950 first-year MBAs. The cross-functional group of faculty that leads FIELD then analyzes and reviews that data regularly.
While these professors look to ensure consistency among student participants, they also explore ways to improve FIELD in areas such as recruiting students for projects, assisting students with team formation and management, designing course content and delivery, and advising other faculty assigned to work with student teams. Each year, faculty gather feedback, review the data, and identify areas for improvement and innovation. 

Close the loop. Closing the loop in AoL represents one of our greatest opportunities as educators to learn from our experiences and translate that learning into course improvements. And, yet, it doesn’t happen nearly enough. It is important to assign clear and specific accountability for AoL in experiential courses and to consider the longitudinal benefits of the learning that arises from these courses.


Click here to view original article 

Kerry Laufer is the Director of OnSite Global Consulting, a second-year elective experiential learning course at the Tuck School of Business at Dartmouth College in Hanover, New Hampshire. Shannon McKeen is an Adjunct Professor at the Kenan-Flagler School of Business at the University of North Carolina in Chapel Hill. Michellana Jester is a Lecturer and Faculty Course Manager in the Global Economics and Management Group at the MIT Sloan School of Management in Cambridge, Massachusetts.