4 Steps to Design an AI-Resilient Learning Experience

Start by thinking about your students and their context. Building empathy for your learners will help you craft a learning experience that’s truly tailored to their needs.

1.1 Review Your Students’ Context

• What program are they enrolled in?
• Are there any prerequisites for this learning experience?
• What prior knowledge and skills do they have?
• What are their career goals?

1.2 Think About Students’ Interactions with Generative AI

• What’s their prior experience with generative AI? Note that in a Fall 2023 survey, 81% of MIT Sloan student respondents said they used generative AI at least once a week.
• What access to generative AI will they have during the learning experience? Students might use free or paid versions of other generative AI tools.
• How might they use generative AI in their future professional roles?

Your learners’ context will provide an essential starting point as you think about outcomes, assessments, and activities for your AI-resilient learning experience.

Now that you know where your students are coming from, it’s time to define the destination: the learning outcomes.

Learning outcomes describe the intended results of the learning experience. They’re the “why” behind your assessments and activities.

In this step, you’ll write learning outcomes that account for your students’ backgrounds and generative AI’s impact.

2.1 Draft Learning Outcomes

Start by responding to this prompt: By the time your students complete this learning experience…

• What should they be able to do?
• What should they know and understand?
• What should their opinions be about these topics?

If it’s helpful, you can also frame this question in terms of time: What should students remember from this experience in six months, or one year, or five years?

When you’re revising outcomes for an existing course, it can help to think about an assessment that you already have that works really well. Why does it work well? What do students demonstrate when they complete that assessment? (J. Rankin, personal communication, February 16, 2024).

2.2 Consider AI’s Impact on Learning Outcomes

Now, it’s time to make sure you’re creating an AI-resilient learning experience. Are there topics that you’d like to 1) retire or 2) promote given AI’s capabilities?

• Retire: Just like fluency with a slide rule became less important after the invention of the pocket calculator, you might want to rethink some topics given generative AI’s capabilities. Consider whether generative AI alone could “achieve” any of your existing learning outcomes (J. Rankin, personal communication, January 11, 2024).
• Promote: Some areas might be newly important in a generative AI-enabled age. For example, would you like students to be able to identify common misconceptions that could surface in AI hallucinations? Or be able to evaluate whether subject-related tasks fall within or outside generative AI’s current capabilities?

MIT Sloan students have told us that they appreciate transparency from faculty around how AI tools might impact their learning. You’ll be better able to guide conversations around AI-resilient learning if you start by getting a clear handle on if, when, how, and why AI could affect your student’s experiences.

2.3 Refine and Rewrite Learning Outcomes

Review your brainstorming. Then write your revised learning outcomes by completing this prompt: “By the time students finish the learning experience, they should be able to [ACTIVE VERB]…”

MIT’s Teaching and Learning Lab blog explains why this format is useful:

(1) by using active verbs, we articulate actions that students will be able to do. These actions can be observed and compared with our desired results;

(2) it puts the focus on students (note: it is not “By the end the course the instructor will have…”), and advances the ultimate goal of backward design–-to facilitate learning, not simply to “cover” content.

Here’s an example: “By the time students finish this course, they should be able to design a digital marketing campaign and analyze its effectiveness through key performance indicators.”

When selecting an active verb for your learning outcome, we recommend using a verb that aligns with one of the levels of Bloom’s Taxonomy.

Bloom’s Taxonomy is a framework that can help you craft effective learning outcomes, align them with assessments, and guide students in working their way from lower-level cognitive skills (e.g., remembering) to higher-level skills (e.g., creating).

This Bloom’s Taxonomy infographic from the Vanderbilt University Center for Teaching shows the levels of Bloom’s Taxonomy and examples of active verbs that align with each level (Armstrong, 2010).

Source: Bloom’s Taxonomy [graphic], by Center for Teaching Vanderbilt University, 2016, Flickr (flickr.com/photos/vandycft/29428436431). CC BY 2.0 DEED.

Would you like your students to be able to “justify a stand or decision”? Consider using a verb like appraise, argue, defend, judge, etc. Alternately, if you’d like to focus on students’ ability to “use information in new situations,” you could use a verb like execute, implement, solve, use, etc.

Given the context of generative AI, you may want to prioritize verbs from the Apply, Analyze, Evaluate, and Create levels of Bloom’s Taxonomy. These verbs align with higher-order thinking skills that could be increasingly essential to students in an AI-enabled world (Genone & Hughes, 2023).

As described in the Teaching & Learning Lab blog, your final learning outcomes should be:

• Specific: Craft outcomes that plainly state what learners should achieve. Use active verbs that refer to tangible skills and knowledge. Avoid vague language that does not give a precise sense of expected abilities and competencies.
• Measurable: There should be tangible evidence that students have achieved the outcomes, whether through rubrics, evaluations, or other methods. Outcomes should focus on demonstrable skills and knowledge that can be meaningfully assessed after the learning experience.
• Realistic: Consider the timeframe, scope, and resources for the learning experience. Make sure you set challenging but feasible outcomes that students can reasonably accomplish given constraints. Align the outcomes with course level, student background, and other contextual factors.

The goal is to craft outcomes that give you and your students clear, focused targets to work towards. The outcomes signal what subject mastery looks like for this learning experience. Well-defined outcomes facilitate AI-resilient learning activity design and help you plan effective assessments of student progress.

2.4 Example Learning Outcomes

These are some example course-level learning outcomes:

• Develop a business plan for a start-up, incorporating market analysis, financial forecasting, and strategic planning.
• Conduct a detailed case study analysis of a multinational corporation, focusing on its operational, financial, and ethical challenges.
• Apply project management methodologies to manage complex business projects, ensuring timely delivery and quality standards.
• Communicate business strategies effectively, crafting persuasive presentations and reports for diverse stakeholders.

These are some example session- or module-level learning outcomes:

• Interpret the key elements of a balance sheet and income statement in a case study.
• Create a financial forecast for a hypothetical business scenario using Excel or a similar tool.
• Design a project plan for a small business initiative, applying basic principles of project management.
• Discuss the implications of cultural differences in international business.
• Prepare and deliver a persuasive business presentation using digital tools.

Additional Resources: Learning Outcomes

• MIT Teaching + Learning Lab – Where to Start: Backward Design – This resource from MIT’s Teaching + Learning Lab guides you through crafting specific, measurable, and student-centered learning outcomes. Understand how to align assessments and activities with these outcomes for a more intentional and effective learning experience.
• Oregon State University Ecampus: Bloom’s Taxonomy Revisited – This resource revisits Bloom’s Taxonomy to help you align course activities and assessments with generative AI capabilities. It emphasizes the need to review and amend learning outcomes, especially at the Remember and Analyze levels, to foster distinctive human skills in an AI-enabled world.

Now, follow these steps to decide what students will produce to show they’ve achieved the learning outcomes.

3.1 Choose the Learning Outcome(s)

Review your list of learning outcomes. Choose the learning outcome(s) you’d like to focus on for this assessment.

3.2 Determine the Assessment Task(s)

Complete this prompt: “Students will show that they’ve achieved the learning outcome(s) by…”

Consider these guidelines:

• Use varied assessment methods. Don’t rely on a single type of assessment throughout a course. Include both formative and summative assessments. Mix traditional methods like quizzes with presentations, projects, case studies, and other formats.
• Incorporate real-world tasks. Include assessments that mimic real-life challenges. How can students apply their skills and knowledge in a way that mirrors tasks they might complete in a professional context?
• Align each assessment with at least one learning outcome. The key to alignment is matching the assessment method to the skill, complexity, and cognitive process that you already selected when crafting your learning outcome. For example, if you want students to be able to make connections between real situations and the 3 Lenses Framework, you can’t just ask them to explain the framework. Instead, you might ask them to use that framework as the basis for analyzing a case study.

Here’s an overview of possible assessment types aligned with Bloom’s Taxonomy active verbs, based on the University of Louisville resource Use Bloom’s Taxonomy to Align Assessments:

3.3 Determine Acceptable AI Use

Think about how your assessment will account for generative AI. At the most basic level, for any given assessment, you can require, allow, or ban generative AI use. Consider both your students’ context and the learning outcomes you’ve developed as you decide which is the best approach for the assessment.

Option 1: Require Generative AI

This means you’ll require students to use generative AI while they demonstrate that they’ve achieved the learning outcomes. Consider this approach when AI use will enable effective new and effective assessment opportunities.

Option 2: Allow Generative AI

This means allowing (but not requiring) students to use generative AI. For example, if your students in an entrepreneurship course are learning about pitching different kinds of products, you might allow them to use generative AI to brainstorm products and create images to include in their pitch materials.

Option 3: Don’t Allow Generative AI

This means telling students they should not use generative AI for the assessment. Here are some contexts where this might be the best approach:

• Foundational knowledge: For assessments focused on foundational knowledge within a field, it could be difficult to tell whether a correct response can be attributed to the student’s knowledge versus a lucky “roll of the dice” with a generative AI tool.
• Real-world constraints: For assessments meant to simulate real-world constraints, the target professional settings might restrict generative AI use due to practical barriers (like time, cost, or data privacy). In those cases, mirroring those parameters could promote authentic skill-building.

Since AI detectors don’t work, you can only enforce an AI ban by asking students to complete assessments in the classroom without a laptop. However, that approach can disadvantage students with disabilities and other marginalized learners—as well as any student who struggles to write by hand or under tight time constraints.

Before facilitating an in-person, no-devices assessment, make sure to review MIT Disability and Access Services’ resource Faculty Procedures for Providing Academic Accommodations. Also, make sure your students have plenty of time and opportunity to request accommodations for these assessments.

Additional Resources: Assessments

• MIT Teaching + Learning Lab – Assess for Learning: This resource from MIT’s Teaching and Learning Lab offers guidance on designing assessments that not only measure learning but also contribute to it. It outlines strategies for aligning assessments with learning outcomes. It also highlights the importance of formative assessments in providing feedback to both students and instructors.
• Vanderbilt University Center for Teaching – Student Assessment in Teaching and Learning: This comprehensive guide covers various aspects of student assessment in higher education. It discusses different methods, the purpose of assessments, and the importance of aligning them with learning objectives. It’s a valuable resource for understanding the multifaceted nature of assessment beyond just grading.

How will you make the best possible use of in-person class time during your AI-resilient learning experience? Consider “flipping” your class.

In a flipped classroom, students build core knowledge outside the classroom through targeted homework assignments. Then in-person class time centers around active learning experiences. On the Vanderbilt University Center for Teaching’s page Active Learning, Cynthia Brame (2016) describes flipped learning in the context of Bloom’s Taxonomy:

This means that students are doing the lower levels of cognitive work (gaining knowledge and comprehension) outside of class, and focusing on the higher forms of cognitive work (application, analysis, synthesis, and/or evaluation) in class, where they have the support of their peers and instructor.

Studies have shown that flipping the classroom and implementing active learning can result in significant learning gains for students (Freeman et al., 2014; Jensen et al., 2017; Deslauriers et al., 2011).

A flipped classroom approach may be especially useful in the era of generative AI. Interactive in-person sessions give you the opportunity to support students’ learning regardless of whether and how they’re using generative AI outside the classroom. These sessions are a time to meet your students where they are—and to help them build the professional expertise they’ll need in their future careers.

This section on AI-resilient learning activities will walk through these steps:

• Choose the Learning Outcome(s): What learning outcome will the prework and activity help students build towards?
• Design the Prework: What pre-class activities and resources will help students build the foundational knowledge to participate in in-class activities? Consider allowing or encouraging AI-use for outside-of-class prework.
• Design the Activities: What in-class activities will students take part in to engage with the learning outcome(s)?
• Determine Acceptable AI Use: Will you require, allow, or ban AI use for the prework and in-class activity?

4.1 Choose the Learning Outcome(s)

Review your list of learning outcomes. Choose the learning outcome you’d like to focus on for this set of learning activities.

4.2 Design the Prework

Decide what pre-class work students will need to do to prepare for the session. Consider readings, videos, case studies, and low-stakes quizzes. Create or curate these resources and activities and post them on your Canvas course site.

A lot of people associate the flipped classroom with lecture videos. However, flipped learning doesn’t need to include any videos. Before class, it might make more sense for students to read a case study, take a low-stakes Canvas quiz, or even have a conversation with an AI chatbot.

Videos can be great—and you’re very welcome to connect with us about creating videos in the Teaching Studio. However, effective prework can take many different forms.

What if students don’t do the prework?

A key challenge in flipped classrooms is that there’s no guarantee students will do the assigned prework. Here are some strategies to tackle the issue:

• Clear Expectations: Start by making it clear to students why prework is crucial. Explain how it will benefit their learning. Make sure they understand that in-class activities will build directly on the prework.
• Accountability: Start class with a quiz about the prework. This approach encourages students to do the work and also gives you a quick overview of their understanding. Frequent low-stakes quizzing also gives students the chance to practice retrieval, which supports the learning process (Rankin, 2024).
• Flexible In-Class Activities: Design class activities to cater to different levels of preparation. You can have tiered activities where students who haven’t done the prework can start at a more basic level, while those who have can engage in more advanced or application-based tasks. This way, you’re not holding back students who are prepared, but you’re also not leaving behind those who aren’t.
• Student Feedback: Gather feedback from your students regularly (not just at the end of the term). Find out what’s working for them and what isn’t. Maybe there are reasons you can address for why they’re not doing the prework.

The goal is to create a learning environment where students see prework as both important and manageable.

4.3 Design the Activities

In a flipped classroom, the real magic happens live. Since students are already familiar with the topic, you can center class time around active learning that helps students build towards the learning outcomes. In this context, insight emerges from students sharing meaningful perspectives together and collaborating to build on prior knowledge.

Consider the examples below of how prework can lead to in-class active learning.

4.4 Additional Resource: Flipped Classrooms and Active Learning

• Flipping the Classroom – Vanderbilt Center for Teaching: This detailed guide to implementing a flipped classroom approach explores transferring foundational learning outside the class and engaging students in higher-order thinking during class sessions.
• Active Learning – Vanderbilt Center for Teaching: This resource offers a deeper dive into the principles and practices of active learning. It includes insights into various active learning strategies, their theoretical underpinnings, and practical applications.

4.5 AI Use

Like with assessments, you have three basic options for how to account for AI: you can require, allow, or ban AI use.

Option 1: Require Generative AI

You can require generative AI use if that will open the door to activities that are closely aligned with your learning outcomes. Consider these examples of AI-integrated learning experiences:

• Harnessing AI in Finance: Eric So’s Innovative Take on Teaching Value Investing – Eric So integrates AI in finance education by having students use AI to write essays in Warren Buffett’s style, fostering a deeper understanding of value investing principles and enhancing classroom dialogue.
• Synergizing Business Communication and AI: Melissa Webster Augments Management Communication – Melissa Webster uses ChatGPT in her managerial communications course, helping students assess AI-generated content and understand its implications in business communication.
• Innovative Pedagogy Meets AI: Jared Curhan’s Modern Twist on Negotiation Training at MIT Sloan – Jared Curhan’s course at MIT Sloan merges AI with negotiation training, using chatbots for practical exercises and feedback, preparing students for real-world negotiation scenarios.

Option 2: Allow Generative AI

You can give students the option of using generative AI without requiring its use. This approach may be best if generative AI use won’t facilitate or impede students’ engagement with the learning outcomes. It also helps if students are familiar enough with the topic that they’ll be able to tell if the AI tools are hallucinating.

For example, in a marketing strategy class, you could give students the option of using AI during certain steps of creating a campaign proposal. Some might choose to use AI for generating initial ideas or ad copy, while others could rely on traditional brainstorming. This flexibility would allow students who are more comfortable with AI to explore its potential, while those who preferred traditional methods could apply those strengths. The learning outcomes—which might be involve aligning with marketing principles and developing a coherent strategy—remain central, regardless of what tools students use as they work to achieve them.

Option 3: Don’t Allow Generative AI

Prework: Since generative AI detectors don’t work, banning AI for prework puts you in the difficult position of having a policy that you can’t enforce (J. Rankin, personal communication, January 11, 2024). If students know that AI use will be banned during the related in-class assessment, that may motivate the majority to complete even outside-of-class prework without AI support. However, for graded outside-of-class prework, you may run into equity issues if some use generative AI regardless of your policies.

In-Class Activities: In-class activities offer more flexibility when it comes to generative AI use. You can prevent students from using generative AI in class if you don’t allow access to devices during activities. Here are some contexts where that might be the best approach:

• Foundational knowledge: For activities focused on building foundational knowledge within a field, permitting AI tools could undermine that objective. Students developing core competencies might not have the context to critically evaluate AI output.
• Real-world constraints: If target professional settings restrict certain tools due to practical barriers (like time, costs, or data privacy), mirroring those parameters could promote authentic skill-building.

4.6 Maximizing the Flipped Classroom

To make the most of your class activities and prework, follow these guidelines:

• Match every activity to specific learning outcomes: Keep your course focused and your students on track by making sure each activity aligns with at least one learning outcome.
• Weave in formative assessments: Incorporate low-stakes formative assessments throughout the course so you and your students can gauge their progress and target areas that need work.
• Provide clear instructions: Make sure students understand the purpose and expectations of both prework and in-class activities.
• Encourage active participation: Design in-class activities that require students to actively engage with the material. This might include discussions, projects, or problem-solving exercises.
• Integrate technology thoughtfully: Use technology to enhance learning, not just for its own sake. Ensure any technology supports students’ progress towards the learning outcomes.

By flipping your class and emphasizing active learning, you create a dynamic learning environment where students engage with complex topics, use what they’ve learned, and think critically. This approach not only makes your course more interactive and engaging but also gives students the chance to apply new knowledge and skills regardless of AI’s impact.