Evaluating Learners in the Age of Ubiquitous AI

0
60

Key Takeaways

  • Assessment in higher education must move beyond recall of facts to evaluate how students think, judge, and apply knowledge in complex, real‑world situations.
  • Employers consistently prioritize “power skills” such as strategic problem‑solving, collaboration, communication, ethical judgement, adaptability, and digital fluency—capabilities that traditional exams struggle to measure.
  • Experiential, case‑based, and project‑focused assessments that present shifting variables reveal students’ reasoning under uncertainty and their ability to adapt.
  • Rather than policing AI use, educators should design assignments that make AI engagement visible, intentional, and evaluable, asking students to compare, critique, and reflect on AI‑generated outputs.
  • Feedback becomes central when assessments are iterative; a “guide‑on‑the‑side” faculty role supports growth, metacognition, and professional‑style supervision.
  • Institutions should explicitly define assessable skills, design evolving assessments, evaluate process over product, make AI use transparent, and invest in faculty development for judgement‑focused grading.
  • Ultimately, assessment signals what we value; aligning it with the complexity and collaboration of modern work prepares graduates to apply their thinking beyond the classroom.

Introduction: The Changing Landscape of Assessment
For much of higher education’s modern history, assessment has followed a familiar formula: a midterm and a final exam, with a heavy emphasis on whether students can retain and reproduce information. That model made sense in a world where knowledge was scarce and expertise lived primarily in textbooks and lectures. As the article notes, “That world no longer exists.” With students’ early access to technology, they can find most information from Google, YouTube, and now AI chatbots. The rapid rise of generative AI hasn’t rendered assessment obsolete, but it has exposed its misalignment with today’s learning realities, forcing educators to ask what they are truly trying to measure.


Employer Expectations and Power Skills
One of the most clarifying moments for the authors came when they stopped debating pedagogy in the abstract and instead examined what employers consistently say they need from graduates. Across multiple industry reports, the themes for necessary business skills are clear: strategic problem‑solving, collaboration, communication, ethical judgement, adaptability, and digital fluency. Rather than label these “soft” skills, the article proposes calling them power skills—competencies that are difficult to capture with traditional exams. If we genuinely believe these capabilities matter, they must be woven throughout the curriculum and assessed through students’ demonstrations of ability, not merely through recall.


From Recall to Judgement: Assessing Thinking
When learning environments become more experiential—grounded in real problems, data, and constraints—assessment inevitably shifts. In professional settings, there is rarely a single correct answer; instead, there are trade‑offs, incomplete information, and evolving conditions. The article urges educators to replace the question “Did you get it right?” with prompts such as: “How did you approach the problem? What assumptions did you make? How did you respond when conditions changed? Can you explain and defend your decision?” This reframing places reasoning, judgement, and adaptability at the heart of evaluation, moving assessment beyond mere correctness.


Experiential Learning and Assessment Design
In simulations, casework, and project‑based assessments, students are often confronted with shifting variables—a reduced budget, a new stakeholder concern, a conflicting data point. The assessment, therefore, is not about perfection but about reasoning under uncertainty. By presenting dynamic scenarios, educators can observe how students gather information, weigh alternatives, revise plans, and justify outcomes. This approach mirrors the iterative nature of professional work and provides a richer picture of student competence than a static exam could ever yield.


AI’s Role in Assessment: From Detection to Evaluation
AI has understandably generated anxiety around academic integrity, but focusing solely on AI detection misses a larger opportunity. The more productive question, according to the article, is: “How do we design assessments where AI use is visible, intentional, and evaluable?” Graduates will inevitably use AI tools in their careers, so educators must train students to employ these tools responsibly and efficiently. Effective assessments can ask learners to:

  • compare their own analysis with AI‑generated output,
  • identify where AI reasoning falls short,
  • explain when they would trust an AI recommendation and when they would not,
  • reflect on bias, limitations, and ethical implications.
    In these cases, AI does not undermine assessment; it reveals students’ thinking, shifting the focus from “Did you use AI?” to “How did you use it, and why?”

Feedback, Iteration, and the Guide‑on‑the‑Side Model
As assessments move away from high‑stakes, single‑moment exams, feedback becomes central. In experiential environments, students learn through iteration: propose, test, revise, and reflect. Assessment, then, becomes cumulative rather than terminal, valuing growth, responsiveness, and metacognition—the ability to understand one’s own decision‑making process. This approach also transforms the traditional “sage on the stage” model into a guide‑on‑the‑side role, where faculty provide structured feedback akin to professional supervision rather than acting as the sole authority delivering judgements from the front of the room.


Practical Advice for Institutions Rethinking Assessment
For institutions beginning this transition, the article outlines several principles that outweigh any specific tool or technology:

  1. Define assessable skills explicitly. If adaptability or collaboration matters, articulate what it looks like in student work.
  2. Design assessments that evolve. Real learning happens when conditions change and students adjust; assessments should reflect that dynamism.
  3. Evaluate process, not just product. Ask students to explain how they arrived at decisions, revealing their reasoning pathways.
  4. Make AI use transparent. Craft assignments that require students to demonstrate and reflect on their engagement with AI.
  5. Support faculty development. Assessing judgement demands different rubrics, norms, and confidence than grading exams; faculty need training and practice in the skills they are teaching, including AI literacy.

Implementing these steps helps align assessment with the competencies employers value and prepares students for lifelong learning.


Assessment as Preparation for Professional Life
Ultimately, assessment is a signal—to students and to employers—about what we value. If we continue to prioritise memorisation in an age of ubiquitous information, we send the wrong message. The article concludes, “AI did not create this challenge; it has accelerated a reckoning.” If our goal is to prepare graduates for a world defined by complexity, collaboration, and constant change, our assessments must do more than measure what students remember. They must reveal how students think—and whether they are ready to apply that thinking beyond the classroom.

Michelle Seref serves as associate dean of undergraduate programmes and professor of business information technology in the Pamplin College of Business at Virginia Tech.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.

https://www.timeshighereducation.com/campus/assessing-students-when-artificial-intelligence-ubiquitous

SignUpSignUp form

LEAVE A REPLY

Please enter your comment!
Please enter your name here