Smart Tech, Smarter Questions: How to Evaluate AI Tools for Accessibility

In an era where artificial intelligence (AI) increasingly shapes educational experiences, developing AI literacy has become an essential leadership competency. For higher education administrators, understanding how to critically evaluate AI tools—especially through an accessibility lens—is crucial for making informed decisions that benefit all students.

AI literacy goes beyond knowing how to use these technologies; it requires a deep understanding of their capabilities, limitations, and potential biases. When evaluating AI solutions, we must look beyond flashy features and ask substantive questions about how these tools serve diverse learner needs.

The EDUCAUSE survey on higher education leaders' concerns about generative AI reveals telling priorities: academic integrity ranks highest (75%), followed by over-reliance on outputs (68%) and inaccurate information (68%). While these concerns certainly merit attention, it's noteworthy that diversity, equity, and inclusion rank eighth at 53%, suggesting we may need to elevate accessibility considerations in our institutional discussions.

Ethical considerations abound when implementing AI in higher education. Beyond issues of academic integrity and accuracy, we must contend with data privacy, latent bias, transparency, and the digital divide. These considerations become even more critical when addressing the needs of students with disabilities, who may be particularly vulnerable to algorithmic discrimination.

For example, predictive analytics that flag students with irregular attendance patterns as "high-risk" might disproportionately impact students with chronic health conditions or disabilities. Similarly, AI-powered proctoring systems using facial recognition can create substantial barriers for neurodiverse students. As leaders, we must question how success is defined in our systems and recalibrate when these definitions inadvertently perpetuate inequity.

Developing a robust accessibility rubric for evaluating AI tools is essential. Such a framework should assess:

• Customizability of outputs: Can users adjust formats, language complexity, and presentation?

• Multimodal interaction options: Does the tool support various input and output modes?

• Transparency of data inputs: Are there clear explanations of data collection and processing?

• Compatibility with assistive technologies: Does it work seamlessly with screen readers and other tools?

• User control: Can users pause, slow down, or customize their experience?

• Plain language support: Does it offer simplified, readable formats?

• Keyboard-only navigation: Is it fully operable without a mouse?

• Cognitive load management: Does it avoid overwhelming users with options?

• Error tolerance: Can users easily recover from mistakes?

• Inclusive defaults: Does it respect and reflect disability identity?

When making procurement decisions, we should insist on vendors demonstrating how their AI solutions address these accessibility considerations. Additionally, we must recognize that AI literacy varies significantly among our student populations. The Inside Higher Ed Student Voice Survey found concerning equity gaps in AI awareness and confidence. Institutional leaders have a responsibility to develop comprehensive AI literacy programs that reach all students, with particular attention to adult learners, first-generation students, and those with disabilities.

By prioritizing AI literacy and robust evaluation frameworks, higher education leaders can make more informed decisions that advance educational equity rather than reinforcing existing disparities. Our commitment to thoughtful implementation will determine whether AI fulfills its promise of enhancing accessibility or creates new barriers in our educational environments.

Ready to move from promise to practice in your AI strategy?

Dr. Claire Brady offers “From Promise to Practice: AI’s Role in Higher Ed Accessibility,” that equips higher education leaders with the tools to design, implement, and evaluate AI technologies through an equity and accessibility lens. Whether you're a disability services professional, a senior leader exploring AI adoption, or a faculty champion for inclusive innovation—this training will challenge your thinking and sharpen your approach. Explore this session and other high-impact AI trainings at www.drclairebrady.com, or reach out directly to schedule a conversation.