A comforting narrative is spreading through engineering organizations: run an AI-powered accessibility scanner, fix the issues it identifies, push the score into the green, and declare the product accessible. The workflow is efficient, measurable, and fits neatly into CI/CD pipelines and executive dashboards. It is also incomplete.
The strongest evidence comes from the web itself. WebAIM’s 2026 Million report, which evaluates the top one million home pages, found that 95.9% still contained automatically detectable WCAG failures, with an average of 56.1 errors per page. More strikingly, that figure rose from the previous year, reversing six consecutive years of gradual improvement. And that statistic captures only what automation can detect — it excludes barriers that depend on meaning, context, interaction, cognitive load, assistive-technology behavior, and human judgment. The measured problem is only the visible portion of a much larger one.
After more than two decades working on accessible web and mobile experiences, I have watched automated testing evolve from relatively simple rule engines into sophisticated systems that scan enormous digital estates, identify regressions, prioritize defects, and increasingly suggest fixes through generative AI. I use these tools, and I recommend them. The problem begins only when organizations confuse better automation with complete accessibility assurance — a conclusion that neither the research nor real-world experience supports.
What the research actually tells us
Automated accessibility testing performs extremely well when a requirement can be reduced to a deterministic question about code: whether an image carries an alt attribute, whether text meets a required contrast ratio, whether a form control exposes an accessible name, whether IDs are duplicated, or whether the rendered DOM contains structural violations. These are measurable conditions, and a machine can evaluate them consistently across thousands — or millions — of pages. That capability has enormous value.
The mistake is assuming that because automation performs these checks reliably, it therefore evaluates accessibility comprehensively. It does not. Research and industry analysis have consistently shown that automated tools identify only a portion of WCAG-related failures. A frequently cited range is roughly 20% to 40% of accessibility issues, depending on methodology and how coverage is defined; a 2020 Deque analysis placed automated detection near 32%, while Carnegie Mellon researchers in 2022 estimated approximately 25% to 37% of WCAG failures. Deque has more recently argued that automated tools may detect closer to 57% when accessibility is measured by the volume of real-world issues rather than by the number of success criteria that can be fully evaluated. That reframing is fair and useful — but even under the more optimistic interpretation, a substantial share of accessibility barriers remains beyond automated detection. This is not an engineering backlog waiting for a larger language model; it reflects a deeper limitation in what automated evaluation is designed to measure.
Accessibility is not only a property of code
To understand that limitation, consider how an automated checker works. The tool inspects the rendered page — typically the DOM after scripts and styles have executed — and evaluates a set of rules, each asking a question that can be answered programmatically: Does every relevant image have an alternative-text attribute? Does this text meet the required contrast threshold? Does this control expose an accessible name? Are the required relationships represented correctly in the markup?
These tests work precisely because they are deterministic. The checker does not need to understand why an image exists, what a button actually does, or how the application feels through a screen reader. It only needs to determine whether the code satisfies a measurable condition. That is automation’s strength — and its boundary. The moment a requirement depends on meaning, purpose, sequence, comprehension, or experience, the problem changes character. “Does the image contain alt text?” is a code-inspection problem. “Does the alt text communicate the information a blind user needs in this context?” is a human-judgment problem. That distinction recurs throughout accessibility testing.
The barriers machines still struggle to evaluate
Alt text can exist and still fail the user. A scanner can verify that an image has an alt attribute; it cannot reliably determine whether the description is useful. Imagine a product image whose alternative text reads simply “shoe.” The attribute exists and the rule passes — but if the page is selling that shoe’s waterproof membrane, tread pattern, and reinforced toe, “shoe” gives a blind shopper almost none of the information the image was meant to convey. The right question is not whether alternative text is present, but what information the user needs from this image in this specific context. Generative AI may improve the fluency of automatically generated alt text, yet it introduces a subtler failure: the model can accurately describe what appears in the frame while missing why the image exists in the user’s journey. Accessibility depends on the second question.
Reading order is a user-experience problem. A page can contain perfectly valid markup while presenting information to assistive technology in an illogical sequence. CSS can visually reposition elements so that a sighted user perceives a clear hierarchy while a screen reader, following the underlying DOM, encounters those ideas out of order. The scanner sees valid markup; the screen-reader user hears a scrambled page. The difficulty compounds in dynamic interfaces: when a dialog opens, does focus move into it, and when it closes, does focus return to the control that launched it? When validation fails, is the user informed immediately, moved to the problem, and given a way to recover? These are interaction questions, and interaction must be experienced to be judged.
A focus indicator can exist and still be invisible. Automated tools can sometimes confirm that a focus style is defined, but accessibility depends on whether users can actually perceive it under real conditions. A focus ring may be technically present yet indistinguishable against a particular component background, and text that passes a contrast calculation in isolation may become unreadable over a photograph, gradient, or video. The machine evaluates the token; the user experiences the interface, and those are not always the same thing.
Labels can be technically valid and functionally meaningless. Accessibility failures often hide behind acceptable strings — “Click here,” “Learn more,” “Invalid input.” Each may satisfy certain automated rules while telling the user nothing about what action is available, where a link leads, or what needs to be corrected. A page with ten links labeled “Learn more” exposes an accessible name for every one of them, yet a screen-reader user navigating a links list encounters ten indistinguishable choices. The code passes; the experience fails.
Assistive technology does not behave like a scanner. The most important limitation is also the most obvious: automated tools do not use a product the way disabled people do. A static rule cannot fully answer whether a live region announces at the right moment, whether a custom dropdown behaves predictably with a screen reader, whether a voice-control user can identify and activate the intended element, whether magnification preserves enough context to stay oriented, or whether a keyboard-only user can complete an entire transaction without losing focus. Accessibility is not experienced one DOM node at a time; it is experienced as a journey. And that journey varies across technologies — JAWS, NVDA, VoiceOver, magnifiers, switch devices, and voice-control systems can each interpret the same implementation differently. Automation can validate patterns against technical expectations; only hands-on testing reveals what happens when those patterns meet real assistive technology.
Cognitive accessibility is even harder to automate. Accessibility is not limited to visual, auditory, or motor disabilities. People with cognitive and learning disabilities may depend on clear language, predictable interactions, consistent navigation, understandable instructions, manageable information density, and effective error recovery. A machine can measure sentence length; it cannot determine whether a complex workflow makes sense to the person trying to complete it, whether instructions impose unnecessary cognitive load, or whether terminology is familiar to the intended audience. Yet those factors frequently decide whether the user succeeds, which makes cognitive accessibility one of the clearest examples of the gap between technical conformance and human usability.
AI introduces a new risk: plausible accessibility
Traditional automated testing primarily identified problems. Generative AI changes the equation because it now proposes solutions — a powerful capability that also creates a new form of risk. When an AI-generated fix removes an automated warning, teams are naturally tempted to assume the underlying problem is solved. The code changes, the scanner turns green, the ticket closes. But satisfying a rule is not the same as satisfying the human need behind it.
Consider an icon-only button with no accessible name. A scanner correctly flags it, and an AI assistant resolves the warning by adding aria-label=”button”. The element now technically has an accessible name, and the check passes — yet the user has learned almost nothing. What does the button do: close a dialog, open a menu, delete an item, add a product to the cart? The correct label depends on intent. A human reviewer would write aria-label=”Add to cart” or aria-label=”Close dialog”, because the difference is semantic, not syntactic. The assistant resolved the machine-detectable symptom without understanding the interaction. This matters all the more because generative AI communicates with such fluency: poor automated advice used to look mechanical, but AI-generated advice often looks authoritative, which makes incorrect recommendations easier to trust — and easier to scale.
ARIA illustrates the point. WebAIM has repeatedly reported an uncomfortable correlation: pages containing more ARIA often contain more detectable accessibility errors. That does not mean ARIA causes poor accessibility; it means ARIA is powerful because it changes the semantic information exposed to assistive technology. Implemented correctly, it can make complex interfaces substantially more usable. Implemented incorrectly, it tells assistive technology something that is simply untrue. The problem is not the presence of accessibility technology but the application of it without sufficient understanding — and generative AI, which lowers the effort required to produce ARIA roles, labels, and descriptions, can now replicate a misunderstanding across hundreds of components almost instantly. Automation accelerates good decisions; it can accelerate bad ones just as easily.
Where AI creates genuine value
None of this argues for removing AI from accessibility engineering. The evidence points toward a more useful conclusion: AI is most powerful when it extends the reach of human expertise rather than attempting to replace it. Used well, AI can triage large digital estates to show experts where to investigate first, detect structural regressions before they reach production, generate initial alt-text suggestions for human review, explain WCAG requirements to engineers in plainer language, surface recurring implementation patterns across a codebase, accelerate remediation, support accessibility education inside engineering teams, and absorb the repetitive work of deterministic checks so that experts can concentrate on the problems that require interpretation and judgment.
That division of labor is not a weakness; it is the architecture accessibility programs should be designing for. Automation provides breadth, speed, consistency, and scale. Human evaluation provides meaning, context, prioritization, and lived experience. AI should function as an accessibility accelerant — never the final accessibility authority.
Stop measuring accessibility only through detectable errors
One of the most important implications is methodological. Organizations frequently gauge progress using automated defect counts or accessibility scores because those metrics are easy to generate — but what is easy to measure is not necessarily what matters most. A reduction in automatically detectable violations demonstrates improvement in the subset of accessibility the tool can observe; it does not, on its own, demonstrate that disabled users can complete critical journeys.
That calls for a broader measurement model. Alongside “How many automated violations remain?”, teams should ask whether users can complete the critical workflow with a keyboard alone, whether the experience holds up across major screen readers, whether users can recover from errors, whether dynamic changes are announced appropriately, whether the content is understandable, and whether people using assistive technologies succeed at comparable rates. That shifts accessibility measurement away from pure defect counting and toward human-outcome measurement — a harder research problem, and a more meaningful one.
The stakes extend beyond compliance scores
The World Health Organization estimates that approximately 1.3 billion people live with significant disability. That is not an edge case; it is an enormous global audience whose ability to participate in commerce, employment, education, healthcare, banking, government services, and social life increasingly depends on accessible technology. The legal environment is tightening in parallel: U.S. federal web-accessibility lawsuits reached 3,117 in 2025 by Seyfarth Shaw’s count, a 27% year-over-year increase, and the European Accessibility Act entered into force in June 2025.
Even so, legal exposure should not be the primary motivation. A product can earn an excellent scanner score and still exclude people. A compliance dashboard cannot tell you whether someone using a screen reader completed checkout; a CI pipeline cannot tell you whether a user with limited vision understood an error message; a perfect automated score cannot tell you whether someone with a cognitive disability finished a complex application without becoming lost. Those are human outcomes, and accessibility exists to improve them. That is why I increasingly regard digital accessibility not as a technical checklist but as a human-rights issue — and rights cannot be reduced to whether a pattern-matching system returned zero errors.
A better accessibility engineering model
The practical alternative is not complicated. It requires changing what an organization considers “done.”
- Automate the floor, not the ceiling. Run automated and AI-assisted testing continuously — in pull requests, CI/CD pipelines, regression suites, component libraries, and production monitoring — to catch structural problems as early and cheaply as possible. But treat a clean scan as the minimum acceptable starting point, never proof of accessibility.
- Require human verification of AI-generated fixes. Review AI-generated accessibility code against the intent of the requirement. The question is not “Did the warning disappear?” but “Did the experience improve?” Remediation is not complete simply because a rule passed after the change.
- Test the journey, not only the component. Component-level testing is essential, but people interact with workflows. Exercise complete experiences with keyboard navigation, screen readers, magnification, and voice control. A checkout button can be accessible while checkout is not; a modal can satisfy its local requirements while breaking focus across the larger flow.
- Include disabled users in the evaluation. Expert testing surfaces technical and experiential barriers automation misses; testing with disabled users reveals whether the product actually works for the people accessibility requirements exist to serve. Their participation is not merely validation — it is evidence.
- Keep accessibility expertise in the decision loop. Accessibility is a specialized discipline spanning standards interpretation, semantic engineering, assistive technology, interaction design, usability, content strategy, disability experience, and regulation. A linter cannot replace that expertise, and neither can a large language model. The organizations that gain the most from AI will be those that give their experts better tools, not those that remove them.
The future is human-guided accessibility intelligence
Accessibility automation will keep improving. Models will interpret interfaces more capably, computer vision will strengthen contextual analysis, agentic systems may eventually navigate applications through simulated assistive-technology interactions, and testing platforms will grow more sophisticated. All of that is welcome. But the central question should not be how much accessibility testing we can automate. The more meaningful question is how automation can help us determine whether people with disabilities can successfully use digital products — a fundamentally different objective. The first optimizes testing efficiency; the second optimizes human outcomes, and the future of the field should pursue both without confusing them.
Automation can find thousands of problems faster than any individual expert, and AI can help engineers understand and remediate them faster than traditional workflows. But neither capability changes the fundamental purpose of accessibility. The purpose is not to satisfy the scanner; it is to enable the person. Until a machine can reliably determine whether a real human being — with a particular disability, assistive technology, context, and goal — can actually accomplish what they came to do, accessibility testing will continue to require human judgment. Not instead of AI, but alongside it, and ultimately responsible for deciding whether the technology has truly served the person on the other side of the screen.
