Most modern ATS platforms now include some form of AI matching, and for what the ATS was designed to do, that layer genuinely helps. The ATS was built to manage application flow, maintain an audit trail, handle compliance, and give the recruiting team one place to move candidates through the pipeline. AI matching sits on top of that and speeds up sorting, which is real value.
What still comes up in most conversations with talent teams is that the shortlist does not always convert into a strong hire. The pattern is usually the same. Matching engines compare the language of a CV against the language of a JD, and CV language was never built to carry a reliable signal of capability. A backend engineer writes "built event-driven services on Kafka" while the JD says "experience with distributed systems at scale," and the engine scores it as weak even though it is a strong fit.
The way I have started thinking about it is that this is less a failure of the ATS and more a mismatch between expectation and design. The ATS is doing what it was built to do. Capability signals have to come from a different layer, whether that is validated skill assessments, verified project work, or evidence from the platforms where the work actually lives.
When you evaluate AI matching layered on your ATS, what test do you run to check whether it is reading skill or reading vocabulary? Would love to hear what has worked for your team.