Abstract: The quality of modern software relies heavily on the effective use of static code analysis tools. To improve their usefulness, these tools should be evaluated using a framework that ...

Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.

AI coding assistants and agentic workflows represent the future of software development and will continue to evolve at a rapid pace. But while LLMs have become adept at generating functionally correct ...