Diversity Measures: Domain-Independent Proxies for Failure in Language Model Queries

Due to challenges such as hallucination, detecting errors in the output of a given prompt becomes an important challenge. In this work, we introduce “diversity measures” that are domain independent and can be used to measure the uncertainty in the result of a language model.

Preprint: https://arxiv.org/abs/2308.11189
Source code: https://github.com/lab-v2/diversity_measures
Lab website: https://labs.engineering.asu.edu/labv2

About the channel:
The Neuro Symbolic Channel provides the tutorials, courses, and research results on one of the most exciting areas in artificial intelligence and machine learning. With content originally from the AI course taught at Arizona State University, this channel brings you the latest at the intersection of symbolic methods (e.g., logic programming) and deep learning. Learn about the latest algorithms, Python packages, and progress toward larger goals such as artificial general intelligence (AGI).