OPDAI at SemEval-2024 Task 6: Small LLMs can Accelerate Hallucination Detection with Weakly Supervised Data¶
Arxiv Link - 2024-02-20 11:01:39
Abstract¶
This paper mainly describes a unified system for hallucination detection of LLMs, which wins the second prize in the model-agnostic track of the SemEval-2024 Task 6, and also achieves considerable results in the model-aware track. This task aims to detect hallucination with LLMs for three different text-generation tasks without labeled training data. We utilize prompt engineering and few-shot learning to verify the performance of different LLMs on the validation data. Then we select the LLMs with better performance to generate high-quality weakly supervised training data, which not only satisfies the consistency of different LLMs, but also satisfies the consistency of the optimal LLM with different sampling parameters. Furthermore, we finetune different LLMs by using the constructed training data, and finding that a relatively small LLM can achieve a competitive level of performance in hallucination detection, when compared to the large LLMs and the prompt-based approaches using GPT-4.
Socials¶
| X | |
|---|---|
| 🚀 Exciting News in AI Research 🚀 Thrilled to share a groundbreaking paper on hallucination detection of Large Language Models (LLMs) that secured the second prize in the model-agnostic track of the SemEval-2024 Task 6! This unified system not only excelled in the model-aware track but also showcased impressive results in detecting hallucinations with LLMs across different text-generation tasks, all without the need for labeled training data. The team leveraged prompt engineering and few-shot learning to evaluate various LLMs on validation data, ultimately identifying those with superior performance to generate high-quality weakly supervised training data. By fine-tuning different LLMs using this data, the study revealed that even smaller LLMs can achieve remarkable levels of performance in hallucination detection, rivaling larger LLMs and prompt-based approaches employing GPT-4. Curious to dive deeper into this cutting-edge research? Check out the full paper here: http://arxiv.org/abs/2402.12913v1 #AI #LLMs #NLP #SemEval2024 #TechResearch #InnovationInAI |
🚀 Exciting news in the field of LLMs! This paper presents a unified system for hallucination detection using LLMs, achieving remarkable results in the SemEval-2024 Task 6. Learn more about their innovative approach and findings at: http://arxiv.org/abs/2402.12913v1 #AI #NLP #LLMs #SemEval2024 🧠🔍📚 |