A Survey on Human Preference Learning for Large Language Models¶
Arxiv Link - 2024-06-17 03:52:51
Abstract¶
The recent surge of versatile large language models (LLMs) largely depends on aligning increasingly capable foundation models with human intentions by preference learning, enhancing LLMs with excellent applicability and effectiveness in a wide range of contexts. Despite the numerous related studies conducted, a perspective on how human preferences are introduced into LLMs remains limited, which may prevent a deeper comprehension of the relationships between human preferences and LLMs as well as the realization of their limitations. In this survey, we review the progress in exploring human preference learning for LLMs from a preference-centered perspective, covering the sources and formats of preference feedback, the modeling and usage of preference signals, as well as the evaluation of the aligned LLMs. We first categorize the human feedback according to data sources and formats. We then summarize techniques for human preferences modeling and compare the advantages and disadvantages of different schools of models. Moreover, we present various preference usage methods sorted by the objectives to utilize human preference signals. Finally, we summarize some prevailing approaches to evaluate LLMs in terms of alignment with human intentions and discuss our outlooks on the human intention alignment for LLMs.
Socials¶
| X | |
|---|---|
| 🚀 Exciting developments in the world of AI and NLP! Check out this insightful survey on how human preferences are introduced into Large Language Models (LLMs) to enhance their effectiveness across diverse contexts. The survey categorizes human feedback sources, preference modeling techniques, usage methods, and evaluation approaches for aligned LLMs. Dive deeper into the relationship between human preferences and LLMs for a comprehensive understanding. Read the full survey here: http://arxiv.org/abs/2406.11191v1 #AI #NLP #LLMs #PreferenceLearning #TechSurvey #HumanIntentions #Innovation #TechTrends |
🚀 Exciting insights on aligning human preferences with Large Language Models (LLMs) for enhanced effectiveness! This survey delves into the sources, formats, modeling, and evaluation of human feedback in LLMs. Dive deeper into the realm of preference-centered learning here: http://arxiv.org/abs/2406.11191v1 #AI #NLP #LLMs #TechResearch |