Personas for role-playing language agents can be categorized into three distinct categories: demographic personas, character personas, and individualized personas (Chen, Wang, Xu, Tuan, Zhang, Shi, & Xie, 2024). Demographic personas focus on groups of people sharing common characteristics. Character personas represent well-established, widely-recognized individuals. Individualized personas refer to digital profiles built and continuously updated based on personalized user data.

Character profiling is summarizing profiles for characters from fictional stories (Yuan, Yuan, Cui, Lin, Wang, Xu, Chen, & Yang, 2024).

A character profile encompasses attributes, relationships, events, and personality. The basic attributes of a character encompass gender, skills, talents, objectives, and background. A character’s interpersonal relationships are a vital aspect of their profile. Events cover the experiences that characters have been part of or impacted by, marking a critical dimension of their profile. Personality refers to the lasting set of characteristics and behaviors that form an individual’s unique way of adapting to life (Yuan, Yuan, Cui, Lin, Wang, Xu, Chen, & Yang, 2024).

Other studies into persona-based decision-making similarly involve descriptions of characters, composed of the characters’ basic situations and storylines, and characters’ memories of current scenes, which can offer more detail (Xu, Wang, Chen, Yuan, Yuan, Liang, Chen, Dong, & Xiao, 2024).

The evaluation of role-playing language agents has two primary categories of criteria: role-playing capability evaluation and persona fidelity evaluation. Role-playing capability evaluations concern aspects such as anthropomorphic abilities, attractiveness, and usefulness, which encompass more granular dimensions including conversation ability, engagement, persona consistency, emotion understanding, theory of mind, and problem-solving ability. Persona fidelity evaluation concentrates on whether individual agents well replicate the intended personas, including their knowledge, linguistic habits, personality, beliefs, and decision-making (Chen, Wang, Xu, Tuan, Zhang, Shi, & Xie, 2024).

With respect to the evaluation of character profiling, there are internal and external evaluation. Internal evaluation involves factual consistency examination, comparing the model-summarized character profiles with reference profiles. External evaluation involves motivation recognition, thoroughly evaluating whether the summarized character profiles enhance models’ understanding of characters’ essences, investigating whether character profiles generated by models effectively aid in comprehending characters’ motivations behind their decisions (Yuan, Yuan, Cui, Lin, Wang, Xu, Chen, & Yang, 2024).

Chen, Jiangjie, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie et al. "From persona to personalization: A survey on role-playing language agents." arXiv preprint arXiv:2404.18231 (2024).

Xu, Rui, Xintao Wang, Jiangjie Chen, Siyu Yuan, Xinfeng Yuan, Jiaqing Liang, Zulong Chen, Xiaoqing Dong, and Yanghua Xiao. "Character is destiny: Can large language models simulate persona-driven decisions in role-playing?." arXiv preprint arXiv:2404.12138 (2024).

Yuan, Xinfeng, Siyu Yuan, Yuhan Cui, Tianhe Lin, Xintao Wang, Rui Xu, Jiangjie Chen, and Deqing Yang. "Evaluating character understanding of large language models via character profiling from fictional works." arXiv preprint arXiv:2404.12726 (2024).