基于大型语言模型的智能体记忆机制综合调研与应用分析资源-CSDN下载

需积分: 5 23 浏览量 2025-01-11 12:11:22 上传评论收藏 1.95MB PDF 举报

资源推荐

资源详情

资源评论

A Survey on the Memory Mechanism of Large

Language Model based Agents

Zeyu Zhang

, Xiaohe Bo

, Chen Ma

, Rui Li

, Xu Chen

, Quanyu Dai

Jieming Zhu

, Zhenhua Dong

, Ji-Rong Wen

Gaoling School of Artiﬁcial Intelligence, Renmin University of China, Beijing, China

Huawei Noah’s Ark Lab, China

[email protected], [email protected]

Abstract

Large language model (LLM) based agents have recently attracted much attention

from the research and industry communities. Compared with original LLMs, LLM-

based agents are featured in their self-evolving capability, which is the basis for

solving real-world problems that need long-term and complex agent-environment

interactions. The key component to support agent-environment interactions is the

memory of the agents. While previous studies have proposed many promising mem-

ory mechanisms, they are scattered in different papers, and there lacks a systemati-

cal review to summarize and compare these works from a holistic perspective, fail-

ing to abstract common and effective designing patterns for inspiring future studies.

To bridge this gap, in this paper, we propose a comprehensive survey on the memory

mechanism of LLM-based agents. In speciﬁc, we ﬁrst discuss “what is” and “why

do we need” the memory in LLM-based agents. Then, we systematically review

previous studies on how to design and evaluate the memory module. In addition, we

also present many agent applications, where the memory module plays an important

role. At last, we analyze the limitations of existing work and show important future

directions. To keep up with the latest advances in this ﬁeld, we create a repository

at https://github.com/nuster1128/LLM_Agent_Memory_Survey.

Personal Assistant

Please help me to explain

“LLM-based agent”.

A LLM-based agent is a

type of artificial ……

In which scenarios does it

have applications?

Personal assistant, game,

code generation, ……

(Knowledge) According to the previous works, large

language model based agents refer to artificial ……

(Context) The current topic is LLM-based agent. “It”

refers to LLM-based agents in this conversation.

Social Simulation

I'm a compassionate physician

specializing in cardiology,

committed to improving patients'

heart health and well-being.

I'm a skilled nurse dedicated to

patient care, ensuring comfort and

supporting health with empathy

and expertise.

Role-playing

I' m a Smurf, and

Smurfs are us!

Have you ever had

a dream?

Magic is all

around the us!

Jarvis, we must

first learn to run!

Wubalubadu.

Bdub Wuc koop.

I' m Batman, the

lights of city.

[Iron Man] My na me is Iron Man,

also known as Tony Stark. I am

the founder of Stark Industries

and a member of the Avengers.

As one of the gen ius inventors

and billionaire, I have created

the most advanced armo r in the

world, which not only protects

me but also gives me incredible

strength and the ability to fly.

Open-world Game

Skills & Knowledge

Code Generation

def bubble_sort(arr):

n = len(arr)

# Traverse through all array elements

for i in range(n):

# Last i elements are already in place

for j in range(0, n-i-1):

# Traverse the array from 0 to n-i-1

# Swap if the element found is greater

than the next element

if arr[j] > arr[j+1]:

arr[j], arr[j+1] = arr[j+1], arr[j]

return arr

Sort the numbers in ascending order.

Bubble sort repeatedly step s through the list,

compares adjacent elements and swaps them

if they are in wrong order. The pass through

the list is repeated until the list is sorted.

Bubble sort can reorder a list of numbers.

Development Group

Recommendation

I want to buy a dress for the

graduation party.

(Context) She just bought a new

blue dress. So she may need a

white accessories to match it.

(Personal Preference) She often buys blue

clothes. She values the cost-effectiv eness of

items, especially on clothes. She likes small

things with light co lors, such as ear pendants.

You may like this blue dress. It is

of good quality and great price.

Would you like to buy a

waistband for your dress?

Great! I like this bule one.

I will buy it for the party.

Medicine

David, a 38-year-old male with a history of allergies and sinus

infections, has a family history of diabetes and hypertension. As

a smoker of about one pack a day and an occasional drinker,

his lifestyle choices may contribute to his health risks. After

traveling to a tropical countr y where mosquito-borne illnesses

are prevalent, he has experienced symptoms such as mild

fatigue, headache, and muscle aches for the past week.

For three days, I’ve had a

fever ranging from

100.5

F to 102

F, a rash

on my limbs, join t pa in

and swelling, especially

in my hands, episodes of

diarrhea, abdominal pain,

and nau sea, leading to a

loss of appetite.

The patient‘s travel history and

symptoms suggest dengue fever,

a mosquito-transmitted illness.

Finance

Yesterday, the financ ial market underwent significant fluctuations.

Equit y markets initially slumped as investors responded to weak

corporate earnings reports and concerns over global economic

growth prospects. Bond yields declined as investors sought safer

assets. However, senti ment improved in the af ternoon following

better-than-expected econom ic data from a major economy, which

boosted investor confidence. As a result, stock prices recovered,

and the market closed with modest gains.

The yield on the

10-year Treasury

note rose by 0.2

percentage points, reaching

its highest level in two years.

Investors sought safer assets,

leading to increased demand

for government bonds.

Corporate bonds experienced

a decline in interest, with their

yields rising by 0.15

percentage points.

LLM

Memory

Agent

Environment

Figure 1: The importance of the memory module in LLM-based agents.

Preprint. Under review.

arXiv:2404.13501v1 [cs.AI] 21 Apr 2024

Contents

1 Introduction 4

2 Related Surveys 5

2.1 Surveys on Large Language Models . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 Surveys on Large Language Model-based Agents . . . . . . . . . . . . . . . . . . 7

3 What is the Memory of LLM-based Agent 7

3.1 Basic Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.2 Narrow Deﬁnition of the Agent Memory . . . . . . . . . . . . . . . . . . . . . . . 9

3.3 Broad Deﬁnition of the Agent Memory . . . . . . . . . . . . . . . . . . . . . . . . 9

3.4 Memory-assisted Agent-Environment Interaction . . . . . . . . . . . . . . . . . . 9

4 Why We Need the Memory in LLM-based Agent 10

4.1 Perspective of Cognitive Psychology . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.2 Perspective of Self-Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.3 Perspective of Agent Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 How to Implement the Memory of LLM-based Agent 11

5.1 Memory Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5.1.1 Inside-trial Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

5.1.2 Cross-trial Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5.1.3 External Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5.2 Memory Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5.2.1 Memory in Textual Form . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5.2.2 Memory in Parametric Form . . . . . . . . . . . . . . . . . . . . . . . . . 16

5.2.3 Advantages and Disadvantages of Textual and Parametric Memory . . . . . 17

5.3 Memory Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5.3.1 Memory Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5.3.2 Memory Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5.3.3 Memory Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

6 How to Evaluate the Memory in LLM-based Agent 20

6.1 Direct Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

6.1.1 Subjective Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

6.1.2 Objective Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

6.2 Indirect Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.2.1 Conversation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.2.2 Multi-source Question-answering . . . . . . . . . . . . . . . . . . . . . . 22

6.2.3 Long-context Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 22

1 Introduction

"Without memory, there is no culture. Without memory, there would be no civilization, no society, no future."

Elie Wiesel, 1928-2016

Recently, large language models (LLMs) have achieved remarkable success in a large number of

domains, ranging from artiﬁcial intelligence and software engineering to education and social sci-

ence [

–

]. Original LLMs usually accomplish different tasks without interacting with environments.

However, to achieve the ﬁnal goal of artiﬁcial general intelligence (AGI), intelligent machines should

be able to improve themselves by autonomously exploring and learning from the real world. For

example, if a trip-planning agent intends to book a ticket, it should send an order request to the ticket

website, and observe the response before taking the next action. A personal assistant agent should

adjust its behaviors according to the user’s feedback, providing personalized responses to improve

user’s satisfaction. To further push the boundary of LLMs towards AGI, recent years have witnessed a

large number of studies on LLM-based agents [

], where the key is to equip LLMs with additional

modules to enhance their self-evolving capability in real-world environments.

Among all the added modules, memory is a key component that differentiates the agents from

original LLMs, making an agent truly an agent (see Figure 1). It plays an extremely important role

in determining how the agent accumulates knowledge, processes historical experience, retrieves

informative knowledge to support its actions, and so on. Around the memory module, people have

devoted much effort to designing its information sources, storage forms, and operation mechanisms.

For example, Shinn et al.

[5]

incorporate both in-trial and cross-trial information to build the memory

module for enhancing the agent’s reasoning capability. Zhong et al.

[6]

store memory information in

the form of natural languages, which is explainable and friendly to the users. Modarressi et al.

[7]

design both memory reading and writing operations to interact with environments for task solving.

While previous studies have designed many promising memory modules, there still lacks a systemic

study to view the memory modules from a holistic perspective. To bridge this gap, in this paper,

we comprehensively review previous studies to present clear taxonomies and key principles for

designing and evaluating the memory module. In speciﬁc, we discuss three key problems including:

(1) what is the memory of LLM-based agents? (2) why do we need the memory in LLM-based

agents? and (3) how to implement and evaluate the memory in LLM-based agents? To begin with,

we detail the concepts of memory in LLM-based agents, providing both narrow and broad deﬁnitions.

Then, we analyze the necessity of memory in LLM-based agents, showing its importance from

three perspectives including cognitive psychology, self-evolution, and agent applications. Based on

the problems of “what” and “why”, we present commonly used strategies to design and evaluate

the memory modules. For the memory design, we discuss previous works from three dimensions,

that is, memory sources, memory forms, and memory operations. For the memory evaluation, we

introduce two widely used approaches including direct evaluation and indirect evaluation via speciﬁc

agent tasks. Next, we discuss agent applications including role-playing, social simulation, personal

assistant, open-world games, code generation, recommendation, and expert systems, in order to show

the importance of the memory module in practical scenarios. At last, we analyze the limitations of

existing work and highlight signiﬁcant future directions.

The main contributions of this paper can be summarized as follows: (1) We formally deﬁne the mem-

ory module and comprehensively analyze its necessity for LLM-based agents. (2) We systematically

summarize existing studies on designing and evaluating the memory module in LLM-based agents,

providing clear taxonomies and intuitive insights. (3) We present typical agent applications to show

the importance of the memory module in different scenarios. (4) We analyze the key limitations of

existing memory modules and show potential solutions for inspiring future studies. To our knowledge,

this is the ﬁrst survey on the memory mechanism of LLM-based agents.

The rest of this survey is organized as follows. First, we provide a systematical meta-survey for the

ﬁelds of LLMs and LLM-based agents in Section 2, categorizing different surveys and summarizing

their key contributions. Then, we discuss the problems of “what is”, “why do we need” and “how

to implement and evaluate” the memory module in LLM-based agents in Section 3 to 6. Next, we

show the applications of memory-enhanced agents in Section 7. The discussions of the limitations of

existing work and future directions come at last in Section 8 and Section 9.

2 Related Surveys

In the past two years, LLMs have attracted much attention from the academic and industry commu-

nities. To systemically summarize the studies in this ﬁeld, researchers have written a lot of survey

papers. In this section, we brieﬂy review these surveys (see Figure 2 for an overview), highlighting

their major focuses and contributions to better position our study.

2.1 Surveys on Large Language Models

In the ﬁeld of LLMs, Zhao et al.

[70]

present the ﬁrst comprehensive survey to summarize the

background, evolution paths, model architectures, training methodologies, and evaluation strategies

of LLMs. Hadi et al.

[71]

and Min et al.

[72]

also conduct LLM surveys from the holistic view,

which, however, provide different taxonomies and understandings on LLMs. Following these surveys,

people dive into speciﬁc aspects of LLMs and review the corresponding milestone studies and key

technologies. These aspects can be classiﬁed into four categories including the fundamental problems,

evaluation, applications, and challenges of LLMs.

Fundamental problems. The surveys in this category aim to summarize techniques that can

be leveraged to tackle fundamental problems of LLMs. Speciﬁcally, Zhang et al.

[8]

provide a

comprehensive survey on the methods of supervised ﬁne-tuning, which is a key technique for better

training LLMs. Shen et al.

[9]

, Wang et al.

[10]

and Liu et al.

[11]

present surveys on the alignment of

LLMs, which is a key requirement for LLMs to produce outputs consistent with human values. Gao

et al.

[12]

propose a survey on the retrieval-augmented generation (RAG) capability of LLMs, which

is key to providing LLMs with factual and up-to-date knowledge and removing hallucinations. Qin

et al.

[18]

summarize the state-of-the-art methods on enabling LLMs to leverage external tools, which

is fundamental for LLMs to expand their capability in domains that require specialized knowledge.

Wang et al.

[13]

, Yao et al.

[14]

, Wang et al.

[15]

, Feng et al.

[16]

and Zhang et al.

[17]

present

surveys on the direction of LLM knowledge editing, which is important for customizing LLMs to

satisfy speciﬁc requirements. Huang et al.

[19]

, Wang et al.

[20]

and Pawar et al.

[21]

focus on

long-context capabilities of LLMs, which is critical for LLMs to process more information at each

time and enhance their application scenarios. Wu et al.

[22]

, Song et al.

[23]

, Caffagni et al.

[24]

and

Yin et al.

[25]

summarize multi-modal LLMs, which expands the capability of LLMs from text to

visual and other modalities. The above surveys mainly focus on the effectiveness of LLMs. Another

important aspect of LLMs is their training and inference efﬁciency. To summarize studies on this

aspect, Zhu et al.

[30]

, Xu and McAuley

[31]

, Wang et al.

[32]

and Park et al.

[33]

systematically

review the techniques of model compression. Ding et al.

[81]

and Xu et al.

[29]

analyze and conclude

the studies on parameter efﬁcient ﬁne-tuning. Bai et al.

[26]

, Wan et al.

[27]

, Miao et al.

[28]

and

Ding et al. [81] put more focuses on the efﬁciency of resource utilization in a general sense.

Evaluation. The surveys in this category focus on how to evaluate the capability of LLMs. Speciﬁ-

cally, Chang et al.

[34]

comprehensively summarize the evaluation methods from an overall perspec-

tive. It encompasses different evaluation tasks, methods, and benchmarks, which serve as critical

parts in assessing LLM performances. Guo et al.

[35]

care more about the evaluation targets and

describe how to evaluate the knowledge, alignment, and safety control capabilities of LLMs, which

supplement evaluation metrics beyond performance.

Applications. The surveys in this category aim to summarize models that leverage LLMs to improve

different applications. More concretely, Zhu et al.

[37]

focus on the ﬁeld of information retrieval

(IR) and summarize studies on LLM-based query processes. Xu et al.

[38]

pay more attention to

information extraction (IE) and provide comprehensive taxonomies for LLM-based models in this

ﬁeld. Li et al. [50], Lin et al. [51] and Wang et al. [52] discuss the applications of LLMs in the ﬁeld

of recommender system, where they utilize agents to generate data and provide recommendations.

Fan et al.

[39]

, Wang et al.

[40]

, and Zheng et al.

[41]

concentrate on how LLMs can beneﬁt software

engineering (SE) in terms of software design, development, and testing. Zeng et al.

[42]

summarize

LLM-based methods in the ﬁeld of robotics. Cui et al.

[43]

and Yang et al.

[44]

focus on the

application of autonomous driving and summarize models in this domain based on LLMs from

different perspectives. Beyond the above domains in artiﬁcial intelligence, LLMs have also been

used in natural and social science. He et al.

[45]

, Zhou et al.

[46]

and Wang et al.

[47]

summarize the

applications of LLMs in medicine. Li et al.

[48]

focus on the applications of LLMs in ﬁnance. He

et al. [49] review the models on leveraging LLMs to improve the development of psychology.

剩余38页未读，继续阅读

评论收藏

内容反馈

码流怪侠

粉丝: 4w+

基于大型语言模型的智能体记忆机制综合调研与应用分析

基于大型语言模型的多智能体协作框架MetaGPT的设计与应用：提升复杂软件开发效率

【AI 人工智能】大型语言模型的实现技术原理与应用.pdf

基于大型语言模型的自主智能体研究综述

基于大型语言模型的多智能体课堂模拟系统SimClass的应用与效果评估

大型语言模型的快速介绍.pptx

基于大型语言模型的工具对电池研究的机遇与挑战.pdf

【人工智能领域】基于Transformer的DeepSeek技术演进与前沿探索：大型语言模型及新一代智能体的设计与应用

一个基于 Java 的 LLM（大语言模型）应用开发（及编排）框架

基于大型语言模型的视频评论回复系统机器人源码

基于LSTM和大语言模型的智能股票对话与分析助手Python实现源码.zip

大型语言模型 (LLM)全解读.pdf

第10章 基于大模型的智能体-大语言模型通识(赵建勇).pptx

2024 Agent+RAG：基于大模型的融合应用探索（八大案例，共146页）.pdf

基于大模型的企业级RAG构建、应用与开发

【教育考试智能化】基于AI辅助评卷系统的智能评阅算法效果综合评价模型设计与应用：从数据分布到跨学科对比分析

【Python】基于大型语言模型的评论回复机器人_pgj.zip

强化大型语言模型作为智能体的能力研究：引入AgentTuning方法及其应用效果评估

### 人工智能基于大语言模型的AI Agent技术进展与行业应用现状综述：从DeepSeek、Manus到企业级智能体解决方案

人工智能DeepSeek技术溯源及前沿探索：大型语言模型与新一代智能体的设计与发展

DeepSeek：从入门到精通，清华大学新闻与传播学院，104页PDF

Ollama 0.5.4版本

清华大学DeepSeek从入门到精通-高清免费

ollama 0.5.7.0安装包，来自git

DeepSeek使用指南：涵盖注册登录、基础对话、文件处理到自动化工作流的技术应用与优化

stable-diffusion部署需要的包

Notepad++ 8.5.6最新版 64位安装包

DeepSeek从入门到精通：大语言模型技术及其应用指南

大规模语言模型：从理论到实践

【Cursor无限.exe】，一个可以帮助你绕过Cursor的试用期限制，轻松继续使用这款强大的AI工具！

容器安全实践（一）：概念篇 - 从“想当然”到“真相”

discuz论坛插件DZ插件 discuz版块横排美化 v1.2.1版 X3.2

最新资源

第10章基于大模型的智能体-大语言模型通识(赵建勇).pptx