ARC-Guard: A High-Precision Multi-Agent Collaborative Framework for Email Classification based on RAG and CoT

Jiajun Deng; Yuanqing Xian

doi:10.54691/f13knk31

Authors

Jiajun Deng
Yuanqing Xian

DOI:

https://doi.org/10.54691/f13knk31

Keywords:

Email Classification; Large Language Models; Multi-Agent; Retrieval-Augmented Generation; Chain-of-Thought.

Abstract

As a critical infrastructure for personal and professional communication, email is facing increasingly severe threats from sophisticated phishing and spam campaigns, making robust email classification systems essential. Traditional classification methods struggle with semantic nuances, while standard Large Language Models (LLMs) often suffer from hallucinations or lack domain-specific context. To address these challenges, we propose ARC-Guard, a multi-agent framework specifically designed for high-precision email classification, which integrates Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) reasoning. The system comprises three dedicated agents: an Initial Analysis Agent for surface-level inspection, a Dual-Path RAG Agent for vector retrieval of similar historical emails, and a Chain-of-Thought Agent that synthesizes retrieved contexts to generate interpretable verdicts. Evaluations on the SecEmail dataset show that ARC-Guard achieves a state-of-the-art (SOTA) accuracy of 90.42%, significantly outperforming baseline models. These results demonstrate that combining retrieval mechanisms with step-by-step reasoning substantially enhances the robustness and interpretability of email threat detection.

Downloads

Download data is not yet available.

References

[1] Zhang J, Bu H, Wen H, et al. When llms meet cybersecurity: A systematic literature review[J]. Cybersecurity, 2025, 8(1): 55.

[2] Lazer S J, Aryal K, Gupta M, et al. A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes[J]. arXiv preprint arXiv:2601.05293, 2026.

[3] Shinn N, Cassano F, Gopinath A, et al. Reflexion: Language agents with verbal reinforcement learning[J]. Advances in neural information processing systems, 2023, 36: 8634-8652.

[4] Gao Y, Xiong Y, Gao X, et al. Retrieval-augmented generation for large language models: A survey[J]. arXiv preprint arXiv:2312.10997, 2023, 2(1): 32.

[5] Sharma C. Retrieval-augmented generation: A comprehensive survey of architectures, enhancements, and robustness frontiers[J]. arXiv preprint arXiv:2506.00054, 2025.

[6] Altwaijry N, Al-Turaiki I, Alotaibi R, et al. Advancing phishing email detection: A comparative study of deep learning models[J]. Sensors, 2024, 24(7): 2077.

[7] Kyaw P H, Gutierrez J, Ghobakhlou A. A systematic review of deep learning techniques for phishing email detection[J]. Electronics, 2024, 13(19): 3823.

[8] He D, Lv X, Xu X, et al. Double-layer detection of internal threat in enterprise systems based on deep learning[J]. IEEE Transactions on Information Forensics and Security, 2024, 19: 4741-4751.

[9] Hosseinzadeh M, Ali U, Ali S, et al. Improving phishing email detection performance through deep learning with adaptive optimization[J]. Scientific Reports, 2025, 15(1): 36724.

[10] Tang R, Chuang Y N, Hu X. The science of detecting LLM-generated text[J]. Communications of the ACM, 2024, 67(4): 50-59.

[11] Koide T, Fukushi N, Nakano H, et al. Chatspamdetector: Leveraging large language models for effective phishing email detection[C]//International Conference on Security and Privacy in Communication Systems. Cham: Springer Nature Switzerland, 2024: 297-319.

[12] Heiding F, Schneier B, Vishwanath A, et al. Devising and detecting phishing emails using large language models[J]. IEEE Access, 2024, 12: 42131-42146.

[13] Goldenits G, Koenig P, Raubitzek S, et al. Small Language Models for Phishing Website Detection: Cost, Performance, and Privacy Trade-Offs[J]. arXiv preprint arXiv:2511.15434, 2025.

[14] Yu Y, Ping W, Liu Z, et al. Rankrag: Unifying context ranking with retrieval-augmented generation in llms[J]. Advances in Neural Information Processing Systems, 2024, 37: 121156-121184.

[15] Nilsson P. Phishing for Trust in the AI Age: A Quasi-Experimental Study on Individual Human Factors Influencing Trust in AI-Driven Phishing Attempts[J]. 2024.

[16] Edge D, Trinh H, Cheng N, et al. From local to global: A graph rag approach to query-focused summarization[J]. arXiv preprint arXiv:2404.16130, 2024.

[17] Qian C, Liu W, Liu H, et al. Chatdev: Communicative agents for software development[C]//Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers). 2024: 15174-15186.

[18] Hong S, Zhuge M, Chen J, et al. MetaGPT: Meta programming for a multi-agent collaborative framework[C]//The twelfth international conference on learning representations. 2023.

[19] Bai J, Bai S, Chu Y, et al. Qwen technical report[J]. arXiv preprint arXiv:2309.16609, 2023.

[20] Bi X, Chen D, Chen G, et al. Deepseek llm: Scaling open-source language models with longtermism[J]. arXiv preprint arXiv:2401.02954, 2024.