Unlocking Ancient Pictographs: A Multi-modal LLM Approach to Dongba Characters Understanding

Abstract

The Dongba script, the only living pictographic writing system, poses unique challenges for computational modeling due to its visual complexity and limited resources. Previous work has primarily relied on convolutional neural networks for image-based recognition, but these approaches struggle with generalization to unseen characters and fail to capture contextual information. This study presents the first systematic evaluation of multimodal large language models (LLMs) for Dongba character recognition. We benchmark state-of-the-art pre-trained multimodal LLMs under zero-shot and two-shot prompting, and further develop DB-LLM, a fine-tuned multimodal model adapted specifically for Dongba scripts. Experimental results reveal that pre-trained models achieve less than 2% accuracy, indicating limited capacity for direct recognition. In contrast, DB-LLM achieves 78.4% accuracy on the seen test set, representing a substantial improvement and demonstrating the effectiveness of targeted adaptation. However, the model shows limited ability to generalize to unseen classes, highlighting the need for future research on cross-inventory generalization and robustness. These findings establish a foundation for computational analysis of Dongba and contribute to the broader study of low-resource pictographic writing systems.