GPT4Graph: Can Large Language Models Understand Graph Structured Data? An Empirical Evaluation and Benchmarking

Large language models (LLM) like ChatGPT have become indispensable to artificial general intelligence (AGI), demonstrating excellent performance in various natural language processing tasks. Graph data is ubiquitous and an essential part of AGI. The training corpus of large language models often includes some algorithmic components, which allows them to achieve certain effects on some graph data-related problems. However, there is still little research on their performance on a broader range of graph-structured data. In this paper, we conduct an empirical study to assess the proficiency of LLMs in comprehending graph data, employing a diverse range of structural and semantic-related tasks that evaluate the LLMs' capabilities in graph understanding. Through our study, we uncover current limitations and future directions of LLMs in comprehending graph and performing associated reasoning tasks.