2024年7月2日
Research report on accounting audit using large-scale language models (LLM)
Research Director Hiroshi Deguchi (Professor, Faculty of Commerce and Business Administration)
Co-researcher: Takao Terano (Vice President / Professor Platform for Arts and Sciences)
Akagi Kaya (Full-time lecturer Platform for Arts and Sciences)
1. Overview of research findings
In recent years, attention has been focused on accounting audits in real time, along with the concepts of real-time economy and real-time audits. In this study, Deguchi was in charge of the possibilities and characteristics of audits using LLM, Nakamura was in charge of the recent trends in audit concepts, and Akagi was in charge of the latest situation regarding the real-time economy. As a result, LLM is changing rapidly, and it is difficult to conclude its evaluation of accounting audits at this point in time.
If we divide accounting work into routine and non-routine parts, with regard to routine work, such as creating standard documents and reports based on journal entries and formats, converting image input into text, and no-coding routine typing work, the LLM can be expected to make a significant contribution, even if company-specific learning is required.
On the other hand, for audit work, which is a non-routine task, it is possible to identify issues in accounting audits without any problems. Based on that, it is possible to point out possible problems with the data, even if the way data is handled differs from company to company. However, this requires a certain degree of standardization and stylization of the data, and when introducing LLM into an ERP or existing accounting processing system, it will be necessary to reconsider the format of the data in the database and whether the data is sufficient for an audit.
In addition, due to its characteristics, LLM allows logical inference with clear premises in natural language, but it cannot generate the objective concept itself. Therefore, even if you ask about the comparison of the importance of financial accounting audits and environmental audits such as greenhouse gas emissions, you will only be able to highlight the problems and issues of each perspective.
From the above, it can be said that the current state of prompt engineering for using LLM in an investigation into compliance with regulations such as the Companies Act, such as accounting audits, is as follows: 1) Identify and organize problems from an auditing perspective from internal documents related to corporate activities such as transactions and information disclosure. 2) Then, for activities that may be problematic, a two-step process can be proposed: point out and raise questions about additional documents necessary to determine whether they are illegal or legal. The latest LLMs such as ChatGPTo have a high ability to point out logical problems, and when conducting an audit, they can raise issues on audit issues from the documents in question (for example, issues such as overstating sales due to fictitious transactions or fictitious circular transactions) from the company's activity data. Then, the LLM can point out and raise questions about additional documents and investigations necessary to determine whether they are legal or illegal, which seems to be an effective way to use it, including documentation for audit reports.
In the medium term, it is considered important to advance the standardization of corporate activity data and to clarify the "calculation" framework that can more directly point out problems with audit issues.
2. Books, papers, academic presentations, etc.
[Books and papers (not peer reviewed)]
Hiroshi Deguchi, Accounting System Theory, Hakutosha, February 2024
New Data, New Statistics and Their Challenges: Invoices and Real-Time Economy, Akagi Kaya, sole author, International Conference on Economic Structures (ICES 2023), Housei University, 2023