A Framework for Assisted Knowledge Management Practices Using Large Language Models: Preliminary Findings

Mohd Nor Akmal Khalid, Nazhatul Hafizah Kamarudin, Wiwied Virgiyanti, Ramayah Thurasamy
dx.doi.org/10.17576/apjitm-2025-1402-05

Abstract

The rapid advancement of large language models (LLMs) has opened up new possibilities for leveraging artificial intelligence to assist with knowledge management tasks. However, the effectiveness and implications of using LLMs for this purpose still need to be fully understood. This paper presents preliminary findings from an exploratory study investigating the use of LLMs for assisted academic writing. This study conduct evaluation of the user experience on various knowledge management tasks, specifically on academic writing such as generating paper outlines, drafting literature reviews, writing some sections, and formulating research questions and hypotheses, supported by the LLM application. These evaluation metrics were also compared with the findings of a short questionnaire survey to assess the user experience using the LLMs applications which is collected and analyzed from participants of a writing workshop. These preliminary results indicate that LLMs can produce coherent and well-structured text for various components, but output quality and accuracy vary considerably across tasks. The findings underscore the promising potential of LLMs for assisted academic writing. While they can enhance and expedite certain writing elements, it is crucial to recognize these models as aids, not substitutes, for the human expertise and critical thinking fundamental to academic writing.

keyword

Artificial Intelligence, Large Language Models, Generative AI, Academic Writing, Preliminary Findings

Area

Knowledge Technology