Moderate guerrilla for the user’s understanding of the artificial intelligence-based user security awareness prototype

Abstract

To enhance user security awareness in response to the numerous cyberattacks targeting users, we developed an artificial intelligence (AI)based prototype utilizing the naive retrieval augmented generation (RAG) method. In pursuit of this objective, we conducted user testing employing the usability testing method to evaluate users' comprehension of the developed prototype. We integrated moderate and guerrilla techniques in usability testing by engaging 20 randomly selected respondents from the government, private sectors, and industries. The majority of participants were male, aged 26-35 years, holding a bachelor's degree, and possessing 510 years of computer experience. The test data were analyzed using the USE assessment matrix, which includes four assessment parameters: usefulness, satisfaction, ease of use, and ease of learning (USE). The data were presented in tabulated form, with total and average values. The test results indicate that usefulness, satisfaction, ease of USE achieved a total value exceeding 4.00 and an average value of 4.29, within an interval range of 4.20-5.00, categorized as very good. The findings of this study have implications for enhancing user security awareness and provide feedback for refining the framework and prototype in subsequent research.