Evaluating Gen AI Use Cases: Takeaways from a Case Study
摘要
Generative artificial intelligence (gen AI) is becoming increasingly popular. However, evaluating it with a multi-faceted, but easy-to-apply model remains a challenge. By cross-referencing findings from information systems (IS) success research, we elaborate on such a model. Taking a case study within a science & technology company, we reveal five takeaways: (1) Gen AI often starts with conversational agents. Its benefits are best measurable by net present value; (2) Enhance “pure” efficiency evaluations with effectiveness criteria; (3) With an experience category take user’s learning curve into account; (4) Towards green IS, think about sustainability as a model’s fourth value category; (5) In a summary, multi-faceted evaluation models should take efficiency, effectiveness, experience, and sustainability criteria into account.