CRFFGAI Algorithm for Handling Uncertainty and Variability in Output Generation of Generative AI Models
摘要
The inherent uncertainty and variability in output generation of artificial intelligence (AI) based models have been long-standing challenges. Generative AI models are founded on vast datasets from diverse data sources like books, articles, websites, research papers as well as structured datasets available over the world wide web (www). Datasets are fundamental in training the significant generative AI models to create comprehensible and contextually relevant responses. Traditional AI models have always struggled to provide consistent results. With the advent of generative AI, these issues have become even more pronounced. This manuscript performs a thematic content analysis of available literature on handling uncertainty and variability in generative AI based solutions. Based on the analysis results, we also propose a novel conditional random field (CRF) based algorithm to mitigate these diverse types of uncertainty and variability in output of Generative AI models. When tested on the CoNLL 2003 dataset, the proposed CRF-based model achieved a 94% accuracy rate in the named entity recognition task, employing a confidence threshold of 0.7%. Ensemble CRF application to the same yielded 89% accuracy. The positive results demonstrate the suitability of the model for controlling the variability of output generated AI models.