Interdisciplinary research efforts in human-powered artificial intelligence (AI) algorithms have contributed to reveal hidden relationships and properties across knowledge graphs. However, the expanding scientific landscape presents challenges in capturing knowledge flows and their impact accurately. For a long time, scholars have made progress in the quantitative analysis of science indicators, historical footprints, and network dynamics. Despite the remarkable strides over the last decade, the pipeline underlying the measurement of scientific output is still difficult to execute in a fully automated way. To overcome these challenges, several academics and practitioners have increasingly explored hybrid intelligent systems that combine machine learning (ML) and crowd-based processing in data-driven research as an instrument of science policy. Building upon these promising advancements, this paper proposes a reinforcement learning from human feedback (RLHF) approach, offering insights for implementing hybrid crowd-algorithmic systems intended to support research evaluation and decision-making. The authors argue that an AI-based RLHF system pipeline can greatly benefit science stakeholders worldwide by enabling new forms of human-AI interactive and continuous sensemaking.