Dependable AI Inference - A Work-in-Progress on CPU, Co-processor and FPGA Approaches
摘要
Current approaches to dependable AI inference in EdgeAI systems are problematic. The massive parallelization implied by GPUs and NPUs does not lend itself to traditional, that is redundant, forms of execution for integrity checking. State of the art inference-runtimes use a data streaming approach to accelerate inference under conditions of low available memory and, implemented in a lock-free manner, exhibit high orchestration efficiency in the face of low computational power. Under these conditions the actual mathematical operations act as a bottleneck. In low-power operations, imperative for the target domain of Space and indeed any battery-operated system, increasing the clock rate simply isn’t an option. Radiation-hardened circuitry, despite its high cost, may offer an alternative. We propose the application of the High-Performance Data Processor (HPDP) as a dedicated mathematical backend integrated in the data streaming pipeline of the Klepsydra AI-inference orchestration framework. Our results, comparing the performance of the HPDP against well-known radiation-hardened CPUs, confirm the validity of the approach. Given the architectural features of the HPDP and lessons-learned we then propose an FPGA architecture for which first experimental results indicate comparable performance estimates.