FusionFrame: A Fusion Dataflow Scheduling Framework for DNN Accelerators via Analytical Modeling
摘要
The growing complexity of DNN models and the widening gap between compute power and memory bandwidth necessitate fusion dataflows to reduce off-chip memory access. However, designing these dataflows across diverse DNNs and DNN accelerator architectures requires a flexible and accurate scheduling framework to explore the vast design space efficiently. Unfortunately, current state-of-the-art (SotA) frameworks struggle to explore these fusion dataflows by mapping limited fusion patterns on less scalable accelerator architectures. In this paper, we present a fusion dataflow framework called FusionFrame. FusionFrame expands the schedule space by characterizing it from three dimensions: operator fusion, loop tiling, and hardware mapping. To capture the extensive schedule space, we design a memory-centric representation to explore various fusion dataflows. Based on this representation, we develop an analytical model that evaluates on-chip data movement and latency for flexible fusion dataflows, enabling hierarchical fusion of operators on DNN accelerators with multi-level memory architectures. This cost model is validated against a taped-out DNN accelerator, Cambricon-Acc. Extensive case studies are then conducted to explore the performance impacts of various schedule space trade-offs, showing that FusionFrame demonstrates a latency improvement of up to 63.9% compared to SotA methods.