Deep Neural Networks (DNNs) are indispensable for AI robots and autonomous driving vehicles. Low power and high performance of DNN processors are critical to realize long battery life and long-term reliable controller operations, keeping their actions flexible. In addition, high program productivity is also essential for their cost-effective product development. We have been developing the OSCAR parallelizing and power-reducing compiler and its co-designed homogeneous and heterogeneous multicore processor chips. In this paper, the TVM, an open-source deep learning compiler, is utilized with the OSCAR compiler to automatically parallelize various DNN inference models. However, the current TVM does not generate a C program compatible with vectorization. In this paper, we propose a code generation method for TVM to generate vectorization-friendly code by transforming the memory layout of tensors to keep a long vector length at an innermost vectorized loop. The parallelized coarse grain task parallelization program is translated into NEC machine code with vector instructions by the NEC compiler. The execution performance of the proposed method with pre-trained DNN inference models is evaluated on NEC SX-Aurora TSUBASA vector multicore. The evaluation result shows that the proposed method achieves \(31.3\times \) speedup on seven cores with a ResNet model and \(37.6\times \) speedup with a VGG model, compared with the compilation flow that does not include the proposed method.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automatic Deep Learning Parallelization for Vector Multicore Chips with the OSCAR Parallelizing and the TVM Open-Source Deep Learning Compiler

  • Fumiaki Onishi,
  • Ryosei Otaka,
  • Kazuki Fujita,
  • Tomoki Suetsugu,
  • Tohma Kawasumi,
  • Toshiaki Kitamura,
  • Hironori Kasahara,
  • Keiji Kimura

摘要

Deep Neural Networks (DNNs) are indispensable for AI robots and autonomous driving vehicles. Low power and high performance of DNN processors are critical to realize long battery life and long-term reliable controller operations, keeping their actions flexible. In addition, high program productivity is also essential for their cost-effective product development. We have been developing the OSCAR parallelizing and power-reducing compiler and its co-designed homogeneous and heterogeneous multicore processor chips. In this paper, the TVM, an open-source deep learning compiler, is utilized with the OSCAR compiler to automatically parallelize various DNN inference models. However, the current TVM does not generate a C program compatible with vectorization. In this paper, we propose a code generation method for TVM to generate vectorization-friendly code by transforming the memory layout of tensors to keep a long vector length at an innermost vectorized loop. The parallelized coarse grain task parallelization program is translated into NEC machine code with vector instructions by the NEC compiler. The execution performance of the proposed method with pre-trained DNN inference models is evaluated on NEC SX-Aurora TSUBASA vector multicore. The evaluation result shows that the proposed method achieves \(31.3\times \) speedup on seven cores with a ResNet model and \(37.6\times \) speedup with a VGG model, compared with the compilation flow that does not include the proposed method.