The Quadric Chimera Graph Compiler (CGC) is a powerful conversion and code optimization tool that accepts a quantized Deep Neural Network (DNN) model as input, performs optimizations, and outputs an optimized C++ code representation of the graph utilizing the Chimera Compute Library (CCL) APIs.
This C++ representation can then be compiled, alongside more traditional C++ algorithms, by the Chimera LLVM C++ Compiler.
CGC is an extension of the popular Apache TVM compiler framework.
How the CGC Works
The high-level goal of the CGC, or any DNN compiler for that matter, is to take abstract DNN graphs and convert them into well-defined C/C++ functions. DNN graphs are abstract by nature because they have no concept of memory allocation or movement. They are a simplified representation of a sequence of compute. However, when deploying these models onto target hardware for inference as a C++ function, we must know how much memory to allocate for each intermediate tensor, weight, and bias; in which type of memory, e.g. LRM or DDR, and at what address those data are stored; and what's the most efficient way of access that data based on the compute.
When you zoom out to see the big picture, this quickly becomes a daunting task. A standard practice for DNN compilers is to perform multiple "passes" or iterations over the graph to reduce the problem scope. During each pass, a series of optimizations are performed and additional runtime context is added to the graph.
Intermediate representations (IR) of the graph are stored by the CGC after each of these optimization passes until finally all of the runtime context is resolved and we're left with C++ code.
