Tutorial: Writing a Kernel in C++ API

Introduction

This tutorial demonstrates how to write a kernel using C++ APIs.

A Chimera program consists of two parts:

• Host code
• Device code

The host code is executed by the host, and it also submits the kernel code to the devices.

Host program

The host program is responsible for performing the following tasks:

• Controlling the execution of kernels on the compute device.
• Copying data from/to the host device.
• Launching the kernel.
• Write CPU version for comparison.

In almost all cases, the host code is identical across all the applications, except for the number of input/output tensors.

The main steps of a host program are as follows.

1. Create DDR tensor objects.
2. Initialize tensors with input data.
3. Package the kernel.
4. Create memory buffer objects.
5. Create a device object.
6. Allocate and copy buffers to the device.
7. Launch the kernel on the device.
8. Copy outputs from the device to host buffers.
9. Compare the device output with the host-side reference model.

Example Host Code

📄 tests/static_data_tests/image_grayscale_interleaved.cpp

78     HOST_MAIN(
79       DdrInputImageShape ddrInputImage; DdrOutputImageShape ddrOutputImage; DdrOutputImageShape ddrExpectedOutputImage;
80
81       DdrInputImageShape::allocate(ddrInputImage);
82       DdrOutputImageShape::allocate(ddrOutputImage);
83       DdrOutputImageShape::allocate(ddrExpectedOutputImage);
84
85       populateTensorRand(ddrInputImage);
86       generateOutput(ddrInputImage, ddrExpectedOutputImage);
87
88       TensorArg<DdrInputImageShape>  argInputTensor{&ddrInputImage};
89       TensorArg<DdrOutputImageShape> argOutputTensor{&ddrOutputImage};
90
91       packageKernel(OUTPUT_PREFIX, FUNC_ARG(grayscaleImage), argInputTensor, argOutputTensor);
92
93       DeviceBufferRef inImage, outImage;
94       auto            device = DeviceManager().getDevice(callKernelGlobals.deviceConfig);
95       device.loadKernel();
96       device.allocateAndCopyToDevice(ddrInputImage, inImage);
97       device.allocate<DdrOutputImageShape>(outImage);
98       device.runKernel(ENTRYPOINT(grayscaleImage), inImage, outImage);
99       device.copyBufferFromDevice(outImage, ddrOutputImage);
100
101      auto nativeCompareVisitor =
102        [&](chimera-software-user-guide/chimera-compute-library-ccl-api/const auto& t1, const auto& t2, const DimensionContext& context) {
103          return nativeCompare<DdrOutputImageShape>(t1, t2, context, 1);
104        };
105      runtime_assert(compareTensors(ddrOutputImage, ddrExpectedOutputImage, nativeCompareVisitor), "tensor mismatch");)

Device Program and Valid C++ Syntax

The device program is the core of Chimera code. All the instructions in the device program run on the Quadric hardware. Therefore, it is vital to remember that only a subset of C++ syntax is valid here.

There is no generic way to write a kernel. But through our experience of writing kernels, we have identified that the steps below can help you in most cases. As you follow these steps, you will able to identify new design patterns:

1. Add the appropriate Chimera C++ includes to the top of your .cpp file.
2. Define the shapes of the tensors you want to flow into the Chimera processor.
3. Define the CPU version of the algorithm that you intend to implement.
4. Define a kernel function with a return type of void, and DDR tensors passed as pointers arguments.
5. Since DDR tensors are passed in as pointers, recreate the tensor objects.
6. Create L2 memory tensors and allocate them, as described in steps 7 to 13.
7. Copy data from DDR to L2 memory using the memCpy API.
8. Define TensorAccessors for reading and writing.
9. Define flow objects for reading and writing data.
10. Create NDArray objects to store tiles, and flow in data from L2 memory to the LRM Array, using V2 APIs.
11. While iterating through the NDArray, perform the required operations on each tile of data.
12. Flow out data from the array to the L2 memory.
13. Finally, write the data to DDR.
14. Perform host-side set up, to launch the kernel with data.
15. Launch the kernel using the Chimera SDK.

Example Kernel Code

📄 tests/static_data_tests/image_grayscale_interleaved.cpp

47     EPU_ENTRY void grayscaleImage(DdrInputImageShape::ptrType  ddrInputImagePtr,
48                                   DdrOutputImageShape::ptrType ddrOutputImagePtr) {
49       MemAllocator ocmMem;
50
51       DdrInputImageShape  ddrInputImage(ddrInputImagePtr);
52       DdrOutputImageShape ddrOutputImage(ddrOutputImagePtr);
53
54       OcmInputImageShape ocmInputImage;
55       ocmMem.allocate(ocmInputImage);
56       OcmOutputImageShape ocmOutputImage;
57       ocmMem.allocate(ocmOutputImage);
58
59       memCpy(ddrInputImage, ocmInputImage);
60       image::grayscale(ocmInputImage, ocmOutputImage);
61       memCpy(ocmOutputImage, ddrOutputImage);
62     }

To find out more about how to write kernels, using the above example, refer to:

• Tutorial: Creating a Simple Kernel