Introduction
Quadric APIs provide two methods for data transfer(bi-directional) between L2 and LRM
Flow APIs
RAU (Random Access Unit) APIs
RAU APIs provide the ability to fetch data in any pattern by computing relevant addresses and fetching data only from those addresses. This feature is particularly useful in cases where we need to access data in a strided pattern. Unlike Flow APIs, which allow us to read/write a block of data from a particular starting address, RAU APIs enable us to fetch data only from relevant addresses, avoiding unnecessary data transfer and computation.
The following are some examples of applications where RAU APIs are useful:
Down-sampling using bilinear-interpolation: In this application, RAU APIs can be used to efficiently access data in a strided pattern to perform bilinear interpolation.
RGB interleaved to RGB planar conversion: In this application, RAU APIs can be used to efficiently access data in a strided pattern to perform RGB interleaved to RGB planar conversion.
RAU APIs
There are several variants of RAU apis.
Load APIs
oneTile APIs
The oneTile function reads data from L2 into LRM and reads only one tile at a time. This function has two variations, one for address-based access and the other for index-based access.
1. Address-based
The address-based function reads data from a tensor located at the given linearized address
/src/core/rau.hppLines 150–168 /**
* @brief Fetches data from a tensor located at the given linearized index.
* @ingroup rau
*
* @tparam TensorType Type of the tensor to fetch data from (auto deduced).
* @tparam outOfRangeConstant Value for elements which are outside the region of interest. Default to `0`.
* @tparam T Element type. Default is `TensorType::elemType`.
* @param addresses Linearized addresses.
* @param tensor Tensor of the type `TensorType`.
* @param qValidAddr The condition if the corresponding core is valid. Default is `true`.
* @return North port contaning the data.
**/
template <typename TensorType,
std::int32_t outOfRangeConstant = 0,
typename T = typename TensorType::UnderlyingType,
std::enable_if_t<(ElementsPerPack<typename TensorType::elemType>::value >= 2), int> = 0>
INLINE qVar_t<T> oneTile(const qVar_t<std::uint32_t>& addresses,
TensorType& tensor,
const qVar_t<bool>& qValidAddr = true) {
2. Index-based
The index-based function reads data from a tensor located at the given index values.
/src/core/rau.hppLines 150–168 /**
* @brief Fetches data from a tensor located at the given linearized index.
* @ingroup rau
*
* @tparam TensorType Type of the tensor to fetch data from (auto deduced).
* @tparam outOfRangeConstant Value for elements which are outside the region of interest. Default to `0`.
* @tparam T Element type. Default is `TensorType::elemType`.
* @param addresses Linearized addresses.
* @param tensor Tensor of the type `TensorType`.
* @param qValidAddr The condition if the corresponding core is valid. Default is `true`.
* @return North port contaning the data.
**/
template <typename TensorType,
std::int32_t outOfRangeConstant = 0,
typename T = typename TensorType::UnderlyingType,
std::enable_if_t<(ElementsPerPack<typename TensorType::elemType>::value >= 2), int> = 0>
INLINE qVar_t<T> oneTile(const qVar_t<std::uint32_t>& addresses,
TensorType& tensor,
const qVar_t<bool>& qValidAddr = true) {
multi-TIle API
The multi-tile function, on the other hand, reads data from L2 into LRM and reads numTiles tiles at a time. This function is useful when users need to fetch multiple tiles of data in a single call. This function has two variations, one for qVar_t data type and the other for NDArray data type.
For qVar_t data type.
/src/core/rau.hppLines 247–261 /**
* @brief Read/write several tiles via RAU.
*
* @tparam TensorType Type of tensor.
* @tparam T Tensor element type. Default is `TensorType::elemType`.
* @param qAddresses Addresses.
* @param qData Data.
* @param tensor Tensor.
* @param numTiles Number of tiles.
*/
template <typename TensorType, typename T = typename TensorType::elemType>
INLINE void tiles(const qVar_t<std::uint32_t>* qAddresses,
qVar_t<T>* qData,
TensorType& tensor,
std::int32_t numTiles) {
For NDArray data type.
/src/core/rau.hppLines 271–286 /**
* @brief Counts the number of tiles.
*
* @tparam TensorType Type of tensor.
* @tparam T Tensor element type. Default is `TensorType::elemType`.
* @tparam size Size.
* @param qAddresses Addresses.
* @param qData Data.
* @param tensor Tensor.
* @param numTiles Number of tiles. Default is `size`.
*/
template <typename TensorType, typename T = typename TensorType::elemType, std::size_t size>
INLINE void tiles(const container::NDArray<qVar_t<std::uint32_t>, size>& qAddresses,
container::NDArray<qVar_t<T>, size>& qData,
TensorType& tensor,
std::int32_t numTiles = size) {
Store APIs
oneTile APIs
The oneTile function is used to store data from LRM into L2, and it can store only one tile at a time. There are two variations of this function: one for address-based access and the other for index-based access.
1. Address-based
The address-based function stores data to a tensor located at the given linearized address in the LRM memory space.
/src/core/rau.hppLines 355–366 /**
* @brief store 1 tile of data to a core-specific address
* @ingroup rau
*
* @tparam dataType data type, deduced
* @tparam TensorType tensor type, deduced
* @param addresses the target addresses
* @param data the data
* @param tensor the target tensor
*/
template <typename dataType, typename TensorType>
INLINE void oneTile(const qVar_t<std::uint32_t>& addresses, const qVar_t<dataType>& data, TensorType& tensor) {
2. Index-based
The index-based function stores data to a tensor located at the given index values.
/src/core/rau.hppLines 442–461
/**
* @brief store 1 tile of data to a core-specific offset
* @ingroup rau
*
* @tparam dataType data type, deduced
* @tparam TensorType tensor type, deduced
* @param bchIndx batch index
* @param chIndx channel index
* @param rowIndx row index
* @param colIndx column index
* @param qData the data
* @param tensor the target tensor
*/
template <typename dataType, typename TensorType>
INLINE void oneTile(const qVar_t<std::uint32_t>& bchIndx,
const qVar_t<std::uint32_t>& chIndx,
const qVar_t<std::uint32_t>& rowIndx,
const qVar_t<std::uint32_t>& colIndx,
const qVar_t<dataType>& qData,
TensorType& tensor) {
multi-TIle API
The multi-tile function stores data from LRM into L2 and can store multiple tiles of data in a single call. This function is useful when there is a need to store multiple tiles of data in the LRM at once. This function has two variations, one for qVar_t data type and the other for NDArray data type.
For qVar_t data type.
/src/core/rau.hppLines 408–418 /**
* @brief Store multiple tiles of data via RAU (delegates to startTiles + qbar; see it for params).
* @ingroup rau
*/
template <typename TensorType,
typename T = typename TensorType::elemType,
std::int32_t elemSize = sizeof(typename TensorType::elemType)>
INLINE void tiles(const qVar_t<std::uint32_t>* qAddresses,
qVar_t<T>* qData,
TensorType& tensor,
std::int32_t numTiles) {
For NDArray data type.
/src/core/rau.hppLines 423–438 /**
* @brief Counts the number of tiles.
*
* @tparam TensorType TensorType Type of the tensor to fetch data from (auto deduced).
* @tparam T Tensor element type. Default is `TensorType::elemType`.
* @tparam size Size.
* @param qAddresses Addresses.
* @param qData Data.
* @param tensor Tensor.
* @param numTiles Number of tiles. Default is `size`.
*/
template <typename TensorType, typename T = typename TensorType::elemType, std::size_t size>
INLINE void tiles(const container::NDArray<qVar_t<std::uint32_t>, size>& qAddresses,
container::NDArray<qVar_t<T>, size>& qData,
TensorType& tensor,
std::int32_t numTiles = size) {
Other Utility functions to use with RAU load/store APIs
RAU Configuration
The config function is a pre-configuration function that sets up the PLS and RAU (Register Access Unit) addresses before calling the RAU load/store functions. This function is designed to simplify the setup process for RAU APIs by providing a single call to configure the PLS and set the RAU address.
To use this function, you need to pass an LRM tensor as input. This API can be used for both load and store operations. If you need to load or store data from/to another LRM tensor, you will need to call the config function again with the new LRM tensor as inp.
The config function must be called before using the RAU load/store functions. It configures the PLS and RAU addresses and sets up the necessary registers for proper operation.
/src/core/rau.hppLines 42–43 template <typename TensorType>
INLINE void config(TensorType& inp) {
Examples
Randomly stores data in qInp at locations specified by qAddrs.
/tests/static_data_tests/rau_load_store_1tile_i16.cppLines 73–86 // Fetch data to qInp[i] using Rau LOAD
rau::config<OcmInOutTensor>(ocmInPtr);
for(std::int32_t i = 0; i < OcmInOutTensor::NUM_TILES; i++) {
qInp[i] = rau::load::oneTile(qAddrs[i], ocmInPtr);
}
// Config rau for writing (Rau STORE) to output
rau::config<OcmInOutTensor>(ocmOutPtr);
for(std::int32_t i = 0; i < OcmInOutTensor::NUM_TILES; i++) {
qVar_t<bool> cond = (((qAddrs[i] / sizeof(typename OcmInOutTensor::elemType)) % core_array::coreDim) != 0);
if(cond) {
rau::store::oneTile(qAddrs[i], qInp[i], ocmOutPtr);
}
}
Calculate linearized addresses
The computeAddr function takes in four parameters, bch, ch, row, and col, which represent the indexes of a tensor. The function then computes a serialized address based on these indexes and returns it as a qVar_t<std::uint32_t> type.
This function is typically used in conjunction with the RAU load/store APIs to load/store data from/to the L2 memory. In the example provided, the computeAddr function is used to compute the serialized address for both the read and write operations. The resulting address is then passed as a parameter to the RAU load/store APIs to perform the respective read/write operation.
/src/core/rau.hppLines 49–54 // returns a serialized address based in the tensor indexes
template <typename TensorType>
INLINE qVar_t<std::uint32_t> computeAddr(qVar_t<std::uint32_t> bch,
qVar_t<std::uint32_t> ch,
qVar_t<std::uint32_t> row,
qVar_t<std::uint32_t> col) {
It is not recommended to use RAU load APIs inside an if/else block. Doing so may lead to unexpected results and computed results may not match the expected results. Instead, qValidAddr parameter can be used to select the cores that should be participating in the load operation.
However, RAU store APIs can be used inside an if/else block.
