Introduction
Quadric APIs provide two methods for data transfer(bi-directional) between L2 and LRM
Flow APIs
RAU (Random Access Unit) APIs
RAU APIs provide the ability to fetch data in any pattern by computing relevant addresses and fetching data only from those addresses. This feature is particularly useful in cases where we need to access data in a strided pattern. Unlike Flow APIs, which allow us to read/write a block of data from a particular starting address, RAU APIs enable us to fetch data only from relevant addresses, avoiding unnecessary data transfer and computation.
The following are some examples of applications where RAU APIs are useful:
Down-sampling using bilinear-interpolation: In this application, RAU APIs can be used to efficiently access data in a strided pattern to perform bilinear interpolation.
RGB interleaved to RGB planar conversion: In this application, RAU APIs can be used to efficiently access data in a strided pattern to perform RGB interleaved to RGB planar conversion.
RAU APIs
There are several variants of RAU apis.
Load APIs
File: /src/core/rau.hppSymbol: oneTileoneTile APIs
/src/core/rau.hppSymbol: oneTileoneTileThe oneTile function reads data from L2 into LRM and reads only one tile at a time. This function has two variations, one for address-based access and the other for index-based access.
1. Address-based
The address-based function reads data from a tensor located at the given linearized address
/src/core/rau.hppLines 148–166 /**
* @brief Fetches data from a tensor located at the given linearized index.
* @ingroup rau
*
* @tparam TensorType Type of the tensor to fetch data from (auto deduced).
* @tparam outOfRangeConstant Value for elements which are outside the region of interest. Default to `0`.
* @tparam T Element type. Default is `TensorType::elemType`.
* @param addresses Linearized addresses.
* @param tensor Tensor of the type `TensorType`.
* @param qValidAddr The condition if the corresponding core is valid. Default is `true`.
* @return North port contaning the data.
**/
template <typename TensorType,
std::int32_t outOfRangeConstant = 0,
typename T = typename TensorType::UnderlyingType,
std::enable_if_t<(ElementsPerPack<typename TensorType::elemType>::value >= 2), int> = 0>
INLINE qVar_t<T> oneTile(const qVar_t<std::uint32_t>& addresses,
TensorType& tensor,
const qVar_t<bool>& qValidAddr = true) {
2. Index-based
The index-based function reads data from a tensor located at the given index values.
/src/core/rau.hppLines 148–166 /**
* @brief Fetches data from a tensor located at the given linearized index.
* @ingroup rau
*
* @tparam TensorType Type of the tensor to fetch data from (auto deduced).
* @tparam outOfRangeConstant Value for elements which are outside the region of interest. Default to `0`.
* @tparam T Element type. Default is `TensorType::elemType`.
* @param addresses Linearized addresses.
* @param tensor Tensor of the type `TensorType`.
* @param qValidAddr The condition if the corresponding core is valid. Default is `true`.
* @return North port contaning the data.
**/
template <typename TensorType,
std::int32_t outOfRangeConstant = 0,
typename T = typename TensorType::UnderlyingType,
std::enable_if_t<(ElementsPerPack<typename TensorType::elemType>::value >= 2), int> = 0>
INLINE qVar_t<T> oneTile(const qVar_t<std::uint32_t>& addresses,
TensorType& tensor,
const qVar_t<bool>& qValidAddr = true) {
multi-TIle API
The multi-tile function, on the other hand, reads data from L2 into LRM and reads numTiles tiles at a time. This function is useful when users need to fetch multiple tiles of data in a single call. This function has two variations, one for
/src/core/rau.hppSymbol: qVar_tqVar_t/src/core/rau.hppSymbol: NDArrayNDArrayFor qVar_t data type.
/src/core/rau.hppLines 245–259 /**
* @brief Read/write several tiles via RAU.
*
* @tparam TensorType Type of tensor.
* @tparam T Tensor element type. Default is `TensorType::elemType`.
* @param qAddresses Addresses.
* @param qData Data.
* @param tensor Tensor.
* @param numTiles Number of tiles.
*/
template <typename TensorType, typename T = typename TensorType::elemType>
INLINE void tiles(const qVar_t<std::uint32_t> qAddresses[],
qVar_t<T> qData[],
TensorType& tensor,
std::int32_t numTiles) {
For NDArray data type.
/src/core/rau.hppLines 269–284 /**
* @brief Counts the number of tiles.
*
* @tparam TensorType Type of tensor.
* @tparam T Tensor element type. Default is `TensorType::elemType`.
* @tparam size Size.
* @param qAddresses Addresses.
* @param qData Data.
* @param tensor Tensor.
* @param numTiles Number of tiles. Default is `size`.
*/
template <typename TensorType, typename T = typename TensorType::elemType, std::size_t size>
INLINE void tiles(const container::NDArray<qVar_t<std::uint32_t>, size>& qAddresses,
container::NDArray<qVar_t<T>, size>& qData,
TensorType& tensor,
std::int32_t numTiles = size) {
Store APIs
oneTile APIs
The oneTile function is used to store data from LRM into L2, and it can store only one tile at a time. There are two variations of this function: one for address-based access and the other for index-based access.
1. Address-based
The address-based function stores data to a tensor located at the given linearized address in the LRM memory space.
/src/core/rau.hppLines 184–204 /**
* @brief fetches data from a tensor located at the given index values
*
* @tparam int32_t Port type (auto deduce)
* @tparam TensorType type of the tensor to fetch data from (auto deduce)
* @tparam outOfRangeConstant value for elements which are outside the region of interest. default to 0
* @param bchIndx batch index
* @param chIndx channel index
* @param rowIndx row index
* @param colIndx column index
* @param tensor tensor of the type TensorType
* @return North port contaning the data
*/
template <typename TensorType,
std::int32_t outOfRangeConstant = 0,
std::enable_if_t<ElementsPerPack<typename TensorType::elemType>::value == 1, int> = 0>
INLINE qVar_t<typename TensorType::elemType> oneTile(const qVar_t<std::uint32_t>& bchIndx,
const qVar_t<std::uint32_t>& chIndx,
const qVar_t<std::uint32_t>& rowIndx,
const qVar_t<std::uint32_t>& colIndx,
TensorType& tensor) {
2. Index-based
The index-based function stores data to a tensor located at the given index values.
/src/core/rau.hppLines 433–452 /**
* @brief store 1 tile of data to a core-specific offset
* @ingroup rau
*
* @tparam dataType data type, deduced
* @tparam TensorType tensor type, deduced
* @param bchIndx batch index
* @param chIndx channel index
* @param rowIndx row index
* @param colIndx column index
* @param qData the data
* @param tensor the target tensor
*/
template <typename dataType, typename TensorType>
INLINE void oneTile(const qVar_t<std::uint32_t>& bchIndx,
const qVar_t<std::uint32_t>& chIndx,
const qVar_t<std::uint32_t>& rowIndx,
const qVar_t<std::uint32_t>& colIndx,
const qVar_t<dataType>& qData,
TensorType& tensor) {
multi-TIle API
The multi-tile function stores data from LRM into L2 and can store multiple tiles of data in a single call. This function is useful when there is a need to store multiple tiles of data in the LRM at once. This function has two variations, one for qVar_t data type and the other for NDArray data type.
For qVar_t data type.
/src/core/rau.hppLines 383–401 /**
* @brief Store multiple tiles of data via RAU.
* @ingroup rau
*
* @tparam TensorType Tensor type, deduced.
* @tparam T TensorType::elemType the array type, deduced.
* @tparam elemSize Size of elements. Default is `sizeof(typename TensorType::elemType)`.
* @param qAddresses The addresses to write.
* @param qData The data to write.
* @param tensor The tensor.
* @param numTiles Number of tiles to write.
*/
template <typename TensorType,
typename T = typename TensorType::elemType,
std::int32_t elemSize = sizeof(typename TensorType::elemType)>
INLINE void tiles(const qVar_t<std::uint32_t> qAddresses[],
qVar_t<T> qData[],
TensorType& tensor,
std::int32_t numTiles) {
For NDArray data type.
/src/core/rau.hppLines 269–284 /**
* @brief Counts the number of tiles.
*
* @tparam TensorType Type of tensor.
* @tparam T Tensor element type. Default is `TensorType::elemType`.
* @tparam size Size.
* @param qAddresses Addresses.
* @param qData Data.
* @param tensor Tensor.
* @param numTiles Number of tiles. Default is `size`.
*/
template <typename TensorType, typename T = typename TensorType::elemType, std::size_t size>
INLINE void tiles(const container::NDArray<qVar_t<std::uint32_t>, size>& qAddresses,
container::NDArray<qVar_t<T>, size>& qData,
TensorType& tensor,
std::int32_t numTiles = size) {
Other Utility functions to use with RAU load/store APIs
RAU Configuration
The config function is a pre-configuration function that sets up the PLS and RAU (Register Access Unit) addresses before calling the RAU load/store functions. This function is designed to simplify the setup process for RAU APIs by providing a single call to configure the PLS and set the RAU address.
To use this function, you need to pass an LRM tensor as input. This API can be used for both load and store operations. If you need to load or store data from/to another LRM tensor, you will need to call the config function again with the new LRM tensor as inp.
The config function must be called before using the RAU load/store functions. It configures the PLS and RAU addresses and sets up the necessary registers for proper operation.
/src/core/rau.hppLines 40–41 template <typename TensorType>
INLINE void config(TensorType& inp) {
Examples
Randomly stores data in qInp at locations specified by qAddrs.
/tests/static_data_tests/rau_load_store_1tile_i16.cppLines 73–86 // Fetch data to qInp[i] using Rau LOAD
rau::config<OcmInOutTensor>(ocmInPtr);
for(std::int32_t i = 0; i < OcmInOutTensor::NUM_TILES; i++) {
qInp[i] = rau::load::oneTile(qAddrs[i], ocmInPtr);
}
// Config rau for writing (Rau STORE) to output
rau::config<OcmInOutTensor>(ocmOutPtr);
for(std::int32_t i = 0; i < OcmInOutTensor::NUM_TILES; i++) {
qVar_t<bool> cond = (qAddrs[i] / sizeof(typename OcmInOutTensor::elemType) % core_array::coreDim) != 0;
if(cond) {
rau::store::oneTile(qAddrs[i], qInp[i], ocmOutPtr);
}
}
Calculate linearized addresses
The computeAddr function takes in four parameters, bch, ch, row, and col, which represent the indexes of a tensor. The function then computes a serialized address based on these indexes and returns it as a qVar_t<std::uint32_t> type.
This function is typically used in conjunction with the RAU load/store APIs to load/store data from/to the L2 memory. In the example provided, the computeAddr function is used to compute the serialized address for both the read and write operations. The resulting address is then passed as a parameter to the RAU load/store APIs to perform the respective read/write operation.
/src/core/rau.hppLines 47–52 // returns a serialized address based in the tensor indexes
template <typename TensorType>
INLINE qVar_t<std::uint32_t> computeAddr(qVar_t<std::uint32_t> bch,
qVar_t<std::uint32_t> ch,
qVar_t<std::uint32_t> row,
qVar_t<std::uint32_t> col) {
It is not recommended to use RAU load APIs inside an if/else block. Doing so may lead to unexpected results and computed results may not match the expected results. Instead, qValidAddr parameter can be used to select the cores that should be participating in the load operation.
However, RAU store APIs can be used inside an if/else block.
