The Flow APIs enable efficient data movement between different memory hierarchies in the Chimera architecture. These APIs abstract the complexities of data transfers and provide optimized buffering strategies for improved performance.
Flow API Overview
The system provides several categories of Flow APIs for different memory transfer scenarios:
OCM ↔ LRM Flows (Internal Flows)
For data transfers between On-Chip Memory (OCM) and Local Register Memory (LRM):
ReadFlow- Buffered read flow from OCM to LRMBufferedReadFlow- Direct read flow from OCM to LRM (no ping-pong buffering)WriteFlow- Buffered write flow from LRM to OCMUnbufferedWriteFlow- Direct write flow from LRM to OCM (no ping-pong buffering)
DDR ↔ LRM Flows (Integrated Flows)
For data transfers that span DDR to LRM with integrated buffering:
DdrArrayFlow- Integrated flow from DDR to LRM (combines DDR→OCM + OCM→LRM)ArrayDdrFlow- Integrated flow from LRM to DDR (combines LRM→OCM + OCM→DDR)
DDR ↔ OCM Flows (External Flows)
For high-level data transfers between DDR and OCM with advanced double buffering:
ExtReadFlow- External read flow from DDR to OCM with double bufferingExtWriteFlow- External write flow from OCM to DDR with double buffering
Memory Hierarchy
The Chimera architecture consists of three main memory levels:
- DDR: External memory for large data storage
- OCM: On-Chip Memory for intermediate buffering
- LRM: Local Register Memory for computation
Flow APIs enable efficient data movement between these levels with optimized buffering strategies.
General Flows
Read Flows
Read flows copy data from L2 memory to LRM.
ReadFlow API helps to read the data stored in an OCM tensor and write it into LRM. The reason this is also called as BufferedReadFlow is, it does not write the read data directly into the output NDArray in LRM. First, it stores the data in a ping-pong buffer NDArray (which is also allocated in the LRM) and then copies the data to the output NDArray. This very useful when it comes to performance improvements because, now it is possible to read the next minROI of input data to the buffer while we process the current minROI of data that we already have in the output NDArray. However, this version takes twice as much space than the BufferedReadFlow version.
UnbufferedReadFlow API can also be used to read the data stored in an OCM tensor and write it into LRM. However, this does not utilize a ping-pong buffer in LRM which prevents concurrent flows with compute and hence degraded performance. Use this API in places where the min ROI needs to take up a large space in LRM and ping-pong (2x the buffer size) is not an option.
Split-phase transfers: issueRead() and issueWrite()
read() and write() on an unbuffered flow barrier until the transfer is done. issueRead(buf) and issueWrite(buf) start a transfer using a caller-owned LrmBuffer<qVar_t<T>, N> and return at once, so independent work can run while it is in flight. flow.collect(buf) then waits for that transfer and pads the lanes of a packed ragged edge with the flow's out-of-range constant exactly as read() does. getElem(index), setElem(index, value) and apply are plain accesses afterwards; in a debug build (the EPU_DEBUG CMake option, on by default) they debug_assert that the buffer was collected, so a read of in-flight data fails loudly, and with -DEPU_DEBUG=OFF the check is not compiled at all. A flow has one transfer in flight at a time: collect it before the flow's next issue, and collect each transfer exactly once. The overlap comes from working on one buffer while the other is in flight, and from separate flows running side by side. A debug build reports a second issue, a second collect, and an issue past the flow's last min-ROI.
flow.collect(buf); // the read issued into buf has landed
qVar_t<std::int32_t> value = buf.getElem(index);
value += 1;
buf.setElem(index, value);
auto gateFlow = createGeneralReadFlow<ReadAccessB, GateBShape, GateBShape, 2, 2, MemAllocator,
StreamSide::SOUTH, false>(gateB, ocmMem);
auto upFlow = createGeneralReadFlow<ReadAccessB, UpBShape, UpBShape, 2, 2, MemAllocator,
StreamSide::SOUTH, false>(upB, ocmMem);
auto outFlow = createGeneralWriteFlow<WriteAccessC, OutShape, OutShape, 2, 2, MemAllocator,
StreamSide::NORTH, false>(out, ocmMem);
LrmBuffer<qVar_t<std::int8_t>, tileElems> qGate; // one min ROI each
LrmBuffer<qVar_t<std::int8_t>, tileElems> qUp;
LrmBuffer<qVar_t<std::int8_t>, tileElems> qOut;
gateFlow.issueRead(qGate);
for(std::size_t k = 0; k < gateFlow.numMROIs(); k++) {
upFlow.issueRead(qUp); // lands while gate is consumed
gateFlow.collect(qGate);
consume(qGate);
if(k + 1 < gateFlow.numMROIs()) {
gateFlow.issueRead(qGate); // lands while up is consumed
}
upFlow.collect(qUp);
consume(qUp);
if(k > 0) {
outFlow.collect(qOut); // the previous tile's store is done
}
produce(qOut);
outFlow.issueWrite(qOut); // stores while the next tiles are read
}
outFlow.collect(qOut);
Write Flows
WriteFlow API helps to write the data stored in LRM into an OCM tensor. The reason this is also called as BufferedWriteFlow is, it does not write the LRM data directly into the output OCM tensor. First, it copies the data in the input NDArray to a ping-pong buffer NDArray (which is also allocated in the LRM) and then writes the data to the output OCM tensor. This very useful when it comes to performance improvements because with this, it is possible to write the previous minROI of processed data to the output OCM tensor while we process the current minROI of data that we already have in the input NDArray. However, this version takes twice as much space than the UnbufferedWriteFlow version.
UnbufferedWriteFlow API can also be used to write the data stored in LRM into an OCM tensor. However, this does not utilize a ping-pong buffer in LRM which prevents concurrent flows with compute and hence degraded performance. Use this API in places where the min ROI needs to take up a large space in LRM and ping-pong (2x the buffer size) is not an option.
Runtime Tensor Support in ReadFlow/WriteFlow (New)
The ReadFlow and WriteFlow APIs now natively support both static tensors (compile-time dimensions) and runtime tensors (runtime dimensions) with a single consistent interface. Runtime tensor support is directly integrated into the flow classes—no separate "unified" flow types are needed.
What ReadFlow/WriteFlow Support
Core Capabilities:
- Static Tensors: Traditional tensors with compile-time known dimensions
- Runtime Tensors: Tensors with dimensions determined at runtime via
RuntimeTensorwrapper - Double Buffering: Ping-pong buffering in LRM for overlapping compute and data movement (default, best performance)
- Unbuffered Mode: Direct transfers without ping-pong buffering to save LRM space
- Tensor Offsets: Partial read/write of tensor sub-regions
- ROI (Region of Interest): Process specific regions of tensors
- MinROI Iteration: Automatic iteration through minimum regions of interest
- Runtime Tensor Slicing: Create smaller views of larger OCM buffers at runtime
- Automatic Address Computation: Runtime dimension-aware address calculations
Current Limitations for Runtime Tensors
- Bordered Flows: Flows with border/halo regions are not yet supported for runtime tensors
- Multi-Layer Synchronization (MLS): Inter-core communication for convolution-like operations with border padding
- Sleep State Optimization: Power optimization by selectively sleeping cores based on data boundaries
- EAST Stream Side: Only NORTH and SOUTH sides are supported
- Constraint:
squareInternalHeightStridemust equal 1 for runtime tensors
Using ReadFlow/WriteFlow with Static Tensors
Static tensors have dimensions known at compile time:
// Define static tensor types
using OcmTensorType = Tensor<int16_t, TensorLocation::OCM, 1, 64, 32, 128>;
// Define access pattern
using MinRoiDesc = MinRoiDescriptor<OcmTensorType,
AxisGroup<Direction::Width, Direction::Height>,
Granularity::Square>;
using Access = TensorAccessor<MinRoiDesc>;
// Create flows with static tensors
OcmTensorType inputTensor;
OcmTensorType outputTensor;
ocmAllocator.allocate(inputTensor);
ocmAllocator.allocate(outputTensor);
ReadFlow<Access, OcmTensorType> readFlow(inputTensor);
WriteFlow<Access, OcmTensorType> writeFlow(outputTensor);
// Use the flows
typename ReadFlow<Access, OcmTensorType>::MinRoiArray lrmBuffer;
for(uint32_t i = 0; i < readFlow.numMROIs(); i++) {
readFlow.read(lrmBuffer);
// Process data in lrmBuffer...
writeFlow.write(lrmBuffer);
}
Using ReadFlow/WriteFlow with Runtime Tensors
Runtime tensors allow dimensions to be specified at runtime, enabling dynamic tensor slicing and flexible memory layouts:
// Allocate a large OCM buffer (base allocation)
using OcmTensorType = Tensor<int16_t, TensorLocation::OCM, 1, 64, 64, 256>;
OcmTensorType ocmBuffer;
ocmAllocator.allocate(ocmBuffer);
// Create RuntimeTensor view with smaller dimensions (runtime slicing)
using RuntimeTensorOcm = RuntimeTensor<OcmTensorType>;
RuntimeTensorOcm runtimeView(
ocmBuffer, // Base OCM buffer
1, 64, 32, 128, // Runtime dimensions (BCH, CHN, ROWS, COLS) - smaller than base
{0, 0, 0, 0} // Start offset
);
// Define access pattern (same MinRoiDesc as static case)
using MinRoiDesc = MinRoiDescriptor<OcmTensorType,
AxisGroup<Direction::Width, Direction::Height>,
Granularity::Square>;
using Access = TensorAccessor<MinRoiDesc>;
// Create flows with runtime tensors
// IMPORTANT: For runtime tensors, RoiShape must equal OcmShape type
ReadFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm> readFlow(runtimeView);
WriteFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm> writeFlow(runtimeView);
// Use flows exactly the same way as static tensors
typename ReadFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm>::MinRoiArray lrmBuffer;
for(uint32_t i = 0; i < readFlow.numMROIs(); i++) {
readFlow.read(lrmBuffer);
// Process data...
writeFlow.write(lrmBuffer);
}
Key Points for Runtime Tensors:
- The
RoiShapetemplate parameter must be the same type asOcmShape(both must beRuntimeTensortypes) - Actual dimensions are specified when constructing the
RuntimeTensorinstance
Buffered vs Unbuffered Flow Usage
Choosing Buffered vs Unbuffered:
| Mode | LRM Usage | Performance | Use Case |
|---|---|---|---|
| Buffered (default) | 2x buffer size | High - overlaps compute with memory ops | Most workloads |
| Unbuffered | 1x buffer size | Lower - sequential operations | LRM-constrained workloads |
Buffered Flow (Default - Recommended):
// Uses ping-pong buffers (2x LRM buffers)
ReadFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm> readFlow(ocmTensor);
WriteFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm> writeFlow(ocmTensor);
typename ReadFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm>::MinRoiArray lrmBuffer;
readFlow.read(lrmBuffer);
// While processing current data, next data is being fetched!
writeFlow.write(lrmBuffer);
Unbuffered Flow (LRM-Constrained):
// Uses single buffer (1x LRM buffer)
UnbufferedReadFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm> readFlow(ocmTensor);
UnbufferedWriteFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm> writeFlow(ocmTensor);
typename UnbufferedReadFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm>::MinRoiArray buffer;
readFlow.read(buffer);
// Sequential operation - no overlap with next fetch
writeFlow.write(buffer);
Complete Example: Static vs Runtime Tensors
This example demonstrates that the same ReadFlow/WriteFlow API works with both static and runtime tensors:
// ===== Static Tensors using ReadFlow/WriteFlow =====
{
// Use Flow directly or the ReadFlow/WriteFlow aliases
using StaticReadFlow = Flow<FlowType::Read, Access, TestTensorOcm,
Epu::StreamSide::SOUTH, TestTensorOcm>;
using StaticWriteFlow = Flow<FlowType::Write, Access, TestTensorOcm,
Epu::StreamSide::NORTH, TestTensorOcm>;
StaticReadFlow readFlow(ocmStaticInput);
StaticWriteFlow writeFlow(ocmStaticOutput);
typename StaticReadFlow::MinRoiArray lrmBuffer;
for(std::uint32_t i = 0; i < readFlow.numMROIs(); i++) {
readFlow.read(lrmBuffer);
writeFlow.write(lrmBuffer);
}
memCpy(ocmStaticOutput, ddrOutStatic);
}
// ===== Runtime Tensors using ReadFlow/WriteFlow =====
{
using RuntimeTensorOcm = RuntimeTensor<TestTensorOcm>;
RuntimeTensorOcm runtimeOcmInp(ocmRuntimeInput,
TestTensorOcm::NUM_BCH,
TestTensorOcm::NUM_CHN,
TestTensorOcm::NUM_ROWS,
TestTensorOcm::NUM_COLS);
RuntimeTensorOcm runtimeOcmOut(ocmRuntimeOutput,
TestTensorOcm::NUM_BCH,
TestTensorOcm::NUM_CHN,
TestTensorOcm::NUM_ROWS,
TestTensorOcm::NUM_COLS);
// Use ReadFlow/WriteFlow directly with runtime tensors
using RuntimeReadFlow = ReadFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm>;
using RuntimeWriteFlow = WriteFlow<Access, RuntimeTensorOcm, RuntimeTensorOcm>;
RuntimeReadFlow readFlow(runtimeOcmInp);
RuntimeWriteFlow writeFlow(runtimeOcmOut);
typename RuntimeReadFlow::MinRoiArray lrmBuffer;
for(std::uint32_t i = 0; i < readFlow.numMROIs(); i++) {
readFlow.read(lrmBuffer);
writeFlow.write(lrmBuffer);
}
memCpy(runtimeOcmOut, runtimeDdrOut);
}
Both code paths produce identical results, validating the unified implementation!
Future Development Plans
Runtime tensor support in ReadFlow/WriteFlow is under active development with planned enhancements:
Upcoming Features:
- Sleep State Support: Power optimization by selectively sleeping cores based on data boundaries
- DdrArrayFlow Integration: Extend runtime tensor capabilities to DDR↔LRM flows
- MLS Support: Multi-layer synchronization for inter-core communication in convolution operations
- Bordered Flow Support: Add support for halo regions and border padding for runtime tensors
Packed Flows (Column and Row Packing)
The MinRoiDescriptor supports packing multiple spatially adjacent elements into a single PE via the packColsVal and packRowsVal template parameters, exposed as the packCols and packRows member constants. This is used for stride-2 convolutions, YUV processing, and other operations where a PE needs to operate on a contiguous spatial block.
MinRoiDescriptor Packing Parameters
template <typename MinRoiVal,
typename AxesVal = DefaultAxisGroup,
Granularity granularityVal = Granularity::Row,
bool hasBorderVal = false,
std::int32_t packColsVal = 1, // consecutive columns packed per PE
std::int32_t packRowsVal = 1, // consecutive rows packed per PE
std::int32_t squareInternalHeightStrideVal = 1,
typename LRMTypeVal = DefaultLRMType,
std::int32_t outOfRangeConstantVal = 0>
struct MinRoiDescriptor;
packColsVal(memberpackCols): Number of consecutive elements in a row packed into one PE. Default is 1 (no packing).packRowsVal(memberpackRows): Number of consecutive rows packed into one PE. Default is 1 (no packing).
When packing is enabled, each PE receives packCols × packRows elements in a single read flow iteration, with no extra OCM round-trip — the packing happens as data streams through the PLS into LRM.
Constraints
| Constraint | Detail |
|---|---|
| Column byte limit | sizeof(elemType) × packCols must be 1, 2, or 4 bytes |
| Row packing granularity | packRows > 1 requires Granularity::Square |
| Height stride exclusion | squareInternalHeightStride cannot be used with packRows > 1 |
| MinRoi dimensions | Width = core_array::coreDim × packCols, Height = core_array::coreDim × packRows |
Supported Element Type / Pack Combinations
| Element type | sizeof | Max packCols | packCols=2 | packCols=4 |
|---|---|---|---|---|
| int8 / uint8 | 1 B | 4 | 2 B ✓ | 4 B ✓ |
| int16 / FP16 | 2 B | 2 | 4 B ✓ | 8 B x |
| int32 / Fx32 | 4 B | 1 | 8 B x | 16 B x |
packRows has no byte-width limit — any element type supports packRows > 1.
Example: 2×2 int8 Packing (Stride-2 Convolution)
// MinRoi sized to hold coreDim × packRows rows and coreDim × packCols cols
using MinRoi = OcmTensor<std::int8_t, 1, 1,
2 * core_array::coreDim, // rows: coreDim × packRows
2 * core_array::coreDim>; // cols: coreDim × packCols
// granularity packCols packRows
// ↓ ↓ ↓
using Desc = MinRoiDescriptor<MinRoi, AxisGroup<>, Granularity::Square, false, 2, 2>;
using Access = TensorAccessor<Desc>;
ReadFlow<Access, OcmInOutShape> readFlow(ocmInp);
WriteFlow<Access, OcmInOutShape> writeFlow(ocmOut);
for (std::uint32_t i = 0; i < readFlow.numMROIs(); i++) {
auto mroi = readFlow.read();
// Each PE now holds a contiguous 2×2 block (4 int8 elements)
// mroi[0..3]: accessible for local computation without neighbor communication
writeFlow.write(mroi);
}
Example: YUV422 4×2 Packing
For YUV422 interleaved data (YUYVYUYV...), packing 4 columns × 2 rows places one complete YUYV × 2-row block (8 int8 elements) into each PE:
using OcmYUVShape = OcmTensor<std::uint8_t, 1, 1, imgH, imgW>;
using MinRoi = OcmTensor<std::uint8_t, 1, 1,
2 * core_array::coreDim, // rows: coreDim × 2
4 * core_array::coreDim>; // cols: coreDim × 4
using Desc = MinRoiDescriptor<MinRoi, AxisGroup<>, Granularity::Square, false, 4, 2>;
using Access = TensorAccessor<Desc>;
ReadFlow<Access, OcmYUVShape> readFlow(ocmYUV);
for (std::uint32_t i = 0; i < readFlow.numMROIs(); i++) {
auto yuv = readFlow.read();
// 8 elements per PE (2 rows of YUYV):
// Row 0: yuv[0]=Y0, yuv[1]=U, yuv[2]=Y1, yuv[3]=V
// Row 1: yuv[4]=Y2, yuv[5]=U, yuv[6]=Y3, yuv[7]=V
// Each PE has 4 pixels + shared U,V → color conversion without neighbor communication
}
Example: 2×2 FP16 Packing
FP16 elements (2 bytes each) support packCols=2 since 2 × 2 = 4 bytes fits the PLS limit:
using MinRoi = OcmTensor<Fp16, 1, 1,
2 * core_array::coreDim,
2 * core_array::coreDim>;
using Desc = MinRoiDescriptor<MinRoi, AxisGroup<>, Granularity::Square, false, 2, 2>;
using Access = TensorAccessor<Desc>;
// Each PE receives a 2×2 FP16 block (4 elements, 8 bytes total in LRM)
ReadFlow<Access, OcmInOutShape> readFlow(ocmInp);
Example: YUV420 (NV12) with Dual Packed Flows
YUV420 stores the Y plane at full resolution and interleaved UV at half resolution in both dimensions. A 2×2 block of Y pixels shares one U and one V value — this maps naturally to two synchronized packed flows:
// Y plane: full resolution, pack 2×2 into each PE
using OcmYShape = OcmTensor<std::uint8_t, 1, 1, imgH, imgW>;
using MinRoiY = OcmTensor<std::uint8_t, 1, 1, 2 * core_array::coreDim, 2 * core_array::coreDim>;
using DescY = MinRoiDescriptor<MinRoiY, AxisGroup<>, Granularity::Square, false, 2, 2>;
using AccessY = TensorAccessor<DescY>;
// UV plane (NV12 interleaved): half height, full width, pack U+V pair per PE
using OcmUVShape = OcmTensor<std::uint8_t, 1, 1, imgH / 2, imgW>;
using MinRoiUV = OcmTensor<std::uint8_t, 1, 1, imgH / 2, imgW>;
using DescUV = MinRoiDescriptor<MinRoiUV, AxisGroup<>, Granularity::Square, false, 2, 1>;
using AccessUV = TensorAccessor<DescUV>;
ReadFlow<AccessY, OcmYShape> readY(ocmY); // numMROIs matches between Y and UV
ReadFlow<AccessUV, OcmUVShape> readUV(ocmUV);
for (std::uint32_t i = 0; i < readY.numMROIs(); i++) {
auto yBlock = readY.read(); // 4 elements: Y00, Y01, Y10, Y11
auto uvBlock = readUV.read(); // 2 elements: U, V
// Each PE now has a 2×2 Y block + its corresponding U,V pair
// → YUV→RGB conversion can proceed locally without neighbor communication
}
The Y read flow (packCols=2, packRows=2) and UV read flow (packCols=2) are synchronized: each PE's 2×2 Y block corresponds exactly to one U,V pair from the half-resolution UV plane.
Partial Read/Write Flows with Tensor Offsets
In some cases, it is required to read/write only a certain part (sub-section) of an OCM tensor. This is where the Tensor offsets come in handy. All types of Flow APIs support tensor offsets. Following image shows what is a ROI and what us a tensor offset. There are two ways of defining the tensor offset.
- By passing required starting indexes of batch, channel, row and column dimensions to the Flow API.
- By creating and passing a TensorOffset object to the Flow API.
DDRArrayFlow
DdrArrayFlow<
typename Access,
typename DdrTensorShape,
typename RoiShape = DdrTensorShape,
StreamSide side = StreamSide::SOUTH,
std::uint32_t numMinRoiPerBufferLimit = 2,
std::uint32_t numBuffers = 2,
typename Buffer = OCMBufferShape<typename Access::MinRoi, RoiShape, typename Access::RoiAxisGroup, numMinRoiPerBufferLimit>,
>
(
DdrTensorShape& ddrTensor,
MemAllocator& allocator,
TensorOffset start = {}
)
Template parameters
Access: the TensorAccessor used to access the data array on cores.
DdrTensorShape: the shape of the DDR tensor.
RoiShape: the shape of the Region of Interest (ROI), which defaults to be the same as DdrTensorShape.
side: side the stream side for the PLS. This defaults to SOUTH for read operations
numMinRoiPerBufferLimit: the maximum number of minimum ROIs (minROIs) in each buffer. The ExtFlow class will determine the exact number to be used, and the value is accessible via ElsFlowType::numMinRoiPerBuffer.
numBuffers: the number of buffers to keep in on-chip memory (OCM). Setting this to 2 will enable asynchronous buffering for better cycle utilization, while setting it to 1 will save OCM usage.
Buffer: the type of the buffer to use.
Parameters
ddrTensor: Source DDR tensor
allocator: ocm allocator to be used locally to allocate ocm memory space
start: Offset for DDR tensor
Example: {bchOffset, chnOffset, rowOffset, colOffset}
ArrayDdrFlow
Functions of DdrArray/ArrayDdr Flows
read() : Return an NDArray with fetched data
read(MinRoiArray& data) : Fetch data from DDR to “data“ NDArray
write(MinRoiArray& data) : Write “data” Array to DDR
numMROIs() : Return the number of min RoIs
Examples
/tests/sweep_tests/tad_flow/ddr_array.cppusing ReadAccess = TensorAccessor<
MinRoiDescriptor<sweeps::MinRoiOcm, AxisGroup<sweeps::axis0, sweeps::axis1, sweeps::axis2>, sweeps::granularity>,
AxisGroup<sweeps::axis3, sweeps::axis4, sweeps::axis5>>;
using WriteAccess = TensorAccessor<
MinRoiDescriptor<sweeps::MinRoiOcm, AxisGroup<sweeps::axis0, sweeps::axis1, sweeps::axis2>, sweeps::granularity>,
AxisGroup<sweeps::axis3, sweeps::axis4, sweeps::axis5>>;
EPU_ENTRY void testDdrArray(sweeps::InOutDdr::ptrType inputDdrPtr, sweeps::InOutDdr::ptrType outputDdrPtr) {
initCycleCount();
MemAllocator ocmMem;
sweeps::InOutDdr ddrInp(inputDdrPtr);
sweeps::InOutDdr ddrOut(outputDdrPtr);
DdrArrayFlow<ReadAccess,
sweeps::InOutDdr,
MemAllocator,
sweeps::InOutDdr,
StreamSide::SOUTH,
sweeps::minRoiInBuffer,
sweeps::numBuffers>
elsRead(ddrInp, ocmMem);
ArrayDdrFlow<WriteAccess,
sweeps::InOutDdr,
MemAllocator,
sweeps::InOutDdr,
StreamSide::NORTH,
sweeps::minRoiInBuffer,
sweeps::numBuffers>
elsWrite(ddrOut, ocmMem);
for(std::uint32_t i = 0; i < elsRead.numMROIs(); ++i) {
auto a = elsRead.read();
elsWrite.write(a);
}
}
ExtReadFlow / ExtWriteFlow
ExtReadFlow and ExtWriteFlow APIs provide double-buffered data movement between DDR and OCM memory. These flows handle the DDR-to-OCM transfer with ping-pong buffering for optimal performance, enabling concurrent data transfer and compute operations.
ExtReadFlow
ExtReadFlow manages the transfer of data from DDR to OCM using double buffering. It prefetches the next OCM buffer while the current buffer is being processed, reducing stalls and improving overall performance.
ExtReadFlow<
typename Access, // TensorAccessor for data access patterns
typename DdrTensorShape, // DDR tensor type
typename OCMBufferShape = void, // OCM buffer shape (auto-determined if void)
typename OcmAllocatorType = void, // OCM allocator type
typename RoiShape = DdrTensorShape, // Region of Interest shape
std::uint32_t numMinRoiPerBufferLimit = 1, // Max minROIs per buffer
std::uint32_t numBuffers = 2 // Number of buffers (2 for double buffering)
>
(
DdrTensorShape& ddrTensor,
OcmAllocatorType& allocator,
TensorOffset start = {}
)
Key Features:
- Double buffering: Enables concurrent DDR-to-OCM transfers and compute operations
- Automatic buffer management: Handles OCM buffer allocation and deallocation
- Configurable buffer sizes: Supports customizable buffer shapes and minROI limits
ExtWriteFlow
ExtWriteFlow manages the transfer of data from OCM to DDR using double buffering. It allows the next() function to return while the outgoing flow for the previous buffer is still in process.
ExtWriteFlow<
typename Access, // TensorAccessor for data access patterns
typename DdrTensorShape, // DDR tensor type
typename OCMBufferShape = void, // OCM buffer shape (auto-determined if void)
typename OcmAllocatorType = void, // OCM allocator type
typename RoiShape = DdrTensorShape, // Region of Interest shape
std::uint32_t numMinRoiPerBufferLimit = 1, // Max minROIs per buffer
std::uint32_t numBuffers = 2 // Number of buffers (2 for double buffering)
>
(
DdrTensorShape& ddrTensor,
OcmAllocatorType& allocator,
TensorOffset start = {}
)
Key Features:
- Asynchronous writes: Returns from write operations while data transfer continues
- Double buffering: Enables concurrent OCM-to-DDR transfers and compute operations
- Pipeline optimization: Reduces overall kernel execution time
External Flow Example
A complete working example demonstrating the definitive double buffering flow pattern using ExtReadFlow can be found in the test suite:
/tests/static_data_tests/ext_read_flow_test.cpp/**
* @brief The main entry point for the EPU kernel demonstrating ExtReadFlow usage with double buffering.
* @param ddrInPtr Pointer to the input tensor in DDR memory.
* @param ddrOutPtr Pointer to the output tensor in DDR memory.
*/
EPU_ENTRY void testExtReadFlow(DdrInShape::ptrType ddrInPtr, DdrInShape::ptrType ddrOutPtr) {
initCycleCount();
blocklist_allocator::MemAllocator mem_allocator;
DdrInShape ddrIn(ddrInPtr);
DdrInShape ddrOut(ddrOutPtr);
// 1. Define the access pattern for the EXTERNAL flow (DDR -> OCM).
using ExtMinRoiDesc = MinRoiDescriptor<OcmInShape>;
using ExtAccess = TensorAccessor<ExtMinRoiDesc, AxisGroup<Direction::Width, Direction::Height, Direction::Channel>>;
// 2. Instantiate the external flow manager. This handles the DDR->OCM double buffering.
ExtReadFlow<ExtAccess, DdrInShape, OcmInShape, MemAllocator> extReadFlow(ddrIn, mem_allocator);
// Allocate output OCM buffer for storing processed results
OcmInShape ocmOut;
mem_allocator.allocate(ocmOut);
// 3. Loop through all the chunks using ExtReadFlow.
for(std::uint32_t i = 0; i < extReadFlow.numMROIs; ++i) {
// This call gets the next ready OCM buffer from the double-buffered DDR stream.
auto currentOcmBuffer = extReadFlow.read</*returnRef=*/false>();
// 4. Define the access pattern for the INTERNAL flows (OCM <-> LRM).
using OcmReadMinRoiDesc = MinRoiDescriptor<OcmInShape, DefaultAxisGroup, Granularity::Row>;
using OcmReadAccess = TensorAccessor<OcmReadMinRoiDesc>;
using OcmWriteMinRoiDesc = MinRoiDescriptor<OcmInShape, DefaultAxisGroup, Granularity::Row>;
using OcmWriteAccess = TensorAccessor<OcmWriteMinRoiDesc>;
// 5. Create the internal flows to stream data from the ready OCM buffer into core registers.
UnbufferedReadFlow<OcmReadAccess, OcmInShape> ocm_to_lrm_flow(currentOcmBuffer);
UnbufferedWriteFlow<OcmWriteAccess, OcmInShape> lrm_to_ocm_flow(ocmOut);
// 6. Process data using MinROI iteration pattern
typename decltype(ocm_to_lrm_flow)::MinRoiArray minROI;
ocm_to_lrm_flow.read(minROI);
// 7. Process the data (simple increment operation) - iterate over MinROI elements
typename decltype(lrm_to_ocm_flow)::MinRoiArray qOut;
for(std::size_t j = 0; j < minROI.size(); ++j) {
qOut[j] = minROI[j] + 1; // Process each element: increment by 1
}
// 8. Write processed data back
lrm_to_ocm_flow.write(qOut);
// 9. Copy this processed chunk back to the correct location in DDR
TensorOffset srcOffset = {0, 0, 0, 0};
TensorOffset dstOffset = {0, 0, 0, static_cast<std::int32_t>(i * OcmInShape::NUM_COLS)};
memCpy(ocmOut, srcOffset, ddrOut, dstOffset, mem_allocator);
}
}
This test demonstrates:
- Complete ExtReadFlow setup with proper tensor definitions and memory allocation
- Double buffering in action with DDR-to-OCM transfers managed automatically
- Data validation ensuring processed results match expected output
- Realistic processing pattern showing how to combine external and internal flows
Flow Architecture:
- DDR → OCM: Managed by
ExtReadFlowwith double buffering - OCM → LRM: Managed by internal
UnbufferedReadFlow - Concurrent operation: While processing current buffer, next buffer is being prefetched
Performance Benefits
External Flow APIs provide several performance advantages:
- Overlapped Execution: Memory transfers and compute operations run concurrently
- Reduced Stalls: Double buffering minimizes wait times for data availability
- Automatic Management: Handles complex buffer management and synchronization
- Scalable Design: Works efficiently with different buffer sizes and access patterns
DDR ↔︎ Array flow API
The Chimera architecture allows the cores to work simultaneously with the memory transfers between the array and the L2 meory, and between the L2 memory and the DDR. The flow APIs DdrArrayFlow and ArrayDdrFlow combine memcpy(DDR ↔︎ L2M) and Read/Write Flow APIs. This has two benefits:
It simplifies the code as the user can use the same code for memory transfers across the system.
It boosts performance as the API also handles the dual buffer needed for parallelization.

However, it is important to note that users still have access to L2 flow and DDR memcpy APIs.

ExtFlow
