Traditional Convolutions
Convolution [Filter NxN, Stride 1x1]
/src/neuralNetBlocks/conv.hppLines 940–952template <typename T,
std::uint32_t numInCh,
std::uint32_t filterSize = 1,
std::uint32_t shiftAmount = 0,
bool useVaping = false,
bool sameWeights = false,
bool clearAccums = true,
typename NDArray>
INLINE qVar_t<T> convTileBlockInt8(NDArray& qData)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | the output type | |
numInCh | std::uint32_t | The number of input channels in the convolution | |
filterSize | std::uint32_t | 1 | The filter size of the convolution |
shiftAmount | std::uint32_t | 0 | The amount to shift the output of the convolution by |
useVaping | bool | false | Whether or not the convolution will implement a Virtual Array Partitioning (VAP) |
sameWeights | bool | false | if True the weights are popped once and used through out the convolution. |
clearAccums | bool | true | if True it clears the qAccumAll variable used in the intrinsic |
NDArray | typename |
| Name | Type | Description |
|---|---|---|
qData | NDArray& | The activations of the input |
INLINE qVar_t<T>
/src/neuralNetBlocks/conv.hppLines 889–898INT8 convolution tile block with a runtime input channel count.
numCenterCh and numRotationCh are derived from numInCh at runtime. Caller ensures numInCh <= NDArray::size().
template <typename T,
std::uint32_t filterSize = 1,
std::uint32_t shiftAmount = 0,
bool clearAccums = true,
typename NDArray>
INLINE qVar_t<T> convTileBlockInt8(NDArray& qData, std::int32_t numInCh)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | ||
filterSize | std::uint32_t | 1 | |
shiftAmount | std::uint32_t | 0 | |
clearAccums | bool | true | |
NDArray | typename |
| Name | Type | Description |
|---|---|---|
qData | NDArray& | |
numInCh | std::int32_t |
INLINE qVar_t<T>
/src/neuralNetBlocks/conv.hppLines 871–880INT8 convolution tile block with pre-computed MAC iteration counts.
Caller supplies numCenterCh and numRotationCh directly, avoiding the per-call division. Use when the values are hoisted out of a hot loop.
template <typename T,
std::uint32_t filterSize = 1,
std::uint32_t shiftAmount = 0,
bool clearAccums = true,
typename NDArray>
INLINE qVar_t<T> convTileBlockInt8(NDArray& qData,
std::uint32_t numCenterCh,
std::uint32_t numRotationCh)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | ||
filterSize | std::uint32_t | 1 | |
shiftAmount | std::uint32_t | 0 | |
clearAccums | bool | true | |
NDArray | typename |
| Name | Type | Description |
|---|---|---|
qData | NDArray& | |
numCenterCh | std::uint32_t | |
numRotationCh | std::uint32_t |
INLINE qVar_t<T>
/src/neuralNetBlocks/conv.hppLines 594–605Convolution with stride 2.
template <typename T,
std::uint32_t numInCh,
std::uint32_t filterSize = 1,
std::uint32_t shiftAmount = 0,
bool useVaping = false,
bool sameWeights = false>
INLINE qVar_t<T> convTileBlockInt8(qVar_t<std::int8_t> qData)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | The output type. | |
numInCh | std::uint32_t | The number of input channels in the convolution. | |
filterSize | std::uint32_t | 1 | The filter size of the convolution. |
shiftAmount | std::uint32_t | 0 | The amount to shift the output of the convolution by. |
useVaping | bool | false | Whether or not the convolution will implement a Virtual Array Partitioning (VAP). |
sameWeights | bool | false | If 'true' the weights are popped once and used through out the convolution. |
| Name | Type | Description |
|---|---|---|
qData | qVar_t<std::int8_t> | The activations of the input. */ // clang-format on |
INLINE qVar_t<T>
Convolution [Filter 5x5, Stride 2x2]
/src/neuralNetBlocks/conv.hppLines 1054–1072Performs a 5x5 conv with stride of 2 on a 2 channel packed input. This operation performs 4 3x3 convolutions to get the 5x5 conv output. The 4 strided input tiles can be mapped as follows:
| 0, 1 | 0, 1 | 0, [1] |
| 2, 3 | 2, 3 | 2, [3] |
---------------------------------
| 0, 1 | 0, 1 | 0, [1] |
| 2, 3 | 2, 3 | 2, [3] |
---------------------------------
| 0, 1 | 0, 1 | 0, [1] |
|[2], [3] | [2], [3] | [2], [3] |
The weights corresponding to indexes inbetween [] need to be zeros for the corresponding convs.
template <typename T,
std::int32_t numInChn,
std::uint32_t filterSize = 1,
std::uint32_t shiftAmount = 0,
bool useVaping = false,
bool sameWeights = false,
typename NDArrayType>
INLINE qVar_t<T> conv5x5StrideOf2x2TileBlockInt8(NDArrayType& qData)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | The output qVar type | |
numInChn | std::int32_t | The number of input channels. | |
filterSize | std::uint32_t | 1 | The size of the convolutional filter. |
shiftAmount | std::uint32_t | 0 | The amount to shift the output of the convolution by |
useVaping | bool | false | Whether or not the convolution will implement a Virtual Array Partitioning (VAP) |
sameWeights | bool | false | if True the weights are popped once and used through out the convolution. |
NDArrayType | typename | The input array type of `qData`. |
| Name | Type | Description |
|---|---|---|
qData | NDArrayType& | The input data fetched, the neighbor moving will be handled inside the function |
INLINE qVar_t<T> — qResult Convolution output with 5x5 filter size and stride of 2x2 for input `qData`. / // clang-format on
Convolution [Filter NxN, Stride 2x2]
/src/neuralNetBlocks/conv.hppLines 1245–1263Computes convolutions in int8 precision with a height, width stride of 2, 2.
template <typename T,
std::uint32_t numInCh,
std::uint32_t filterSize = 1,
std::uint32_t shiftAmount = 0,
bool useVaping = false,
bool sameWeights = false,
typename NDArrayType>
INLINE qVar_t<T> convStrideOf2x2TileBlockInt8(NDArrayType& qData)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | The type of the input values. | |
numInCh | std::uint32_t | The type of the input values. | |
filterSize | std::uint32_t | 1 | The filter size of the convolution. |
shiftAmount | std::uint32_t | 0 | The amount to shift the output of the convolution by. |
useVaping | bool | false | Whether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy. |
sameWeights | bool | false | If 'True' the weights are popped once and used through out the convolution. |
NDArrayType | typename | The input array type of `qData`. |
| Name | Type | Description |
|---|---|---|
qData | NDArrayType& | The activations of the input. |
INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on
Convolution [Filter NxN, Stride 2x2]
/src/neuralNetBlocks/conv.hppLines 1245–1263Computes convolutions in int8 precision with a height, width stride of 2, 2.
template <typename T,
std::uint32_t numInCh,
std::uint32_t filterSize = 1,
std::uint32_t shiftAmount = 0,
bool useVaping = false,
bool sameWeights = false,
typename NDArrayType>
INLINE qVar_t<T> convStrideOf2x2TileBlockInt8(NDArrayType& qData)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | The type of the input values. | |
numInCh | std::uint32_t | The type of the input values. | |
filterSize | std::uint32_t | 1 | The filter size of the convolution. |
shiftAmount | std::uint32_t | 0 | The amount to shift the output of the convolution by. |
useVaping | bool | false | Whether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy. |
sameWeights | bool | false | If 'True' the weights are popped once and used through out the convolution. |
NDArrayType | typename | The input array type of `qData`. |
| Name | Type | Description |
|---|---|---|
qData | NDArrayType& | The activations of the input. |
INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on
Depthwise Convolutions
Depthwise Convolution [Filter NxN, Stride 1x1]
/src/neuralNetBlocks/conv.hppLines 1665–1709Computes Groupwise convolutions in int8 precision
template <typename T,
std::uint32_t groupSize = 4,
std::uint32_t filterSize = 3,
std::uint32_t stride = 1,
FracRepType shiftAmount = 0,
bool useVaping = false,
bool sameWeights = false,
typename NDArrayType>
INLINE qVar_t<T> groupwiseConvTileBlockInt8(NDArrayType& qData,
std::int32_t outputIteration = 0)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | The type of the input values. | |
groupSize | std::uint32_t | 4 | The number of channels in a group. |
filterSize | std::uint32_t | 3 | The filter size of the convolution. |
stride | std::uint32_t | 1 | The stride of the convolution. |
shiftAmount | FracRepType | 0 | The amount to shift the output of the convolution by. |
useVaping | bool | false | Whether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy. |
sameWeights | bool | false | If 'True' the weights are popped once and used through out the convolution. |
NDArrayType | typename | The input array type of `qData`. |
| Name | Type | Description |
|---|---|---|
qData | NDArrayType& | The activations of the input. |
outputIteration | std::int32_t | If function is looped for a very large convolution that must run in multiple iterations, `outputIteration` is used to calculate the offset in the output tensor's memory where partial convolutional outputs should be stored. |
INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on
Depthwise Convolution [Filter 3x3, Stride 2x2]
/src/neuralNetBlocks/conv.hppLines 1549–1645Depthwise conv function for 3x3 and stride 2x2 case.
The data is laid out in such a way that two outputs are computed at a time using two intrinsic calls:
A Typical 3x3 filter is labeled as:
0 │ 1 │ 2 3 │ 4 │ 5 6 │ 7 │ 8
The Strided flows bring in packed rows and columns in a single core Each core has the bottom right 2x2 grid of activations present for the conv. Due to the v2i8 nature of the packing, consecutive channels are brought in together. This yields the following sequential data layout in qData: 4_0 4_1 5_0 5_1 7_0 7_1 8_0 8_1
To compute the rest of the 3x3 convolution, local data movement is used to pull data in 4 byte increments from the West neighbor core which contains 3_0 3_1 and 6_0 6_1, then data from the North neighbor core which contains 1_0 1_1 2_0 2_1 and finally data from the North-West neighbor core which contains 0_0 0_1. Post Local Data Movement, the data is arranged in qMove as: x x 3_0 3_1 x x 6_0 6_1 1_0 1_1 2_0 2_1 x x 0_0 0_1
heightOffset/widthOffset > 0 implies how much to move the data North/West respectively. For example, if the border values are 0 then we would want the input (not represented below in its strided form which is the actual input) to go from.
> <b>Note:</b> The double lines are border cores('||' or '=') and a square made of single lines represents the same convolution across 4 tiles corresponding to different filter/input indices. Each box corresponds to a conv in original representation without any offset.
0 0 ║ 0 0 0 0 0 0 ... 0 0 ║ 0 0 0 0 0 0 ... ════╬═════════════════════════ ║ ┌─────┐ ┌─────┐ ┌─────┐ 0 0 ║ │ 4 5 │ │ 4 5 │ │ 4 5 │ ... 0 0 ║ │ 7 8 │ │ 7 8 │ │ 7 8 │ ... ║ └─────┘ └─────┘ └─────┘ ║ ┌─────┐ ┌─────┐ ┌─────┐ 0 0 ║ │ 4 5 │ │ 4 5 │ │ 4 5 │ ... 0 0 ║ │ 7 8 │ │ 7 8 │ │ 7 8 │ ... ║ └─────┘ └─────┘ └─────┘ ║ ┌─────┐ ┌─────┐ ┌─────┐ 0 0 ║ │ 4 5 │ │ 4 5 │ │ 4 5 │ ... 0 0 ║ │ 7 8 │ │ 7 8 │ │ 7 8 │ ... ║ └─────┘ └─────┘ └─────┘
to as follows for heightOffset = widthOffset = 1
0 0 ║ 0 0 0 0 0 ... ┌───╫───┐ ┌─────┐ ┌─────┐ 0 │ 4 ║ 5 │ │ 4 5 │ │ 4 5 │ ... ══╪═══╬═══╪═╪═════╪═╪═════╪═ 0 │ 7 ║ 8 │ │ 7 8 │ │ 7 8 │ ... └───╫───┘ └─────┘ └─────┘ ┌───╫───┐ ┌─────┐ ┌─────┐ 0 │ 4 ║ 5 │ │ 4 5 │ │ 4 5 │ ... 0 │ 7 ║ 8 │ │ 7 8 │ │ 7 8 │ ... └───╫───┘ └─────┘ └─────┘ ┌───╫───┐ ┌─────┐ ┌─────┐ 0 │ 4 ║ 5 │ │ 4 5 │ │ 4 5 │ ... 0 │ 7 ║ 8 │ │ 7 8 │ │ 7 8 │ ... └───╫───┘ └─────┘ └─────┘
In the first diagram the top left corner conv gets converted to: 0 ║ 0 0 4 ║ 5 4 ══╬════ ➤ ══╬════ 0 ║ 4 5 7 ║ 8 7 0 ║ 7 8 4 ║ 5 4
which means the 8 7 5 4 should be on the same core across the 4 tiles in that order
As can be deciphered for the above diagram, this would require the following operations
4_0, 4_1 -> Move West, Move North 5_0, 5_1 -> Move North 7_0, 7_1 -> Move West 8_0, 8_1 ->
and would also lead to a change in the of ops to 8_0 8_1 7_0 7_1 5_0 5_1 4_0 4_1. All this done with the help of applyCoreOffsetWE/NS calls for neighbor movements.
template <std::int32_t heightOffset = 0,
std::int32_t widthOffset = 0,
bool useVaping = false,
typename InNDArrayType>
INLINE depthwiseConv3x3StrideOf2x2TileBlock(InNDArrayType& qData,
std::uint32_t outputIteration)| Name | Type | Default | Description |
|---|---|---|---|
heightOffset | std::int32_t | 0 | Padding related offset in height. |
widthOffset | std::int32_t | 0 | Padding related offset in width. |
useVaping | bool | false | Use VAPing (Virtual Array Partitioning) in the convolution. |
InNDArrayType | typename | Type of the input array (`qData`). |
| Name | Type | Description |
|---|---|---|
qData | InNDArrayType& | [in] The input data fetched, the neighbor movement within the function. Must be at least 2-byte aligned: when numMacsPerCore > 8 the input reorder reads it in whole 2-byte units. Every current caller satisfies this with room to spare -- the numMacsPerCore <= 8 path already reads the same buffer through an int32 pointer, a stricter requirement that ships today. |
outputIteration | std::uint32_t | The output channel index for the first channel to be processed. |
INLINE — A vector of `core_array::numAccumSums` integers. / // clang-format on
Depthwise Dilated Convolution [Filter NxN, Dilation Factor DxD]
/src/neuralNetBlocks/conv.hppLines 2560–2624Computes Depthwise dilated convolutions in int8 precision. This operation is DDR only.
template <std::int32_t dilationFactor,
std::uint32_t filterSize = 3,
FracRepType numFracBits = 31,
typename DdrInOutShape,
typename DdrWeightShape,
typename OcmAllocatorType>
INLINE void depthwiseDilatedConvInt8(DdrInOutShape& ddrIn,
DdrWeightShape& ddrWeights,
DdrInOutShape& ddrOut,
OcmAllocatorType& ocmMemAlloc,
FixedPoint32<numFracBits> scale,
std::int8_t xZeroPoint,
std::int8_t wZeroPoint,
std::int8_t yZeroPoint)| Name | Type | Default | Description |
|---|---|---|---|
dilationFactor | std::int32_t | In pytorch this is the spacing between kernel elements. In the quadric implementation this is the spacing between input tensor elements. Effectively both are same. Currently this operation only supports 1, 2, and 4 dilation factors. | |
filterSize | std::uint32_t | 3 | The filter size of the convolution. Only filterSize of 3 is supported. |
numFracBits | FracRepType | 31 | number of fractional bits for quantization parameters |
DdrInOutShape | typename | The shape of the input/output DDR tensor | |
DdrWeightShape | typename | The shape of the weight DDR tensor | |
OcmAllocatorType | typename | Type of OCM memory allocator |
| Name | Type | Description |
|---|---|---|
ddrIn | DdrInOutShape& | The DDR input tensor |
ddrWeights | DdrWeightShape& | The DDR weight tensor (contains weights for conv as well as bias) |
ddrOut | DdrInOutShape& | The DDR output tensor |
ocmMemAlloc | OcmAllocatorType& | ocm allocator to be used locally to allocate ocm memory space |
scale | FixedPoint32<numFracBits> | Scale parameter for quantization |
xZeroPoint | std::int8_t | Zero point quantization parameter for input tensor (needs to be 0) |
wZeroPoint | std::int8_t | Zero point quantization parameter for weight tensor (needs to be 0) |
yZeroPoint | std::int8_t | Zero point quantization parameter for output tensor / // clang-format on |
INLINE void
Groupwise Convolutions
Groupwise Convolution [Filter NxN, Stride MxM]
/src/neuralNetBlocks/conv.hppLines 1665–1709Computes Groupwise convolutions in int8 precision
template <typename T,
std::uint32_t groupSize = 4,
std::uint32_t filterSize = 3,
std::uint32_t stride = 1,
FracRepType shiftAmount = 0,
bool useVaping = false,
bool sameWeights = false,
typename NDArrayType>
INLINE qVar_t<T> groupwiseConvTileBlockInt8(NDArrayType& qData,
std::int32_t outputIteration = 0)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | The type of the input values. | |
groupSize | std::uint32_t | 4 | The number of channels in a group. |
filterSize | std::uint32_t | 3 | The filter size of the convolution. |
stride | std::uint32_t | 1 | The stride of the convolution. |
shiftAmount | FracRepType | 0 | The amount to shift the output of the convolution by. |
useVaping | bool | false | Whether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy. |
sameWeights | bool | false | If 'True' the weights are popped once and used through out the convolution. |
NDArrayType | typename | The input array type of `qData`. |
| Name | Type | Description |
|---|---|---|
qData | NDArrayType& | The activations of the input. |
outputIteration | std::int32_t | If function is looped for a very large convolution that must run in multiple iterations, `outputIteration` is used to calculate the offset in the output tensor's memory where partial convolutional outputs should be stored. |
INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on
FP16 Convolutions
FP16 Convolution [Filter NxN, Stride 1x1 - Auto Buffer]
/src/neuralNetBlocks/conv.hppLines 670–683this is a wrapper of the other version of convTileBlockFp16, this one allocates its internal buffer when and should be preferred in handwritten code.
template <typename OutT,
std::uint32_t numInCh,
std::uint32_t filterSize = 1,
bool useVaping = false,
bool sameWeights = false,
std::size_t inSize,
bool drainBroadcastFifo = false,
FracRepType numFracBits>
INLINE convTileBlockFp16(container::NDArray<qVar_t<FixedPoint32<numFracBits>>, inSize>& qData)| Name | Type | Default | Description |
|---|---|---|---|
OutT | typename | The output data type, which must be a FixedPoint type. | |
numInCh | std::uint32_t | The number of input channels for the convolution. | |
filterSize | std::uint32_t | 1 | The size of the convolution filter. Defaults to 1. Supported sizes are 1, 3, and 5. |
useVaping | bool | false | A boolean flag to enable vaping. Defaults to false. |
sameWeights | bool | false | if True the weights are popped once and used through out the convolution. |
inSize | std::size_t | inferred, the size of input array | |
drainBroadcastFifo | bool | false | When true, consumes leftover broadcast FIFO elements after the MAC operation. Defaults to false. |
numFracBits | FracRepType | inferred, the number of frac bits of input |
| Name | Type | Description |
|---|---|---|
qData | container::NDArray<qVar_t<FixedPoint32<numFracBits>>, inSize>& | An NDArray of qVar_t<FixedPoint32<numFracBits>> representing the input data for the convolution. |
INLINE — A qVar_t<OutT> containing the result of the convolution after shifting.
FP16 Groupwise Convolutions
FP16 Groupwise Convolution [Filter NxN, Stride MxM]
/src/neuralNetBlocks/conv.hppLines 1823–1945Computes Groupwise convolutions in fp16 precision
template <typename T,
std::uint32_t groupSize = 4,
std::uint32_t filterSize = 3,
std::uint32_t stride = 1,
FracRepType shiftAmount = 0,
bool useVaping = false,
bool sameWeights = false,
typename NDArrayType>
INLINE qVar_t<T> groupwiseConvTileBlockFp16(NDArrayType& qData,
std::int32_t outputIteration = 0)| Name | Type | Default | Description |
|---|---|---|---|
T | typename | The type of the input values. | |
groupSize | std::uint32_t | 4 | The number of channels in a group. |
filterSize | std::uint32_t | 3 | The filter size of the convolution. |
stride | std::uint32_t | 1 | The stride of the convolution. |
shiftAmount | FracRepType | 0 | The amount to shift the output of the convolution by. |
useVaping | bool | false | Whether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy. |
sameWeights | bool | false | If 'True' the weights are popped once and used through out the convolution. |
NDArrayType | typename | The input array type of `qData`. |
| Name | Type | Description |
|---|---|---|
qData | NDArrayType& | The activations of the input. |
outputIteration | std::int32_t | If function is looped for a very large convolution that must run in multiple iterations, `outputIteration` is used to calculate the offset in the output tensor's memory where partial convolutional outputs should be stored. |
INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on
MAC Issue and Collect
Every convTileBlock* function launches the MEU and immediately reads the accumulators with accsum, which stalls until the MAC has finished. MacDescriptor splits the two so independent work can run in the MAC's shadow. Declare the descriptor with the MAC's shape, call start(qData) to launch, do the independent work, then collect(qData) to wait and read the result. Each fused function is exactly that: a descriptor followed by start and collect, so the fused and the split form are the same MAC.
MacDescriptor
/src/neuralNetBlocks/conv.hppLines 169–578Compile-time description of one MEU launch: the MAC unit, the window and how to read the result. Declare it, start<numInCh>(qData) or start(qData, k) launches, collect waits. Operands and FX32 scratch may be NDArrays, NDArrayViews, or LrmBufferHandles. Collect any pending flow transfer before passing a buffer, and keep the launch operand unchanged until MAC collection completes.
| Name | Type | Default | Description |
|---|---|---|---|
type | MacType | MAC unit. | |
OutT | typename | Result type: an integer for Int8, a FixedPoint32 for Fp16. | |
filterSize | std::uint32_t | 1 | Convolution window, one of {1, 3, 5}; 1 for matmul and fully connected layers. |
shiftAmount | std::uint32_t | 0 | Right shift applied to the accumulator sum. |
drainBroadcastFifo | bool | false | Fp16 1x1: pop the broadcast lines staged beyond the channel count after the MAC; a windowed launch consumes every line it stages and never drains. |
clearAccums | bool | true | Clear the accumulators before the launch; false chains onto an earlier launch of either MAC type. |
start derives everything from the template parameters: which MEU path (MacType::Int8 or MacType::Fp16), the window, the centre and rotation counts from the channel count, and which operand layout it was given (an Fp16 array or view or an int16_t array launch directly; an FX32 array is converted by the two-argument start(qFx32, qScratch) into a caller-owned fp16 scratch, which is then the operand collect takes). A compile-time count can be passed as a template argument of the calls, start<K>(qData) and, for fp16, collect<K>(qData); a runtime count is a call argument, start(qData, k) and, for fp16, collect(qData, k). Int8 collects with collect(qData) for both static and runtime counts. A second int8 launch with clearAccums = false adds into the accumulators of an uncollected first launch, and one collect(qData) then returns the sum of both.
using namespace chimera::nn;
const MacDescriptor<MacType::Int8, std::int32_t> mac; // int8, 1x1, no shift
mac.start<K>(qTile); // MEU starts over K channels
nextTileFlow.issueRead(qNext); // lands while the MAC runs
qAcc += mac.collect(qTile); // wait only here
const MacDescriptor<MacType::Fp16, FixedPoint32<20>, 3> conv3; // fp16 3x3 window
conv3.start<K>(qFp16Tile); ... qOut = conv3.collect<K>(qFp16Tile);
const MacDescriptor<MacType::Fp16, FixedPoint32<20>> fromFx32; // FX32 operand, converted into qScratch
fromFx32.start<K>(qFx32Tile, qScratch); ... qOut = fromFx32.collect<K>(qFx32Tile, qScratch);
mac.start(qTile, k); ... qAcc += mac.collect(qTile); // int8, K at runtime
fromFx32.start(qFp16Tile, k); ... qOut = fromFx32.collect(qFp16Tile, k); // fp16, K at runtime
