Unlock this feature

This feature isn’t part of your plan yet

Contact sales to get upgraded to the full DevStudio experience.

Unlock this feature

This feature isn't part of your plan yet.

Convolutions

Traditional Convolutions

Convolution [Filter NxN, Stride 1x1]

convTileBlockInt8function
File: /src/neuralNetBlocks/conv.hppLines 940–952
template <typename T,
          std::uint32_t numInCh,
          std::uint32_t filterSize = 1,
          std::uint32_t shiftAmount = 0,
          bool useVaping = false,
          bool sameWeights = false,
          bool clearAccums = true,
          typename NDArray>
INLINE qVar_t<T> convTileBlockInt8(NDArray& qData)
NameTypeDefaultDescription
Ttypenamethe output type
numInChstd::uint32_tThe number of input channels in the convolution
filterSizestd::uint32_t1The filter size of the convolution
shiftAmountstd::uint32_t0The amount to shift the output of the convolution by
useVapingboolfalseWhether or not the convolution will implement a Virtual Array Partitioning (VAP)
sameWeightsboolfalseif True the weights are popped once and used through out the convolution.
clearAccumsbooltrueif True it clears the qAccumAll variable used in the intrinsic
NDArraytypename
NameTypeDescription
qDataNDArray&The activations of the input

INLINE qVar_t<T>

convTileBlockInt8function
File: /src/neuralNetBlocks/conv.hppLines 889–898

INT8 convolution tile block with a runtime input channel count.

numCenterCh and numRotationCh are derived from numInCh at runtime. Caller ensures numInCh <= NDArray::size().

template <typename T,
          std::uint32_t filterSize = 1,
          std::uint32_t shiftAmount = 0,
          bool clearAccums = true,
          typename NDArray>
INLINE qVar_t<T> convTileBlockInt8(NDArray& qData, std::int32_t numInCh)
NameTypeDefaultDescription
Ttypename
filterSizestd::uint32_t1
shiftAmountstd::uint32_t0
clearAccumsbooltrue
NDArraytypename
NameTypeDescription
qDataNDArray&
numInChstd::int32_t

INLINE qVar_t<T>

convTileBlockInt8function
File: /src/neuralNetBlocks/conv.hppLines 871–880

INT8 convolution tile block with pre-computed MAC iteration counts.

Caller supplies numCenterCh and numRotationCh directly, avoiding the per-call division. Use when the values are hoisted out of a hot loop.

template <typename T,
          std::uint32_t filterSize = 1,
          std::uint32_t shiftAmount = 0,
          bool clearAccums = true,
          typename NDArray>
INLINE qVar_t<T> convTileBlockInt8(NDArray& qData,
                                   std::uint32_t numCenterCh,
                                   std::uint32_t numRotationCh)
NameTypeDefaultDescription
Ttypename
filterSizestd::uint32_t1
shiftAmountstd::uint32_t0
clearAccumsbooltrue
NDArraytypename
NameTypeDescription
qDataNDArray&
numCenterChstd::uint32_t
numRotationChstd::uint32_t

INLINE qVar_t<T>

convTileBlockInt8function
File: /src/neuralNetBlocks/conv.hppLines 594–605

Convolution with stride 2.

template <typename T,
          std::uint32_t numInCh,
          std::uint32_t filterSize = 1,
          std::uint32_t shiftAmount = 0,
          bool useVaping = false,
          bool sameWeights = false>
INLINE qVar_t<T> convTileBlockInt8(qVar_t<std::int8_t> qData)
NameTypeDefaultDescription
TtypenameThe output type.
numInChstd::uint32_tThe number of input channels in the convolution.
filterSizestd::uint32_t1The filter size of the convolution.
shiftAmountstd::uint32_t0The amount to shift the output of the convolution by.
useVapingboolfalseWhether or not the convolution will implement a Virtual Array Partitioning (VAP).
sameWeightsboolfalseIf 'true' the weights are popped once and used through out the convolution.
NameTypeDescription
qDataqVar_t<std::int8_t>The activations of the input. */ // clang-format on

INLINE qVar_t<T>

Convolution [Filter 5x5, Stride 2x2]

conv5x5StrideOf2x2TileBlockInt8function
File: /src/neuralNetBlocks/conv.hppLines 1054–1072

Performs a 5x5 conv with stride of 2 on a 2 channel packed input. This operation performs 4 3x3 convolutions to get the 5x5 conv output. The 4 strided input tiles can be mapped as follows:

| 0,   1  |  0,   1  |  0,  [1] |
| 2,   3  |  2,   3  |  2,  [3] |
---------------------------------
| 0,   1  |  0,   1  |  0,  [1] |
| 2,   3  |  2,   3  |  2,  [3] |
---------------------------------
| 0,   1  |  0,   1  |  0,  [1] |
|[2], [3] | [2], [3] | [2], [3] |

The weights corresponding to indexes inbetween [] need to be zeros for the corresponding convs.

template <typename T,
          std::int32_t numInChn,
          std::uint32_t filterSize = 1,
          std::uint32_t shiftAmount = 0,
          bool useVaping = false,
          bool sameWeights = false,
          typename NDArrayType>
INLINE qVar_t<T> conv5x5StrideOf2x2TileBlockInt8(NDArrayType& qData)
NameTypeDefaultDescription
TtypenameThe output qVar type
numInChnstd::int32_tThe number of input channels.
filterSizestd::uint32_t1The size of the convolutional filter.
shiftAmountstd::uint32_t0The amount to shift the output of the convolution by
useVapingboolfalseWhether or not the convolution will implement a Virtual Array Partitioning (VAP)
sameWeightsboolfalseif True the weights are popped once and used through out the convolution.
NDArrayTypetypenameThe input array type of `qData`.
NameTypeDescription
qDataNDArrayType&The input data fetched, the neighbor moving will be handled inside the function

INLINE qVar_t<T> — qResult Convolution output with 5x5 filter size and stride of 2x2 for input `qData`. / // clang-format on

Convolution [Filter NxN, Stride 2x2]

convStrideOf2x2TileBlockInt8function
File: /src/neuralNetBlocks/conv.hppLines 1245–1263

Computes convolutions in int8 precision with a height, width stride of 2, 2.

template <typename T,
          std::uint32_t numInCh,
          std::uint32_t filterSize = 1,
          std::uint32_t shiftAmount = 0,
          bool useVaping = false,
          bool sameWeights = false,
          typename NDArrayType>
INLINE qVar_t<T> convStrideOf2x2TileBlockInt8(NDArrayType& qData)
NameTypeDefaultDescription
TtypenameThe type of the input values.
numInChstd::uint32_tThe type of the input values.
filterSizestd::uint32_t1The filter size of the convolution.
shiftAmountstd::uint32_t0The amount to shift the output of the convolution by.
useVapingboolfalseWhether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy.
sameWeightsboolfalseIf 'True' the weights are popped once and used through out the convolution.
NDArrayTypetypenameThe input array type of `qData`.
NameTypeDescription
qDataNDArrayType&The activations of the input.

INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on

Convolution [Filter NxN, Stride 2x2]

convStrideOf2x2TileBlockInt8function
File: /src/neuralNetBlocks/conv.hppLines 1245–1263

Computes convolutions in int8 precision with a height, width stride of 2, 2.

template <typename T,
          std::uint32_t numInCh,
          std::uint32_t filterSize = 1,
          std::uint32_t shiftAmount = 0,
          bool useVaping = false,
          bool sameWeights = false,
          typename NDArrayType>
INLINE qVar_t<T> convStrideOf2x2TileBlockInt8(NDArrayType& qData)
NameTypeDefaultDescription
TtypenameThe type of the input values.
numInChstd::uint32_tThe type of the input values.
filterSizestd::uint32_t1The filter size of the convolution.
shiftAmountstd::uint32_t0The amount to shift the output of the convolution by.
useVapingboolfalseWhether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy.
sameWeightsboolfalseIf 'True' the weights are popped once and used through out the convolution.
NDArrayTypetypenameThe input array type of `qData`.
NameTypeDescription
qDataNDArrayType&The activations of the input.

INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on

Depthwise Convolutions

Depthwise Convolution [Filter NxN, Stride 1x1]

groupwiseConvTileBlockInt8function
File: /src/neuralNetBlocks/conv.hppLines 1665–1709

Computes Groupwise convolutions in int8 precision

template <typename T,
          std::uint32_t groupSize = 4,
          std::uint32_t filterSize = 3,
          std::uint32_t stride = 1,
          FracRepType shiftAmount = 0,
          bool useVaping = false,
          bool sameWeights = false,
          typename NDArrayType>
INLINE qVar_t<T> groupwiseConvTileBlockInt8(NDArrayType& qData,
                                            std::int32_t outputIteration = 0)
NameTypeDefaultDescription
TtypenameThe type of the input values.
groupSizestd::uint32_t4The number of channels in a group.
filterSizestd::uint32_t3The filter size of the convolution.
stridestd::uint32_t1The stride of the convolution.
shiftAmountFracRepType0The amount to shift the output of the convolution by.
useVapingboolfalseWhether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy.
sameWeightsboolfalseIf 'True' the weights are popped once and used through out the convolution.
NDArrayTypetypenameThe input array type of `qData`.
NameTypeDescription
qDataNDArrayType&The activations of the input.
outputIterationstd::int32_tIf function is looped for a very large convolution that must run in multiple iterations, `outputIteration` is used to calculate the offset in the output tensor's memory where partial convolutional outputs should be stored.

INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on

Depthwise Convolution [Filter 3x3, Stride 2x2]

depthwiseConv3x3StrideOf2x2TileBlockfunction
File: /src/neuralNetBlocks/conv.hppLines 1549–1645

Depthwise conv function for 3x3 and stride 2x2 case.

The data is laid out in such a way that two outputs are computed at a time using two intrinsic calls:

A Typical 3x3 filter is labeled as:

0 │ 1 │ 2 3 │ 4 │ 5 6 │ 7 │ 8

The Strided flows bring in packed rows and columns in a single core Each core has the bottom right 2x2 grid of activations present for the conv. Due to the v2i8 nature of the packing, consecutive channels are brought in together. This yields the following sequential data layout in qData: 4_0 4_1 5_0 5_1 7_0 7_1 8_0 8_1

To compute the rest of the 3x3 convolution, local data movement is used to pull data in 4 byte increments from the West neighbor core which contains 3_0 3_1 and 6_0 6_1, then data from the North neighbor core which contains 1_0 1_1 2_0 2_1 and finally data from the North-West neighbor core which contains 0_0 0_1. Post Local Data Movement, the data is arranged in qMove as: x x 3_0 3_1 x x 6_0 6_1 1_0 1_1 2_0 2_1 x x 0_0 0_1

heightOffset/widthOffset > 0 implies how much to move the data North/West respectively. For example, if the border values are 0 then we would want the input (not represented below in its strided form which is the actual input) to go from.

> <b>Note:</b> The double lines are border cores('||' or '=') and a square made of single lines represents the same convolution across 4 tiles corresponding to different filter/input indices. Each box corresponds to a conv in original representation without any offset.

0 0 ║ 0 0 0 0 0 0 ... 0 0 ║ 0 0 0 0 0 0 ... ════╬═════════════════════════ ║ ┌─────┐ ┌─────┐ ┌─────┐ 0 0 ║ │ 4 5 │ │ 4 5 │ │ 4 5 │ ... 0 0 ║ │ 7 8 │ │ 7 8 │ │ 7 8 │ ... ║ └─────┘ └─────┘ └─────┘ ║ ┌─────┐ ┌─────┐ ┌─────┐ 0 0 ║ │ 4 5 │ │ 4 5 │ │ 4 5 │ ... 0 0 ║ │ 7 8 │ │ 7 8 │ │ 7 8 │ ... ║ └─────┘ └─────┘ └─────┘ ║ ┌─────┐ ┌─────┐ ┌─────┐ 0 0 ║ │ 4 5 │ │ 4 5 │ │ 4 5 │ ... 0 0 ║ │ 7 8 │ │ 7 8 │ │ 7 8 │ ... ║ └─────┘ └─────┘ └─────┘

to as follows for heightOffset = widthOffset = 1

0 0 ║ 0 0 0 0 0 ... ┌───╫───┐ ┌─────┐ ┌─────┐ 0 │ 4 ║ 5 │ │ 4 5 │ │ 4 5 │ ... ══╪═══╬═══╪═╪═════╪═╪═════╪═ 0 │ 7 ║ 8 │ │ 7 8 │ │ 7 8 │ ... └───╫───┘ └─────┘ └─────┘ ┌───╫───┐ ┌─────┐ ┌─────┐ 0 │ 4 ║ 5 │ │ 4 5 │ │ 4 5 │ ... 0 │ 7 ║ 8 │ │ 7 8 │ │ 7 8 │ ... └───╫───┘ └─────┘ └─────┘ ┌───╫───┐ ┌─────┐ ┌─────┐ 0 │ 4 ║ 5 │ │ 4 5 │ │ 4 5 │ ... 0 │ 7 ║ 8 │ │ 7 8 │ │ 7 8 │ ... └───╫───┘ └─────┘ └─────┘

In the first diagram the top left corner conv gets converted to: 0 ║ 0 0 4 ║ 5 4 ══╬════ ➤ ══╬════ 0 ║ 4 5 7 ║ 8 7 0 ║ 7 8 4 ║ 5 4

which means the 8 7 5 4 should be on the same core across the 4 tiles in that order

As can be deciphered for the above diagram, this would require the following operations

4_0, 4_1 -> Move West, Move North 5_0, 5_1 -> Move North 7_0, 7_1 -> Move West 8_0, 8_1 ->

and would also lead to a change in the of ops to 8_0 8_1 7_0 7_1 5_0 5_1 4_0 4_1. All this done with the help of applyCoreOffsetWE/NS calls for neighbor movements.

template <std::int32_t heightOffset = 0,
          std::int32_t widthOffset = 0,
          bool useVaping = false,
          typename InNDArrayType>
INLINE depthwiseConv3x3StrideOf2x2TileBlock(InNDArrayType& qData,
                                            std::uint32_t outputIteration)
NameTypeDefaultDescription
heightOffsetstd::int32_t0Padding related offset in height.
widthOffsetstd::int32_t0Padding related offset in width.
useVapingboolfalseUse VAPing (Virtual Array Partitioning) in the convolution.
InNDArrayTypetypenameType of the input array (`qData`).
NameTypeDescription
qDataInNDArrayType&[in] The input data fetched, the neighbor movement within the function. Must be at least 2-byte aligned: when numMacsPerCore > 8 the input reorder reads it in whole 2-byte units. Every current caller satisfies this with room to spare -- the numMacsPerCore <= 8 path already reads the same buffer through an int32 pointer, a stricter requirement that ships today.
outputIterationstd::uint32_tThe output channel index for the first channel to be processed.

INLINE — A vector of `core_array::numAccumSums` integers. / // clang-format on

Depthwise Dilated Convolution [Filter NxN, Dilation Factor DxD]

depthwiseDilatedConvInt8function
File: /src/neuralNetBlocks/conv.hppLines 2560–2624

Computes Depthwise dilated convolutions in int8 precision. This operation is DDR only.

template <std::int32_t dilationFactor,
          std::uint32_t filterSize = 3,
          FracRepType numFracBits = 31,
          typename DdrInOutShape,
          typename DdrWeightShape,
          typename OcmAllocatorType>
INLINE void depthwiseDilatedConvInt8(DdrInOutShape& ddrIn,
                                     DdrWeightShape& ddrWeights,
                                     DdrInOutShape& ddrOut,
                                     OcmAllocatorType& ocmMemAlloc,
                                     FixedPoint32<numFracBits> scale,
                                     std::int8_t xZeroPoint,
                                     std::int8_t wZeroPoint,
                                     std::int8_t yZeroPoint)
NameTypeDefaultDescription
dilationFactorstd::int32_tIn pytorch this is the spacing between kernel elements. In the quadric implementation this is the spacing between input tensor elements. Effectively both are same. Currently this operation only supports 1, 2, and 4 dilation factors.
filterSizestd::uint32_t3The filter size of the convolution. Only filterSize of 3 is supported.
numFracBitsFracRepType31number of fractional bits for quantization parameters
DdrInOutShapetypenameThe shape of the input/output DDR tensor
DdrWeightShapetypenameThe shape of the weight DDR tensor
OcmAllocatorTypetypenameType of OCM memory allocator
NameTypeDescription
ddrInDdrInOutShape&The DDR input tensor
ddrWeightsDdrWeightShape&The DDR weight tensor (contains weights for conv as well as bias)
ddrOutDdrInOutShape&The DDR output tensor
ocmMemAllocOcmAllocatorType&ocm allocator to be used locally to allocate ocm memory space
scaleFixedPoint32<numFracBits>Scale parameter for quantization
xZeroPointstd::int8_tZero point quantization parameter for input tensor (needs to be 0)
wZeroPointstd::int8_tZero point quantization parameter for weight tensor (needs to be 0)
yZeroPointstd::int8_tZero point quantization parameter for output tensor / // clang-format on

INLINE void

Groupwise Convolutions

Groupwise Convolution [Filter NxN, Stride MxM]

groupwiseConvTileBlockInt8function
File: /src/neuralNetBlocks/conv.hppLines 1665–1709

Computes Groupwise convolutions in int8 precision

template <typename T,
          std::uint32_t groupSize = 4,
          std::uint32_t filterSize = 3,
          std::uint32_t stride = 1,
          FracRepType shiftAmount = 0,
          bool useVaping = false,
          bool sameWeights = false,
          typename NDArrayType>
INLINE qVar_t<T> groupwiseConvTileBlockInt8(NDArrayType& qData,
                                            std::int32_t outputIteration = 0)
NameTypeDefaultDescription
TtypenameThe type of the input values.
groupSizestd::uint32_t4The number of channels in a group.
filterSizestd::uint32_t3The filter size of the convolution.
stridestd::uint32_t1The stride of the convolution.
shiftAmountFracRepType0The amount to shift the output of the convolution by.
useVapingboolfalseWhether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy.
sameWeightsboolfalseIf 'True' the weights are popped once and used through out the convolution.
NDArrayTypetypenameThe input array type of `qData`.
NameTypeDescription
qDataNDArrayType&The activations of the input.
outputIterationstd::int32_tIf function is looped for a very large convolution that must run in multiple iterations, `outputIteration` is used to calculate the offset in the output tensor's memory where partial convolutional outputs should be stored.

INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on

FP16 Convolutions

FP16 Convolution [Filter NxN, Stride 1x1 - Auto Buffer]

convTileBlockFp16function
File: /src/neuralNetBlocks/conv.hppLines 670–683

this is a wrapper of the other version of convTileBlockFp16, this one allocates its internal buffer when and should be preferred in handwritten code.

template <typename OutT,
          std::uint32_t numInCh,
          std::uint32_t filterSize = 1,
          bool useVaping = false,
          bool sameWeights = false,
          std::size_t inSize,
          bool drainBroadcastFifo = false,
          FracRepType numFracBits>
INLINE convTileBlockFp16(container::NDArray<qVar_t<FixedPoint32<numFracBits>>, inSize>& qData)
NameTypeDefaultDescription
OutTtypenameThe output data type, which must be a FixedPoint type.
numInChstd::uint32_tThe number of input channels for the convolution.
filterSizestd::uint32_t1The size of the convolution filter. Defaults to 1. Supported sizes are 1, 3, and 5.
useVapingboolfalseA boolean flag to enable vaping. Defaults to false.
sameWeightsboolfalseif True the weights are popped once and used through out the convolution.
inSizestd::size_tinferred, the size of input array
drainBroadcastFifoboolfalseWhen true, consumes leftover broadcast FIFO elements after the MAC operation. Defaults to false.
numFracBitsFracRepTypeinferred, the number of frac bits of input
NameTypeDescription
qDatacontainer::NDArray<qVar_t<FixedPoint32<numFracBits>>, inSize>&An NDArray of qVar_t<FixedPoint32<numFracBits>> representing the input data for the convolution.

INLINE — A qVar_t<OutT> containing the result of the convolution after shifting.

FP16 Groupwise Convolutions

FP16 Groupwise Convolution [Filter NxN, Stride MxM]

groupwiseConvTileBlockFp16function
File: /src/neuralNetBlocks/conv.hppLines 1823–1945

Computes Groupwise convolutions in fp16 precision

template <typename T,
          std::uint32_t groupSize = 4,
          std::uint32_t filterSize = 3,
          std::uint32_t stride = 1,
          FracRepType shiftAmount = 0,
          bool useVaping = false,
          bool sameWeights = false,
          typename NDArrayType>
INLINE qVar_t<T> groupwiseConvTileBlockFp16(NDArrayType& qData,
                                            std::int32_t outputIteration = 0)
NameTypeDefaultDescription
TtypenameThe type of the input values.
groupSizestd::uint32_t4The number of channels in a group.
filterSizestd::uint32_t3The filter size of the convolution.
stridestd::uint32_t1The stride of the convolution.
shiftAmountFracRepType0The amount to shift the output of the convolution by.
useVapingboolfalseWhether or not the convolution will implement a Virtual Array Partitioning (VAP) strategy.
sameWeightsboolfalseIf 'True' the weights are popped once and used through out the convolution.
NDArrayTypetypenameThe input array type of `qData`.
NameTypeDescription
qDataNDArrayType&The activations of the input.
outputIterationstd::int32_tIf function is looped for a very large convolution that must run in multiple iterations, `outputIteration` is used to calculate the offset in the output tensor's memory where partial convolutional outputs should be stored.

INLINE qVar_t<T> — qOutput A single output channel. */ // clang-format on

MAC Issue and Collect

Every convTileBlock* function launches the MEU and immediately reads the accumulators with accsum, which stalls until the MAC has finished. MacDescriptor splits the two so independent work can run in the MAC's shadow. Declare the descriptor with the MAC's shape, call start(qData) to launch, do the independent work, then collect(qData) to wait and read the result. Each fused function is exactly that: a descriptor followed by start and collect, so the fused and the split form are the same MAC.

MacDescriptor

MacDescriptorstruct
File: /src/neuralNetBlocks/conv.hppLines 169–578

Compile-time description of one MEU launch: the MAC unit, the window and how to read the result. Declare it, start<numInCh>(qData) or start(qData, k) launches, collect waits. Operands and FX32 scratch may be NDArrays, NDArrayViews, or LrmBufferHandles. Collect any pending flow transfer before passing a buffer, and keep the launch operand unchanged until MAC collection completes.

NameTypeDefaultDescription
typeMacTypeMAC unit.
OutTtypenameResult type: an integer for Int8, a FixedPoint32 for Fp16.
filterSizestd::uint32_t1Convolution window, one of {1, 3, 5}; 1 for matmul and fully connected layers.
shiftAmountstd::uint32_t0Right shift applied to the accumulator sum.
drainBroadcastFifoboolfalseFp16 1x1: pop the broadcast lines staged beyond the channel count after the MAC; a windowed launch consumes every line it stages and never drains.
clearAccumsbooltrueClear the accumulators before the launch; false chains onto an earlier launch of either MAC type.

start derives everything from the template parameters: which MEU path (MacType::Int8 or MacType::Fp16), the window, the centre and rotation counts from the channel count, and which operand layout it was given (an Fp16 array or view or an int16_t array launch directly; an FX32 array is converted by the two-argument start(qFx32, qScratch) into a caller-owned fp16 scratch, which is then the operand collect takes). A compile-time count can be passed as a template argument of the calls, start<K>(qData) and, for fp16, collect<K>(qData); a runtime count is a call argument, start(qData, k) and, for fp16, collect(qData, k). Int8 collects with collect(qData) for both static and runtime counts. A second int8 launch with clearAccums = false adds into the accumulators of an uncollected first launch, and one collect(qData) then returns the sum of both.

using namespace chimera::nn;
const MacDescriptor<MacType::Int8, std::int32_t> mac;   // int8, 1x1, no shift
mac.start<K>(qTile);                                    // MEU starts over K channels
nextTileFlow.issueRead(qNext);                          // lands while the MAC runs
qAcc += mac.collect(qTile);                             // wait only here

const MacDescriptor<MacType::Fp16, FixedPoint32<20>, 3> conv3;   // fp16 3x3 window
conv3.start<K>(qFp16Tile);            ...            qOut = conv3.collect<K>(qFp16Tile);

const MacDescriptor<MacType::Fp16, FixedPoint32<20>> fromFx32;   // FX32 operand, converted into qScratch
fromFx32.start<K>(qFx32Tile, qScratch); ...          qOut = fromFx32.collect<K>(qFx32Tile, qScratch);

mac.start(qTile, k);                  ...            qAcc += mac.collect(qTile);       // int8, K at runtime
fromFx32.start(qFp16Tile, k);         ...            qOut = fromFx32.collect(qFp16Tile, k);  // fp16, K at runtime

Sign in to your account

Don't have an account? 
By signing in, you are agreeing to our Terms of Use and Privacy Policy.
Quadric // One architecture. Every algorithm.

Develop.

Simulate.

Profile.

Collaborate.