SoftmaxKernels

Collaboration diagram for SoftmaxKernels:

Classes
class	SoftmaxScalar< INP_H, INP_W, INP_W_PAD >
	Scalar implementation, SoftmaxScalar<10,10,16> takes. More...
class	SoftmaxSingleaxis< INP_H, INP_W, INP_W_PAD >
	Vector implementation for single axis, SoftmaxSingleaxis<10,10,16> takes 1241 cycles requires INP_W_PAD%8=0. More...
class	SoftmaxMultiaxis< INP_H, INP_W, INP_W_PAD >
	Vector implementation for multiple axis, SoftmaxMultiaxis<10,10,16> takes 1082 cycles requires INP_H%2==0 && INP_W_PAD%8=0. More...

Functions
double	fastexp (float val)
	See https://nic.schraudolph.org/pubs/Schraudolph99.pdf.
float	fastexp2 (float val, int precision)
	approximation with (1 + x/256)^256 Error is small for [-512. 1.5]

Detailed Description

• Softmax(input, axis) = Exp(input) / ReduceSum(Exp(input), axis=axis, keepdims=1)

Function Documentation

fastexp()

double fastexp

(

float

val

)

out = in * y + (b - c) where

• y = 2**20/math.log(2)
• b = 1023 * 2**20
• c = 2**20 * math.log(3 / (8*math.log(2)) + 0.5) / math.log(2)