Softplus — nnf_softplus • torch

Applies element-wise, the function $S o f t p l u s (x) = 1 / β * l o g (1 + e x p (β * x))$ .

Usage

nnf_softplus(input, beta = 1, threshold = 20)

Arguments

input: (N,*) tensor, where * means, any number of additional dimensions
beta: the beta value for the Softplus formulation. Default: 1
threshold: values above this revert to a linear function. Default: 20

Details

For numerical stability the implementation reverts to the linear function when $i n p u t * β > t h r e s h o l d$ .