Softplus module — nn_softplus • torch

Applies the element-wise function: $Softplus (x) = \frac{1}{β} * \log (1 + \exp (β * x))$

Usage

nn_softplus(beta = 1, threshold = 20)

Arguments

beta: the $β$ value for the Softplus formulation. Default: 1
threshold: values above this revert to a linear function. Default: 20

Details

SoftPlus is a smooth approximation to the ReLU function and can be used to constrain the output of a machine to always be positive. For numerical stability the implementation reverts to the linear function when $i n p u t \times β > t h r e s h o l d$ .

Shape

Input: $(N, *)$ where * means, any number of additional dimensions
Output: $(N, *)$ , same shape as the input

Examples

if (torch_is_installed()) {
m <- nn_softplus()
input <- torch_randn(2)
output <- m(input)
}