SELU module

Usage

nn_selu(inplace = FALSE)

Arguments

inplace

(bool, optional): can optionally do the operation in-place. Default: FALSE

Details

$$\text{SELU}(x)=\text{scale}\ast (max(0,x)+min(0,\alpha \ast (\exp (x)-1)))$$

with $\alpha =1.6732632423543772848170429916717$ and $\text{scale}=1.0507009873554804934193349852946$.

More details can be found in the paper Self-Normalizing Neural Networks.

Shape

• Input: $(N,\ast )$ where * means, any number of additional dimensions

• Output: $(N,\ast )$, same shape as the input

Examples

if (torch_is_installed()) {
m <- nn_selu()
input <- torch_randn(2)
output <- m(input)
}