GLU module — nn_glu • torch

Applies the gated linear unit function $G L U (a, b) = a \otimes σ (b)$ where $a$ is the first half of the input matrices and $b$ is the second half.

Usage

nn_glu(dim = -1)

Arguments

dim: (int): the dimension on which to split the input. Default: -1

Shape

Input: $(*_{1}, N, *_{2})$ where * means, any number of additional dimensions
Output: $(*_{1}, M, *_{2})$ where $M = N / 2$

Examples

if (torch_is_installed()) {
m <- nn_glu()
input <- torch_randn(4, 2)
output <- m(input)
}