Orthogonal initialization — nn_init_orthogonal

Fills the input Tensor with a (semi) orthogonal matrix, as described in Exact solutions to the nonlinear dynamics of learning in deep linear neural networks - Saxe, A. et al. (2013). The input tensor must have at least 2 dimensions, and for tensors with more than 2 dimensions the trailing dimensions are flattened.

Usage

nn_init_orthogonal_(tensor, gain = 1)

Arguments

tensor: an n-dimensional Tensor
gain: optional scaling factor

Examples

if (torch_is_installed()) {
w <- torch_empty(3, 5)
nn_init_orthogonal_(w)
}
#> torch_tensor
#> -0.4669  0.6767  0.5106  0.1811  0.1747
#>  0.8529  0.4220  0.2186 -0.1472  0.1583
#> -0.2042  0.1474  0.0232 -0.9457 -0.2041
#> [ CPUFloatType{3,5} ]