Skip to contents

Kaiming uniform initialization

Source: R/nn-init.R
nn_init_kaiming_uniform_.Rd

Fills the input Tensor with values according to the method described in Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification - He, K. et al. (2015), using a uniform distribution.

Usage

nn_init_kaiming_uniform_(
  tensor,
  a = 0,
  mode = "fan_in",
  nonlinearity = "leaky_relu"
)

Arguments

tensor

an n-dimensional torch.Tensor

a

the negative slope of the rectifier used after this layer (only used with 'leaky_relu')

mode

either 'fan_in' (default) or 'fan_out'. Choosing 'fan_in' preserves the magnitude of the variance of the weights in the forward pass. Choosing 'fan_out' preserves the magnitudes in the backwards pass.

nonlinearity

the non-linear function. recommended to use only with 'relu' or 'leaky_relu' (default).

Examples

if (torch_is_installed()) {
w <- torch_empty(3, 5)
nn_init_kaiming_uniform_(w, mode = "fan_in", nonlinearity = "leaky_relu")
}
#> torch_tensor
#>  0.0135  0.4357 -1.0775 -0.5297 -0.5172
#> -0.9761 -0.6444  0.4427  0.8794 -0.0924
#>  0.1444 -0.3599  1.0510  1.0316 -0.3599
#> [ CPUFloatType{3,5} ]