tensorplay.nn.functional.gelu

tensorplay.nn.functional.gelu(input, approximate='none') → Tensor[source]

When approximate is ‘none’, applies $\text{GELU}(x) = x * \Phi(x)$; ‘tanh’ uses the tanh estimation.