RMSStandardize

class penzai.nn.standardization.RMSStandardize

Bases: Layer

Root-mean-squared standardization layer.

As proposed by Zhang & Sennrich (2019): https://arxiv.org/abs/1910.07467. This layer does not include the learnable parameter.

Variables:

• across (str | tuple[str, ...]) – Axis names to standardize across.

• epsilon (float | jax.Array) – Small constant to prevent division by zero.

Methods

__init__(across[, epsilon])
__call__(value, **_unused_side_inputs)	Root-mean-square standardizes a named array across the given axes.

Attributes

epsilon
across

Inherited Methods

(expand to view inherited methods)

attributes_dict()	Constructs a dictionary with all of the fields in the class.
bind_variables(variables[, allow_unused])	Convenience function to bind variables to a layer.
from_attributes(**field_values)	Directly instantiates a struct given all of its fields.
key_for_field(field_name)	Generates a JAX PyTree key for a given field name.
select()	Wraps this struct in a selection, enabling functional-style mutations.
stateless_call(variable_values, argument, /, ...)	Calls a layer with temporary variables, without modifying its state.
tree_flatten()	Flattens this tree node.
tree_flatten_with_keys()	Flattens this tree node with keys.
tree_unflatten(aux_data, children)	Unflattens this tree node.
treescope_color()	Computes a CSS color to display for this object in treescope.

__call__(value: NamedArray, **_unused_side_inputs) → NamedArray

Root-mean-square standardizes a named array across the given axes.