RMSStandardize

RMSStandardize#

class penzai.deprecated.v1.nn.standardization.RMSStandardize[source]#

Bases: Layer

Root-mean-squared standardization layer.

As proposed by Zhang & Sennrich (2019): https://arxiv.org/abs/1910.07467. This layer does not include the learnable parameter.

Variables:

  • across (str | tuple[str, ...]) – Axis names to standardize across.

  • epsilon (float | jax.Array) – Small constant to prevent division by zero.

Methods

__init__(across[, epsilon])

input_structure()

output_structure()

__call__(value)

Root-mean-square standardizes a named array across the given axes.

Attributes

epsilon

across

Inherited Methods

(expand to view inherited methods)

attributes_dict()

Constructs a dictionary with all of the fields in the class.

from_attributes(**field_values)

Directly instantiates a struct given all of its fields.

key_for_field(field_name)

Generates a JAX PyTree key for a given field name.

select()

Wraps this struct in a selection, enabling functional-style mutations.

tree_flatten()

Flattens this tree node.

tree_flatten_with_keys()

Flattens this tree node with keys.

tree_unflatten(aux_data, children)

Unflattens this tree node.

treescope_color()

Computes a CSS color to display for this object in treescope.
__call__(value: NamedArray) → NamedArray[source]#

Root-mean-square standardizes a named array across the given axes.

Contents