llama_from_huggingface_model

llama_from_huggingface_model#

penzai.experimental.v2.models.transformer.variants.llama.llama_from_huggingface_model(model: LlamaForCausalLM, upcast_activations_to_float32: bool = False, use_layer_stack: bool = False) → model_parts.TransformerLM[source]#

Converts a HuggingFace Llama model to a Penzai model.

This function converts Llama models from their HuggingFace implementations to Penzai. (Other models with the same architecture may also be supported if they use the same configuration, but this has not been tested.)

Parameters:

  • model – The HuggingFace Llama model.

  • upcast_activations_to_float32 – Whether to cast activations to float32 when the model runs. This allows analyzing activations at higher precision without consuming additional memory for parameters.

  • use_layer_stack – Whether to use a layer stack for the decoder blocks.

Returns:

A Transformer model containing the loaded parameters.

Contents