> ## Documentation Index
> Fetch the complete documentation index at: https://docs.biohub.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# ESM3

> ESM3 is a multimodal generative protein language model that jointly models sequence, structure, and function.

ESM3 is a multimodal generative protein language model that jointly models sequence, structure, and
function. It enables controllable generation of novel proteins by conditioning on any combination of
these modalities.

<Columns cols={2}>
  <Card title="Hugging Face" icon="https://mintlify.s3.us-west-1.amazonaws.com/biohub-8940d923/icons/hugging-face.svg" href="https://huggingface.co/collections/biohub/esm3-model-family" horizontal />

  <Card title="GitHub" icon="github" href="https://github.com/Biohub/esm/tree/main#esm-3-" horizontal />

  <Card title="Paper" icon="file" href="https://www.science.org/doi/10.1126/science.ads0018" horizontal />

  <Card title="API Reference" icon="code" href="https://docs.biohub.ai/api/protein/models/logits" horizontal />
</Columns>

<Note>
  **ESM3-open** is released under the
  [MIT License](https://github.com/Biohub/esm/blob/main/LICENSE.md). See the **Variants** tab for
  availability and access methods for all ESM3 variants.
</Note>

<h2 id="get-started">
  Get Started
</h2>

Get started using ESM3 with our Quickstart and Tutorials.

### Quickstart Guide

<Steps>
  <Step title="Install the `esm` Python package">
    ```python theme={null}
    pip install esm
    ```
  </Step>

  <Step title="Create an API key">
    [Generate an API key](https://biohub.ai/developer-console/api-keys) from your Biohub account.
    This API key manages your access to credits and tokens, and the term API key/token is often used
    interchangeably within documentation.
  </Step>

  <Step title="Connect to the Biohub Platform API">
    Call the ESM3 inference client with the selected model of choice and replace `<your API token>`
    with your token name.

    ```python theme={null}
    from esm.sdk.forge import ESM3ForgeInferenceClient

    client = ESM3ForgeInferenceClient(model="esm3-open-2024-03", url="https://biohub.ai", token="<your API token>")
    ```
  </Step>

  <Step title="Run your inference">
    Now you are ready to use your model. For examples of generating novel proteins with ESM3, check
    out our [Tutorials](#tutorials).
  </Step>
</Steps>

<h3 id="tutorials">
  Model Tutorials
</h3>

<Columns cols={2}>
  <Card title="Understanding the ESMProtein class" icon="https://mintlify.s3.us-west-1.amazonaws.com/biohub-8940d923/icons/colab.svg" href="https://colab.research.google.com/github/Biohub/esm/blob/main/cookbook/tutorials/esmprotein.ipynb">
    Get familiar with how ESM3 represents proteins.
  </Card>

  <Card title="Generating proteins with ESM3" icon="https://mintlify.s3.us-west-1.amazonaws.com/biohub-8940d923/icons/colab.svg" href="https://colab.research.google.com/github/Biohub/esm/blob/main/cookbook/tutorials/esm3_generate.ipynb">
    Learn how to scaffold a functional motif, edit secondary structure, and guide design using
    solvent exposure.
  </Card>

  <Card title="Designing a novel GFP with ESM3" icon="https://mintlify.s3.us-west-1.amazonaws.com/biohub-8940d923/icons/colab.svg" href="https://colab.research.google.com/github/Biohub/esm/blob/main/cookbook/tutorials/gfp_design.ipynb">
    Walk through the exact prompting strategy used to design a novel fluorescent protein with no
    close natural relatives.
  </Card>

  <Card title="Guided generation with ESM3" icon="https://mintlify.s3.us-west-1.amazonaws.com/biohub-8940d923/icons/colab.svg" href="https://colab.research.google.com/github/Biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb">
    Add scoring functions into the generation process, such as structural quality, sequence
    constraints, or other properties.
  </Card>
</Columns>

<h2 id="model-details">
  Model Details
</h2>

For additional information, see the Hugging Face link.

### Model Card

<Accordion title="Cite this Model">
  [Simulating 500 million years of evolution with a language model](https://www.science.org/doi/10.1126/science.ads0018)

  ```bibtex theme={null}
  @article{esm3_2025,
    title={Simulating 500 million years of evolution with a language model},
    author={Hayes, Thomas and Rao, Roshan and Akin, Halil and Sofroniew, Nicholas J. and Oktay, Deniz and Lin, Zeming and Verkuil, Robert and Tran, Vincent Q. and Deaton, Jonathan and Wiggert, Marius and Badkundri, Rohil and Shafkat, Irhum and Gong, Jun and Derry, Alexander and Molina, Raul S. and Thomas, Neil and Khan, Yousuf A. and Mishra, Chetan and Kim, Carolyn and Bartie, Liam J. and Nemeth, Matthew and Hsu, Patrick D. and Sercu, Tom and Candido, Salvatore and Rives, Alexander},
    journal={Science},
    volume={387},
    number={6736},
    pages={850--858},
    year={2025},
    doi={10.1126/science.ads0018}
  }
  ```
</Accordion>

<Tabs>
  <Tab title="Overview">
    | | |
    | - | - |
    | **Version** | 2024-03 |
    | **Architecture** | Transformer |
    | **Supported Modalities** | Sequence, structure, function |
    | **Training Data** | 3,000+ sequences and 700,000+ unique training tokens |

    #### Intended Use

    ESM3 is designed for prompt-driven generation of sequences and structures based on inputs of
    motifs, partial coordinates, secondary structure (SS) constraints, or function keywords.

    #### Limitations & Risks

    Novel sequence generation can lead to designs with hazardous properties. Model proposals may not
    be physically realizable; pLDDT/pTM are helpful but imperfect. Not intended for clinical or
    therapeutic applications without further validation.

    **ESM3-open** is released under the
    [MIT License](https://github.com/Biohub/esm/blob/main/LICENSE.md). See the **Variants** tab for
    availability and access methods for all ESM3 variants.
  </Tab>

  <Tab title="Details">
    | | |
    | - | - |
    | **Parameters** | 1.4, 7, and 98 billion parameters for ESM3 Small, Medium, and Large respectively. |
    | **Transformer layers** | 48, 96, 216 layers for ESM3 Small, Medium, and Large respectively. |
    | **Type of model architecture** | Pre-LN transformer architecture with rotary embeddings and SwiGLU non-linearities. The first transformer block includes an SE(3)-invariant Geometric Attention layer which conditions on backbone atomic coordinates and encodes geometric relationships. |
    | **Training FLOPs** | 10<sup>21</sup>, 10<sup>22</sup>, and 10<sup>24</sup> for ESM3 Small, Medium, and Large respectively. |
    | **Training datasets** | ESM3 was trained on protein sequences from PDB, UniRef, OAS, metagenomic sequences from MGnify and JGI, and synthetic data from AFDB, ESM Atlas (inverse folded), and ESM Atlas (inverse folded). Sequences, structures, and annotations were part of the training set. |
  </Tab>

  <Tab title="Variants">
    To obtain API access to ESM3 variants aside from ESM3-open, fill out
    [this form](https://info.biohub.org/biohub-additional-compute-credits) with your project details
    and desired model.

    **Flagship Models**

    Most users will be interested in using one of these models.

    | Model | Model Size | Max Context Length | Release Date | Availability |
    | - | - | - | - | - |
    | esm3-open-2024-03 | 1.4B | 2048 | 2024-03 | Biohub Platform API, GitHub, and Hugging Face |
    | esm3-large-2024-03 | 98B | 2048 | 2024-03 | Biohub Platform API |
    | esm3-medium-2024-08 | 7B | 2048 | 2024-08 | Biohub Platform API |
    | esm3-small-2024-08 | 1.4B | 2048 | 2024-08 | Biohub Platform API |

    **Published Models**

    These models were used to generate all of the results in the ESM3 paper and are provided to
    facilitate reproducibility.

    | Model | Model Size | Max Context Length | Release Date | Availability |
    | - | - | - | - | - |
    | esm3-large-2024-03 | 98B | 2048 | 2024-03 | Biohub Platform API |
    | esm3-medium-2024-03 | 7B | 2048 | 2024-03 | Biohub Platform API |
    | esm3-small-2024-03 | 1.4B | 2048 | 2024-03 | Biohub Platform API |
  </Tab>

  <Tab title="Usage">
    | | |
    | - | - |
    | **Primary Use Case** | Protein representation learning and embeddings, generating protein embeddings that can be fine-tuned for various downstream prediction tasks including functional annotation, mutational effect analysis, and the design of novel proteins and peptides, and predicting the functional impact of mutations and amino acid substitutions on protein function. |
    | **Supported Input Modalities** | Protein sequences |
    | **Access** | Access ESM3-open through [Biohub](#get-started) or [Hugging Face](https://huggingface.co/collections/biohub/esm3-model-family). To obtain API access to ESM3 variants aside from ESM3-open, fill out [this form](https://info.biohub.org/biohub-additional-compute-credits) with your project details and desired model. |
    | **License** | **ESM3-open** is released under the [MIT License](https://github.com/Biohub/esm/blob/main/LICENSE.md). See the **Variants** tab for availability and access methods for all ESM3 variants. |
    | **Not recommended for** | ESM3 can exhibit poor performance when modifying sequences on which it is highly confident. Not recommended for clinical diagnosis or treatment recommendations. Computational metrics do not replace wet-lab validation. Treat model outputs as machine-generated hypotheses that require further experimental validation, not as established biological facts. |
  </Tab>
</Tabs>