The Number of Parameters of GPT-4o and Claude 3.5 Sonnet

Recently, I saw a paper from Microsoft that surprisingly revealed the parameter counts for models such as GPT-4o and Claude 3.5 Sonnet.

Figure 1: First page of the paper. [Source].

From Figure 2, we can get a glimpse of the details.

Figure 2: The parameter counts for models such as GPT-4o and Claude 3.5 Sonnet. [Source].

According to Figure 2, we can see that GPT-4o contains approximately 200 billion parameters, while GPT-4o-mini has about 8 billion parameters. Claude 3.5 Sonnet operates with roughly 175 billion parameters.

ChatGPT uses approximately 175 billion parameters, while GPT-4 is significantly larger with about 1.76 trillion parameters.

For the newest models, o1-mini contains approximately 100 billion parameters, and o1-preview features about 300 billion parameters.

However, I remember that Microsoft previously published a paper claiming ChatGPT had only 20 billion parameters, but they withdrew the paper later.

Therefore, this time I'm somewhat skeptical, and waiting for more follow-up materials.