HuaRenCa
Back to Forum
Community

Can an AI specialist explain why or what made mythos class models special?

meisan
meisan

2 months ago

My guess is that they changed the tokenizer in one way or another. But i would like some perspective from fellow ai enthusiasts.

4
13

Comments (4)

Your avatar
Sign in to comment
zhezhe
zhezhe2 months ago

10 Trillion parameters by itself doesn't hurt. It's also been trained to be relentless in task execution, and has a great balance of the personality and writing chops of Opus 4.6 with the vision, UI, and agentic engineering nous of Opus 4.8.

It's a great, all-round intelligence, which can handle almost all code and design tasks without breaking a sweat.

rjhuantan
rjhuantan2 months ago

To add a bit of context, Opus was 2 trillion parameters so Mythos was ~5x bigger. But we don't know exactly how those parameters are used. There's some kind of internal mixture of experts approach so it might have some optimizations where it doesn't use all the parameters all the time.

sanqi
sanqi2 months ago

Mythos/Fable are about 10x larger than Opus, they're bigger models. Mythos is likely ~10T, Opus ~1T, Sonnet ~100B, as a very rough ballpark.

Emma Tcherkezian
Emma Tcherkezian2 months ago

Short version: 1) many small and secret technical improvements to data, training, and model architecture, and RL. Not one breakthrough. 2) its just a very BIG model

Long Version Amodei said in an interview that most capability gains come from the combination of many small improvements to every piece of the puzzle. Better kv-cache lookup, a better attention mechanism, higher quality data, better RLHF methods, and it all adds up ,there's usually not one giant breakthrough.

The large size is probably where most of the Mythos 'wow' factor comes from. Karpathy said on twitter that he gets the 'big model' feeling from talking to Fable. This is a known (if not a scientific) phenomenon: Despite smaller models benchmarking closer to larger models, there's something different about large models in their ability to 'just get' things, the ability to do good work from increasingly vague and bad prompts, that's very hard to measure. Fable is also really expensive and generates tokens slowly, which ALSO points to 'its a large model'.