Did Open-Source YuE2 Beat Suno? Read the Footnote
YuE2 tops WildSongBench at 6.9632 against Suno v5’s 6.8721 — but that run selected from eight candidates against Suno’s two. At matched budget Suno still leads. The uncontested win is zero-shot covers, and the weights are non-commercial.
TL;DR — An open-weight music model, YuE2, now tops WildSongBench at 6.9632 against Suno v5's 6.8721. The headline is real and the footnote undoes half of it: that score is best-of-8, while Suno's is best-of-2. Matched at two candidates, YuE2 scores 6.7316 and Suno v5 still wins. The uncontested result is somewhere else entirely — zero-shot cover generation, where YuE2 beats the next open model by 1.5× and ACE-Step 1.5 by 27×. And the weights are non-commercial, which matters more than the benchmark for most people reading this.
The claim, and the footnote under it
YuE2 is from Multimodal Art Projection (M-A-P). The repository is Apache-2.0, has 7,508 stars, and the model card's own summary reads:
YuE2 (best-of-8) achieves the highest SongBench average among all evaluated open and proprietary models: 6.9632, compared with 6.8721 for Suno v5, 6.5562 for Suno v6, and 6.4195 for Suno v6 Wild.
That is accurate. It is also the number you should not use on its own, because two rows below it in the same table is this footnote:
192 prompts. Standard YuE2 selects from two candidates; best-of-8 selects from eight.
Suno's rows were produced by selecting from two candidates. So the like-for-like comparison is not the one in the headline:
| SongBench Avg | Candidates | |
|---|---|---|
| YuE2 (best-of-8) | 6.9632 | 8 |
| Mureka 9 | 6.9377 | not stated |
| Suno v5 | 6.8721 | 2 |
| YuE2 | 6.7316 | 2 |
| Suno v5.5 | 6.7150 | 2 |
| Suno v4.5 | 6.6995 | 2 |
| Suno v6 | 6.5562 | 2 |
| Suno v6 Wild | 6.4195 | 2 |
| LeVo 2 | 6.3247 | open |
| MiniMax Music 3 | 6.2830 | open |
| ACE-Step 1.5 | 6.0118 | open, MIT |
At two candidates each, Suno v5 leads by 0.14. The open model wins by spending four times the compute at inference and keeping the best result.
That is not cheating — best-of-N is a legitimate technique and the model card labels it plainly. But "open source beat Suno" and "open source beats Suno when allowed four times as many attempts" are different sentences, and only the second one is supported.
Two more things the table says that the headline does not:
A proprietary model is still ahead of YuE2's matched-budget score, and level with its best-of-8. Mureka 9 sits at 6.9377, within 0.03 of YuE2's best run. The story is not open-versus-closed; it is that the top of the field has bunched up.
Best-of-8 makes pronunciation worse. YuE2's phoneme error rate goes from 8.44% at two candidates to 9.79% at eight. Selecting for the overall score picks takes that sing better and enunciate worse.
The result that is not contested
Buried below the headline table is a comparison where YuE2 is not narrowly ahead — it is in a different category.
Zero-shot cover generation, on SHS100K: 948 works, two styles, two seeds, 3,792 songs per method, and — importantly — no candidate selection at all.
| Method | CLEWS mAP ↑ | Hit@1 ↑ | Musicality ↑ |
|---|---|---|---|
| YuE2 (full score) | 0.647 | 71.3% | 5.104 |
| YuE2 (without chords) | 0.598 | 67.3% | 5.490 |
| SongEcho | 0.419 | 48.4% | 3.286 |
| ACE-Step 1.5 | 0.024 | 2.4% | 3.689 |
| YuE2 (without score) | 0.006 | 0.3% | 5.691 |
Read the top and bottom rows together. With its symbolic score, YuE2 identifies as a cover of the original 0.647 of the time. Strip the score out and it collapses to 0.006 — while its musicality rises to the best figure in the table.
That is a clean ablation. Without the score it makes better-sounding music that is no longer the same song. The score is doing the work, and the mechanism is visible rather than asserted.
It is also why ACE-Step 1.5 scores 0.024 here while scoring respectably on full-song generation: it has no equivalent stage.
What is actually new: the score in the middle
The architectural idea is the reason to care, benchmark or not.
Most music generators are black boxes — prompt in, audio out, and if the third chorus is wrong your options are re-roll or accept it. YuE2 puts an editable symbolic score in ABC notation between planning and rendering. The model card's phrasing: you can "read, play, and change the composition before rendering it," with melody and chords as explicit controls.
That enables three things no closed product currently offers together:
- Symbolic planning — inspect and edit the melody and chords before any audio exists
- Zero-shot covers — transcribe a recording to a score, then re-render it in another style
- Agentic editing — hand the exported score to an agent, have it propose harmonic or melodic revisions, re-render
For anyone who has tried to fix one bar of a Suno track, that is the difference between a slot machine and an instrument.
The catch nobody puts in the headline
The weights are CC BY-NC 4.0. Non-commercial.
The repository code is Apache-2.0, but the model weights are not, and the output of a non-commercially-licensed model is not something to put in a product you sell. Compare:
| Weights licence | Commercial use | |
|---|---|---|
| YuE2 | CC BY-NC 4.0 | No |
| ACE-Step 1.5 | MIT | Yes |
| Suno | commercial terms | per subscription tier |
This matters more than 0.14 benchmark points for most readers. If you are scoring a game — and Korean developers are already building whole BGM pipelines around Suno — YuE2's licence is the reason it may not replace what you are doing, however it scores.
ACE-Step 1.5 is MIT, 12,673 stars, and sits at 6.0118 on the same table. That is meaningfully behind, and it is usable in a product.
Running it
| Model | YuE2-3B, plus YuE2-Vae / YuE2-Vae-legacy decoders |
| Requirements | Linux, Python 3.10+, 24 GB NVIDIA GPU with BF16, 24 GB host RAM |
| Speed | 3.6-minute song in 71 seconds on an RTX 4090 |
| Peak VRAM | 11.18 GiB typical; 14.08 GiB under maximum-context testing |
| Output | 48 kHz stereo, no quantisation needed |
| Languages | English and Mandarin demonstrated |
Seventy-one seconds for a three-and-a-half-minute song on a consumer card is the number that will decide adoption, not the benchmark. It brings iteration cost close to zero for anyone who already owns the hardware.
Note the benchmark runs used YuE2-Vae-legacy, while the default decoder is YuE2-Vae, which the card says "delivers better perceptual audio quality." The published numbers therefore come from a configuration chosen for reproducibility, not the one you get out of the box.
Reasons to distrust the whole table
Not as a hedge — these are specific.
Suno v6 scores below Suno v5. 6.5562 against 6.8721, with v6 Wild lower still at 6.4195. A commercial product's newer release scoring materially worse than its predecessor is a strong hint that the benchmark is not measuring what that release changed. Either v6 optimised for something SongBench does not reward, or the metric disagrees with the people who shipped it. Either way it should lower your confidence in small gaps elsewhere in the table.
The benchmark is published by the people who won it. WildSongBench is an M-A-P dataset and the comparison is on M-A-P's model card. The protocol is documented and the full CSV is linked, which is better practice than most, but it is not an independent evaluation.
Automatic metrics, not listeners. SongBench Avg is composed from automatic measures. No human preference study is reported in the table above.
Suno's numbers are of a moving target. Hosted products change without version bumps. A score for "Suno v6" is a score for whatever it served on 12 September 2026.
What I would actually conclude
An open 3B model is now genuinely in the conversation. A year ago that was not true. At matched sampling it is within 0.14 of the best Suno version on this benchmark and ahead of most of the field. That is the real news, and it survives every caveat above.
It is not "the open model won." At matched budget it did not, and a proprietary model — Mureka 9 — is level with its best run.
The score interface is the more durable advantage. Benchmarks close; architecture persists. Editable ABC notation in the middle of the pipeline is a capability, and the cover-generation ablation shows it is load-bearing rather than decorative.
The licence is the practical blocker. Non-commercial weights rule it out of most of the situations where people currently pay Suno. For commercial work the open option on this table is ACE-Step 1.5, nearly a full point behind.
What this article does not establish
I have not run YuE2 or Suno. Every number here comes from the YuE2-3B model card, its linked WildSongBench results, or the GitHub API, all read on 13 September 2026. I did not listen to samples from either model and this is not a quality judgement of my own.
The parameter count is stated inconsistently across sources — the repository calls it YuE2-3B, and one description gives 4B. I have used the repository's own naming.
The "27× ACE-Step" figure is a ratio of CLEWS mAP scores on the cover-generation benchmark (0.647 against 0.024), not a claim about general capability. ACE-Step 1.5 is not built to do that task.
The claim that no closed product offers an editable score interface is based on what those products document publicly; I have not audited every competitor.
FAQ
Did the open-source YuE2 really beat Suno?
On WildSongBench, YuE2's best-of-8 score (6.9632) is above Suno v5 (6.8721). But Suno's figures come from selecting between two candidates, and YuE2 at two candidates scores 6.7316 — below Suno v5. The win requires four times the sampling budget, which the benchmark's own footnote states.
What is YuE2 actually better at?
Zero-shot cover generation, decisively and with no candidate selection: 0.647 CLEWS mAP against 0.419 for the next open method and 0.024 for ACE-Step 1.5. Its own ablation shows that removing the symbolic score collapses this to 0.006, which is good evidence the mechanism is real.
Can I use YuE2 commercially?
No. The weights are CC BY-NC 4.0 — non-commercial — even though the repository code is Apache-2.0. For commercial use the open alternative on the same table is ACE-Step 1.5 under MIT, which scores 6.0118 against YuE2's 6.7316.
What hardware do I need?
Linux, Python 3.10+, and a 24 GB NVIDIA GPU with BF16 support, plus 24 GB of host RAM. On an RTX 4090 it generates a 3.6-minute song in about 71 seconds, peaking around 11 GiB of VRAM in normal use.
Why does Suno v6 score lower than Suno v5?
Nobody has explained it, and it is the best reason to treat the whole table cautiously. A newer commercial release scoring 0.32 below its predecessor suggests the benchmark is not capturing what that release changed.
What is the editable score, in practice?
YuE2 writes the composition as ABC notation before rendering audio. You can read it, play it, change the melody or chords, and then render — or hand it to an agent to revise and re-render. That is the part no closed music product currently offers.
← Back to all posts