YouTube
How to A/B Test YouTube Thumbnails with Test & Compare
How YouTube’s Test & compare works: who can use it, testing up to three thumbnails or titles, reading the results, and designing variants worth testing.
By The Mocknex team · Published · 7 min read
YouTube has a built-in A/B test called Test & compare. You give a long-form video up to three thumbnails, three titles, or three title-and-thumbnail pairs; YouTube shows them to different viewers for up to about two weeks and then tells you which one earned the most watch time. This guide covers who can use it, how to set it up, how to read a result that says “Inconclusive”, and how to design variants that teach you something instead of three near-identical images.
What YouTube’s Test & compare does
Each viewer who sees the video in a feed is shown one of your variants. YouTube keeps track of how much those viewers go on to watch, and when the test ends it picks the variant with the highest watch time share — not the highest click-through rate. That distinction matters: a thumbnail that gets more clicks but makes people leave in ten seconds can lose to one that gets fewer, better-matched clicks. YouTube does it this way on purpose, because a clicky thumbnail that misleads isn’t a win.
When the test finishes, the winning variant becomes the video’s thumbnail (and title, if you tested titles) for everyone.
Who can use it, and on which videos
| Requirement | Detail |
|---|---|
| Where | YouTube Studio on a computer — not the mobile app |
| Channel | Advanced features turned on (YouTube verifies your channel with a phone number, ID or channel history) |
| What you can test | Up to 3 thumbnails, 3 titles, or 3 title + thumbnail pairs |
| How long | Usually a few days, up to about 2 weeks; low-traffic videos may never get a clear result |
| Not available for | Shorts, Premieres, scheduled live streams, made-for-kids videos, mature-audience videos and private videos |
If you don’t see the option in the thumbnail section of a video’s details, check those two things first: you’re on desktop Studio, and advanced features are enabled under Settings → Channel → Feature eligibility.
How to set up a test
- Open YouTube Studio on a computer, go to Content and open the video’s details (or start a new upload).
- In the Thumbnail section, choose Test & compare.
- Choose whether to test thumbnails, titles, or both.
- Upload two or three thumbnails (they must meet the normal rules — 16:9, JPG or PNG, under the size limit).
- Confirm and save. The test starts when the video is public.
Every variant has to pass YouTube’s normal checks, so it’s worth running files through the thumbnail size checker first; it catches wrong ratios and oversize PNGs before Studio does. The full list of requirements is in our YouTube thumbnail size guide.
Reading the results
Results appear in the video’s Details page and under Analytics → Reach once the test ends. You’ll see each variant’s share of watch time and one of three outcomes:
| Result | What it means | What to do |
|---|---|---|
| Winner | One variant clearly earned more watch time | Keep it, and note what was different about it for your next video |
| Performed the same | Variants did roughly equally well | The difference you tested doesn’t matter much to your viewers — test something bigger next time |
| Inconclusive | Not enough data to be confident | Usually too few impressions; YouTube applies the first variant. Try again on a video with more traffic |
Designing variants worth testing
The most common reason a test teaches nothing is that the variants are too similar — the same photo with a slightly different shade of yellow. Viewers decide in a fraction of a second, so small changes rarely move watch time enough to measure. Change one big thing:
- The subject — your face reacting vs. the object or result the video is about.
- The promise — different text: a number (“$40 vs $400”), a question, or no text at all.
- The composition — a close crop vs. a wide scene; a before/after split vs. a single image.
- The contrast — a dark, moody version vs. a bright, saturated one.
Keep everything else the same so you know what caused the result. And make sure every variant is honest about the video — the test measures watch time, so a thumbnail that over-promises tends to lose anyway.
Narrowing drafts down before you publish
Test & compare only takes three variants and needs real traffic, so it pays to pick strong finalists first. Before publishing, look at your drafts the way viewers will meet them — small, surrounded by other videos, for under a second:
- ✓Put each draft in a realistic home feed and sidebar with the thumbnail preview tool, next to the videos you compete with.
- ✓Compare drafts side by side, then flash each for 0.6 seconds with the blink test in the thumbnail A/B tester. Note what you remember.
- ✓Check each in grayscale and blurred. If it turns to mush, it will be scrolled past.
- ✓Read the title and thumbnail together: the thumbnail should add something the title doesn’t say.
This kind of pre-publish comparison can’t predict click-through rate — only real impressions can. What it does well is throw out the weak drafts so your real test is between good options.
Mistakes that waste a test
- Testing on a video with little traffic. A few hundred views won’t produce a clear result. Use tests on videos you expect to be seen widely.
- Changing the variants mid-test. It restarts the comparison. Decide first, then leave it alone.
- Judging by the first day. Early viewers are mostly subscribers, who behave differently from people discovering you in browse or search.
- Testing titles and thumbnails together when you want to learn about one. Pairs are useful when the title and thumbnail work as a set; read how titles and thumbnails work together before choosing.
FAQ
Can I A/B test a Shorts thumbnail?
No. Test & compare isn’t available for Shorts, and neither are Premieres, scheduled live streams, made-for-kids or mature-audience videos.
Why does YouTube pick the winner by watch time instead of CTR?
Because a thumbnail can raise clicks while lowering how long people watch — for example, if it misrepresents the video. Watch time rewards the thumbnail that attracts viewers who actually want the video.
Can I run a test from my phone?
No, it’s only in YouTube Studio on a computer. You can view the video’s analytics on your phone afterwards.
Does a test hurt the video’s performance?
Viewers see one of your own variants either way, so a test doesn’t penalise the video. The cost is that some viewers see your weaker option while the test runs — another reason to pick strong finalists first.
Can I test an old video?
Yes, if it’s eligible. Older videos with steady search or suggested traffic are often good candidates, because traffic is predictable.