When a creator tells me they want to try a few designers before committing, I think that is a completely sensible instinct, and I would do the same thing in their seat, but the way most people run that trial tells them almost nothing useful, because they ask three designers for a free sample on an imaginary video, pick the prettiest picture, and then discover two months later that the prettiest picture came from someone who misses deadlines, ignores the title, and cannot keep a channel looking consistent from one upload to the next.
So this is how I would run a paid test project with a thumbnail designer if I were hiring one today, from the side of someone who has been hired this way many times and who also hires designers for our own channels. The short version is that a fair test is small, real and paid, it judges how the person works as much as what they make, and it is honest about the questions no test can answer.
Why pay for a test instead of asking for a free sample?
Because a free sample tests how much effort someone is willing to give away, not how they work on a real deadline, and the designers you most want to hire are usually the ones too busy to do unpaid spec work.
There is also a quality problem built into free samples. A designer doing work for free on a made up video has every reason to make something flashy and generic, because they are trying to impress you in one image, not trying to make a specific video perform for a specific audience. So you end up comparing portfolios dressed up as tests, right, which is not what you set out to do. Paying, even a modest amount, changes the dynamic completely, right, the designer treats it as client work, asks the questions they would ask a client, and gives you the same process you would get if you hired them for a year, which is exactly what you are trying to see.
The other reason is fairness, and I care about this one more than people expect. Thumbnails are skilled work, and a test that asks a dozen people for free work so you can use the best one is not a test, it is a contest with no prize, and good designers can smell it from the first message.
How many videos should a test project cover?
One or two real upcoming videos is the right size, enough to see the whole process twice without turning the test into a commitment you are not ready to make.
One video tells you whether the designer can make a good image. Two videos tell you something much more important, whether the second one feels like it belongs to the same channel as the first, and whether the designer learned anything from your notes on the first one. That second data point is where most of the real information lives. If you can, pick two videos that are different in kind, for instance a talking head explainer and a more visual story video, because a designer who handles both well is far more useful to a real channel than one who is great at only one look.
What I would avoid is testing on an old video that already has a thumbnail, unless the test is specifically about repackaging, because the designer is then working against an existing image and you are comparing a new version to something your audience has already seen. Real, unreleased videos keep the test honest, and whatever comes back should be delivered at the size YouTube recommends, which is set out on the custom thumbnail help page.
What should the brief for a test look like?
Exactly like the brief you would send if you had already hired them, the real topic, the working title, the audience, the frames or photos you have, and the one thing that makes this video worth clicking.
A test is only as good as the brief, right, and a lot of creators send a thinner brief to a test designer than they would to someone they trust, which means the test ends up measuring how well the designer guesses. Send the real thing. We have a whole page on how to brief a thumbnail designer so the first draft lands, and a shorter post with the brief template I wish every creator sent, and either of those is a good structure for a test brief.
One more thing on the brief. Tell the designer it is a test and what you will be judging, because a good designer will appreciate the honesty and a bad one will reveal themselves by not asking any questions about it.
What should you actually judge during the test?
Judge the process first and the image second, how they handle the brief, whether they think about the title, how they respond to notes and whether the file arrives when they said it would.
Think back to the last person you hired for anything creative, and whether the thing that went wrong later was the work itself or the way they worked, because I would bet it was the second. The final thumbnail is the most visible thing a designer gives you, but it is also the least informative about the next fifty videos, because you are only seeing one moment of their best effort. The process tells you what working with them will feel like every week, and that is what you are really buying. So I would watch for a few specific things as the test runs.
That is one, the questions they ask when they get the brief. A designer who asks about your audience, your best performing videos and what the title is going to be is thinking about packaging, while one who just asks for your logo and brand colours is thinking about decoration. Secondly, whether they show you more than one direction before polishing, because rough concepts early mean they are willing to explore before committing. And the third thing is how they take notes, whether the second round actually reflects what you asked for or just changes something at random.
Then the image itself, right, and here the most useful thing you can do is look at it at the size your viewers will, on your phone, in your actual feed next to the other videos in your niche. Trust me on any level, a thumbnail that looks great full screen on a monitor and turns to mush at feed size is the most common way a test fools people.
What can a paid test not tell you?
A short test cannot tell you whether the designer will lift your click-through rate over time, because one or two videos are far too few to separate the thumbnail's effect from the topic, the timing and plain luck.
This is the honest limit, and I would rather say it plainly than pretend otherwise. If one test thumbnail does well and the other does badly, that is almost no information about the designer, because a single video's performance swings with so many things that have nothing to do with the packaging. We have written about this in detail, how most thumbnail tests end long before they mean anything, and how long a thumbnail test needs to run before you trust it, and the same logic applies even more strongly to a hiring test, where you have two data points and a lot of hope.
Even YouTube's own title and thumbnail test picks its winner by watch time across a whole audience, right, which tells you how much data a fair comparison needs, and a hiring test has nowhere near that. A test also cannot show you consistency over months, whether the designer keeps the channel's look coherent across thirty videos, whether they build on what worked, and whether they stay reliable when they get busy. You only learn those things by working together for a while, which is why I would treat the test as a filter, a way to rule people out, rather than a way to prove someone is the one.
Hire on the process you watched, and let the numbers come later, because two videos have never proven anything about anybody.
How do you compare two designers fairly?
Give both of them the same two videos and the same brief, then look at the results side by side at feed size, ideally without knowing whose is whose until you have picked.
The blind part sounds like overkill, but it helps, because by the time you are comparing finals you have already formed opinions about who was nicer to work with, and those opinions leak into how you see the images. Mix them into a mock feed with real competitor videos, scroll past at normal speed and notice which one actually stops you. Then reveal and weigh that against everything you noted about the process. If one designer made the slightly better image but missed the deadline and ignored half your notes, I would choose the other one almost every time. The longer list of warning signs to look for while you do this is in red flags I watch for when hiring a thumbnail designer, and any test that turns up two or three of them has already given you its answer.
How would a test with ClickTheory work?
The simplest fair test with us is a pack itself, 250 dollars for the next eight videos, because it gives you far more than two data points and there is no retainer to escape if it does not suit you.
I know a pack is bigger than a two video test, and I would rather say so at the start. The reason I think it works as a test is that eight videos show you everything a short trial shows, and a fair bit of what a short trial cannot, whether the look holds together across a run of uploads and whether the second batch of thumbnails is sharper than the first because of what we learned. Each video gets a strategist thinking about the title and the promise and a designer building the image, you can send notes as many times as you need, and the work typically comes back in two or three days, weekends excepted. If you want to see the kind of work we do before you commit to anything, the portfolio shows it without client names, and the services page has the add-ons and the rest of the details.
If all you need is one thumbnail for one video, honestly, a freelancer marketplace may be a better fit for that, and there is nothing wrong with starting there. And I would be wary of anyone who offers a guaranteed click-through rate as part of a test, because nobody can honestly promise that, and I explain why in what a thumbnail CTR guarantee is actually worth.
Is a test project ever the wrong idea?
It can be, when you already know what you want and have seen enough of a designer's real work to trust them, because then a test just delays a decision you have effectively already made.
Tests are most useful when you are choosing between unknowns. If someone has been recommended by a creator you trust, has a portfolio full of work in your niche and has run channels of their own, a test can still be reassuring, but the main things it would tell you you already know, and every week you spend testing is a week of uploads going out with packaging you are not happy with. In that situation I would rather start the real work and agree to review it honestly after a month.
The point of all of this is really simple, a test exists to show you how someone works, right, not to give you a magic number, and when you run it with real videos, a real brief and a fair fee, it does that job well. So yeah. That's my way of saying it.