YouTube has a new trick up its sleeve, and Marques ‘MKBHD’ Brownlee is not impressed. The platform is rolling out video A/B testing, letting creators upload different versions of the same video and let YouTube measure which one holds viewers longest. Brownlee calls it the craziest, riskiest idea the platform has tried in recent years.
The feature builds on existing tools that let creators test different thumbnails and titles. Now the test moves to the video itself. Creators can upload different edits — different openings, different cuts — and YouTube will measure which one holds viewers longest. Brownlee says creators can compare retention graphs across up to three versions before picking the final cut.
What the A/B Test Actually Does
The setup is simple on paper. A creator uploads a video with up to three variations, each with a different opening or edit. YouTube shows each version to a slice of the audience and tracks how long they stay. The creator sees the numbers, picks the winner, and that version becomes the published one.
Brownlee spoke with YouTube engineers about the feature and learned how the company plans to stop creators from testing completely different videos against each other. YouTube will use a semantic analysis model to check how similar the uploaded videos are. Only videos that score highly enough on similarity will be allowed into the test.
The company also plans to cap testing to the launch period of new uploads. Older videos cannot be pulled apart and tested against replacement cuts. That restriction is meant to keep the test from turning into a free-for-all where anything goes.
Why Brownlee Calls It Crazy
Brownlee’s main concern is the viewer. He wants to know whether anyone watching a video knows they are part of a test. He wants to know whether comments on a video reflect the version that person saw. He wants to know whether timestamps in comments still work if the video gets changed later.
His questions are direct:
- As a viewer, if I’m watching a video, do I know if that video is part of an A/B test or not?
- If I’m reading the comments on a video, how do I know which version of the upload they watched?
- If someone’s comment has a timestamp, does that break if I’m watching the other version of that video?
Brownlee also worries about the shared experience. There is an unspoken rule on YouTube that everyone is watching the same thing. A/B testing breaks that.
“There’s this sort of unspoken rule of community on YouTube that, like, we’re all having the same experience together,” Brownlee said. “We’re all watching the same video.”
“So letting people put up two or three different versions of the same video at the same time just kind of fundamentally breaks that.”
The Tool’s Limits
Brownlee argues that the tool is being sold as a way to make better videos, but it does not actually help creators build them. It helps them find which version of a video holds viewers longest, which is not the same as building a better one.
“This is being framed as a tool to help you improve your videos, but I think for 99.9% of creators, well, to be literal, it’s not actually helping you make better videos,” he said. “It’s helping you find which version of your video has the best retention, which doesn’t necessarily mean it’s a better video.”
The distinction matters. A video that holds viewers for longer might be edited badly or paced poorly. A video that loses viewers quickly might be brilliant. The tool rewards retention without rewarding quality.
The Rollout Timeline
YouTube is expected to roll out video A/B testing more broadly early next year. The similarity requirements are designed to stop creators from testing completely different uploads against one another.
Brownlee’s reservations are not about the technology itself. He is worried about how it changes the relationship between creator and viewer. A viewer who watches a video today might see a version that is gone tomorrow, replaced by the cut that held viewers longest. Comments on that video could reference a version that no longer exists.
The feature also raises practical questions. How do timestamps work across versions? What happens if someone watches one version of a video during testing and returns later to find a different cut has become the permanent one?
Other Announcements at the Event
YouTube’s Made on YouTube event also announced other additions, including custom homepages, new livestream features, and AI-powered Shorts editing tools. Brownlee acknowledged that some of YouTube’s recent changes have been useful.
But the A/B testing feature stands apart. Brownlee called it the craziest, riskiest, and possibly worst idea yet. He is not saying YouTube should stop it. He is saying the platform should answer his questions before it launches.
The Bigger Picture
YouTube’s A/B testing feature is a bold move. It gives creators a powerful tool to refine their videos. It also puts the audience in an uncertain position, watching a video that may not be what it seems.
Brownlee’s questions are fair. They deserve answers before the feature goes live. The platform has said it will cap testing to the launch period of new uploads and use a similarity model to stop creators from testing wildly different videos. Whether those safeguards are enough remains to be seen.
Key Facts
- Video A/B testing is coming to YouTube early next year
- Creators can compare retention graphs across up to three versions
- YouTube will use a semantic analysis model to check video similarity
- Testing is capped to the launch period of new uploads
- Older videos cannot be pulled apart and tested against replacement cuts
Brownlee is waiting to see how it lands.
See the video the story is built around at Dexerto.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.