CTR is not the story. It's the first sentence.
An advertiser selling vintage pocket watches was pulling thousands of impressions from people searching for luxury cars. The same keyword had two meanings, and the click-through rate on those searches was close to zero. Every relevancy heuristic we had pointed the same way: cut these placements. I came across this case while optimizing search ranking for sponsored ads at Kleinanzeigen.
The data said something stranger. When we cut the low-CTR bucket, we lost roughly 10% of clicks and gained only around 1% in conversion rate. The clicks we removed had converted no worse than the clicks we kept. CTR had told us who was willing to click, and we had been reading it as who was worth reaching.
The proxy problem
Search ranking at scale needs signals that are fast and always available. Click-through rate is one of them: it is measurable in real time, it accumulates quickly, and in most cases it correlates with relevance. On Kleinanzeigen, a sponsored ad sits between ordinary listings, marked with a small label. Clicks on it arrive in real time, while purchases arrive later, if the advertiser reports them at all.
CTR measures the willingness to click on a specific ad in a specific context, and nothing more. It does not measure conversion. For high-intent searches the gap between the two is small: someone knows what they want, types it in, sees the ad and clicks it. The gap widens in discovery categories, for products people do not know they want until they see them. That gap matters more than a dashboard suggests.
One keyword, two meanings
The keyword described exactly what the collectibles advertiser sold. In an unrelated context, the same word referred to something automotive, so their ads appeared at high volume on queries from people looking for luxury cars. The result was high impressions with near-zero CTR. That is exactly the profile a relevancy cutoff analysis flags for removal.
The hypothesis going in was simple. Accidental clicks do not convert, so removing irrelevant placements should raise the conversion rate. That assumption is the reason to cut on CTR in the first place.
Why the cut didn’t work
We ran the analysis and cut the low-CTR, low-impression bucket. We lost roughly 10% of clicks and gained around 1% in conversion rate, which looked like a poor trade on its own. The reason behind it was more interesting than the trade. The conversion rate of the low-CTR clicks was not measurably worse than that of the high-CTR clicks, so the data did not support the hypothesis.
For a discovery advertiser, that changes what a bad placement means. Someone searching for luxury cars sees an ad for a vintage pocket watch they were not looking for. They stop, remember that they have been meaning to look at one of these, click and buy. That conversion happened because of the bad placement. Most people scrolled past, as expected, and the few who stopped were reachable precisely because they found something where they did not expect it.
The low CTR was the signature of this dynamic, so the metric flagged the very thing that made these placements work. Cut the placement, and you don’t cut waste. You cut the discovery surface.
Two kinds of advertiser
An intent-based advertiser, someone selling sofas to people searching “buy sofa”, has a simple relationship between CTR and conversion. The signal is clean, and optimizing on it works. A discovery advertiser has the opposite dynamic, because their best customers include people who were not looking for them. Their CTR was low everywhere except where intent already existed. Treating both the same and cutting on CTR removed exactly the inventory that discovery advertisers depended on.
Any marketplace with different kinds of advertisers carries this tension, and it does not resolve cleanly. It also stays invisible in aggregate metrics and only shows up when you trace individual campaigns back to the person searching on the other end.
The right signal was downstream conversion, and conversion data is hard. It is slow to accumulate and depends on advertiser cooperation. It is also noisy enough that acting on it too early creates more problems than it solves. So the bucketing ran on the best signal available, which was the wrong one for this problem.
The cost of the majority case
The cutoff analysis was optimized for the majority case, and for that case it was correct. Its cost lands on the advertisers the average was never describing. Over time, any ranking system built on engagement signals will underserve advertisers whose value to users is not obvious at the moment of search. That covers discovery categories, niche specialists and anything where purchase intent develops after exposure rather than before.
CTR stays useful as long as you know when it is the right proxy and when it is not. Telling the two apart takes conversion signal infrastructure that makes the distinction measurable instead of guesswork. Until that exists, the pocket-watch buyer on a luxury-car search looks like noise in a low-CTR bucket. Building it is a harder problem than the ranking algorithm, and the more important one.
Related: We could only manually review a fraction of campaigns each week: what happens when you build the infrastructure to stop relying on engagement signals as a proxy.