impressions on one of my sites dropped and i did what everyone does. i assumed something external had happened. an update, a penalty, a competitor.
it was two things, and i caused one of them.
the part that was not my fault
new sites get a short burst of visibility and then lose it. you launch, you get impressions for a couple of weeks on terms you have no business ranking for, and then the search engine finishes its first real assessment and puts you where you actually belong.
that dip is not a penalty. it is the honeymoon ending. if you launched recently and your graph goes up then falls off, check the calendar before you change anything, because the worst outcome here is panic-editing a site that is behaving exactly as expected.
i know this. i still spent a day looking for a cause.
the part that was my fault
there was a real problem underneath it, and it was in my robots.txt.
i had a rules setup at the cdn layer and a robots file on the site, and between them they were blocking crawlers i actually wanted. specifically the ai crawlers. the ones that feed the assistants that a growing number of people now ask for recommendations instead of typing a query into a search box.
i had blocked them without deciding to block them. it came from a default configuration and a copied file, and i never read the result carefully because robots.txt is four lines long and looks like something that cannot be wrong.
so for the first weeks of a product’s life, the systems most likely to surface a brand new tool to somebody asking “what should i use for x” could not read it.
the other self-inflicted one
on a different site i found the reverse problem. too much indexed, not too little.
demo profiles, created so an empty product would not look empty, were indexed and carrying self-referencing canonical tags. so they were competing with real user pages. and my comparison pages were competing with my own landing page for the same terms.
that is cannibalisation and it is entirely a self harm. nobody outside did it to me. i built pages faster than i thought about what each one was supposed to rank for, and ended up with several of my own urls arguing over the same query.
what i changed
read the robots file out loud. every line, against the list of crawlers you actually want. decide about ai crawlers deliberately, in either direction. blocking them can be a legitimate choice. blocking them by accident is not.
check the cdn layer too. the file on your site is not the only thing that answers for it. i had rules in two places and only ever looked at one.
one page, one query. before creating any comparison or alternatives page, write down the exact phrase it is supposed to win. if that phrase is already assigned to another page, you are not adding a page, you are splitting one.
noindex your demo content. or better, delete it once you have real users. seeded personas are scaffolding, and scaffolding left on a finished building looks like an unfinished building.
go long tail, and specific. i stopped writing pages aimed at head terms i cannot win against companies with actual marketing budgets. the pages that work for me are narrow, specific, and aimed at a person with a particular problem. fewer visitors, all of them relevant.
why i am writing this down
because both of my launches had the same shape of mistake. not a strategy failure. a configuration i never read after copying it, and a set of pages i created faster than i thought about them.
i audit exactly this for other people. i know the checklist. i did not run it on myself until the graph made me.
that is the pattern i keep finding in my own work, and the only fix i have found is to schedule the audit like a client engagement, on somebody else’s checklist, treating my own site as if a stranger built it and cannot be trusted.