My thought is, that the algorithms (created by people) are built to mime the hundreds of “human decision-making factors” — things like “authority” and “trust”. Humans base decisions in part on a person’s or brand’s name — SE’s weight results on keyword inclusion or CTR (influenced by brand equity). SEO is more a craft than science — totally — but it (like advertising) is understood through social science and behavioral science — just sometimes through the lens of computer science.

I think the most important thing you said is that SEO is like looking at a big black box and trying to figure out what’s inside. Scientists have developed a lot of really great formal processes for doing exactly that, but the processes are often implemented incorrectly and assumptions are made — of course those attempts fall apart.

There’s a lot to be said for going with your gut and trail and error, but the people who have the most success at going with their instincts have developed those instincts with repetition and study.

Formal processes and informal techniques both have a place, and the people who use all the tools appropriately will always have an advantage over the people who throw away half the toolbox.