If you have questions about whether or not the Over-Optimization Penalty (OOP) exists or not, then you obviously haven't pushed the limits enough with your own website optimization or organic SEO efforts to have experienced the joys of running into the OOP (Over-Optimization Penalty).

I'm sure you're probably thinking: "prove it". If that's what you're thinking, let me tell you about a specific example of what I call the Over-Optimization Penalty, or OOP:

Over-Optimization penalty because of too much anchor text.
You decide that your site is not ranking well. So, you change the navigation on your site so that all the home page links say "keyword keyword home" instead of "home". A few days later you lose your rankings, which is a serious blow to your traffic. Since that was the only thing you changed on your site you change those links to your home page back so it says, "home" instead of "keyword keyword home". BINGO! All of your lost rankings and your lost traffic comes back a few days later.

If that's not proof of an over-optimization penalty then I don't know what is--I would call that an OOP: you do something to try to boost your rankings by over optimizing some element of your website, something that backfires and causes a loss of rankings.

I personally can name tons of other examples just like this that prove that there's an OOP.

One more thing--I think that you can over-optimize a web page or site without going as far as actually spamming. In the example above, I truly believe that having too much of the same anchor text isn't spamming, it's just over-optimization. Spamming, in my book, would be something like keyword stuffing or something like repeating a keyword over and over again. Over optimization is different than spamming.

White Hat, Gray Hat, and Black Hat
Whether or not you think the OOP (an over-optimization penalty) exists depends on what type of "hat" you wear.

If you're a true "white hat" SEO that won't dare test the limits then you're going to believe that the OOP doesn't exist. After all, you'll never experience any over-optimization penalties because you won't dare do anything that would possibly have a negative effect on search engine rankings.

If you're a true "grey hat" SEO then you've definitely experienced the OOP, just like I've described in a previous post. For example, you've gone a little too heavy on your anchor text so you have to "back off" on using your keywords in the anchor text. Or, you realize that your keyword density on a page is too high--OOPS! you've just experienced the over-optimization penalty (OOP).

If you're a true "black hat" SEO then you definitely know that there's a difference between an over-optimization penalty and being a spammer. If you've never had a site banned in a search engine then you're not a true "black hat" SEO. And if you've experienced rankings that have suddenly vanished because of too much over-optimization then yes, you probably believe in the OOP, just like I do.

There is a very big difference between over-optimization and spamming. Spamming will get your site banned. Over-optimization will not. The "penalty", if you will, is a loss of search engine rankings due to over-optimization. Thus, that's what gray hat and black hat Search Engine Optimization Specialists call the OOP.