It's fun to theorize about more realistic and complex AI, but there's probably a reason many games don't include such things.
First of all, the game can become too difficult. Especially a game in which you are intended to fight dozens of enemies and come out alive. When each individual enemy becomes more competent and more clever, it becomes harder.
Related to that, the game can become frustrating. When AI is sneaky, the player will often be hit without knowing where from, or attacked from behind. Or an enemy marksman who picks a good hiding spot will easily get shots away before the player can locate him.
And finally, complicated AI can, to the player, be indistinguishable from random or even buggy behaviour. Such things as Arkage's example of minute changes in an enemy's "anger" level, possibly in conjunction with other dynamic attributes, will never be perceptible to the player. He will never be able to notice patterns, and will often become frustrated. The only remedies are 1) to dumb down the behaviours and make them obvious, for example to make the enemies say out loud "I'm really mad!"; or 2) explain it in the readme so the player has a starting point to understand what is random/buggy behaviour and what is the programmed AI behaviour (the latter being what he should pay attention to for learning patterns). It doesn't help that most game players are conditioned for bad AI, and will refuse to recognize advanced AI most times, passing it off as random or glitchy behaviour and complaining about it.
Oh, and of course, there's the "problem" that complex/emergent AI takes control away from the game/level designer, but that is not a negative thing to us, because we aren't of the Half-Life school of linearity, anyway!
However, all these "compromises" don't sit well with me, and probably not you either. I still want to make sophisticated AI systems based on reality, and I would like nothing more than to include in the readme only the disclaimer "this game has advanced AI, forget what you have learned from other games and treat the AIs as you would real people". I always hate the heavy-handed approach of games like FEAR, which has somewhat "advanced" AI, but the enemies always tend to announce all their decisions out loud: "Flank him!" or "I see a flashlight!". But still, if you design a game from the start with an advanced level of AI in mind, and keep the areas roomy enough and the enemies not too numerous, you should be able to train the player to always look over his shoulder for sneaking enemies, and to forget the old method of "clear a room once, it's safe forever". And if the game is still too stressful to play, add subtle AI handicaps like louder footsteps and slower decision making (and of course a well-designed difficulty slider), but nothing so heavy-handed as fear.
F. A. Špork, an enlightened nobleman and a great patron of art, had a stately Baroque spa complex built on the banks of the River Labe.