Is annoy/be annoyed an action only preformed by A.I? It's been so long inspecting the game without any CC.

No. If you make your sim a mean sim, you can have the annoy social in the pie menu.

I thought the annoy action appeared regardless of nice points, because I remember being able to direct a nice sim to annoy someone. I think the relationship has to be neutral (0/0), or negative, but that's in my experience...

For me, the annoy option doesn't show regardless of low or high nice points and I'm playing the game without any CC. By default, what I start off with is the irritate option, which leads to throw drink or argue.
I always thought annoy was under the prank category. So my assumption was low nice and high playfulness but it doesn't make any difference.

'Annoy' is usually either under the 'Irritate,' or 'Mean/Fight' menu, forgot which one, but it's under one of them. So it may have to do with relationship level, as new options unlock the lower the relationship gets.

Annoy is under irritate, appears for mean sims only with relationship score 0 or under. I remember Olive Specter annoyed every stranger.