You can give Ward a lot of labels but "dirty" should not be among them. He was aggressive, violent, and, yes, often annoying to linebackers and defensive backs alike.
It's true that Ward was named the NFL's dirtiest player in a 2009 Sports Illustrated poll. So, it's not just Steeler-hating fans who are calling Ward out. It's his peers.
Still, this doesn't mean that Ward is dirty. When you play Ward's style, you're not going to gain many friends around the league. Players don't like it when they're running along minding their own business and find themselves face first in the ground a second later. And they really don't like it when it's a 6-foot wide receiver who put them there.
There is also a poll with the article, "Was Hines Ward a Dirty Player?" Go vote, "NO!
The poll is on this page