If you think the Dunmers have it bad, the Argonians have it worse. They are not permitted into the city and are restricted to the docks. Although Windhelm does not welcome the Argonians, the Argonians’ aquatic skills are invaluable to the smooth running of the docks. Which is why they are allowed only at the docks, for that is where the Nords need the Argonians.
But indifference breeds indifference. Just as Windhelm does not care for the Argonians, the Argonians do not care for Windhelm either.