As with previous Civilization games, all civilizations have at least one unique unit. However, there are also some civilizations that have a second unique unit instead of a unique building or a unique improvement. Rome, for example, has two unique units (the Legion and the Ballista), whereas Russia has only one (the Cossack).
Unique units usually have bonuses related to terrain or combat strength that set them apart from the units they replace. Most unique units' bonuses, however, are not too significant - none of them are indestructible juggernauts, but they'll tip the odds in their owner's favor while they're available. The Conquistador, for example, is not much more powerful than the Knight it replaces (provided that the two have the same promotions, are on the same terrain, etc.), so Spain is not advised to build Conquistadors en masse and go to war with all its neighbors after unlocking them.
With the Gods & Kings expansion, allied Militaristic City-States will occasionally gift you some unique units along with standard ones. This makes it easier to gain control of other civilizations' unique units and can help you or just annoy and hinder you, depending on what they give. The Mohawk Warrior, for example, has bonuses in forest and jungle that benefit the Aztecs but are practically useless to Arabia.