Aliens spawn from Alien Nests and will zealously defend them. Disturbing an Alien Nest is the easiest way to antagonize them. If you come within 2 hexes they almost always attack.
To clear an Alien Nest in vanilla Beyond Earth, simply move a unit onto the tile containing the nest. If an alien unit is present on said tile, you will need to defeat the unit or wait until it moves. Nests yield a bounty of Energy and, with the Scavenging Virtue, Science. In the Rising Tide expansion, players must manually select the "Destroy Alien Nest" option in order to destroy one. Destroying an Alien Nest will anger nearby Aliens.
In vanilla Beyond Earth, cleared Alien Nests leave a Xenomass, Firaxite, and Floatstone that may later be improved on and used. Unoccupied Xenomass, Firaxite, and Floatstone in unobserved tiles can spawn a new nest at any time, on land or at sea, regardless of miasma. Clearing all nests will not prevent new ones from spawning later, making it impossible to exterminate the aliens.
Alternatively, leaving a nest intact whilst not attacking aliens at melee range will allow them to calm down and consider you non-threatening, and they will not attack your colonists or military units when they pass by. Eventually, aliens will come to regard your people as their own, giving you a sort of independent free militia force. Additionally, the rate at which you gain positive "influence" with aliens may be partially determined by how many nests you have within your borders. If you get a really lucky colony with access to 3 Alien Nests and you don't attack the aliens, you can sometimes get them to be allies after just a dozen turns or so.
Be advised that aliens are not barbarians, so they are highly feral and not tactical, and you can lure them into ambushes. After antagonizing the aliens for some time, nests can spawn Siege Worms.