This whole thing contains SGA spoilers through "First Strike", but the comments could contain more... who knows?
The Ancients started out in another galaxy where they in the Ori were effectively the same race. However, the Ori believed that they had a right to 'play God' over unenlightened mortals, and were willing to go to the war with the Ancients over this. But rather than let this happen, the Ancients left for the Milky Way, and while they were there they spent their time seeding human life - and Stargates - throughout the galaxy. They were able to repress the Ori's knowledge that the Milky Way was populated, at least for a time.
Atlantis may have been constructed on Earth, and at one point a second-generation Gate (the big turny stone kind) was placed on Earth as well as the one already present inside Atlantis.
But at some point "several million years ago" most of the Ancients left on Atlantis. This may have had something to do with the virus that Ayiana was carrying and exposed Jack and the others to (a plague in the ancient Milky Way galaxy has been mentioned in several SG-1 episodes, including Window of Opportunity and Threads). Maybe they just wanted a new galaxy to seed with life. At least one Ancient, later called Aiyana, was left behind in the vicinity of the Stargate (which, depending on how many millions of years ago this all happened, may not have been a icy wasteland it would become after the supercontinent broke up and Antarctica moved to the southern pole).
Eventually, the Ancients landed Atlantis on the world referred to as Lantea, in the Pegasus Galaxy, which they also began to seed with life and Stargates. Unfortunately, one of the worlds they peopled was home to the Iratus bug, which eventually mutated into the Wraith. The Wraith set out to cull an unsuspecting galaxy, and their technology, either invented/grown or adapted, was effective even against the Ancients. After a long war, the Lanteans decided to sink the city (with only a time-traveling Elizabeth Weir left behind to rotate the three ZPMs, thereby saving the future expedition) and used the Stargate to travel back to Earth. These survivors mingled with the human population, spreading stories of a fantastical underwater city and presumably spreading some genes around as well, leading to a small number of present-day humans having the ATA gene.
At some point during all of this, two things happened. First, the Ancients became interested in the prospect of Ascension. There is evidence of their work on Ascension in both galaxies. It is thought that the majority of Ancients eventually 'moved on' to this higher plane, where the most important rule was non-intervention in the affairs of lesser beings.
Second, while at war with the Wraith, the Ancients created the Asurans. In the beginning they were only nanites that would infect and kill the Wraith, but they quickly evolved into something more sophisticated, eventually taking the form of their creators. The Ancients attempted to use this to their own ends, and did not remove the subroutine in the Asurans' base code which made them aggressive. They also implanted directives that would keep the Asurans from tampering with their base code or turning on their creators. Eventually, however, the Ancients decided that the Asurans were a failure and destroyed what had become their homeworld. Enough of the nanites survived that they were able to rebuild an even surpass the scope and technology of Atlantis at its peek, and when they discovered from Weir, Sheppard and his team that Atlantis had not been destroyed, they went looking for revenge.
McKay hypothesizes, during all this, that the Asurans may be related to the Replicators known in the Milky Way. These Replicators were initially encountered by SG-1 as technologically-advanced bugs which could make more of themselves if provided with the necessary raw materials, and whose basic units were kerons. These Replicators, it was discovered, where created by an android called Reese, which contained nanites. Reese's origins were vague, but her creator, or 'father', might have been an Ancient in the Milky Way still working on perfecting the Asuran technology, or a scientist from another world who found remnants of their experiments and decided to give it a try himself.
Bored - and emotionally unstable - Reese created "toys", the first insect-form Replicators, which eventually killed Reese's father. The 'toys' were programmed onto to make more of themselves, as Reese directed them to, and anything or anyone who gets in the way of this is at risk.
The Asgard, embroiled in a war with the insect-Replicators in their own galaxy - the consequence of the Asgard fiddling with the Replicators and not fully appreciating their destructive power - took the remains of Reese's body and used it to set a trap for their enemy. They activated a command in Reese telling all of the Replicators to "come forth" and intended to trap them within a time-dilation bubble. Unfortunately, the Replicators were able to counteract and even reverse the device, giving them plenty of time to investigate/consume Reese's body as well as the rest of the Asgard planet and evolve into microscopic nanites rather than large and cumbersome blocks. They created more of themselves, more or less in Reese's image, eventually escaping the Asgard trap and setting out to conquer the universe.
They crippled the Goa'uld, took on Samantha Carter's form, and seemed unstoppable until SG-1 discovered and used the massive Ancient-built weapon (which was originally built by the Ancients to recreate life "after the whole plague thing"). It seemed that the threat from the Replicators was at an end. Unfortunately, the IOA considered the Pegasus Replicators to be a threat to Earth and launched a preemptive strike against them, which failed.
Thanks to beatrice_otter for her help!