Gregory Clark's The Son Also Rises: Surnames and the History of Social Mobility has a timeline of the Galtonian question of the association between status and fertility, using the famous Darwin family, of which Galton was a member on his mother's side, as an example.

- ... in the preindustrial world, fertility was typically strongly positively associated with status. ...In England before 1780, this effect was so strong that the wealthiest parents had twices as many children as the average family.

- ... for parents who married between 1780 and 1880, there is no association between fertility and social status.

- ... there was a strong negative association between social status and fertility ... in England for parents who married between 1890 and 1960.  ... by 1880 in England, upper class-men seem to have produced far fewer children than those of the middle or lower classes. Indeed, from 1880 to 1940, the richest English families seem to have been dying out.

- .... Currently ... the correlation is relatively weak, with high-status parents having as many children as lower-status parents, or modestly fewer.

This timeline suggests that Galton (who didn't have children himself) was doing what he did best—systematically noticing—when he became worried in the later 19th Century about fertility among the most accomplished.

Here's Clark's example of the bearers of the Darwin surname. The Darwins are fascinating because even Charles wasn't exactly a genius. Instead, they are more like the 1960s astronauts: examples of a solid system functioning properly. Neil Armstrong wasn't some unique superhero (and he never claimed to be): he was the product of a healthy culture.

The lineage of Charles Darwin is a nice illustration of how large the families of the middle and upper classes could be in preindustrial England. He descended from a line of successful and prosperous forebears. His great-grandfather Robert Darwin (1682-1754) produced seven children, all of whom survived to adulthood. His grandfather Erasmus Darwin (1731-1802) produced fifteen children (born to two wives and two mistresses), twelve of whom survived to adulthood. His father, Robert Waring Darwin (1766 - 1848), produced six children, all of whom survived to adulthood. ...

Charles Darwin, marrying in 1839, had ten children, though only seven surived childhood.

These seven children produced only nine grandchildren, an average of only 1.3 per child. (This figure is unusually low for this era, but there was great randomness in individual fertility.)

The nine grandchildren produced in turn only 20 great-grandchildren, 2.2 per grandchild. This figure was less than the population average for this period. The great-grandchildren, born on average in 1918, produced 28 great-great-grandchildren, 1.4 each. ... The Darwin lineage failed to maintain itself in genetic terms.

Interestingly, with respect to social mobility rates, the 27 great-great grandchildren of Charles Darwin, born on average nearly 150 years after Darwin, are still a surprisingly distinguished cohort. Eleven are notable enough to have Wikipedia pages, or the like, such as Times obituaries. 

I looked them up for my recent Taki's column on Clark's book and found that 16 of the 28 great-great-grandchildren of Darwin now have Wikipedia pages.