During NaNoWriMo 2010, I attempted to write a serious work of historical adventure set in Tudor England, just as the Dissolution of the Monasteries was taking place. I gave my lead character a pretty outlandish first name--but it was a very Catholic first name, so I figured I could get away with it, since the character was born in a convent.
But as I was scrabbling around for a name that wasn't Elizabeth, Anne, Mary, or Catherine for a female character, I decided to ask a friend to help me brainstorm first names. Her suggestions weren't usuable for the time and place and that's when I realized that figuring out historically appropriate names for your characters is probably one of those odd historical fiction writer quirks.
(For the record, among her suggestions at the time were Avery and Shirley, both of which were last names, then became first names given to boys in the mid-to-late nineteenth century, then more recently, given to girls as first names).
More recently, as I was tapping away at the draft I'm currently almost 12K words into, I realized that I had major characters named Jane, John, and James and a minor character named Jones. So that's a problem. And then I went back and looked up the types of names that would've been given to people born in the Georgian era, who would've been adults in the early 1800s, in England.
Unsurprisingly, many of the names are what we would consider traditional English names: William, John, Elizabeth, Jane, Anne, Catherine, Henry, Charles, Mary, George, James, Robert, Edward. Men sometimes had surnames as first names (think of Fitzwilliam Darcy, whose first name is his mother's maiden name because his mother's family had more money and prestige). There were many feminizations of male names: Georgette, Georgiana, Georgina, Edwina; feminization got even more intense as the 19th century wore on, resulting in names like Thomasina, Wilhelmina, Frederica, and Benjamina (!).
Of course, in this time period, naming also went by class, so lower or working class were more likely to have Biblical (Old Testament Biblical) names than the upper classes, while the upper and middle classes may have also had more Classical names because Greek and Roman culture was in vogue in the Georgian and Regency periods--think of Cassandra Austen, for instance, who was named after her mother, or babies named Julius, Augustus, or Octavius.
As the Napoleonic Wars heated up, babies were also named for battles and famous people, like Horatio Nelson, Lord Byron, and Arthur Wellesley, the Duke of Wellington. Names of the kings (who were named George for over a hundred years until George IV died) and the royal family were used, too: Caroline, Augusta, Charlotte, Sophia, and later on, of course, Victoria, who was born in 1819.
Many names we think of as girls' names were boys' names in the olden days: Evelyn, Ashley, Whitney, Aubrey, Beverly, Hilary, Meredith, Shannon, etc.
Granted, historical names are one thing and historical romance names are another. As long as the name you choose for your born-in-the-late-eighteenth-century character isn't utterly implausible, too modern, actually an animal name, or you can explain it some way--and it was in use at the time--then it seems like it can fly in historical romance.
For the record, while I had to keep Jane and John (they've been called those names for years, though John's often referred to by his last name), the other names in my new story changed. James (my default name for an English heir to anything) has become Richard and Jones, Jane's maid, was re-surnamed Griffith, because I finished reading a Welsh-set book not long ago and decided I could do a shout-out by using a Welsh last name for a maid.
What's in a Name?
Top Female First Names in 1800
Georgian Girls Names