Blame Noah Webster.
A Separated Population
There are some word differences we can't lay at Webster's feet. For example, "while" and "whilst" mean the same thing, but as far as I can tell, nobody really knows why "whilst" survived in Britain but withered in America. According to World Wide Words (3), "whilst" is considered more formal than "while," even in Britain. So if I had to guess, I'd say "whilst" probably fell out of favor in America because we are a less formal nation, and geographic separation of the two populations also let the language change differently in the two countries, but really, I'm just making things up at this point. If anyone has a better answer, please post it in the comments.
Why Do Britons and Americans Use Single and Double Quotation Marks Differently?
On to a difference where I at least have a hint of an answer.
In America we use double quotation marks to enclose a quotation, and single quotation marks if we need to enclose another quotation inside the first quotation. In British English, it's the opposite. Single quotation marks are used for everyday purposes such as enclosing a stand-alone quotation (4, 5).
In 1908, an influential British style guide called The King's English, stated that "The prevailing [metho is to use double marks for most purposes, and single ones for quotations within quotations." So to spell it out for you, the author, Fowler, was saying that at the time the British did it the same way we do it now in America. But Fowler went on to advocate for single quotations marks, saying it is more logical to use them for regular quotations, and to reserve double quotation marks for quotations within quotations (6). He didn't explain why he thought it was more logical; he just said it was. Given that the British method now follows Fowler's stated preference, I presume that Fowler is the reason the British now use single quotation marks where Americans primarily use double quotation marks--that he was influential enough to make that change happen. But that one little paragraph from The King's English is the only proof I have, so if you know of some other reason Britons made the change, please leave a comment below.
Next: Why Quotation Marks Are Handled Differently