Robert Lane Greene considers how little we use the word – and why we keep avoiding it:
Geoffrey Pullum makes a distinction between Normal and Formal language, and most English-speakers today, when in Normal mode, steer clear of whom. We leave out the relative pronoun (That's the friend I'm inviting to dinner) or just use who. Children are rarely exposed to Formal, and have little concept of register. Whom is just weird for them. A search of the Spoken category of the Corpus of Contemporary English finds that I is about eight times more common than me—but who is 57 times more common than whom. It appears just 53 times out of every million words.
He goes on to note that because "whom is becoming less common, many people can't use it properly even when they are aiming for Formal."