Anyone who steals your wallet -- or even anyone who finds your wallet -- has a gold mine. Your driver's license is in there, your credit card is in there, and if you carry your social security card, they've got that, as well. That's a big deal. The thing of it is, they don't even need to physically take anything out of your wallet to rob you blind. If they write down your credit card info and your address from your driver's license, they can clean you out shopping online. And with your social security number, there's not much a person can't do by way of opening bank accounts and new lines of credit. You could get your wallet back later, with everything in it, and still be hung out to dry.
One instance of a lost wallet turning into full-on identity theft is the case of Margot Somerville, who lost her wallet on a San Francisco trolley in 2006. That kicked off an ordeal that lasted for a full five years, costing her $20,000 in fraudulent charges, another $60,000 trying to prove her innocence, and held the constant fear of prison in the back of her mind during her last years with her husband. All that because a woman was depositing fraudulent checks in Somerville's name, all the way over in California.
If you do lose your wallet, you need to go into lockdown mode, even if you do get it back. Have all of your credit cards reissued, notify credit bureaus of potential fraud, call the social security office and the IRS, and generally follow the stolen identity recovery handbook in full, because you're about as compromised as you can be.