When you suddenly find yourself funding your daughter’s 600-guest wedding or covering unexpected expenses (like major storm damage), you may be tempted to raid your retirement savings. But you know that’s probably a bad idea. As AARP notes, “If you’re under 59 ½ years of age, you could face a 10 percent tax penalty plus have to pay ordinary income taxes on any amount you withdraw.” Adds Forbes, the money you withdraw from your retirement account could actually push you into a higher income bracket, raising your tax rate.

Also, if you withdraw a large sum from your IRA (say, $30,000 for that wedding), you’ll have a tough time putting it back: you’re still limited in terms of the annual amount you can legally add to the fund. In 2013, according to the IRS, those under age 50 are limited to contributing up to $5,500 annually into their traditional and Roth IRAs; those over 50 can go as high as $6,500 annually.

Finally, if you do spend retirement money now, you’ll have other things to consider: Can you replace those savings soon? Can you make up for the interest and earnings you’ll lose while your money is out of the retirement account?

Of course, there can be times when withdrawing from your retirement fund would be the lesser of two (or more) evils. Here’s when you should consider dipping into your retirement savings:

If you’re stuck with credit card debt

“One case where tapping retirement accounts early can really make sense is if you have a lot of credit card debt and are paying interest on it at high rates,” says Forbes. The publication goes on to quote Paul Jacobs, a certified financial planner with Palisades Hudson in Atlanta, who says that if you pay off the balance with retirement funds, “you’re likely to end up in a better place financially.”

If you have great hardship

You can actually withdraw, as opposed to borrow, money from a 401(k) or Traditional IRA if you’re suffering from certain hardships. According to Bankrate, these hardships include:

  • Deductible medical bills that exceed 7.5 percent of your adjusted gross income. (You’ll owe income taxes on this money, but no penalty.)
  • Disability, if you’re totally and permanently disabled. (Taxes; no penalty)
  • Health insurance premiums, if you’re unemployed and have been so for at least 12 weeks. (Taxes; no penalty)
  • An IRS tax levy against your account. (Taxes; no penalty)
  • A first-time home purchase, or a home purchase if you haven’t owned one in two years. (Taxes and a penalty if you’re withdrawing from your 401(k); Taxes and no penalty, if you’re withdrawing from your Traditional IRA)
  • Higher education costs for you, your spouse, children, grandchildren or immediate family members. (401(k): taxes and penalty; IRA: taxes and no penalty)

If none of the above describes your situation, and you still must raid your retirement savings, a couple of scenarios will allow you to access your money penalty-free: 

If you have a Roth IRA

With a Roth IRA, you’ve already paid income tax on the money you’ve deposited into your retirement account. That means you can withdraw any Roth IRA contributions – though not their earnings – whenever you’d like, penalty- and tax-free, explains Kiplinger.

If you have a 401(k)

Most 401(k) plans allow you to borrow money from them, according to SmartMoney. The site explains that, typically, you can borrow as much as half your vested balance, up to $50,000 – without penalty. You will have to pay current interest rates on the loan, though, and you must pay back the outstanding amount within five years.

But once again, remember, you want to avoid withdrawing money from your retirement plan unless you have to. One way to do this is by creating an emergency fund. If you have 6 to 12 months worth of living expenses in the bank (or more, if your situation warrants it), you’ll be prepared for job loss, illness and other unexpected costs, should they arrive.

Have you ever dipped into your retirement savings? How did you replace that money?