But in recent years, companies have introduced "no cost" and low-cost refinancing packages that minimize or completely eliminate the out-of-pocket expenses of refinancing. (These refinancing packages compensate with a higher interest rate, or by including some of the costs in the amount that is financed.)

With traditional refinancing, the most often cited rule-of-thumb is that the interest rate for your new mortgage must be about 2 percentage points below the rate of your current mortgage for refinancing to make sense. However, with the newer low- and no-cost refinancing programs, it can be worth your while to refinance to obtain a smaller reduction in interest rates.

How long you expect to stay in your home is also a factor to consider. If you'll be moving in a few years, the month-to-month savings may never add up to the costs that are involved in a refinancing.