When shopping for a home or looking to refinance, everyone wants to know what kind of deal they can get on a mortgage. Of course, some people end up paying considerably more than others. Some of the reasons for that are fairly obvious, others not so much. Let's take a look at some of the main factors.

Credit score

If you've done any research into getting a mortgage at all, you ought to know that your credit score doesn't just affect your ability to get a mortgage, it has a major impact on how much you're going to end up paying.

In fact, for scores down to a certain level, around the lower 600s, the cost of your mortgage will be the key impact your credit score will have! Many people with lower scores find it's not so much that they can't get a mortgage per se, but that they can't afford one.

A borrower with a credit score in the mid-600s will pay about a full percentage point more in mortgage interest on a 30-year fixed-rate mortgage than a borrower with an excellent credit score of 760 and above, according to figures from the Fair Isaac Co., developer of the widely used FICO credit scoring system. That's a difference of about $120 a month.

Even good, but not great, credit, in the 700-759 range, will cost you more. Borrowers in that range tend to pay about two-tenths of a percent more on a 30-year loan, or an additional $25 a month on a $200,000 mortgage. Have a score in the lower 600s? Figure on paying an additional one and a half percent, if not more, boosting your mortgage payment nearly $200 a month on a 30-year loan.

Down payment

A lower credit score won't necessarily require a larger down payment, until you get to a certain point. Many lenders will approve FHA mortgages for borrowers with credit scores as low as 600, and FHA allows down payments as small as 3.5 percent. Go lower than that, however, and you're stuck in the specialty lending market, and can expect to be required to put up 20-30 percent or more.

However, the size of your down payment, or the equity you have in the home (when refinancing) can affect the interest rate you pay. A borrower who makes only a 5 percent down payment is a bigger risk than one who puts down 20 percent; so the interest rate or fees charged may be higher.

Another key thing to remember is that if you put less than 20 percent down is that you'll have to pay for mortgage insurance as a hedge against the increased risk of default (exceptions: VA and USDA mortgages do not require additional insurance on no-money down loans). Depending on the loan program and how much you put down, this can cost anywhere from 0.35 to 1.5 percent of your loan amount per year - which is like raising your interest rate by that much. So it can be costly.

Loan type

Choosing the wrong type of mortgage for your situation can be costly. Yes, you can get an FHA home loan with as little as 3.5 percent down - but were you aware there's an upfront insurance premium of 1.75 percent of the loan amount, plus up an annual insurance premium that often exceeds the cost of private mortgage insurance?

VA insurance is renowed for allowing veterans and active-duty service members to buy a home with no money down, but also includes an upfront "funding fee" of roughly 2 percent? If you have the money for a down payment, a conforming mortgage by Fannie Mae or Freddie Mac could be a better deal.

The terms of a mortgage can also make a significant difference. A 30-year fixed-rate mortgage will give you stable monthly payments and is great if you're planning to own the home for a long time, but if you're planning to move in a few years, you can probably get a better deal with an ARM (adjustable-rate mortgage). The interest rate on a 5/1 ARM is currently running about a full percentage point lower than on a 30-year fixed-rate mortgage and a 7-year ARM isn't much higher, so if you expect to upgrade or otherwise move again in a few years, an ARM will save you money.

Loan amount

The amount you borrow will affect the mortgage rate that you pay. Generally speaking, a jumbo mortgage (one exceeding the maximum loan amounts allowed by Fannie Mae, Freddie Mac or the FHA) will require an interest rate about one-half to a full percent higher than on a comparable conforming loan, because jumbo loans can't be guaranteed by one of those agencies.

The conforming loan limits are $647,200 in most of the country for a single-family home, though in certain high-cost areas they go as high as $970,800. Anything above that is considered a jumbo loan.

Ironically, jumbo mortgages are currently running at or just a bit cheaper than conforming loans. That's partly because lenders see less risk in making loans to well-heeled borrowers at present, even without the guarantees. But historically, that's an aberration.

You'll also likely pay more on a mortgage with a relatively small loan balance, say around $150,000 or less. This is because lenders face certain fixed costs in originating any type of mortgage and smaller loans don't generate as much interest income. So they need to charge a higher rate or fees to compensate. In fact, you might not even be able to obtain a mortgage or refinance for a loan amount of $50,000 or less.

Property type

The kind of residence you're buying or refinancing will also affect the interest rate you pay on a mortgage. The mortgage rate on a condominium, for example, will often run about one-eighth to one-quarter of a percent higher than on a single-family home of a similar price. Rates on manufactured homes (aka mobile homes) are even higher, particularly if they're sited on leased land and you have to get a chattel loan, rather than a mortgage, to buy one (See this article for more information).

You'll also pay more for a mortgage to purchase a vacation home or other second residence, which will often carry a rate about a quarter of a percent higher than a mortgage to buy a primary residence. Down payment requirements tend to be higher as well, with most buyers putting up at least 20-25 percent and some lenders requiring 35 percent or more.

You'll also pay a higher interest rate on an investment property than you would on a primary residence.

Remember though, that your interest rate and down payment are linked: a larger down payment helps you get a lower interest rate, while accepting a higher interest rate will allow you to get by with a smaller down payment. So while some lenders will accept 10 percent down on a mortgage for a second home, you'll pay a higher interest rate; or if you want the same interest rate you could get on a primary residence, you'll need to make a larger down payment.