Per IRS Publication 17 Your Federal Income Tax (For Individuals), page 171:
The term "points" is used to describe certain charges paid, or treated as paid, by a borrower to obtain a home mortgage. Points may also be called loan origination fees, maximum loan charges, loan discount, or discount points.
A borrower is treated as paying any points that a home seller pays for the borrower's mortgage. See Points paid by the seller, later.
You generally can't deduct the full amount of points in the year paid. Because they are prepaid interest, you generally deduct them ratably over the life (term) of the mortgage. See Deduction Allowed Ratably.
For exceptions to the general rule, see Deduction Allowed in Year Paid.
Refinancing. Generally, points you pay to refinance a mortgage aren't deductible in full in the year you pay them. This is true even if the new mortgage is secured by your main home.
However, if you use part of the refinanced mortgage proceeds to improve your main home and you meet the first 6 tests listed under Deduction Allowed in Year Paid, you can fully deduct the part of the points related to the improvement in the year you paid them with your own funds. You can deduct the rest of the points over the life of the loan.