In order to avoid an IRS penalty, you must be 59 1/2 or older to withdraw funds. To avoid a surrender charge from COF, review your contract. All surrender charges are listed there. To see the surrender charges for annuities COF currently offers, please visit the product disclosures page.