Little things add up in a big way: take the stairs, park at the back of the parking lot, do exercises at your desk (google it,) I remember someone mentioned lifting the milk jug a few times like a barbell while making the kids' lunches.
If you live in a small enough town, walk to do your errands.
One little fitness contraption that I'm in love with is the Gazelle. It's got an infomercial, and I've had the chance to try it out, though I don't own one myself. It functions like a treadmill, minus the impact, so you can save your back and neck and maybe even stretch out some more.
Stretch as soon as you get up in the morning, and before bed. That doesn't have to take more than 5-10 minutes; just get yourself limbered up.
Swimming is already a great low-impact exercise for you to do. Cardio for 30-minute sessions 3+ times a week will be enough for you to not just lose weight, but build muscle and keep you fit.
And make sure you're getting all your vitamins! Don't cut carbs completely; you need to get all the energy-giving vitamins fruits and veggies have to offer you.
Good luck