Piedmont Park in the afternoons is nice when it's not to cold or hot. It's like a downsized central park, but for Atlanta.
Atlantic Station is a new live, work, play district that includes restaurants, a movie theater, and mall-esque quality apparel stores (ie GAP, AE, H&M). The district is walkable and accessible from Arts Center Station (MARTA) where a free shuttle operates between the station and the district.
Buckhead in the evening is a must. If you have a car I suggest driving the length of Peachtree St. If you're into upscale shopping centers then try Lenox Square and Phipps Plaza. Both are across from each other and accessible via the local rail transit (MARTA).
Little Five Points is a funky disctrict that deserves attention from any shopping hipster guru.
Freedom Park has walking trails and is a great place to see the Atlanta skyline. Try the corner of Freedom Parkway and Boulevard