Taking a day trip by boat is always a good idea. It costs about €15, (inc. lunch). A couple of tips:
1. Don't take the large boats, mid-size ones with max. 20 passengers tend to be more quiet while the larger ones are mostly chaotic.
2. Most boats make the Kara Ada (Black Island) tour. But my suggestion is the Orak Island tour. It's a bit further and a bit more expensive, but the sea is beautiful and the bays are more tranquil.