Hotel de la Cathedral offers romantic rooms in a not too kitschy French style, some with a great view of the Cathedral. All that at a good price (€80-120). Breakfast was €11 per person with lots of fresh fruit, cold meats, cheeses and the obvious bread, baguette, croissant, and marmalade.
A good choice of restaurants for dinner are on the same street.
Parking underneath the nearby market hall is only € 9 for 24h.