One week in a place hardly qualifies me to tell what to do, but I have recommendations to share. Please do yourself a favor and visit this place for breakfast EVERY DAY OF YOUR LIFE. Get something with pears. Or apricots. Or get one of each because I am not one of those judge-y people.
We loved our hotel, El Patio, for it's kind staff and it's perfect location: two blocks from the beach, three blocks from strong cuban coffee, and a 5 minute walk to the main drag - Duval street. The rooms were clean and....retro-ish. Their website sucks, so trust me: this place is really great.
In a weird Holyoke and Massachusetts connection, the architect of Holyoke's historic Train Station, HH Richardson, also designed Key West's Custom House.