For foreign travelers, Muttrah is probably Muscat's most charming district. In any case, it was my favorite. The old town provides an overview of what an Omani town might have looked like before the economic boom that followed the exploitation of natural resources. This white city punctuated by domes and minarets facing the sea in the midst of the ochre mountains is an image that can only remain etched in the memory of visitors.
Muttrah souq is one of the best places to find items that wil bring back memories of your stay in Oman : from incense (which provided Oman's wealth for centuries) to scarves and pottery.
A little farther on the coastal road, it's also worth the detour to the district where the sultan lives so you can see his palace, visit some museums (including the Franco-Omani Museum) or anything simply to soak up the peaceful atmosphere that prevails there.