I've only been there once and I went to these five cities: Nagoya, Kyoto, Nara, Osaka, and Kobe.

Out of them all, Kyoto was by far my favorite. Full of temples, great food, culture, and beautiful nature. The locals will also go there in traditional dress (kimonos and wooden sandals) so it adds to the allure. I'd also recommend going to Arashiyama Bamboo Forest: http://www.insidekyoto.com/arashiyama-bamboo-grove

Nearby, there is a place where you can feed monkeys as well.

Other than that, I'd recommend Nara. It's near Kyoto and it used to be a capital of Japan. There are temples and wild deer EVERYWHERE!. You can buy little biscuits and feed them. It's quite a trip.

Nagoya was really beautiful and I enjoyed it there. While in Japan, you should really try going to an onsen (hot spring). It was an amazing experience. The nakedness takes some getting used to but nobody pays you any attention.

As for Osaka and Kobe...there were just okay to me. Osaka is like any other big city, though it has Universal Studios if you're into that.