When we left the house for Chora Church, we had an elaborate plan in mind that would get us from the church to Istanbul Design Week at the old Galata Bridge. Actually, we had Lonely Planet's plan in mind: In the Istanbul City Guide, they had more or less laid out a path that would get us from the church to the water, via several historic sites (though oddly the book doesn't mention that the neighborhood is borderline seedy).
The Lonely Planet path laid out eight points of interest and we took the route backwards (which actually makes more sense as heading toward the Golden Horn, instead of away, is all downhill), and started about halfway. (The sites we skipped were a spot where you could climb the walls, the Mihrimah Sultan mosque visible on the map, and the Roma neighborhood of Sulukule which doesn't exist anymore).
So after we explored Chora Church (point 1 on the map), we headed to the Byzantine city walls, which had been built Emperor Theodosius II to contain the burgeoning city. The walls stretch from point 2 to 3 and beyond in both directions, but point 2 is more or less where we figured out that you can tackle the steep steps and actually climb UP on the walls. The view from the top was absolutely spectacular - we could see Istanbul Design Week to the left, the skyscrapers in the our general neighborhood straight ahead, and Galata Tower to the right.
From there, we attempted to reach point 4, the former location of the palace's dungeons. Apparently some of the Byzantine emperors were tortured and murdered there. It was difficult following the specific path laid out in the book because neither the book nor the streets much bothered with street signs, but after wandering through the neighborhood, we eventually found the general location. We couldn't manage to figure out what the dungeons were supposed to be though, if they were against the walls or across the street.
After that, it was just about a block to the Golden Horn and its waterfront park, which was gorgeously illuminated by the amber afternoon light. From there, we could clearly see the Aya Sofya and again the Galata Tower to the right, as well as the Rahmi Koc Museum and its ships directly across the way.