Fifth Ave. is lined with colleges, like Pitt, CMU, and Chatham. Colleges have reputations for being party areas, so he's probably talking about going to a party at one of the colleges, probably at one of the apartments.