Madison Square Garden
Madison Square Garden is an indoor arena jammed between 7th and 8th avenues and 31st and 34th streets. It is the 4th arena of its name. A MSG has not been located in Madison Square since the second building with the name, a McKim, Mead, and White work, was torn down in 1925 to make way for the New York Life Insurance Building. It is best known among urban historians and New York City history buffs for having replaced McKim, Mead, and White's iconic Penn Station.
The New York Rangers play here, although tickets are rare and expensive. The New York Knicks also play here, but these tickets, while easier to obtain, are not that much cheaper. Both teams fail at their respective sports despite oversized payrolls.
The Columbia basketball team actually plays a few exhibition games here in January. MSG also hosts some games of the NCAA basketball tournament, which are definitely worth watching (especially for rooting for or against the Ivy League NCAA tournament entrant of that year).
Getting there is easy: take the 1, 2, or 3 to 34th Street/Penn Station.