The building at 800 N. Charles St. was designed by Jacob and Ephraim Wall for residential use starting in 1842 for J.D. Kremelberg, a German merchant. In 1883, it was purchased by Arunah S. Abell, founder of The Baltimore Sun, and was Abell's "in town" residence until he died 1888. In a 1989 biography, author William S. Abell described the house as a "magnificent mansion" and "a grand twenty-five room, four-story marble and brick building with a splendid winding staircase rising to a skylight in the roof." The house also has a level below grade.

After Abell's death, the building had a variety of uses. The Baltimore Club, a private social group, occupied it between 1908 and 1932. It then became the Longfellow Hotel, with 27 "sleeping rooms." In the 1940s, owner O.L. Bonifay renamed it the Park Plaza and continued to use it as a hotel. During the 1950s, Martin J. Welsh operated the building as Marty's Park Plaza, with a restaurant, bar and banquet rooms. There were whispers of high-stakes card games upstairs and secret rooms, including an arena for cockfighting on the top floor.

In 1966, four businessmen from Washington, D.C., including Kingdon Gould Jr., former ambassador to Luxembourg, bought it for use as a restaurant and possible future redevelopment. Also in the late 1960s, a portion of the property was transformed to house one of Baltimore's first discotheques, with music played on 45 rpm records.

In the early 1970s, architects Seymour Tatar and W. Boulton Kelly acquired the Park Plaza and redeveloped it as a collection of offices, shops and restaurants patterned after San Francisco's Ghiradelli Square.

It is considered a "contributing building" within Baltimore's Mount Vernon Historic District and the federal government's Mount Vernon National Historic Landmark District. By law, any changes to the exterior must be approved by the city's Commission for Historical and Architectural Preservation.