After the British burned the Capitol in 1814 during their invasion of the new Federal City, many in Congress felt it was time to move back to Philadelphia and abandon L'Enfant's city in the wilderness. Unable to meet in the ruins of the Capitol, Congress met for a brief time in Blodgett's Hotel at 7th and F Streets, NW, where the Patent Office Building (now the National Potrait Gallery and the Museum of American Art) stands. Hoping to forestall the rising movement to take the government back north, the citizens of Washington built a brick building on the site that now holds the Supreme Court building.

Known as the Brick Capitol, it was used by Congress as a meeting place from 1815 until 1819. After that, it served as a supply depot and briefly as a Federal jail during the Civil War (1861-1865). This small bronze plaque appears on the east side of the marble pillar at the steps to the Supreme Court building.