During the Civil War, Mrs. Bailey was held at gunpoint by Stoneman soldiers in the Lee House on Cartner Street when she resisted demands for money; a bed pillow was set on fire, but the soldiers left without harming her. The fire was quickly extinguished, with the only remaining evidence of the incident a charred wallboard on display at the library. The house was built in 1838 by Braxton Bryant Bailey.