Stephen Lewis Howell, business partner to Thomas McNeely, built the Greek Revival house around the 1850s. Howell served as Registrar of Deeds. Howell sold the house in 1863 to William Lafayette Brown, who ran a tobacco factory to the north of the house. When the Browns moved to Winston-Salem in about 1880, they sold the property to Calvin Cowles Sanford, a prominent businessman and merchant. The house descended to the Larew family upon C.C. Sanford's death.