For more than two centuries, this house has witnessed celebrations, business, departures and homecomings.