The ornamental Pavilion was part of a 9.5-hectare waterfront estate owned by the famous Grace family from 1907 to 1917 (purchased from Peter McIntosh). It was officially opened on April 4, 1911 for weekend recreational use by Grace Bros employees, along with such sporting facilities as bowling greens, tennis courts, cricket pitches and a small golf course.
In 1917 the property was sold to Nestlé, known as the Anglo-Swiss Condensed Milk Company (for 25,000 pounds). The site was purchased to build the ‘largest chocolate factory in the Southern Hemisphere’. Abbotsford House was used for the offices,and the factory was built on either sides of the house.
Nestlé restored the pavilion to a staff recreational clubhouse in 1924, which explains why the main hall has a double timber floor – the second layer simply built over the first when the pavilion reverted to its original use.
Following the the closure of the factory on 18 December 1991, the site was sold (to IPM) and redeveloped for medium density housing. IPM in conjunction with the Heritage Council of NSW and Drummoyne Council, relocated the Pavilion to its publicly visible spot in October 1996. After 11 painstaking months the Pavilion was faithfully restored to its original glory, including a geometrically-patterned exposed timber frame, decorative metal and slate roof, and remarkable interior detailing and timber joinery. Once again Abbotsford House is used as a private house.