The oldest part of the present Bossenstein Castle is the square keep, probably built before the 14th century by a Joannes van Busco or Van den Bossche. Not much later it went to the Van Berchem family. They are supposed to have made some major alterations to the castle. They sold it in 1544 to Guilelmus van der Rijt, who was a member of the city council of nearby Antwerp. In the deed of sale the castle was described as an impressive estate, with a lot of outbuildings. Of these outbuildings nothing survives.
In 1655 it was sold again, this time to Willem van Halmale. He rebuilt the castle into a beautiful residence, which was still fortified. During the 18th century the castle changed hands several times between several noble families. Between 1798 and 1843 the castle was used by religious institutions. In 1906 the castle underwent restoration works, saving it from slow decay.
At present the castle is surrounded by the grounds of an 18-hole golf and polo club. The terrain of this club is probably not freely accessible. There is however a long distance walking path leading through the golf course which is. As far as I know the castle itself can not be visited.
References:Inverness Castle sits on a cliff overlooking the River Ness. The red sandstone structure evident today was built in 1836 by architect William Burn. It is built on the site of an 11th-century (c. 1057) defensive structure. Today, it houses Inverness Sheriff Court.
The castle is said to have been built by Máel Coluim III of Scotland, after he had razed to the ground the castle in which Macbeth of Scotland according to much later tradition, murdered Máel Coluim"s father Donnchad I of Scotland, and which stood on a hill around 1 km to the north-east.
The first Inverness Castle was partially destroyed by King Robert I of Scotland and a replacement castle was sacked in the 15th century by the Clan Donald during the Siege of Inverness (1429). The castle was occupied during the Raid on Ross in 1491.
In 1548 another castle with tower was completed by George Gordon, 4th Earl of Huntly (1514–1562). He was constable of the castle until 1562.