The oldest parts of the Croy castle were probably built in the 15th century. There is not much known about the history of the castle. Jacob van Croÿ, Bishop of Cambrai was in 1477 owner of the land, but a house is not mentioned. In the 16th century villages in the neighbourhood were several times demolished, probably also the castle (but this is not recorded).
The last inhabitant was Freule (Lady) Constance van der Brugghen. Her uncle bought Croy in 1772, In 1773 he died and left the castle to his half brother Johan Karel Gideon. He married Margaretha Gertuda Falck and they had three sons and a daughter. In 1820, 1826 and 1864 the sons died. Freule (Lady) Constance van der Brugghen died in 1873 and left the building to municipality of Stiphout with the obligation to use the building for the help of poor elderly people. This was led by the sisters of Geloof, Hoop en Liefde. In 1977 the firemen disapprove the house as being a house for elderly people because of the safety regulation. The castle is currently no longer inhabited but in use as an office.
Research of the castle points that the size was more or less the same all those years. The building history seems complex but not much is know from written sources. Its seems that the original building has been severely damaged. The Cellar, the right wing and the round tower are probably from the beginning of the construction. The last changes e.g. the entrance and the Crow-stepped gable were build in the 18th century.
References:Sigmaringen Castle was first mentioned in the year 1077 in the chronicles of Petershausen monastery. The oldest parts of the castle are concealed beneath the alterations made during the 17th and the 19th centuries. The secret of the earliest settlement built on this defendable rock will never be fully revealed: large-scale excavation work would be necessary, which the extensive land development renders impossible. Judging from the many Roman remains unearthed in the area around Sigmaringen, the 12th century keep known as the 'Roman Tower' could be traced back to a Roman predecessor.
The castle remains that have been preserved (gate, great hall and keep) date back to the Staufer period around 1200. The castle remains were integrated into subsequent buildings. The foundations of the castle buildings are to a large extent identical to the surrounding castle wall.
These remains give us a good idea of how the castle might have looked during the 12th century.