The oldest parts of the Croy castle were probably built in the 15th century. There is not much known about the history of the castle. Jacob van Croÿ, Bishop of Cambrai was in 1477 owner of the land, but a house is not mentioned. In the 16th century villages in the neighbourhood were several times demolished, probably also the castle (but this is not recorded).
The last inhabitant was Freule (Lady) Constance van der Brugghen. Her uncle bought Croy in 1772, In 1773 he died and left the castle to his half brother Johan Karel Gideon. He married Margaretha Gertuda Falck and they had three sons and a daughter. In 1820, 1826 and 1864 the sons died. Freule (Lady) Constance van der Brugghen died in 1873 and left the building to municipality of Stiphout with the obligation to use the building for the help of poor elderly people. This was led by the sisters of Geloof, Hoop en Liefde. In 1977 the firemen disapprove the house as being a house for elderly people because of the safety regulation. The castle is currently no longer inhabited but in use as an office.
Research of the castle points that the size was more or less the same all those years. The building history seems complex but not much is know from written sources. Its seems that the original building has been severely damaged. The Cellar, the right wing and the round tower are probably from the beginning of the construction. The last changes e.g. the entrance and the Crow-stepped gable were build in the 18th century.
References:The Château du Lude is one of the many great châteaux of the Loire Valley in France. Le Lude is the most northerly château of the Loire Valley and one of the last important historic castles in France, still inhabited by the same family for the last 260 years. The château is testimony to four centuries of French architecture, as a stronghold transformed into an elegant house during the Renaissance and the 18th century. The monument is located in the valley of Le Loir. Its gardens have evolved throughout the centuries.