The Château des Tours was built in the 14th century. At that time it had three towers: one had a polygonal interior and was the dungeon, and the others were circular flanking an oblong building on the east and west corners of its south face; then, at the same corners on the north face, there are sentry boxes with Corbels. The oldest part is in the south of the present castle and is also a little higher.
North of the original castle there was a courtyard formed by the castle and crenellated walls outside with wide and deep ditches below. These ditches remain but the walls were replaced in the 16th century by rectangular buildings. At the east and west corners of the north facade there are two round towers, less high than the others, but topped like them with machicolations. There is a door between these two towers above which are the Arms of Calvimont carved on a large stone: they have lain since 1793 a few steps in front of the door.
The castle was restored by Viollet-le-Duc in the 19th century.
References:Inverness Castle sits on a cliff overlooking the River Ness. The red sandstone structure evident today was built in 1836 by architect William Burn. It is built on the site of an 11th-century (c. 1057) defensive structure. Today, it houses Inverness Sheriff Court.
The castle is said to have been built by Máel Coluim III of Scotland, after he had razed to the ground the castle in which Macbeth of Scotland according to much later tradition, murdered Máel Coluim"s father Donnchad I of Scotland, and which stood on a hill around 1 km to the north-east.
The first Inverness Castle was partially destroyed by King Robert I of Scotland and a replacement castle was sacked in the 15th century by the Clan Donald during the Siege of Inverness (1429). The castle was occupied during the Raid on Ross in 1491.
In 1548 another castle with tower was completed by George Gordon, 4th Earl of Huntly (1514–1562). He was constable of the castle until 1562.