The earliest records of Kellie go back to 1150 where it is mentioned in a charter issued by King David I. The first known owner was Robert of London, the illegitimate son of King William the Lion. None of the early buildings to have survived.
The estate was signed over to a Siward relative, Walter Oliphant, in 1360 and the castle remained in the ownership of the Oliphant family until 1613. It was purchased by Sir Thomas Erskine, who had saved the life of King James VI during the Gowrie Conspiracy. The King stayed at Kellie in 1617 during his only visit to Scotland after the Union of the Crowns, and he appointed Erskine as Earl of Kellie in 1619.
Originally a simple tower house, the lower section of what now constitutes the northwest tower is the oldest part of the castle, dating from around 1360, and is said to be haunted. In 1573 a new tower was built by the 4th Lord Oliphant to the east of the original tower. It is believed that the 4th Lord built the east tower as a jointure-house (a property set aside for the wife after the husband's death) for his wife Margaret. Between 1573 and 1606 the two towers were linked by a new range, terminated by another tower in the south-west, creating the T-plan layout that remains today. The castle is a fine example of Scots Baronial domestic architecture, with an imposing mix of gables, corbelled towers, and chimneys.
The walled garden is 17th century, with late Victorian additions, contains a fine collection of old-fashioned roses, fruit trees and herbaceous plants.
References:Celje Castle was once the largest fortification on Slovenian territory. The first fortified building on the site (a Romanesque palace) was built in the first half of the 13th century by the Counts of Heunburg from Carinthia on the stony outcrop on the western side of the ridge where the castle stands. It had five sides, or four plus the southern side, which was a natural defence. The first written records of the castle date back to between 1125 and 1137; it was probably built by Count Gunter. In the western section of the castle, there was a building with several floors. Remains of the walls of this palatium have survived. In the eastern section, there was an enclosed courtyard with large water reservoirs. The eastern wall, which protects the castle from its most exposed side, was around three metres thicker than the rest of the curtain wall. The wall was topped with a parapet and protected walkway.