The abbey of Saint-Andre-le-Bas was founded in the 8th century by Duke Ansemund. The church was originally a chapel of the palace of kings of Burgundy built in the end of the 9th century. The abbey flourished in the High Middle Ages.
The troubles of the Hundred Years' War and the competition of the new religious orders reduced the power of the convent and it was unable to recover from the Wars of religion. The monastery was dissolved in the late 18th century. The abbey church, built in the 11th century, became a parish church and the convent buildings were sold and partly dismembered. The church, the bell tower and the cloister are still remarkable for their harmonious Romanesque sculpted ornementation.
Celje Castle was once the largest fortification on Slovenian territory. The first fortified building on the site (a Romanesque palace) was built in the first half of the 13th century by the Counts of Heunburg from Carinthia on the stony outcrop on the western side of the ridge where the castle stands. It had five sides, or four plus the southern side, which was a natural defence. The first written records of the castle date back to between 1125 and 1137; it was probably built by Count Gunter. In the western section of the castle, there was a building with several floors. Remains of the walls of this palatium have survived. In the eastern section, there was an enclosed courtyard with large water reservoirs. The eastern wall, which protects the castle from its most exposed side, was around three metres thicker than the rest of the curtain wall. The wall was topped with a parapet and protected walkway.