According to an inscription in the courtyard, the Vélez-Blanco Castle was built between 1506 and 1515 after Fajardo received the lordship of the town from Ferdinand II and Isabella, took up residence there and was given the title of Marqués (1507) by Ferdinand, who was then regent. The heraldry of the principal areas of the castle, including the coat of arms of his second wife, Mencía de la Cueva, belongs to this period.
The castle is situated on a hill overlooking the town. An important aspect of this castle is its huge tower, Torre del Homenaje, which is over 20 m high, an emblematic element of the castle and a symbol of the power over the dominion. The stone structure, had wooden stairs, which could be removed in case of danger isolating the upper level as a last defense. There is a vast number of decorative elements topping its battlements. The building has been restored in stages during the second half of the 20th century, including the floors of the towers.
The fortified areas of the castle, particularly the defences and the Late Gothic east gallery of the courtyard, are examples of late 15th-century Spanish art.