Originally known as Yestred (from the Brythonic Ystrad, meaning strath or dale), the barony of Yester was granted by King William the Lion to Hugo de Giffard, a Norman immigrant given land in East Lothian during the reign of King David I. Latin language documents note the location as Zester in vital records.
The original stone keep, built before 1267, is generally considered to be by Sir Hugo de Giffard. A grandson of the first Laird of Yester, he served as a guardian of the young Alexander III of Scotland, and was by repute a magician and necromancer. Alexander III is known to have been at Yester on and around 24 May 1278, where he corresponded with Edward I of England. Following the Scots Wars of Independence, Yester was rebuilt as a castle of enceinte.
In 1298, during the Battle of Falkirk, Alexander de Welles, Master of Torphichen Preceptory, was killed. Based on the heraldic evidence there is very little doubt that Alexander de Welles was a member of the Lincolnshire Welle(s) family. Also at Falkirk was Adam de Welle(s) of Lincolnshire and of the Castle of Yester in Lothian, to whom the English King Edward I, during his occupation of Scotland, gave various properties confiscated from the Lothian nobility and gentry.
In 1357, there being no male line left of the Giffards, Joanna, a daughter and co-heiress, of the last Sir Hugo de Giffard, married Sir William [or Thomas] de la Haye of Peebles, of Locherworth (Locherwart), the Sheriff of Peebles. He was invested with the barony and lands of Yester through his wife. The barony has stayed with the Hay family ever since.
John Hay of Yester was in 1487 created a Lord of Parliament, as the first Lord Hay of Yester. In 1513 during the disastrous Battle of Flodden, John, second Lord Hay, was killed along with a great proportion of the country's fighting men. In May 1544 during the conflict known as the Rough Wooing, the castle, village, and harvest were burnt by the English army returning from the burning of Edinburgh.
John, 4th Lord Hay defended the castle from an English force in 1547, and was captured later that year at the Battle of Pinkie and held in the Tower of London for three years. In February 1548 the English commander Grey of Wilton captured the castle and put George Douglas of Pittendreich in charge of Yester and Dalkeith. French soldiers taken at Yester were used at hostages by the English. According to Ulpian Fulwell, the captured garrison was mostly Scottish or Spanish. Two men suspected of shouting insults defaming Edward VI were made to fight a duel at the market place of Haddington.
1557 saw the death of the 4th Lord Hay. His son William abandoned the castle and moved into a new tower house on the site of the present day mansion of Yester House. In 1646 the 8th Lord John Hay was created Marquess of Tweeddale. The castle gradually fell into disrepair, and by the late 17th century was in a very parlous state, the stones having been much quarried for building material. Although the castle almost disappeared completely, Sir Hugo's original Goblin Ha' was tenanted by the Marquess' falconer until 1737. Yester House with its Adam interiors was sold in 1972 to Italian/American operatic composer Gian-Carlo Menotti then left to his son, Chip, who sold it in 2013.
