The castle passed into the Seymour family one of whose family members was Jane Seymour, wife of Henry VIII and mother of King Edward VI. Indeed, Edward Seymour was Lord Protector of the young king.
The property saw massive building programmes in both the Tudor periods and the later Jacobite periods and the home that was created was astoundingly beautiful and beyond anything else that had been built in the south west of England. In particular, at a time when glass windows were a luxury, the new parts of the building were full of windows which meant that at night light blazed from the castle in a show of wealth.
By the late 1600's the property had been stripped of anything that could used in the family's other properties elsewhere and it was allowed to fall into ruin. The then head of the family had a Parliamentary career in London as speaker of the House of Commons and had little interest in the castle.
The fortunes of each of the families had waxed and waned but the Seymours were adroit at steering their way through difficult times and still own the property today.