Newport's castle was the residence of John de Somerie who married the last Paganell, where the town's name comes from. It was likely to have been erected soon after the Norman Conquest, as William erected castles all over the kingdom. The site was believed to have been on the eminence of the Ouse and Lovat, forming a natural moat around the part that faced the meadow - still known as Castle Meadow.

It is not known whether the castle was one of defence or just residence, but it fell into a state of decay with the abolition of the feudal system and the discovery of gunpowder.