The Pennyman family originally came from Stokesley but were in Ormesby, North Yorkshire by the early sixteenth century. Robert Pennyman was hanged at York in 1569 for his part in the Pilgrimage of Grace. His older brother, William, had two sons; James Pennyman (d. 1624) who lived in Ormesby and another William Pennyman (d. 1589) who lived in Stokesley. The family expanded the Ormesby estate in the early seventeenth century, through the marriage of James Pennyman's daughter, Elizabeth, into the Strangewayes family and by buying more land in 1600 and leasing the rectory and tithes of Ormesby church in 1613 from Toby Matthews, archbishop of York (Pennyman, Records, p.4; DPE/7/1).
Apart from Elizabeth, James Pennyman (d. 1624) had six sons and three other daughters by his wife, Agnes Burnet of Cleveland, and one illegitimate son, William Pennyman (will dated 1628), who became one of the six clerks in chancery. William married Anne, daughter of Sir Robert Aske of Aughton, and their son, also William (1603-1643), followed his father into law after matriculating from Christ Church, Oxford, in 1623. He was a student of the Inner Temple and was made baronet in 1628 by Charles I. He succeeded to large estates in Yorkshire and built Marske Hall in 1625. He had a successful career (though his sale of land in Ormesby in the 1630s suggests he was heavily indebted), gaining the support of Thomas Wentworth (earl of Strafford) and becoming a bencher of Gray's Inn in 1639. He sat as MP for Richmond in the short and long parliaments of 1640 and was called as a witness during the trial of Strafford. He demonstrated his royalism early by voting against the bill of attainder. He later raised a troop of horse for the king, fought at Edgehill and ultimately was made governor of Oxford, where he died in the epidemic of 1643(Ambler, The old halls and manor houses of Yorkshire, pp. 75-6; U DPE/7/2, U DPE/12/5).
James Pennyman's legitimate line also produced some of the most committed royalists of the English civil wars. His eldest surviving legitimate son, James Pennyman (1608-1655), married first Catherine Kingsley, daughter of the archdeacon of Canterbury and second Joan Smith of London, and by them had a total of twelve children, seven of whom died young. He marched against the Scots with Charles I and he and his oldest son, also James Pennyman (1609-1679), sold part of their estate at Ormesby to William Pennyman's London creditors, Gervase and Jeremy Elwes, to finance their royalist support in the civil wars of the 1640s. A younger son, John Pennyman (1628-1706), also went into the king's service, becoming an ensign in the king's army at 15. With the defeat of Charles I in the first civil war, John Pennyman was forced to flee abroad. His father and brother managed to avoid sequestration of their estates but were crippled by composition fines until they gave in and finally took the covenant (Legard, The Legards, p. 207).
John Pennyman was able to return to London upon payment of the fines and he went into apprenticeship with a royalist wool draper called Mr Fabian. During the 1650s he was one of Christopher Feake's fifth monarchists and joined the Quakers in about 1658. But his religion became too mystical even for the Quakers and he spent the remainder of his life claiming a special portion of 'the inner light' and writing small tracts attacking George Fox and William Penn. He was aided towards the end of his life by his third wife, whom he had married in a clandestine ceremony, much to the irritation of Quaker leadership.
His older brother, James Pennyman, was rewarded for his support during the civil wars by Charles II in 1664 when he was awarded a baronetcy. This passed to his son, Thomas Pennyman (1642-1708), who was High Sheriff of Yorkshire 1702-3 and spent most of his life very involved in local government. He married Frances, daughter of Sir John Lowther of Westmorland.
Their eldest son, James Pennyman (1662-1745), succeeded as third baronet. He married Mary, daughter of Michael Warton of North Bar House in Beverley. Mary was the coheiress of her brother, who was the richest merchant in Beverley, but there were so many heirs that after he died in 1725 it took fifty years for a property settlement to be finalised. Her inheritance enabled the Pennyman family to build a large townhouse at Lairgate in Beverley which became, in the twentieth century, the offices of East Riding County Council. In the meantime, James and Mary Pennyman chose mostly to live at Thornton in Cleveland. Their eldest son, James (born circa 1693), married Dorothy, the daughter of Archbishop Wake. James pre-deceased his father by two years, dying without issue in 1743. Dorothy built the family house at Ormesby, but it was only completed the year of her death in 1754 and it stood empty for sixteen years (Roebuck, Yorkshire baronets, p.300; Pevsner, The buildings of England: Yorkshire, the North Riding, pp.276-7).
When the older James Pennyman died in 1745, the baronetcy passed to his second son (who died unmarried and childless in 1768) and then to his third son, Warton Pennyman, who married Charlotte, the daughter of Sir Charles Hotham of Beverley, Scorborough and South Dalton. The marriage brought £2000 into the family. They had eleven children, nine of whom were female. Their second son died at the age of six leaving their eldest son, another James Pennyman (born circa 1737).
James Pennyman, 6th baronet, carved out a political career. He was MP for Scarborough 1770-1774 and for Beverley 1774-1796. He voted with Rockingham's party consistently, but rarely spoke in the house. On Rockingham's death he supported Shelburne and then Pitt. Rockingham described him as 'A Yorkshire gentleman of very good principles and very large fortune'. The fortune partly came from the Warton inheritance which finally reached the family in the 1770s, though his descendants complained that he was a profligate spender who wasted every penny. In 1770 he also moved into the hall at Ormesby built by his aunt Dorothy sixteen years before and he took the opportunity of buying back the land sold by his forebears to finance their civil war efforts (Roebuck, Yorkshire baronets, p. 289). He married Elizabeth, daughter of Sir Henry Grey and they had ten children. Of six male heirs, only one survived to adulthood - William Henry Pennyman (b.1764). The baronetcy passed to him upon his father's death in 1808, but it also expired with him because he died without issue in 1852. He willed his estate to his sister, Mrs Robinson. However, she died within the year and the baronetcy and estate moved laterally (Namier & Brooke, The House of Commons, iii, pp. 264-5 citing Rockingham MSS; Pennyman, Records, pp.7-8).
The property at Ormesby passed through the female line of Ralph and Bridget Pennyman, to his cousin, James White Worsley (1792-1870), who assumed the name, Pennyman, under the terms of the will. The Worsley branch of the Pennyman's continued to own Ormesby Hall. Some of their land in Normanby passed down through the Consett family (U DPE/6/50, 52; U DPE/12/18) who were cousins by marriage and the papers also reveal that the Pennymans were related by marriage to the Addison family. The Addisons were recusants and in the 1640s and 1650s the Pennymans helped them with financial difficulties (U DPE/6/29, 37-8).