Try and get to a couple of medieval churches and/or old monasteries. The ones in Norfolk are better than anywhere else in the country in my very biased opinion.
My favourites around there are:
Mary Magdalane, Warham - amazing medieval stained glass from the Norwich school which was one of the most prolific in Europe.
Binham Priory - Originally a priory from the 12th century which is still very spectacular inside. Early examples of English gothic tracery - predates Westminster Abbey. Interesting also to see the pictures of saints defaced and painted over with bible verses in English from the reformation.
Burnham Deepdale and Burnham Norton - both with late Saxon/ early Norman round towers and lots of interesting monuments inside.
Bale - a bit further away but some of the best medieval glass I've seen anywhere in England.