Glass is a poor insulator, and windows are a prime entry point for cold or hot air. Especially if they are dated or have not been properly sealed the outside air will seep into the indoor conditioned ...
You probably already have a roll of bubble wrap sat around the house.