Will swimming make my cold worse?

If it were warm water and nice and humid, I would say yes. but since you say it's cold, I would suggest you skip it