I would like to find a device to measure humidity levels where I keep my guitars, but I don't even know what to look for. Is there such a device and if so, what is it called?
What you're looking for is called a hygrometer. You should be able to find one at your local hardware store, look for them near temperature and weather measurement devices.
For example, here's an inexpensive unit available at Walmart:
Ideally you will want to try and achieve humidity between 30 and 50 percent. How to do that depends on the climate in your area, as well as other factors such as your home's heating and cooling methods.