They're a problem because we're overconsuming them. They're made from nonrenewable sources, require a lot of water to make, and emissions from plastic equals nearly 1.8 billion metric tons of CO2.
Tap water is generally a better option because an estimated 25 percent or more of bottled water is just tap water. Also, tap water is monitored constantly whereas bottled water goes through no kind of regulation or monitoring.
Each come with pros and cons. For instance steel bottles are usually most expensive, but are also most durable. Glass bottles can be infinitely recycled and doesn't alter the taste of your drink. Plastic bottles are affordable and durable.