We own a 3-6 month swim suit. We lived in Phoenix when my son was born, and it was so dang hot that we went to the public pool many times. The first time I took him into the pool he was 6 weeks old. He didn't like it at all. But he loved me carrying him around in the pool above the water.
When he was 3 months old, we took him to the neighbor's pool and he LOVED it. We have a "Baby Float" which is a ring floaty with straps underneath so he won't fall out. We always kept a hand on the float, but he didn't mind the waves or little splashes or anything.
We went camping with our oldest son when he was 2 months old. The campground had several hot tubs of different temperatures. The coolest one was like bathwater. He LOVED it, and was floating on his back with our hand only under his head.
We went swimming every day when we were at a hotel with an indoor pool when he was 4 months old.
I've never heard anything about not taking a kid to the pool. Actually, there are swim lessons for tiny infants to get them used to the water. At this age, they actually possess the innate ability to swim.
So I wouldn't worry about it.
Oh, and by the way, those baby floaties are less than $5. Absolutely worth it!