you might be misinformed about the pill making you infertile. that isnt true, all women are different and if you've been on the pill for a long time it just means that it COULD take your body a while to get back to its NATURAL cycle (not fertile means). You could get pregnant within weeks of going off the pill.
When you are trying to get pregnant, you are tracking ovulation. That can be hard to do if your cycle is whack. One month could be 33 days, the next 28, the next 22. So you never know when you ovulate! But that doesn't mean you didn't ovulate or get pregnant.
Don't stress.
If you don't want to get pregnant, I'd stay on the pill and take them very vigilantly (same time everyday! try and be precise! this will help the transition of coming off of them!) and then go off the pill about 3 months prior to trying to concieve so that you can begin learning how to read your body.