In my opinion polymer clay is not toxic, only fumes when it's baking could be a bit nasty, but only when you overbake clay or bake it on too high temperature. I have been using polymer clay for over a year and haven't notice anything wrong with it. But if you're concerned, you can read this article I found:
Hope it helped.
"Only two things are infinite, the universe and human stupidity, and I'm not sure about the former."