Why are things that shouldn't go in your mouth so appealing to babies?

Moving has encouraged this little guy to get into and taste all types of things I don't want him to.
While I was cleaning the kitchen of the old apartment I was happy that Wesley was doing so well in the living room alone. Most of the things were our of the apartment so I figured all was well...
then I discovered why he was playing by himself so well. He pulled a roll or toilet paper our the my bag and was chewing on it...gross!

He also found the little clear plastic that hooks a tag to clothing and choked on it. In fact that was the first time he choked.

He pulled a piece off this basket and choked on it. I have had to say prayers of gratitude that I was able to hit his back and it came out. His toys are no longer in that basket.
He chewed on a rawhide bone at the neighbors. Luckily it was on the dog hadn't gotten to yet, but still.
He chewed on closed pocket knives on my key chain. Time to take those off.
He's chewed on shoes.
Bret stopped him right before he ate an ant, and I think he may have eaten one just before that. I had no idea his fine motor skills were that developed.
I keep having to take away carpet shreds from him, and sometimes they make it in his mouth. I don't want to know if he's actually swallowed any.
He's chewed on clean diapers, and thinks I am cruel for keeping the dirty ones away from him.
Power cords are his very favorite, and that scares me.
I am sure there are wonderfully inedible things I am leaving off the list.

Wes, why aren't your toys more appealing to you? I don't mind if you chew on those.

I guess I just have to do my best, pray for protection, and trust in the Lord.