This question was one of the Big Ones for me, that helped drive me away from fundamentalist Christianity many years ago. Catholicism--not exactly what you think of as fundy nowadays--came up with the idea of limbo, a place for newborn babies who'd died before being baptized. It wasn't hell, exactrly, but it wasn't heaven, and it assuredly was not a concept founded on the scriptures!
But the fundamentalist worldview, as I understand it, admits, first, that ONLY baptized Christians can get into heaven ("I am the Way, the Truth, and the Life. No one comes to the Father, but by me." --John 14:6 "He that believeth and is baptized shall be saved; but he that believeth not shall be damned."--Mark 16:16.) There is absolutely NO wiggle room here for unbaptized babies, innocent children, or pagans who've never heard the Gospel.
Some groups soften the above with the sentiment that God will take care of the ones who fall through the cracks, or by saying that we can't judge, God is responsible for it all, and His ways are mysterious, not fathomable to mere humans. Granted. I still feel the whole doctrine of hellfire is an afront to civilization and to all men of good will, and I find the fact that some DO condfemn unbaptized children to hell starkly appalling.