When I define a probabilistic Turing machine I give it a special "coin state" which it enters and magically lands in a special "heads" state and "tails" state uniformly and independently each time. I imagine a computer hooked up to a little box with a coin inside that gets flipped and some sensor or camera determines whether it landed heads or tails.
I have no problems thinking about probabilistic computation just like I have no issues with quantum machines which haven't been built yet or nondeterministic machines which will never exist.
We don't care where those random bits come from as long as they fulfill the right properties. Of course our computers don't have little coin boxes so they generate randomness using pseudorandom generators which don't fulfill all the properties we expect from true randomness. So we developed theories of PRGs and under what assumptions good PRGs exist. Whether we can use them depends on whether we use randomness for searching or hiding.
We can't disprove that BPP = NEXP (everything in nondeterministic exponential time can be solved in probabilistic polynomial time). Then true randomness will give us the secrets of the universe and PRGs won't help much. Random bits would be worth their weight in gold but can we get them? I'd make a fortune selling little coin boxes.