Simple reaction time to visual stimuli depends on several stimulus properties. Recently, converging evidence showed that larger stimulus size evokes faster reactions and that this effect seemingly depends on the stimulus’ perceived size rather than on physical stimulus properties. Size–distance scaling usually is regarded as the main functional mechanism underlying size perception. Yet, the role of stimulus depth (distance to a target) has often been neglected in previous studies. Hence, in the present investigation, stimuli were generated using stereo head mounted displays to manipulate stimulus depth. In Experiment 1, a large or small target was presented within the center of a reference plane, either in the same depth plane or displaced (near, far) while participants had to perform a simple reaction time task. At the same time, the target was modulated such that either retinal size was constant or variable across depth planes. In Experiments 2 and 3 the reference plane was shifted along with the target (blocked or on a trial-by-trial basis), while retinal size modulation was equal to Experiment 1. As expected, participants reacted faster to physically larger targets. Also Experiment 1 revealed faster reaction times for closer targets, while the commonly described connection between perceived size (i.e., size–distance scaling) was not apparent in any experiment. Thus, unlike past findings using a virtual three-dimensional task-setting (as induced by binocular disparity) reaction times are not affected by variations of perceived stimulus size.