In this paper we consider the situation in which a robot is deployed in an unknown scenario and has to explore the entire environment without possibility of measuring its absolute position. The robot can take relative position measurements (from odometry and from place revisiting episodes) and can then estimate autonomously its trajectory. Therefore, the quality of the resulting estimate depends on the motion strategy adopted by the robot. The problem of uncertainty-constrained exploration is then to explore the environment while satisfying given bounds on the admissible uncertainty in the estimation process. We adopt a moving horizon strategy in which the robot plans its motion T steps ahead. Our formulation leads to a mixed-integer linear problem that has several desirable properties: (i) it guarantees that the robot motion is collision free, (ii) it guarantees that the uncertainty constraints are met, (iii) it enables the design of algorithms that efficiently solve moderately sized instances of the exploration problem. We elucidate on the proposed formulation with numerical experiments.