Adverse circumstances, such as extreme weather events, can cause significant disruptions to normal operation of electric distribution systems (DSs), which include isolating parts of the DS due to damaged transmission equipment. In this article, we consider the problem of load restoration in a microgrid (MG) that is islanded from the upstream DS. The MG contains sources of distributed generation, such as microturbines and renewable energy sources as well as energy storage systems (ESSs). We formulate the load restoration task as a nonconvex optimization problem. This problem embodies the physics of the MG by leveraging a branch flow model while incorporating salient phenomena in islanded MGs, such as the need for internal frequency regulation and complementarity requirements arising in ESS operations. Since the formulated optimization problem is nonconvex, we introduce a convex relaxation that can be solved through model predictive control as a baseline method. However, in order to solve the problem considering its full nonconvexity, we leverage a policy-learning method called constrained policy optimization, a tailored version of which is used as our proposed algorithm. The aforementioned approaches, along with an additional deep learning method, are compared through extensive simulations.