For the first time, we present Neural Inverse Space Mapping (NISM) optimization for EM-based design of microwave structures. The inverse of the mapping from the fine to the coarse model parameter spaces is exploited for the first time in a Space Mapping algorithm. NISM optimization does not require: up-front EM simulations, multipoint parameter extraction or frequency mapping. The inverse of the mapping is approximated by a neural network whose generalization performance is controlled through a network growing strategy. We contrast our new algorithm with Neural Space Mapping (NSM) optimization.