Pseudoinvertible Neural Networks

This work introduces novel architecture components and training procedures to create augmented neural networks with the ability to process data bidirectionally via an end-to-end approximate inverse. We develop pseudoinvertible neural network (PsI-NN) layers which function as drop-in replacements for corresponding convolutional and fully connected layers; by using these, existing architectures gain a pseudoinverse function which, with training, approximately reverses the forward function. For cases where learning both a task and its inverse are necessary or desirable, we show that PsI-NN enabled models match or exceed the quality of results generated by systems that use two separate models while drastically reducing system parameter count. We demonstrate this on two tasks: unpaired image translation and semisupervised classification. In both, PsI-NN greatly reduces parameter count without any loss of output quality; in semisupervised image classification, PsI-NN improves classification performance beyond the baseline autoencoder method. Our approach creates pseudoinvertible versions of existing architectures, circumventing the stringent constraints required to create a true inverse while allowing a single neural network to learn a task in two directions simultaneously.