Capacity-Driven Autoencoders for Commun...

The autoencoder concept has fostered the reinterpretation and the design of moderncommunication systems. It consists of an encoder, a channel and a decoder block that modify theirinternal neural structure in an end-to-end learning fashion. However, the current approach to train anautoencoder relies on the use of the cross-entropy loss function. This approach can be prone to overfittingissues and often fails to learn an optimal system and signal representation (code). In addition, less is knownabout the autoencoder ability to design channel capacity-approaching codes, i.e., codes that maximize theinput-output mutual information under a certain power constraint. The task being even more formidablefor an unknown channel for which the capacity is unknown and therefore it has to be learnt. In thispaper, we address the challenge of designing capacity-approaching codes by incorporating the presenceof the communication channel into a novel loss function for the autoencoder training. In particular, weexploit the mutual information between the transmitted and received signals as a regularization termin the cross-entropy loss function, with the aim of controlling the amount of information stored. Byjointly maximizing the mutual information and minimizing the cross-entropy, we propose a theoreticalapproach that a) computes an estimate of the channel capacity and b) constructs an optimal coded signalapproaching it. Theoretical considerations are made on the choice of the cost function and the ability ofthe proposed architecture to mitigate the overfitting problem. Simulation results offer an initial evidenceof the potentiality of the proposed method.