MLSE and MMSE-DFE en/decoding for modulated code coded ISI channels

In this work we present two decoding methods, maximum likelihood sequence estimation (MLSE) and minimum mean-squared-error decision feedback equalizer (MMSE-DFE), and their corresponding code designs of a new transmitter-assisted technique, modulated code (MC) encoding, for mitigating intersymbol interference (ISI) in digital communications. An MC is a convolutional code defined on the real/complex field, and an MC coded system is able to achieve coding gain on an ISI channel as compared to the ideal AWGN channel.; When MLSE is used as MC decoding, the Euclidean free distance between the noiseless received signals is most of the concern, and thus it is maximized during the MC design procedure. We formulate the MC optimization as a quadratic program, where the objective function and the constraints are all quadratic. We then propose a fast searching algorithm of the optimal MC for a given ISI channel. With this algorithm the MC optimization procedure is able to select eligible candidates fast and eliminate catastrophic MCs. Tests show that, as the computational complexity increases exponentially with the code sizes, the MC optimization procedure is very efficient for small size codes.; To search for MC distance spectra and bound error-rates of MLSE detection, we extend the bidirectional algorithm for distance spectra of trellis codes to that of MCs on given ISI channels.; Considering MCs as controlled ISI, we can employ equalization as a suboptimum decoding method with the advantage of a much lower complexity. Known results include Li and Ding's joint transmitter-receiver optimization, in which an MMSE linear equalizer (LE) was employed. Following the well-known MMSE-DFE results for single-input-single-output (SISO) systems, we determine the MMSE solution of the infinite-length DFE coefficients for given MCs and channels. The solution shows that the feedforward filter consists of a matrix filter, matched to the combination of the MC and the channel, followed by an anticausal transversal filter. This MMSE-DFE achieves a mean-squared-error (MSE) less than that of the MMSE-LE unless the combined channel is ISI-free. Based on a lower bound of the above MSE a code design is developed for the MMSE-DFE detection and the solution agrees with the water-pouring argument of power allocation. For practical use, we determine the optimum decision delay of a finite-length MMSE-DFE and give a simple form of least-mean-square (LMS) adaptive DFE. An example of applications on multipath fading channels is also given.