From f56960ef48a6772b036033d9f6c2cbf18c146748 Mon Sep 17 00:00:00 2001 From: Andreas Tsouchlos Date: Sun, 25 Aug 2024 00:05:21 +0200 Subject: [PATCH] Update text to text of final submission --- letter.tex | 906 +++++++++++++++++++++++++++++------------------------ 1 file changed, 497 insertions(+), 409 deletions(-) diff --git a/letter.tex b/letter.tex index 0d83258..6b0a84c 100644 --- a/letter.tex +++ b/letter.tex @@ -34,10 +34,10 @@ % -\newcommand{\reviewone}[1]{{\textcolor{KITblue}{#1}}} -\newcommand{\reviewtwo}[1]{{\textcolor{KITpalegreen}{#1}}} -\newcommand{\reviewthree}[1]{{\textcolor{KITred}{#1}}} - +\newcommand{\reviewone}[1]{{\textcolor{black}{#1}}} +\newcommand{\reviewtwo}[1]{{\textcolor{black}{#1}}} +\newcommand{\reviewthree}[1]{{\textcolor{black}{#1}}} +\newcommand{\reviewfour}[1]{{\textcolor{KITorange}{#1}}} % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -47,7 +47,7 @@ \newif\ifoverleaf -%\overleaftrue % When enabled, this option allows the document to be compiled +\overleaftrue % When enabled, this option allows the document to be compiled % on overleaf: % - common.tex is sourced from a different directory % - TikZ Externalization is disabled @@ -69,7 +69,7 @@ \pgfplotsset{colorscheme/cel} \newcommand{\figwidth}{\columnwidth} -\newcommand{\figheight}{0.7\columnwidth} +\newcommand{\figheight}{0.65\columnwidth} \pgfplotsset{ FERPlot/.style={ @@ -102,10 +102,16 @@ \begin{document} -\title{List-based Optimization of Proximal Decoding for Linear Block Codes} +\title{List-based Optimization of Proximal Decoding for LDPC Codes }%Linear Block Codes} +%\author{Author 1, Author 2, Author 3} \author{Andreas Tsouchlos, Holger Jäkel, and Laurent Schmalen -\thanks{The authors are with the Communications Engineering Lab (CEL), Karlsruhe Institute of Technology (KIT), corresponding author: \texttt{holger.jaekel@kit.edu}}} +\thanks{The authors are with the Communications Engineering Lab (CEL), Karlsruhe Institute of Technology (KIT), corresponding author: \texttt{holger.jaekel@kit.edu} +% +This work has received funding in part from the European Research Council +(ERC) under the European Union’s Horizon 2020 research and innovation +program (grant agreement No. 101001899).} + } \markboth{IEEE Communications Letters}{List-based Optimization of Proximal Decoding for Linear Block Codes} @@ -120,7 +126,7 @@ \begin{abstract} -In this paper, the proximal decoding algorithm described in, e.g., \cite{proximal_paper}, is considered within the +In this paper, the proximal decoding algorithm is considered within the context of \textit{additive white Gaussian noise} (AWGN) channels. An analysis of the convergence behavior of the algorithm shows that proximal decoding inherently enters an oscillating behavior of the estimate @@ -152,11 +158,6 @@ Optimization-based decoding, Proximal decoding, ML-in-the-list. %%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Introduction} -\reviewone{Test1} -\reviewtwo{Test2} -\reviewthree{Test3} - - \IEEEPARstart{C}{hannel} coding using binary linear codes is a way of enhancing the reliability of data by detecting and correcting any errors that may occur during its transmission or storage. @@ -168,14 +169,14 @@ decoding. While the established decoders for LDPC codes, such as belief propagation (BP) and the min-sum algorithm, offer good decoding performance, they are generally not optimal and exhibit an error floor for high -\textit{signal-to-noise ratios} (SNRs) \cite{channel_codes_book}, making them +\textit{signal-to-noise ratios} (SNRs) \cite{channel_codes_book}, rendering them inadequate for applications with extreme reliability requirements. Optimization based decoding algorithms are an entirely different way of -approaching the decoding problem; +approaching the decoding problem: they map the decoding problem onto an optimization problem in order to leverage the vast knowledge from the field of optimization theory. -A number of different such algorithms have been introduced. +A number of different such algorithms have been introduced in the literature. The field of \textit{linear programming} (LP) decoding \cite{feldman_paper}, for example, represents one class of such algorithms, based on a relaxation of the \textit{maximum likelihood} (ML) decoding problem as a linear program. @@ -186,7 +187,7 @@ Wadayama \textit{et al.} \cite{proximal_paper}. Proximal decoding relies on a non-convex optimization formulation of the \textit{maximum a posteriori} (MAP) decoding problem. -The aim of this work is to improve upon the performance of proximal decoding by +The aim of this work is to improve the performance of proximal decoding by first presenting an analysis of the algorithm's behavior and then suggesting an approach to mitigate some of its flaws. This analysis is performed for @@ -194,7 +195,7 @@ This analysis is performed for We first observe that the algorithm initially moves the estimate in the right direction; however, in the final steps of the decoding process, convergence to the correct codeword is often not achieved. -Subsequently, we attributed this behavior to the nature +Subsequently, we attribute this behavior to the nature of the decoding algorithm itself, comprising two separate gradient descent steps working adversarially. @@ -204,7 +205,8 @@ In this additional step, the components of the estimate with the highest probability of being erroneous are identified. New codewords are then generated, over which an ``ML-in-the-list'' \cite{ml_in_the_list} decoding is performed. -A process to conduct this identification is proposed in this paper. +The main point of the paper at hand is to improve list generation such that +it is especially tailored to the nature of proximal decoding. Using the improved algorithm, a gain of up to $\SI{1}{dB}$ can be achieved compared to conventional proximal decoding, depending on the decoder parameters and the code. @@ -213,6 +215,7 @@ depending on the decoder parameters and the code. %%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Preliminaries} + %%%%%%%%%%%%%%%%%%%%% \subsection{Notation} @@ -228,16 +231,14 @@ number of parity-checks: \mathcal{C} := \left\{ \boldsymbol{c} \in \mathbb{F}_2^n : \boldsymbol{H}\boldsymbol{c}^\text{T} = \boldsymbol{0} \right\} \end{align*} -% -The check nodes $j \in \mathcal{J}:=\left\{1, \ldots, m\right\}$ each -correspond to a parity check, i.e., a row of $\boldsymbol{H}$. -The variable nodes $i \in \mathcal{I}:=\left\{1, \ldots, n\right\}$ correspond -to the components of a codeword being subjected to a parity check, i.e., -to the columns of $\boldsymbol{H}$. -The neighborhood of a parity check $j$, i.e., the set of indices of components +The check nodes indexed by $j \in \mathcal{J}:=\left\{1, \ldots, m\right\}$ +correspond to the parity checks, i.e., to the rows of $\boldsymbol{H}$. +The variable nodes indexed by $i \in \mathcal{I}:=\left\{1, \ldots, n\right\}$ correspond +to the components of a codeword, i.e., to the columns of $\boldsymbol{H}$. +The neighborhood of a parity check $j$, i.e., the set of component indices relevant for the according parity check, is denoted by -$\mathcal{N}_c(j) := \left\{i \in \mathcal{I}: \boldsymbol{H}\negthinspace_{j,i} = 1 \right\}, +$\mathcal{N}_\mathrm{c}(j) := \left\{i \in \mathcal{I}: \boldsymbol{H}\negthinspace_{j,i} = 1 \right\}, \hspace{2mm} j \in \mathcal{J}$. In order to transmit a codeword $\boldsymbol{c} \in \mathbb{F}_2^n$, it is @@ -246,61 +247,69 @@ $\boldsymbol{x} = 1 - 2\boldsymbol{c}$, with $ \boldsymbol{x} \in \left\{\pm 1\right\}^n$, which is then transmitted over an AWGN channel. The received vector $\boldsymbol{y} \in \mathbb{R}^n$ is decoded to obtain an -estimate of the transmitted codeword, denoted as -$\hat{\boldsymbol{c}} \in \mathbb{F}_2^n$. +estimate $\hat{\boldsymbol{c}} \in \mathbb{F}_2^n$ of the transmitted codeword. A distinction is made between $\boldsymbol{x} \in \left\{\pm 1\right\}^n$ and $\tilde{\boldsymbol{x}} \in \mathbb{R}^n$, -the former denoting the BPSK symbols transmitted over the channel and +the former denoting the transmitted BPSK symbols and the latter being used as a variable during the optimization process. -The posterior probability of having transmitted $\boldsymbol{x}$ when receiving -$\boldsymbol{y}$ is expressed as a \textit{probability mass function} (PMF) -$P_{\boldsymbol{X}\mid\boldsymbol{Y}}(\boldsymbol{x} \mid \boldsymbol{y})$. -Likewise, the likelihood of receiving $\boldsymbol{y}$ upon transmitting -$\boldsymbol{x}$ is expressed as a \textit{probability density function} (PDF) +The likelihood of receiving $\boldsymbol{y}$ upon transmitting +$\boldsymbol{x}$ is expressed by the \textit{probability density function} (PDF) $f_{\boldsymbol{Y}\mid\boldsymbol{X}}(\boldsymbol{y} \mid \boldsymbol{x})$. + %%%%%%%%%%%%%%%%%%%%% \subsection{Proximal Decoding} - -Proximal decoding was proposed by Wadayama et al. as a novel formulation -of optimization-based decoding \cite{proximal_paper}. With proximal decoding, the proximal gradient method \cite{proximal_algorithms} is used to solve a non-convex optimization formulation of the MAP decoding problem. - With the equal prior probability assumption for all codewords, MAP and ML decoding are equivalent and, specifically for AWGN channels, correspond to a nearest-neighbor decision. For this reason, decoding can be carried out using a figure of merit that describes the distance from a given vector to a codeword. One such expression, formulated under the assumption of BPSK, is the -\textit{code-constraint polynomial} \cite{proximal_paper} +\textit{code-constraint polynomial} +\reviewone{ +defined in +} +\cite{proximal_paper} % \begin{align*} h( \tilde{\boldsymbol{x}} ) = \underbrace{\sum_{i=1}^{n} \left( \tilde{x}_i^2-1 \right) ^2}_{\text{Bipolar constraint}} + \underbrace{\sum_{j=1}^{m} \left[ - \left( \prod_{i\in \mathcal{N}_c \left( j \right) } \tilde{x}_i \right) + \left( \prod_{i\in \mathcal{N}_\mathrm{c} \left( j \right) } \tilde{x}_i \right) -1 \right] ^2}_{\text{Parity constraint}} .\end{align*}% % -Its intent is to penalize vectors far from a codeword. -It comprises two terms: one representing the bipolar constraint due to transmitting BPSK -and one representing the parity constraint, incorporating all -information regarding the code. +Its intent is to penalize vectors far from a codeword +\reviewone{ +and, thus, it serves as an objective function describing the quality of possible +estimates. +Please note that all valid codewords are local minima of $h(\tilde{\boldsymbol{x}})$ +}. +The code-constraint polynomial comprises two terms: the first part is +representing the bipolar constraint due to using BPSK, whereas the second part +is representing the parity constraint, incorporating all information regarding +the code. +\reviewthree{ +Please note that the first part of the code-constraint polynomial +may be easily adapted to higher order constellations, whereas the second part of +the code-constraint polynomial requires bit values in $\mathbbm{F}_2$. +This can be achieved by employing a bit-metric decoder. +} -The channel model can be considered using the negative log-likelihood -% -\begin{align*} - L \mleft( \boldsymbol{y} \mid \tilde{\boldsymbol{x}} \mright) = -\ln\mleft( +The channel can be characterized using the negative log-likelihood +$ + L \mleft( \boldsymbol{y} \mid \tilde{\boldsymbol{x}} \mright) = -\ln ( f_{\boldsymbol{Y} \mid \tilde{\boldsymbol{X}}} \mleft( - \boldsymbol{y} \mid \tilde{\boldsymbol{x}} \mright) \mright) -.\end{align*} -% -Then, the information about the channel and the code are consolidated in the objective -function \cite{proximal_paper} + \boldsymbol{y} \mid \tilde{\boldsymbol{x}} \mright)) +$ +. +Then, the information about the channel and the code are consolidated in the +objective function \cite{proximal_paper} % \begin{align*} g \mleft( \tilde{\boldsymbol{x}} \mright) @@ -309,41 +318,49 @@ function \cite{proximal_paper} \hspace{5mm} \gamma > 0% .\end{align*} % -The objective function is minimized using the proximal gradient method, which -amounts to iteratively performing two gradient-descent steps \cite{proximal_paper} -with the given objective function and considering AWGN channels. -To this end, two helper variables, $\boldsymbol{r}$ and $\boldsymbol{s}$, are +The objective function $g \mleft( \tilde{\boldsymbol{x}} \mright)$ is minimized +using the proximal gradient method, which amounts to iteratively performing two +gradient-descent steps \cite{proximal_paper} with the given objective function +in AWGN channels. +To this end, two helper variables $\boldsymbol{r}$ and $\boldsymbol{s}$ are introduced, describing the result of each of the two steps: % \begin{alignat}{3} \boldsymbol{r} &\leftarrow \boldsymbol{s} - - \omega \mleft( \boldsymbol{s} - \boldsymbol{y} \mright) - \hspace{5mm }&&\omega > 0 \label{eq:r_update}\\ + - \omega \nabla L\mleft( \boldsymbol{y} \mid \boldsymbol{s} \mright), + \hspace{5mm }&&\omega > 0, \label{eq:r_update}\\ \boldsymbol{s} &\leftarrow \boldsymbol{r} - \gamma \nabla h\mleft( \boldsymbol{r} \mright), \hspace{5mm} &&\gamma > 0 \label{eq:s_update} .\end{alignat} % -An equation for determining $\nabla h(\boldsymbol{r})$ is given in -\cite{proximal_paper}, where it is also proposed to initialized $\boldsymbol{s}=\boldsymbol{0}$. -It should be noted that the variables $\boldsymbol{r}$ and $\boldsymbol{s}$ -represent $\tilde{\boldsymbol{x}}$ during different -stages of the decoding process. +Derivation of +$\nabla L\mleft( \boldsymbol{y} \mid \boldsymbol{s} \mright) = \boldsymbol{s} - \boldsymbol{y}$ +for AWGN and an equation for determining $\nabla h(\boldsymbol{r})$ are given +in \cite{proximal_paper}, where it is also proposed to initialize +$\boldsymbol{s}=\boldsymbol{0}$. +It should be noted that $\boldsymbol{r}$ and $\boldsymbol{s}$ represent +$\tilde{\boldsymbol{x}}$ during different stages of the decoding process. As the gradient of the code-constraint polynomial can attain very large values -in some cases, an additional step is introduced in \cite{proximal_paper} to ensure numerical stability: -every estimate $\boldsymbol{s}$ is projected onto +in some cases, an additional step is introduced in \cite{proximal_paper} to +ensure numerical stability: +every estimate $\boldsymbol{s}$ is projected onto the hypercube $\left[-\eta, \eta\right]^n$ by a projection -$\Pi_\eta : \mathbb{R}^n \rightarrow \left[-\eta, \eta\right]^n$, where $\eta$ -is a positive constant larger than one, e.g., $\eta = 1.5$. -The resulting decoding process as described in \cite{proximal_paper} is -presented in Algorithm \ref{alg:proximal_decoding}. +$\Pi_\eta : \mathbb{R}^n \rightarrow \left[-\eta, \eta\right]^n$ +\reviewone{ +defined as component-wise clipping, i.e., +$\Pi_\eta(x_i) = \arg \min_{-\eta\leq \xi \leq \eta } |x_i-\xi|$ +as in \cite{proximal_paper}, +} +where $\eta$ is a positive constant larger than one, e.g., $\eta = 1.5$. +The resulting decoding process is given in Algorithm \ref{alg:proximal_decoding}. \begin{algorithm} - \caption{Proximal decoding algorithm for an AWGN channel \cite{proximal_paper}.} + \caption{Proximal decoding in AWGN \cite{proximal_paper}} \label{alg:proximal_decoding} - \begin{algorithmic} + \begin{algorithmic}[1] \STATE $\boldsymbol{s} \leftarrow \boldsymbol{0}$ \STATE \textbf{for} $K$ iterations \textbf{do} \STATE \hspace{5mm} $\boldsymbol{r} \leftarrow \boldsymbol{s} - \omega \left( \boldsymbol{s} - \boldsymbol{y} \right) $ @@ -351,48 +368,43 @@ presented in Algorithm \ref{alg:proximal_decoding}. \STATE \hspace{5mm} $\boldsymbol{\hat{c}} \leftarrow \mathbbm{1}_{\left\{ \boldsymbol{s} \le 0 \right\}}$ \STATE \hspace{5mm} \textbf{if} $\boldsymbol{H}\boldsymbol{\hat{c}} = \boldsymbol{0}$ \textbf{do} \STATE \hspace{10mm} \textbf{return} $\boldsymbol{\hat{c}}$ - \STATE \hspace{5mm} \textbf{end if} - \STATE \textbf{end for} \STATE \textbf{return} $\boldsymbol{\hat{c}}$ \end{algorithmic} \end{algorithm} - %%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Improved algorithm} + %%%%%%%%%%%%%%%%%%%%% \subsection{Analysis of the Convergence Behavior} In Fig. \ref{fig:fer vs ber}, the \textit{frame error rate} (FER), \textit{bit error rate} (BER), and \textit{decoding failure rate} (DFR) of -proximal decoding are shown for an LDPC code with $n=204$ and $k=102$ -\cite[204.33.484]{mackay}. -Hereby, a \emph{decoding failure} is defined as returning a \emph{non valid codeword}, i.e., as non-convergence of the algorithm. -The parameters chosen for this simulation are $\gamma=0.05, \omega=0.05, -\eta=1.5$ and $K=200$ ($K$ describing the maximum number of iterations). -They were determined to offer the best performance in a preliminary examination, -where the effect of changing multiple parameters was simulated over a wide -range of values. -It is apparent that the DFR completely dominates the FER after a certain SNR. -This means that most frame errors are not due to the algorithm converging -to the wrong codeword, but due to the algorithm not converging at all. +proximal decoding are shown for the LDPC code [204.33.484] \cite{mackay} with +$n=204$ and $k=102$. +Hereby, a \emph{decoding failure} is defined as returning a +\emph{non valid codeword}, i.e., as non-convergence of the algorithm. +The parameters chosen in this simulation are $\gamma=0.05$, $\omega=0.05$, +$\eta=1.5$, and $K=200$ ($K$ describing the maximum number of iterations). +They +\reviewfour{adhere to \cite{proximal_paper} and} +were determined to offer the best performance in a preliminary examination, where +the effect of changing multiple parameters was simulated over a wide range of +values. +It is apparent that the DFR completely dominates the FER for sufficiently high +SNR. +This means that most frame errors are not due to the algorithm converging to the +wrong codeword, but due to the algorithm not converging at all. -As proximal decoding is an optimization-based decoding method, one possible -explanation for this effect might be that during the decoding process, convergence -to the final codeword is often not achieved, although the estimate is moving into -the right direction. -This would suggest that most frame errors occur due to only a few incorrectly -decoded bits.% -% -\begin{figure}[t] +\begin{figure}[htb] \centering - \ifoverleaf - \includegraphics{figs/letter-figure0.pdf} - \else + \ifoverleaf + \includegraphics{figs/letter-figure0.pdf} + \else \begin{tikzpicture} \begin{axis}[ grid=both, @@ -400,7 +412,7 @@ decoded bits.% ymode=log, xmin=1, xmax=8, ymax=1, ymin=1e-6, - % ytick={1e-0, 1e-2, 1e-4, 1e-6}, + ytick={1e-0, 1e-2, 1e-4, 1e-6}, width=\figwidth, height=\figheight, legend pos = south west, @@ -429,33 +441,54 @@ decoded bits.% \caption{FER, DFR, and BER for $\left( 3, 6 \right)$-regular LDPC code with $n=204, k=102$ \cite[\text{204.33.484}]{mackay}. - Parameters used for simulation: $\gamma =0.05,\omega = 0.05, + Parameters: %used for simulation: + $\gamma =0.05,\omega = 0.05, \eta = 1.5, K=200$. } \label{fig:fer vs ber} \end{figure}% % +As proximal decoding is an optimization-based decoding method, one possible +explanation for this effect might be that during the decoding process, convergence +to the final codeword is often not achieved, although the estimate is moving +into the right direction. +This would suggest that most frame errors occur due to only a few incorrectly +decoded bits. An approach for lowering the FER might then be to add an ``ML-in-the-list'' \cite{ml_in_the_list} step to the decoding process shown in Algorithm \ref{alg:proximal_decoding}. -This step consists in determining the $N \in \mathbb{N}$ most probably -erroneous bit positions $\mathcal{I}'$, generating a list of $2^N$ codeword candidates out of the current estimate $\hat{\boldsymbol{c}}$ with bits in $\mathcal{I}'$ adopting all possible values, i.e., $\mathcal{L}'=\left\{ \hat{\boldsymbol{c}}'\in\mathbb{F}_2^n: \hat{c}'_i=\hat{c}_i, i\notin \mathcal{I}'\text{ and } \hat{c}'_i\in\mathbb{F}_2, i\in \mathcal{I}' \right\}$, and performing ML decoding on this list. +This step consists in determining the $N \in \mathbb{N}$ positions +$\mathcal{I}'\subset \mathcal{I}$ of bits that are most probably erroneous, +generating a list of $2^N$ codeword candidates out of the current estimate +$\hat{\boldsymbol{c}}$ with bits in $\mathcal{I}'$ adopting all possible values,i.e., +% +\begin{equation}\label{eq:def:L_prime} + \mathcal{L}'=\left\{ \hat{\boldsymbol{c}}'\in\mathbb{F}_2^n: \hat{c}'_i=\hat{c}_i, i\notin \mathcal{I}'\text{ and } \hat{c}'_i\in\mathbb{F}_2, i\in \mathcal{I}' \right\}, +\end{equation} +% +and performing ML decoding on this list. +\reviewone{ +Whereas list decoding is usually based on the analysis of received +values, e.g., ML-in-the-list decoding or Chase decoding \cite{chase_decoding}, +the following consideration aims at generating this list by exploiting +characteristic properties of proximal decoding. +} -This approach crucially relies on identifying the most probably erroneous bits. +The aforementioned process crucially relies on identifying the positions of bits +that are most likely erroneous. Therefore, the convergence properties of proximal decoding are investigated. -Considering (\ref{eq:s_update}) and (\ref{eq:r_update}), Fig. -\ref{fig:grad} shows the two gradients along which the minimization is -performed for a repetition code with $n=2$. -It is apparent that a net movement will result as long as the two gradients -have a common component. +Fig. \ref{fig:grad} shows the two gradients performed for a repetition code with +$n=2$. +It is apparent that a net movement will result as long as the two gradients have +a common component. As soon as this common component is exhausted, they will work in opposing directions resulting in an oscillation of the estimate. This behavior supports the conjecture that the reason for the high DFR is a failure to converge to the correct codeword in the final steps of the -optimization process.% +optimization process. % -\begin{figure}[t] +\begin{figure}[htb] \centering \ifoverleaf @@ -468,7 +501,7 @@ optimization process.% ylabel={$\tilde{x}_2$}, y label style={at={(axis description cs:-0.06,0.5)},anchor=south}, width=\figwidth, - height=\figheight, + height=0.6\columnwidth, grid=major, grid style={dotted}, view={0}{90}] \addplot3[point meta=\thisrow{grad_norm}, @@ -496,7 +529,7 @@ optimization process.% legend style={draw=white!15!black, legend cell align=left, empty legend, - at={(0.9775,0.97)},anchor=north east}] + at={(0.9775,0.77)},anchor=north east}] \addlegendimage{mark=none} \addlegendentry{ $\nabla L\left(\boldsymbol{y} @@ -506,7 +539,7 @@ optimization process.% \end{tikzpicture} \fi - \vspace{3mm} + \vspace{2mm} \ifoverleaf \includegraphics{figs/letter-figure2.pdf} @@ -515,7 +548,7 @@ optimization process.% \begin{axis}[xmin = -1.25, xmax=1.25, ymin = -1.25, ymax=1.25, width=\figwidth, - height=\figheight, + height=0.6\columnwidth, xlabel={$\tilde{x}_1$}, ylabel={$\tilde{x}_2$}, y label style={at={(axis description cs:-0.06,0.5)},anchor=south}, @@ -546,7 +579,7 @@ optimization process.% legend style={draw=white!15!black, legend cell align=left, empty legend, - at={(0.9775,0.97)},anchor=north east}] + at={(0.9775,0.77)},anchor=north east}] \addlegendimage{mark=none} \addlegendentry{$\nabla h\left(\tilde{\boldsymbol{x}}\right)$}; \end{axis} @@ -559,20 +592,17 @@ optimization process.% Shown for $\boldsymbol{y} = \begin{pmatrix} -0.5 & 0.8 \end{pmatrix}$. } \label{fig:grad} -\end{figure}% -% +\end{figure} -In Fig. \ref{fig:prox:convergence_large_n}, we consider only component -$\left(\tilde{\boldsymbol{x}}\right)_1$ of the estimate during a -decoding operation for the LDPC code used also for Fig. 1. -Two qualities may be observed. -First, we observe that the average absolute values of the two gradients are equal, -however, they have opposing signs, -leading to the aforementioned oscillation. -Second, the gradient of the code constraint polynomial itself starts to -oscillate after a certain number of iterations.% -% -\begin{figure} +In Fig. \ref{fig:prox:convergence_large_n}, we show the component $\tilde{x}_1$ +and corresponding gradients during decoding for the [204.33.484] LDPC code. +We observe that both gradients start oscillating after a certain number of +iterations. +Furthermore, it can be observed the both gradients have approximately equal +average magnitudes, but possess opposing signs, leading to an oscillation of +$\tilde{x}_1$. + +\begin{figure}[htb]%[H] \centering \ifoverleaf @@ -603,38 +633,64 @@ oscillate after a certain number of iterations.% table [col sep=comma, x=k, y=grad_h_0, discard if gt={k}{300}] {res/extreme_components_20433484_combined.csv}; - \addlegendentry{$\left(\tilde{\boldsymbol{x}}\right)_1$} + \addlegendentry{$\tilde{x}_1$} \addlegendentry{$\left(\nabla L\right)_1$} \addlegendentry{$\left(\nabla h\right)_1$} \end{axis} \end{tikzpicture} \fi - \caption{Visualization of component $\left(\tilde{\boldsymbol{x}}\right)_1$ + \caption{Visualization of component $\tilde{x}_1$ for a decoding operation for a (3,6) regular LDPC code with $n=204, k=102$ \cite[\text{204.33.484}]{mackay}. - Parameters used for simulation: $\gamma = 0.05, \omega = 0.05, - \eta = 1.5, E_b/N_0 = \SI{4}{dB}$. + Parameters: + $\gamma = 0.05, \omega = 0.05, + \eta = 1.5, E_b/N_0 = \SI{6}{dB}$. } \label{fig:prox:convergence_large_n} \end{figure}% + %%%%%%%%%%%%%%%%%%%%% \subsection{Improvement Using ``ML-in-the-List'' Step} - -Considering the magnitude of the oscillation of the gradient of the code constraint -polynomial, some interesting behavior may be observed. Let $\boldsymbol{i}'=(i'_1, \ldots, i_n')$ be a permutation of $\{1,\ldots, n\}$ such that $\left(\nabla h\right)_{i'}$ is arranged according to increasing variance of oscillation of its magnitude, i.e., $\text{Var}_\text{iter}(|\left(\nabla h\right)_{i'_1}|)\leq \cdots \leq \text{Var}_\text{iter}(|\left(\nabla h\right)_{i'_n}|)$ with $\text{Var}_\text{iter}(\cdot)$ denoting the empirical variance along the iterations. - -Hereafter, Fig. \ref{fig:p_error} shows Monte Carlo simulations of the probability that decoded bit $\hat{c}_i'$ at position $i'$ of the estimated codeword -is wrong. %, when the components of -%$\boldsymbol{c}$ are ordered from smallest to largest oscillation of -%$\left(\nabla h\right)_i$. -It can be observed that lower magnitudes of oscillation correlate with higher probability that the corresponding bit was not decoded correctly. -Thus, this magnitude might be used as a feasible indicator -%for determining the probability that a given component was decoded incorrectly and, thus, -for identifying erroneously decoded bit positions as $\mathcal{I}'=\{i_1', \ldots, i_N'\}$.% + +\reviewone{ +Based on the observations depicted in Fig. \ref{fig:grad} and Fig. +\ref{fig:prox:convergence_large_n}, it seems a meaningful approach to tag the +$N\in\mathbb{N}$ most likely erroneous bits based on the oscillation of the +gradient of the code-constraint polynomial. +To this end, let +$\Delta_i^{(h)}:=| \left(\nabla h\right)_{i}[K] - \left(\nabla h\right)_{i}[K-1]|$ +be the oscillation height at the last iteration with $\left(\nabla h\right)_{i}[K]$ +denoting the gradient at position $i$ and iteration $K$. +Now, let $\boldsymbol{i}'=(i'_1, \ldots, i_n')$ be a permutation of +$\left\{1, \ldots, n\right\}$ such that $\Delta_{i'}^{(h)}$ is arranged according +to increasing oscillation height and select its $N$ smallest indices, i.e., % -\begin{figure}[H] +\begin{align}\label{eq:def:i_prime} + & \boldsymbol{i}'=(i_1',\ldots, i_n')\in S_n : + \left| \Delta_{i_1'}^{(h)}\right| + \leq\cdots\leq + \left| \Delta_{i_n'}^{(h)} \right| + \\ + \label{eq:def:set_I} + &\mathcal{I}'=\{i_1', \ldots, i_N'\} \text{ with } \boldsymbol{i}' \text{ as defined in (\ref{eq:def:i_prime})} +\end{align} +% +with $S_n$ denoting the symmetric group of $\{1,\ldots, n\}$. +To reason this approach, Fig. \ref{fig:p_error} shows Monte Carlo simulations of +the probability that the decoded bit +\reviewfour{ +$\hat{c}_{i'}$ +} +at position $i'$ of the estimated codeword is wrong. +It can be observed that lower magnitudes of oscillation height correlate with a +higher probability that the corresponding bit was not decoded correctly. +Thus, the oscillation height might be used as a feasible indicator for +identifying the $N$ bits that are most likely erroneous. +} + +\begin{figure}[hbt]%[H] \centering \ifoverleaf @@ -643,118 +699,150 @@ for identifying erroneously decoded bit positions as $\mathcal{I}'=\{i_1', \ldot \begin{tikzpicture} \begin{axis}[ grid=both, - ylabel=$P(\hat{c}_{i'} \ne c_{i'})$, xlabel=$i'$, ymode=log, xmin=0,xmax=200, + ymin=1e-8,ymax=1e-2, + ytick={1e-2, 1e-4, 1e-6, 1e-8}, width=0.95\figwidth, height=\figheight, ] + \addplot+ [scol0, mark=none, line width=1] table [col sep=comma, y=p_error]{res/p_error.csv}; \end{axis} + \begin{axis}[hide axis, + width=\figwidth, + height=\figheight, + xmin=10, xmax=50, + ymin=0, ymax=0.4, + legend style={draw=white!15!black, + legend cell align=left, + empty legend, + at={(0.91,0.96)},anchor=north east}] + \addlegendimage{mark=none} + \addlegendentry{ + $\nabla L\left(\boldsymbol{y} + \mid \tilde{\boldsymbol{x}}\right)$ + }; + \end{axis} \end{tikzpicture} \fi - \caption{Probability that a component of the estimated codeword - $\boldsymbol{\hat{c}}\in \mathbb{F}_2^n$ is erroneous for a (3,6) regular + \caption{Probability that $P(\hat{c}_{i'} \ne c_{i'})$ for a (3,6) regular LDPC code with $n=204, k=102$ \cite[\text{204.33.484}]{mackay}. - Indices $i'$ are ordered such that $|\left(\nabla h\right)_{i'_1}|\leq \cdots \leq |\left(\nabla h\right)_{i'_n}|$. - Parameters used for the simulation: $\gamma = 0.05, \omega = 0.05, - \eta = 1.5, E_b/N_0 = \SI{6}{dB}$. - Simulated with $\SI{100000000}{}$ iterations using the all-zeros codeword.} + Indices $i'$ are ordered as in eq. (\ref{eq:def:i_prime}). + Parameters: $\gamma = 0.05, \omega = 0.05, \eta = 1.5, + \reviewthree{ + E_\text{b}/N_0 = \SI{6}{dB} + } + $, + \reviewthree{ + $10^9$ + } + codewords. + } \label{fig:p_error} \end{figure} -The complete improved algorithm is given in Algorithm \ref{alg:improved}. -First, the proximal decoding algorithm is applied. +The proposed improved algorithm is given in Algorithm \ref{alg:improved}. +First, the proximal decoding Algorithm \ref{alg:proximal_decoding} is applied. If a valid codeword has been reached, i.e., if the algorithm has converged, we return this solution. -Otherwise, $N \in \mathbb{N}$ components are selected based on the criterion -presented above. -Originating from $\boldsymbol{\hat{c}} \in \mathbb{F}_2^n$ resulting from proximal decoding, +Otherwise, $N \in \mathbb{N}$ components are selected as described in eq. (\ref{eq:def:set_I}). +Originating from $\boldsymbol{\hat{c}} \in \mathbb{F}_2^n$, the result of proximal decoding, the list $\mathcal{L}'$ of codeword candidates with bits in $\mathcal{I}'$ modified is generated and an ``ML-in-the-list'' step is performed. +\reviewthree{ +If the list $\mathcal{L}'$ does not contain a valid codeword and, thus, +$\mathcal{L}'_\text{valid}=\emptyset$, the additional step boils down to the +maximization of $2^N$ correlations +$\langle 1-2\boldsymbol{c}_l', \boldsymbol{y}\rangle $, $\boldsymbol{c}_l'\in\mathcal{L}'$, +in which ties happen with probability zero and are solved arbitrarily. +Note that $2^N$ parity checks have to be evaluated for elements in $\mathcal{L}'$ +in order to determine $\mathcal{L}'_\text{valid}$ either way. +Restricting the correlations to the (non-empty) list $\mathcal{L}'_\text{valid}$ +may reduce the computational burden and ensure that a valid codeword is returned. +} \begin{algorithm} - \caption{Improved proximal decoding algorithm. - } + \caption{Proposed improved proximal decoding in AWGN } \label{alg:improved} - \begin{algorithmic} - \STATE $\boldsymbol{s} \leftarrow \boldsymbol{0}$ - \STATE \textbf{for} $K$ iterations \textbf{do} - \STATE \hspace{5mm} $\boldsymbol{r} \leftarrow \boldsymbol{s} - \omega \left( \boldsymbol{s} - \boldsymbol{y} \right) $ - \STATE \hspace{5mm} $\boldsymbol{s} \leftarrow \Pi_\eta \left(\boldsymbol{r} - \gamma \nabla h\left( \boldsymbol{r} \right) \right)$ - \STATE \hspace{5mm} $\boldsymbol{\hat{c}} \leftarrow \mathds{1}_{ \left\{ \boldsymbol{s} \leq 0 \right\}}$ - \STATE \hspace{5mm} \textbf{if} $\boldsymbol{H}\boldsymbol{\hat{c}} = \boldsymbol{0}$ \textbf{do} - \STATE \hspace{10mm} \textbf{return} $\boldsymbol{\hat{c}}$ - \STATE \hspace{5mm} \textbf{end if} - \STATE \textbf{end for} - \STATE $\textcolor{KITblue}{\text{$\mathcal{I}'\leftarrow \{i_1',\ldots, i_N'\}$ (indices of $N$ probably wrong bits) - %$\mathcal{I} = \{i_1,\ldots,i_N\}$ - } - }$ + \begin{algorithmic}[1] - \STATE $\textcolor{KITblue}{\text{%Generate candidates - $\mathcal{L}'\leftarrow\left\{ \boldsymbol{\hat{c}}'\in\mathbb{F}_2^n: \hat{c}'_i=\hat{c}_i, i\notin \mathcal{I}' \text{ and } \hat{c}'_i\in\mathbb{F}_2, i\in \mathcal{I}' \right\} - %\left(\boldsymbol{\hat{c}}_{l}\right)_{l=1}^{2^N} - $ - %by varying bits in $\mathcal{I}$ + \STATE \text{$\hat{\boldsymbol{c}}\leftarrow\text{proximal decoding}(\boldsymbol{y})$} + + \STATE \textbf{if} $\boldsymbol{H}\boldsymbol{\hat{c}} = \boldsymbol{0}$ \textbf{do} + \STATE \hspace{5mm} \textbf{return} $\boldsymbol{\hat{c}}$ + \STATE $\textcolor{black}{\text{$\mathcal{I}'\leftarrow \{i_1',\ldots, i_N'\}$ (indices of $N$ probably wrong bits) } }$ + + \STATE $\textcolor{black}{\text{ + $\mathcal{L}'\leftarrow\left\{ \boldsymbol{\hat{c}}'\in\mathbb{F}_2^n: \hat{c}'_i=\hat{c}_i, i\notin \mathcal{I}' \text{ and } \hat{c}'_i\in\mathbb{F}_2, i\in \mathcal{I}' \right\}$ }} $\vspace{1mm} - %\STATE \hspace{20mm} \textcolor{KITblue}{(list of codeword candidates)} - \STATE $\textcolor{KITblue}{\textbf{return ML\textunderscore in\textunderscore the\textunderscore list}\left( - %\left(\boldsymbol{\hat{c}}_l\right)_{1=1}^{2^N} - \mathcal{L}' - \right)}$ + + \STATE \textcolor{black}{$\mathcal{L}'_\text{valid} \leftarrow \{ \boldsymbol{\hat{c}}'\in\mathcal{L}': \boldsymbol{H}\boldsymbol{\hat{c}}'=\boldsymbol{0}\}$ (select valid codewords) } + \STATE \textcolor{black}{\textbf{if} $\mathcal{L}'_\text{valid}\neq\emptyset$ \textbf{do}} + \STATE \hspace{5mm} + \textcolor{black}{\textbf{return} $\arg\max \{ \langle 1-2\boldsymbol{\hat{c}}'_l, \boldsymbol{y} \rangle : \boldsymbol{\hat{c}}'_l\in\mathcal{L}'_\text{valid}\}$} + \STATE \textcolor{black}{\textbf{else}} + \STATE \hspace{5mm} + \textcolor{black}{\textbf{return} $\arg\max \{ \langle 1-2 \boldsymbol{\hat{c}}'_l, \boldsymbol{y} \rangle : \boldsymbol{\hat{c}}'_l\in\mathcal{L}'\}$} \end{algorithmic} \end{algorithm} -\begin{algorithm} - \caption{ML-in-the-List algorithm.} - \label{alg:ml-in-the-list} - - \begin{algorithmic} - \STATE $\mathcal{L}'_\text{valid} \leftarrow \{ \boldsymbol{\hat{c}}'\in\mathcal{L}': \boldsymbol{H}\boldsymbol{\hat{c}}'=\boldsymbol{0}\}$ (select valid codewords) - % Find valid codewords within $\mathcal{L}'$ - %under $\left(\boldsymbol{\hat{c}}_{l}\right)_{1=1}^{2^N}$ - \STATE \textbf{if} $\mathcal{L}'_\text{valid}\neq\emptyset$ \textbf{do} - %no valid codewords exist - \STATE \hspace{5mm} - \textbf{return} $\arg\max \{ \langle 1-2\boldsymbol{\hat{c}}'_l, \boldsymbol{y} \rangle : \boldsymbol{\hat{c}}'_l\in\mathcal{L}'_\text{valid}\}$ - %Compute $\langle \boldsymbol{\hat{c}}'_l, \boldsymbol{\hat{c}} \rangle$ for all variations $\boldsymbol{\hat{c}}'_l\in\mathcal{L}$ - \STATE \textbf{else} - \STATE \hspace{5mm} - \textbf{return} $\arg\max \{ \langle 1-2 \boldsymbol{\hat{c}}'_l, \boldsymbol{y} \rangle : \boldsymbol{\hat{c}}'_l\in\mathcal{L}'\}$ - %Compute $\langle \boldsymbol{\hat{c}}'_l, \boldsymbol{\hat{c}} \rangle$ for valid codewords $\boldsymbol{\hat{c}}'_l\in\mathcal{L}$ - \STATE \textbf{end if} - %\STATE \textbf{return} $\boldsymbol{\hat{c}}_l$ with highest $\langle \boldsymbol{\hat{c}}_l, \boldsymbol{\hat{c}} \rangle$ - \end{algorithmic} -\end{algorithm}% -% - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Simulation Results \& Discussion} + Fig. \ref{fig:results} shows the FER and BER resulting from applying -proximal decoding as presented in \cite{proximal_paper} and the improved -algorithm presented in this work, when both are applied to a $\left( 3,6 \right)$-regular LDPC -code with $n=204$ and $k=102$ \cite[204.33.484]{mackay}. -The parameters chosen for the simulation are -$\gamma = 0.05, \omega=0.05, \eta=1.5, K=200$. -Again, these parameters were chosen, -as a preliminary examination -showed that they provide the best results for proximal decoding as well as -the improved algorithm. -All points were generated by simulating at least 100 frame errors. -The number of possibly wrong components selected was selected as $N=8$, -since this provides reasonable gain without requiring an unreasonable amount -of memory and computational resources. +proximal decoding as presented in \cite{proximal_paper} and the proposed improved +algorithm, when both are applied to the $\left( 3,6 \right)$-regular LDPC +code [204.33.484] \cite{mackay} with $n=204$ and $k=102$. +The parameters chosen for the simulation are $\gamma = 0.05, \omega=0.05, \eta=1.5, K=200$ +\reviewfour{ +as for proximal decoding, since those parameters also turned out close-to-optimum +for the improved algorithm in our simulations. +} +The number of possibly wrong components was selected as $N=8$. +\reviewfour{ +To reason this choice, Table \ref{N Table} shows the SNRs required for $N\in\{4, 6, 8, 10, 12\}$ to achieve an FER of $10^{-2}$ and $10^{-3}$, respectively. +} +\begin{table}[hbt] + \centering + \reviewfour{ + \caption{SNR (in dB) to achieve target FERs $10^{-2}$ and $10^{-3}$} + \label{N Table} + \begin{tabular}{c|ccccc} + $N$ & 4 & 6 & 8 & 10 & 12 \\ \hline + $\text{FER} = 10^{-2}$ & $4.94$ & $4.76$ & $4.67$ & $4.60$ & $4.54$ \\ + $\text{FER} = 10^{-3}$ & $5.84$ & $5.61$ & $5.49$ & $5.39$ & $5.32$ \\ + \end{tabular} + } +\end{table} + +A noticeable improvement can be observed both in the FER and the BER. +The gain varies significantly with the SNR, which is to be expected since higher +SNR values result in a decreased number of bit errors, making the correction of +those errors in the ``ML-in-the-list'' step more likely. +For an FER of $10^{-6}$, the gain is approximately $\SI{1}{dB}$. +\reviewone{As shown in Fig. \ref{fig:results}, it can be seen that BP decoding +\reviewfour{ +with $200$ iterations +} +outperforms the improved scheme by approximately $\SI{1.7}{dB}$. +Nevertheless, note that Algorithm \ref{alg:improved} requires only linear +operations and could be favorable in applications as, e.g., massive MIMO, in +which application of BP is prohibitive \cite{proximal_paper}. +} +Similar behavior to Fig. \ref{fig:results} was observed with a number of +different codes, e.g., \cite[\text{PEGReg252x504, 204.55.187, 96.3.965}]{mackay}. +Furthermore, we did not observe an immediate relationship between the code length +and the gain during our examinations. % -\begin{figure} +\begin{figure}[hbt]%[H] \centering \ifoverleaf @@ -776,12 +864,12 @@ of memory and computational resources. ymode=log, xmin=1, xmax=8, ymax=1, ymin=1e-6, + ytick={1e-0, 1e-2, 1e-4, 1e-6}, width=\figwidth, height=\figheight, legend pos=north east, - legend cell align={left}, - ylabel={BER (\lineintext{}), FER (\lineintext{dashed})}, ] + \addplot+[FERPlot, mark=o, mark options={solid}, scol1, forget plot] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}, @@ -808,6 +896,7 @@ of memory and computational resources. {res/improved_ber_fer_dfr_20433484.csv}; \addlegendentry{Improved}; + \addplot+[FERPlot, mark=o, mark options={solid}, scol0, forget plot] table [x=SNR, y=FER, col sep=comma, discard if gt={SNR}{9}] @@ -818,52 +907,45 @@ of memory and computational resources. discard if gt={SNR}{7.5}] {res/bp_20433484.csv}; \addlegendentry{BP}; + \end{axis} \end{tikzpicture} \fi - \caption{FER and BER of proximal decoding \cite{proximal_paper} and the + \caption{FER (-\,-\,-) and BER (---) of proximal decoding \cite{proximal_paper} and the improved algorithm for a $\left( 3, 6 \right)$-regular LDPC code with $n=204, k=102$ \cite[\text{204.33.484}]{mackay}. - Parameters used for simulation: $\gamma=0.05, \omega=0.05, \eta=1.5, - K=200, N=8$. - } + Parameters: + $\gamma=0.05, \omega=0.05, \eta=1.5, + K=200, N=8$.} \label{fig:results} -\end{figure}% -% +\end{figure} -A noticeable improvement can be observed both in the FER as well as the BER. -The gain varies significantly -with the SNR, which is to be expected since higher SNR values result in a decreased number -of bit errors, making the correction of those errors in the -``ML-in-the-list'' step more likely. -For an FER of $10^{-6}$, the gain is approximately $\SI{1}{dB}$. -Similar behavior was observed with a number of different codes, e.g., \cite[\text{PEGReg252x504, 204.55.187, 96.3.965}]{mackay}. -Furthermore, no immediate relationship between the code length and the gain was observed -during our examinations. %%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Conclusion} + In this paper, an improvement on proximal decoding as presented by -Wadayama et al. \cite{proximal_paper} is introduced for AWGN channels. +Wadayama \emph{et al.} \cite{proximal_paper} is proposed for AWGN channels. It relies on the fact that most errors observed in proximal decoding stem from only a few components of the estimate being wrong. These few erroneous components can mostly be corrected by appending an additional step to the original algorithm that is only executed if the algorithm has not converged. A gain of up to $\SI{1}{dB}$ can be observed, depending on the code, -the parameters considered, and the SNR. +the code parameters, and the SNR. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Acknowledgements} +% \section{Acknowledgements} -This work has received funding in part from the European Research Council -(ERC) under the European Union’s Horizon 2020 research and innovation -programme (grant agreement No. 101001899) and in part from the German Federal -Ministry of Education and Research (BMBF) within the project Open6GHub -(grant agreement 16KISK010). +% This work has received funding in part from the European Research Council +% (ERC) under the European Union’s Horizon 2020 research and innovation +% programme (grant agreement No. 101001899) and in part from the German Federal +% Ministry of Education and Research (BMBF) within the project Open6GHub +% (grant agreement 16KISK010). % @@ -887,100 +969,51 @@ Ministry of Education and Research (BMBF) within the project Open6GHub \section{Authors' Response to the Editor resp. the Reviewers} -\subsection{Review 1} +Before addressing the comments of the reviewers, the authors would first like to thank the editor and the reviewers for their tremendous efforts and their valuable comments which will help us to improve the paper. We hope that the upcoming changes meet your approval and address all the concerns. In the following section we respond to the comments of the reviews and mark the according changes in the paper. +\subsection{Editor} + \begin{itemize} - \item \textbf{Comment 1:} This paper proposes a combination of proximal decoding and ML-in-the-list decoding. There are several issues with the paper in its current form that need to be addressed. + \item \textbf{Comment 1:} [...] Your manuscript requires revisions, as outlined below, before the paper can be published. If these revisions are satisfactorily made (including meeting the length guidelines), the paper will be accepted for publication. [...] + + \vspace{0.25cm} + \textbf{Authors:} + Thank you for your positive feedback. The ``length-issue" stems from the fact that the submitted version has been compiled without the thanks and the acknowledgments. It should be solved by now. + \vspace{0.75cm} + + + \item \textbf{Comment 2:} Reviewer 2 suggests to add some more simulations. + + \vspace{0.25cm} + \textbf{Authors:} + We added simulation results for different values of $N$, as the reviewer suggested. + Unfortunately, due to the short revision time and temporal constraints due to the + summer break, we were not able to add further results regarding the complexity and + different codes and parameters. +\end{itemize} + + + +\subsection{Review 1} + +\begin{itemize} + \item \textbf{Comment 1:} I believe that the paper deserves publication. I think the paper has been considerably improved. \vspace{0.25cm} \textbf{Authors:} - Thank you for your feedback. \reviewone{The according changes will be marked by accordingly coloring the changes in the paper and be listed below. - } + Thank you for your positive feedback. We also would like to thank you for your insightful comments which helped to improve the paper. \vspace{0.75cm} - \item \textbf{Comment 2:} The definition of code-constraint polynomial is baseless. The authors should explain why we use the code-constraint polynomial. Also, I think the code-constraint polynomial cannot be used to replace the prior PDF of $\boldsymbol{x}$, since $h(\boldsymbol{0})$ is the minimum value of $h(\boldsymbol{x})$. + \item \textbf{Comment 2:} + A side note: I could not find the point-by-point letter of reply. Nevertheless, I could track the changes applied to the manuscript, for the issues I've pointed out in the first version of the manuscript. \vspace{0.25cm} \textbf{Authors:} - Thank you for your feedback. The definition of the code-constraint polynomial is directly according to \cite{proximal_paper}. There the authors state that: - - \vspace{.1cm} - "[...] The first term on the right-hand side of this equation represents the bipolar constraint [...] and the second term corresponds to the parity constraint induced by $\boldsymbol{H}$ [...]. Since the polynomial $h(x)$ has a sum-of-squares (SOS) form, it can be regarded - as a penalty function that gives positive penalty values for non-codeword vectors in $\mathbb{R}^n$. The code-constraint polynomial $h(x)$ is inspired by the non-convex parity constraint function used in the GDBF objective function [4]. [...]" + That is strange; we uploaded the comments together with the revised version of the paper, coloring the added phrases according to the request of the reviews. \vspace{.1cm} - Please note that $\boldsymbol{0}$ is not a global minimum for the code-constraint polynomial, but every codeword constitutes a local minimum. Therefore, an iterative algorithm can converge to one of those local minima and, thus, approximate the nearest neighbor decision. - - \vspace{0.75cm} - - - \item \textbf{Comment 3:} The definition of the projection $\prod_\eta$ should be provided. - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your feedback. We added the following description: - - \vspace{.1cm} - "[...] every estimate $\boldsymbol{s}$ is projected onto $\left[-\eta, \eta\right]^n$ by a projection $\Pi_\eta : \mathbb{R}^n \rightarrow \left[-\eta, \eta\right]^n$ - \reviewone{ - defined as component-wise clipping, i.e., $\Pi_\eta(x_i)=x_i$ if $-\eta\leq x_i\leq \eta$, $\Pi_\eta(x_i)=\eta$ if $x_i>\eta$, and $\Pi_\eta(x_i)=-\eta$ if $x_i<\eta$, - } - where $\eta$ is a positive constant larger than one, e.g., $\eta = 1.5$. [...]" - \vspace{0.75cm} - - - \item \textbf{Comment 4:} The proposed improved proximal decoding algorithm is just a combination of proximal decoding and ML-in-the-list decoding. Then, the process of the ML-in-the-list decoding used in this paper is similar to that of chase decoding, which is commonly used in decoding. ML-in-the-list decoding. - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your feedback. Yes, this is correct. The idea is pretty similar to Chase decoding. The paper at hand is not claiming the introduction of ML-in-the-list or Chase decoding, but to provide a way how the list can be being generated in proximal decoding. We tried to clarify by adding the following statement: - - \vspace{.1cm} - \reviewone{asdf} - \vspace{0.75cm} - - - \item \textbf{Comment 5:} - The criterion to construct the index set ${\mathcal I}’$ with $N$ elements should be explained clearly. - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your feedback. We added the following parts in the according paragraph for more clarity: - - \vspace{.1cm} - "[...] \reviewone{Tagging the $N\in\mathbb{N}$ most likely erroneous bits can be based on} considering the \reviewone{oscillation of the gradient magnitudes $|\left(\nabla h\right)_{i}|$, $i=1,\ldots, n$ \sout{of the magnitude of the gradient oscillation}} of the code-constraint polynomial \reviewone{by determining the empirical variances along the iterations $\text{Var}_\text{iter}(|\left(\nabla h\right)_{i}|)$, $i=1,\ldots, n$}. - \reviewone{\sout{some interesting behavior may be observed}}. - \reviewone{Now,} let \reviewone{$\boldsymbol{i}'=(i'_1, \ldots, i_n')=(\tau(1),\ldots, \tau(n))$ with $\tau: \{1,\ldots, n\}\to\{1,\ldots,n\}$} be a permutation of $\{1,\ldots, n\}$ such that $\left| \left(\nabla h\right)\right|_{i'}$ is arranged according to increasing \reviewone{empirical} variances \reviewone{\sout{gradient's magnitude oscillation of its magnitude}}, i.e., - \begin{equation}\label{eq:def:i_prime} - \text{Var}_\text{iter}(|\left(\nabla h\right)_{i'_1}|)\leq \cdots \leq \text{Var}_\text{iter}(|\left(\nabla h\right)_{i'_n}|). - \end{equation} - \reviewone{\sout{with $\text{Var}_\text{iter}(\cdot)$ denoting the empirical variance along the iterations.}} - - \reviewone{To reason the approach in eq. (\ref{eq:def:i_prime}) \sout{Hereafter}}, Fig. \ref{fig:p_error} shows Monte Carlo simulations of the probability that decoded bit $\hat{c}_i'$ at position $i'$ of the estimated codeword - is wrong. %, when the components of - %$\boldsymbol{c}$ are ordered from smallest to largest oscillation of - %$\left(\nabla h\right)_i$. - It can be observed that lower magnitudes of oscillation correlate with higher probability that the corresponding bit was not decoded correctly. - Thus, this magnitude might be used as a feasible indicator - %for determining the probability that a given component was decoded incorrectly and, thus, - for identifying \reviewone{the $N$ most likely} erroneously decoded bit positions as \reviewone{the first $N$ indices of $\boldsymbol{i}'$}: - \[ - \mathcal{I}'=\{i_1', \ldots, i_N': \boldsymbol{i}' \text{ as defined in (\ref{eq:def:i_prime})} \}.%[...]" - \] - \vspace{0.75cm} - - \item \textbf{Comment 6:} - The performance of BP decoding should be provided as the baseline. - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your feedback. We added the according BP behavior in the figure and added the following comment: - - \vspace{.1cm} - \reviewone{As shown in Fig., it can that BP decoding performs...} - \vspace{0.75cm} \end{itemize} @@ -989,43 +1022,150 @@ Ministry of Education and Research (BMBF) within the project Open6GHub \begin{itemize} - \item \textbf{Comment 1:} I believe that the paper makes a nice contribution to the topic of optimization-based decoding of LDPC codes. The topic is especially relevant, nowadays, for the applicability of this kind of decoders to quantum error correction - where classical BP decoding may yield limited coding gains, due to the loopy nature of the graphs. - - The work is nicely-presented, solid, and the results are convincing. My only comment would be to try to put the use of this decoder in some perspective: - - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your positive feedback. \reviewtwo{The according changes will be marked by accordingly coloring the changes in the paper and be listed below.} - - \vspace{0.75cm} - - \item \textbf{Comment 2:} [...] adding, on the performance charts, the performance of a standard BP decoder (it will beat your decoding algorithm, but this is not the point) - - + \item \textbf{Comment 1:} The authors have made substantial revisions to the manuscript based on the reviewers' feedback. Nevertheless, there are some issues that need to be addressed. \vspace{0.25cm} \textbf{Authors:} - Thank you for your feedback. We added the according BP behavior in the figure and added the following comment: + Thank you for the positive assessment that several helpful changes have been made. We will address the remaining concerns in the following and add \reviewfour{according changes to the paper}. + \vspace{0.75cm} - \vspace{.1cm} - \reviewtwo{As shown in Fig., it can seen that BP decoding performs...} - \vspace{0.75cm} - - \vspace{0.75cm} - - - \item \textbf{Comment 3:} [...] explaining when this class of algorithms should be preferred to BP decoding + \item \textbf{Comment 2:} It seems that the PMF defined in Section II.A is not used. If not, delete it. + \vspace{0.25cm} \textbf{Authors:} - Thank you for your feedback. We added the following statement: + The according sentence has been removed. + \vspace{0.75cm} - \vspace{.1cm} - \reviewtwo{something concerning effort?!?} + + + \item \textbf{Comment 3:} The definition of DFR is not provided. Decoding failure seems that the decoded result is not the valid codeword. + + \vspace{0.25cm} + \textbf{Authors:} + At the beginning of section III.A---upon introducing the abbreviation DFR---it is stated that ``[...] Hereby, a \emph{decoding failure} is defined as returning a \emph{non valid codeword}, i.e., as non-convergence of the algorithm. [...]" + \vspace{0.75cm} + + + \item \textbf{Comment 4:} In line 36 of page 3, ${c}'_i$ should be $c_{i’}$. + + \vspace{0.25cm} + \textbf{Authors:} + Thank you for noting this typo. The according equation has been corrected: + + \vspace{0.25cm} + ``[...] To reason this approach, Fig. \ref{fig:p_error} shows Monte Carlo simulations of the probability that the decoded bit \reviewfour{$\hat{c}_{i'}$} at position $i'$ of the estimated codeword is wrong. [...]" \vspace{0.75cm} + + + + + \item \textbf{Comment 5:} The values of $\gamma$, $\omega$ and $\eta$ used in this paper are identical to [11], which should be indicated in Section IV. + + \vspace{0.25cm} + \textbf{Authors:} + We added a short comment: + + \vspace{0.25cm} + ``[...] They \reviewfour{adhere to \cite{proximal_paper} and} were determined to offer the best performance in a preliminary examination, + where the effect of changing multiple parameters was simulated over a wide + range of values. [...]" \vspace{0.75cm} + + + + \item \textbf{Comment 6:} $\gamma$, $\omega$ and $\eta$ are the parameters of proximal decoding. Are the values of these parameters optimal for list proximal decoding? The authors should provide some simulation results to illustrate the influence of changing these parameters on the performance of list proximal decoding. + + \vspace{0.25cm} + \textbf{Authors:} + Thank you for this hint. In deed, these parameters were chosen since they are close-to-optimum for stand-alone proximal decoding and turned out to be at least well functioning for the improved list-based algorithm. Optimization of those values for the improved list-version has not been conducted up to now. This is subject to future research. To clarify, we added the following explanation: + + \vspace{0.25cm} + ``[...] + The parameters chosen for the simulation are $\gamma = 0.05, \omega=0.05, \eta=1.5, K=200$ % + \reviewfour{as for proximal decoding, since those parameters also turned out close-to-optimum for as the improved algorithm in our simulations.} + [...]'' + \vspace{0.75cm} + + + + \item \textbf{Comment 7:} $N$ is the main parameter for list proximal decoding. More simulation results with different $N$ should be provided. + + \vspace{0.25cm} + \textbf{Authors:} + Thank you for the comment. Several analyses showed that $N=8$ is a very good compromise between effort resp. list size and performance. We added the following comment: + + \setcounter{table}{0} + + \vspace{0.25cm} + ``[...] + The number of possibly wrong components was selected as $N=8$. + \reviewfour{ + To reason this choice, Table \ref{N Table} shows the SNRs required for $N\in\{4, 6, 8, 10, 12\}$ to achieve an FER of $10^{-2}$ and $10^{-3}$, respectively. + } + \begin{table}[hbt] + \centering + \reviewfour{ + \caption{SNR (in dB) to achieve target FERs $10^{-2}$ and $10^{-3}$} + \label{N Table} + \begin{tabular}{c|ccccc} + $N$ & 4 & 6 & 8 & 10 & 12 \\ \hline + $\text{FER} = 10^{-2}$ & $4.94$ & $4.76$ & $4.67$ & $4.60$ & $4.54$ \\ + $\text{FER} = 10^{-3}$ & $5.84$ & $5.61$ & $5.49$ & $5.39$ & $5.32$ \\ + \end{tabular} + } + \end{table} + [...]" + \vspace{0.75cm} + + + + \item \textbf{Comment 8:} More simulation results with different $R$ should be provided, such as $R = 0.25$ and $R = 0.75$. + + \vspace{0.25cm} + \textbf{Authors:} + Unfortunately, due to the short revision time and temporal constraints due to the summer break as well as---even more important---the space constraint inherent to a letter, we were not able to add these results to the current paper. + \vspace{0.75cm} + + + + \item \textbf{Comment 9:} The iteration number of BP should be provided. + + \vspace{0.25cm} + \textbf{Authors:} + We added the number of iterations: + + \vspace{0.25cm} + ``As shown in Fig. \ref{fig:results}, it can be seen that BP decoding \reviewfour{with $200$ iterations} outperforms the improved scheme by approximately $\SI{1.7}{dB}$. + [...]'' + \vspace{0.75cm} + + + + \item \textbf{Comment 10:} The complexity comparison among proximal decoding, list proximal decoding and BP should be provided. + + \vspace{0.25cm} + \textbf{Authors:} Thank you for encouraging a more detailed comparison of according complexity. In our opinion, a more in-depth analysis would go beyond the space limitations of this letter, which primarily aims at describing the idea of improving proximal decoding by means of using a suitably defined list. + \vspace{0.75cm} + + + + \item \textbf{Comment 11:} In Section IV, the authors said “Nevertheless, note that Algorithm 2 requires only linear operations and could be favorable in applications as, e.g., massive MIMO, in which application of BP is prohibitive [11].”, but no simulation results support this conclusion. In [11], the proximal decoding has better performance than the BP decoding, which can not indicate that the proposed list proximal decoding also shows better performance for LDPC coded massive MIMO systems. + + \vspace{0.25cm} + \textbf{Authors:} This is perfectly true. As of now, our comparison has been conducted for SISO systems only and it is not yet analyzed whether they carry over to (massive) MIMO systems. This sentence is directly based on \cite{proximal_paper} and had been added after the first review in order to motivate that the consideration of proximal decoding makes sense after all. + \vspace{0.75cm} + + + \item \textbf{Comment 12:} “Figure” and “Fig.” are mixed in this paper, such Figure 5 and Fig. 5. The authors should used unified representation. + + \vspace{0.25cm} + \textbf{Authors:} Thank you for pointing out this inconsistency. We changed it to ``Fig.'' throughout the paper. + \vspace{0.75cm} + + + \end{itemize} @@ -1033,68 +1173,16 @@ Ministry of Education and Research (BMBF) within the project Open6GHub \subsection{Review 3} \begin{itemize} - \item \textbf{Comment 1:} The paper describes an enhancement and mitigate essential flaws found in the recently reported proximal decoding algorithm for LDPC codes, mentioned in the references section. At first the algorithm subject to the paper is interesting because the published material a few years back seem to have no substantial performance improvement, and did not seem to make any influence. It is therefore interesting to see that this paper addresses this fact and fixes the issues around the originally proposed algorithm and demonstrating up to a 1 dB coding gain as a result of these corrections and enhancements. - - While I find the paper is interesting and relevant, here are my essential comments that would prevent me in favor of publication. - + \item \textbf{Comment 1:} All my comments have been addressed carefully. I have no further comments. \vspace{0.25cm} \textbf{Authors:} - Thank you for your positive feedback. \reviewthree{The according changes will be marked by accordingly coloring the changes in the paper and be listed below.} + Thank you for your positive feedback. We also would like to thank you for your insightful comments which helped to improve the paper. \vspace{0.75cm} - - \item \textbf{Comment 2:} The work is titled after linar block codes, however both the original proximal decoding paper and this work go after LDPC codes only. Clarification required as in whether the proposed method would work for any linear block code, and if so, elaboration and proof is needed as well. Currently, linear codes are only mentioned in the first two sentences of the Introduction section other than the title. - - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your feedback. We also analysezed the proposed scheme for BCH codes. There it turned out that... - - - \vspace{.1cm} - \reviewthree{Some comment regarding the applicability of the proposed scheme to BCH...} - \vspace{0.75cm} - - - \item \textbf{Comment 3:} Does this work (and the original work) based on BPSK modulation only? How would the code constraint polynomial change with higher order modulations? It would be interesting to see how this would change given that the polynomial is based on a nearest neighbor decision. - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your feedback. - - \vspace{.1cm} - \reviewthree{one sentence regarding bit-metric decoder mapping higher valued symbols to elementwise bit-LLRs. } - \vspace{0.75cm} - - - \item \textbf{Comment 4:} The decoding failure rate stands out as a good analysis as in explaining the FER behavior. But if the codeword is not really converging at all, wouldn't there be simpler approaches than ML decoding to find out which one of $2^N$ codewords is the valid one? - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your feedback. - - \vspace{.1cm} - \reviewthree{one sentence concerning the feasability of using $N$ bit candidates and choosing $N$ according to the complexity; comment on trade-off w.r.t. $N$} - \vspace{0.75cm} - - - \item \textbf{Comment 5:} If you can, please have a more comprehensive simulation to smooth out the curve in Fig.4. Otherwise, please explain the odd behavior in the middle of the figure. I would also recommend a bar graph over a line graph for a better representation of the data. - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your feedback. The behavior is due to only few errors occurring in this setting (please mind the $y$-axis. Since the relevant information is contained in only the lower values of $i'$, which will finally be chosen for constituting $\mathcal{I}'$, only indices up to, e.g. $N=12$ are relevant. The figure has been complemented by focussing the relevant region. - \vspace{0.75cm} - - - \item \textbf{Comment 6:} How does your algorithm handle the case when there is more than one ML in your final list? It is not shown in the algorithm. - - \vspace{0.25cm} - \textbf{Authors:} - Thank you for your feedback. Since Algorithm 3 (ML-in-the-List) is dealing with real-valued numbers, the probability of two correlations being equal is zero almost surely. \textcolor{red}{@AT: Do we need to check this?} Even if the event of a draw would happen, choosing either of the candidates is equivalent with respect to the ML decision rule. - \vspace{0.75cm} \end{itemize} - \end{document} + +