\chapter{Comparison of Proximal Decoding and \acs{LP} Decoding using \acs{ADMM}}% \label{chapter:comparison} In this chapter, proximal decoding and \ac{LP} Decoding using \ac{ADMM} are compared. First, the two algorithms are studied on a theoretical basis. Subsequently, their respective simulation results are examined and their differences are interpreted based on their theoretical structure. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Theoretical Comparison}% \label{sec:comp:theo} \ac{ADMM} and the proximal gradient method can both be expressed in terms of proximal operators \cite[Sec. 4.4]{proximal_algorithms}. Additionally, the two algorithms show some striking similarities with regard to their general structure and the way in which the minimization of the respective objective functions is accomplished. The \ac{LP} decoding problem in equation (\ref{eq:lp:relaxed_formulation}) can be slightly rewritten using the \textit{indicator functions} $g_j : \mathbb{R}^{d_j} \rightarrow \left\{ 0, +\infty \right\} \hspace{1mm}, j\in\mathcal{J}$ for the polytopes $\mathcal{P}_{d_j}, \hspace{1mm} j\in\mathcal{J}$, defined as% % \begin{align*} g_j\left( \boldsymbol{t} \right) := \begin{cases} 0, & \boldsymbol{t} \in \mathcal{P}_{d_j} \\ +\infty, & \boldsymbol{t} \not\in \mathcal{P}_{d_j} \end{cases} ,\end{align*} % by moving the constraints into the objective function, as shown in figure \ref{fig:ana:theo_comp_alg:admm}. Both algorithms are composed of an iterative approach consisting of two alternating steps. The objective functions of the two problems are similar in that they both comprise two parts: one associated to the likelihood that a given codeword was sent, stemming from the channel model, and one associated to the constraints the decoding process is subjected to, stemming from the code used. % \begin{figure}[h] \centering \begin{subfigure}{0.42\textwidth} \centering \begin{align*} \text{minimize}\hspace{2mm} & \underbrace{L\left( \boldsymbol{y} \mid \tilde{\boldsymbol{x}} \right)}_{\text{Likelihood}} + \underbrace{\gamma h\left( \tilde{\boldsymbol{x}} \right)} _{\text{Constraints}} \\ \text{subject to}\hspace{2mm} &\tilde{\boldsymbol{x}} \in \mathbb{R}^n \end{align*} \begin{genericAlgorithm}[caption={}, label={}, basicstyle=\fontsize{10}{18}\selectfont ] Initialize $\boldsymbol{r}, \boldsymbol{s}, \omega, \gamma$ while stopping critierion unfulfilled do $\boldsymbol{r} \leftarrow \boldsymbol{r} + \omega \nabla L\left( \boldsymbol{y} \mid \boldsymbol{s} \right) $ $\boldsymbol{s} \leftarrow \textbf{prox}_{\scaleto{\gamma h}{7.5pt}}\left( \boldsymbol{r} \right) $|\Suppressnumber| |\Reactivatenumber| end while return $\boldsymbol{s}$ \end{genericAlgorithm} \caption{Proximal decoding} \label{fig:ana:theo_comp_alg:prox} \end{subfigure}\hfill% \begin{subfigure}{0.55\textwidth} \centering \begin{align*} \text{minimize}\hspace{5mm} & \underbrace{\boldsymbol{\gamma}^\text{T}\tilde{\boldsymbol{c}}} _{\text{Likelihood}} + \underbrace{\sum\nolimits_{j\in\mathcal{J}} g_j\left( \boldsymbol{T}_j\tilde{\boldsymbol{c}} \right) } _{\text{Constraints}} \\ \text{subject to}\hspace{5mm} & \tilde{\boldsymbol{c}} \in \left[ 0, 1 \right]^n \end{align*} \begin{genericAlgorithm}[caption={}, label={}, basicstyle=\fontsize{10}{18}\selectfont ] Initialize $\tilde{\boldsymbol{c}}, \boldsymbol{z}, \boldsymbol{u}, \boldsymbol{\gamma}, \rho$ while stopping criterion unfulfilled do $\tilde{\boldsymbol{c}} \leftarrow \argmin_{\tilde{\boldsymbol{c}}} \left( \boldsymbol{\gamma}^\text{T}\tilde{\boldsymbol{c}} + \frac{\rho}{2}\sum_{j\in\mathcal{J}} \left\Vert \boldsymbol{T}_j\tilde{\boldsymbol{c}} - \boldsymbol{z}_j + \boldsymbol{u}_j \right\Vert \right)$ $\boldsymbol{z}_j \leftarrow \textbf{prox}_{g_j} \left( \boldsymbol{T}_j\tilde{\boldsymbol{c}} + \boldsymbol{u}_j \right), \hspace{5mm}\forall j\in\mathcal{J}$ $\boldsymbol{u}_j \leftarrow \boldsymbol{u}_j + \tilde{\boldsymbol{c}} - \boldsymbol{z}_j, \hspace{15.25mm}\forall j\in\mathcal{J}$ end while return $\tilde{\boldsymbol{c}}$ \end{genericAlgorithm} \caption{LP decoding using \ac{ADMM}} \label{fig:ana:theo_comp_alg:admm} \end{subfigure}% \caption{Comparison of the proximal gradient method and \ac{ADMM}} \label{fig:ana:theo_comp_alg} \end{figure}% % Their major difference is that while with proximal decoding the constraints are regarded in a global context, considering all parity checks at the same time, with \ac{ADMM} each parity check is considered separately and in a more local context (line 4 in both algorithms). Furthermore, while with proximal decoding the step considering the constraints is realized using gradient descent - amounting to an approximation - with \ac{ADMM} it reduces to a number of projections onto the parity polytopes $\mathcal{P}_{d_j}, \hspace{1mm} j\in\mathcal{J}$, which always provide exact results. The contrasting treatment of the constraints (global and approximate with proximal decoding as opposed to local and exact with \ac{LP} decoding using \ac{ADMM}) also leads to different prospects when the decoding process gets stuck in a local minimum. With proximal decoding this occurs due to the approximate nature of the calculation, whereas with \ac{LP} decoding it occurs due to the approximate formulation of the constraints - independent of the optimization method itself. The advantage which arises because of this when employing \ac{LP} decoding is the \ac{ML} certificate property: when a valid codeword is returned, it is also the \ac{ML} codeword. This means that additional redundant parity-checks can be added successively until the codeword returned is valid and thus the \ac{ML} solution is found \cite[Sec. IV.]{alp}. In terms of time complexity the two decoding algorithms are comparable. Each of the operations required for proximal decoding can be performed in $\mathcal{O}\left( n \right) $ time for \ac{LDPC} codes (see section \ref{subsec:prox:comp_perf}). The same is true for \ac{LP} decoding using \ac{ADMM} (see section \ref{subsec:admm:comp_perf}). Additionally, both algorithms can be understood as message-passing algorithms, \ac{LP} decoding using \ac{ADMM} as similarly to \cite[Sec. III. D.]{original_admm} and \cite[Sec. II. B.]{efficient_lp_dec_admm}, and proximal decoding by starting with algorithm \ref{alg:prox}, substituting for the gradient of the code-constraint polynomial and separating the $\boldsymbol{s}$ update into two parts. The algorithms in their message-passing form are depicted in figure \ref{fig:comp:message_passing}. $M_{j\to i}$ denotes a message transmitted from \ac{CN} j to \ac{VN} i. % \begin{figure}[h] \centering \begin{subfigure}{0.48\textwidth} \centering \begin{genericAlgorithm}[caption={}, label={}, % basicstyle=\fontsize{10}{16}\selectfont ] Initialize $\boldsymbol{r}, \boldsymbol{s}, \omega, \gamma$ while stopping critierion unfulfilled do for j in $\mathcal{J}$ do $p_j \leftarrow \prod_{i\in N_c\left( j \right) } r_i $ $M_{j\to i} \leftarrow p_j^2 - p_j$|\Suppressnumber| |\vspace{0.22mm}\Reactivatenumber| end for for i in $\mathcal{I}$ do $s_i \leftarrow s_i + \gamma \left[ 4\left( s_i^2 - 1 \right)s_i \phantom{\frac{4}{s_i}}\right.$|\Suppressnumber| |\Reactivatenumber|$\left.+ \frac{4}{s_i}\sum_{j\in N_v\left( i \right) } M_{j\to i} \right] $ $r_i \leftarrow r_i + \omega \left( s_i - y_i \right)$ end for end while return $\boldsymbol{s}$ \end{genericAlgorithm} \caption{Proximal decoding} \label{fig:comp:message_passing:proximal} \end{subfigure}% \hfill \begin{subfigure}{0.48\textwidth} \centering \begin{genericAlgorithm}[caption={}, label={}, % basicstyle=\fontsize{10}{16}\selectfont ] Initialize $\tilde{\boldsymbol{c}}, \boldsymbol{z}, \boldsymbol{u}, \boldsymbol{\gamma}, \rho$ while stopping criterion unfulfilled do for j in $\mathcal{J}$ do $\boldsymbol{z}_j \leftarrow \Pi_{P_{d_j}}\left( \boldsymbol{T}_j\tilde{\boldsymbol{c}} + \boldsymbol{u}_j\right)$ $\boldsymbol{u}_j \leftarrow \boldsymbol{u}_j + \boldsymbol{T}_j\tilde{\boldsymbol{c}} - \boldsymbol{z}_j$ $M_{j\to i} \leftarrow \left( z_j \right)_i - \left( u_j \right)_i, \hspace{3mm} \forall i \in N_c\left( j \right) $ end for for i in $\mathcal{I}$ do $\tilde{c}_i \leftarrow \frac{1}{d_i} \left(\sum_{j\in N_v\left( i \right) } M_{j\to i} - \frac{\gamma_i}{\mu} \right)$|\Suppressnumber| |\vspace{7mm}\Reactivatenumber| end for end while return $\tilde{\boldsymbol{c}}$ \end{genericAlgorithm} \caption{\ac{LP} decoding using \ac{ADMM}} \label{fig:comp:message_passing:admm} \end{subfigure}% \caption{The proximal gradient method and \ac{LP} decoding using \ac{ADMM} as message passing algorithms} \label{fig:comp:message_passing} \end{figure}% % This message passing structure means that both algorithms can be implemented very efficiently, as the update steps can be performed in parallel for all \acp{CN} and for all \acp{VN}, respectively. In conclusion, the two algorithms have a very similar structure, where the parts of the objective function relating to the likelihood and to the constraints are minimized in an alternating fashion. With proximal decoding this minimization is performed for all constraints at once in an approximative manner, while with \ac{LP} decoding using \ac{ADMM} it is performed for each constraint individually and with exact results. In terms of time complexity, both algorithms are linear with respect to $n$ and are heavily parallelisable. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Comparison of Simulation Results}% \label{sec:comp:res} The decoding performance of the two algorithms is compared in figure \ref{fig:comp:prox_admm_dec} in the form of the \ac{FER}. Shown as well is the performance of the improved proximal decoding algorithm presented in section \ref{sec:prox:Improved Implementation}. The \ac{FER} resulting from decoding using \ac{BP} and, wherever available, the \ac{FER} of \ac{ML} decoding taken from \cite{lautern_channelcodes} are plotted as a reference. The parameters chosen for the proximal and improved proximal decoders are $\gamma=0.05$, $\omega=0.05$, $K=200$, $\eta = 1.5$ and $N=12$. The parameters chosen for \ac{LP} decoding using \ac{ADMM} are $\mu = 5$, $\rho = 1$, $K=200$, $\epsilon_\text{pri} = 10^{-5}$ and $\epsilon_\text{dual} = 10^{-5}$. For all codes considered in the scope of this work, \ac{LP} decoding using \ac{ADMM} consistently outperforms both proximal decoding and the improved version, reaching very similar performance to \ac{BP}. The decoding gain heavily depends on the code, evidently becoming greater for codes with larger $n$ and reaching values of up to $\SI{2}{dB}$. These simulation results can be interpreted with regard to the theoretical structure of the decoding methods, as analyzed in section \ref{sec:comp:theo}. The worse performance of proximal decoding is somewhat surprising, considering the global treatment of the constraints in contrast to the local treatment in the case of \ac{LP} decoding using \ac{ADMM}. It may be explained, however, in the context of the nature of the calculations performed in each case. With proximal decoding, the calculations are approximate, leading to the constraints never being quite satisfied. With \ac{LP} decoding using \ac{ADMM}, the constraints are fulfilled for each parity check individualy after each iteration of the decoding process. It should be noted that while in this thesis proximal decoding was examined with respect to its performance in \ac{AWGN} channels, in \cite{proximal_paper} it is presented as a method applicable to non-trivial channel models such as \ac{LDPC}-coded massive \ac{MIMO} channels, perhaps broadening its usefulness beyond what is shown here. The timing requirements of the decoding algorithms are visualized in figure \ref{fig:comp:time}. The datapoints have been generated by evaluating the metadata from \ac{FER} and \ac{BER} simulations using the parameters mentioned earlier when discussing the decoding performance. The codes considered are the same as in sections \ref{subsec:prox:comp_perf} and \ref{subsec:admm:comp_perf}. While the \ac{ADMM} implementation seems to be faster the the proximal decoding and improved proximal decoding implementations, infering some general behavior is difficult in this case. This is because of the comparison of actual implementations, making the results dependent on factors such as the grade of optimization of each of the implementations. Nevertheless, the run time of both the proximal decoding and the \ac{LP} decoding using \ac{ADMM} implementations is similar and both are reasonably performant, owing to the parallelisable structure of the algorithms. % \begin{figure}[h] \centering \begin{tikzpicture} \begin{axis}[grid=both, xlabel={$n$}, ylabel={Time per frame (ms)}, width=0.6\textwidth, height=0.45\textwidth, legend style={at={(0.5,-0.52)},anchor=south}, legend cell align={left},] \addplot[RedOrange, only marks, mark=square*] table [col sep=comma, x=n, y=spf, y expr=\thisrow{spf} * 1000] {res/proximal/fps_vs_n.csv}; \addlegendentry{Proximal decoding} \addplot[Gray, only marks, mark=*] table [col sep=comma, x=n, y=spf, y expr=\thisrow{spf} * 1000] {res/hybrid/fps_vs_n.csv}; \addlegendentry{Improved proximal decoding ($N=12$)} \addplot[NavyBlue, only marks, mark=triangle*] table [col sep=comma, x=n, y=spf, y expr=\thisrow{spf} * 1000] {res/admm/fps_vs_n.csv}; \addlegendentry{\acs{LP} decoding using \acs{ADMM}} \end{axis} \end{tikzpicture} \caption{Comparison of the timing requirements of the different decoder implementations} \label{fig:comp:time} \end{figure}% % \footnotetext{asdf} % \begin{figure}[h] \centering \begin{subfigure}[t]{0.48\textwidth} \centering \begin{tikzpicture} \begin{axis}[ grid=both, xlabel={$E_b / N_0$ (dB)}, ylabel={FER}, ymode=log, ymax=1.5, ymin=8e-5, width=\textwidth, height=0.75\textwidth, ] \addplot[RedOrange, line width=1pt, mark=*, solid] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/proximal/2d_ber_fer_dfr_963965.csv}; \addplot[RedOrange, line width=1pt, mark=triangle, densely dashed] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/hybrid/2d_ber_fer_dfr_963965.csv}; \addplot[Turquoise, line width=1pt, mark=*] table [x=SNR, y=FER, col sep=comma, discard if not={mu}{3.0}] %{res/hybrid/2d_ber_fer_dfr_963965.csv}; {res/admm/ber_2d_963965.csv}; \addplot[Black, line width=1pt, mark=*] table [col sep=comma, x=SNR, y=FER,] {res/generic/fer_ml_9633965.csv}; \addplot [RoyalPurple, mark=*, line width=1pt] table [x=SNR, y=FER, col sep=comma] {res/generic/bp_963965.csv}; \end{axis} \end{tikzpicture} \caption{$\left( 3, 6 \right)$-regular \ac{LDPC} code with $n=96, k=48$ \cite[\text{96.3.965}]{mackay_enc}} \end{subfigure}% \hfill% \begin{subfigure}[t]{0.48\textwidth} \centering \begin{tikzpicture} \begin{axis}[ grid=both, xlabel={$E_b / N_0$ (dB)}, ylabel={FER}, ymode=log, ymax=1.5, ymin=8e-5, width=\textwidth, height=0.75\textwidth, ] \addplot[RedOrange, line width=1pt, mark=*, solid] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/proximal/2d_ber_fer_dfr_bch_31_26.csv}; \addplot[RedOrange, line width=1pt, mark=triangle, densely dashed] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/hybrid/2d_ber_fer_dfr_bch_31_26.csv}; \addplot[Turquoise, line width=1pt, mark=*] table [x=SNR, y=FER, col sep=comma, discard if not={mu}{3.0}] {res/admm/ber_2d_bch_31_26.csv}; \addplot[Black, line width=1pt, mark=*] table [x=SNR, y=FER, col sep=comma, discard if gt={SNR}{5.5}, discard if lt={SNR}{1},] {res/generic/fer_ml_bch_31_26.csv}; \addplot [RoyalPurple, mark=*, line width=1pt] table [x=SNR, y=FER, col sep=comma] {res/generic/bp_bch_31_26.csv}; \end{axis} \end{tikzpicture} \caption{BCH code with $n=31, k=26$} \end{subfigure}% \vspace{3mm} \begin{subfigure}[t]{0.48\textwidth} \centering \begin{tikzpicture} \begin{axis}[ grid=both, xlabel={$E_b / N_0$ (dB)}, ylabel={FER}, ymode=log, ymax=1.5, ymin=8e-5, width=\textwidth, height=0.75\textwidth, ] \addplot[RedOrange, line width=1pt, mark=*, solid] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}, discard if gt={SNR}{5.5}] {res/proximal/2d_ber_fer_dfr_20433484.csv}; \addplot[RedOrange, line width=1pt, mark=triangle, densely dashed] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}, discard if gt={SNR}{5.5}] {res/hybrid/2d_ber_fer_dfr_20433484.csv}; \addplot[Turquoise, line width=1pt, mark=*] table [x=SNR, y=FER, col sep=comma, discard if not={mu}{3.0}, discard if gt={SNR}{5.5}] {res/admm/ber_2d_20433484.csv}; \addplot[Black, line width=1pt, mark=*] table [col sep=comma, x=SNR, y=FER, discard if gt={SNR}{5.5}] {res/generic/fer_ml_20433484.csv}; \addplot [RoyalPurple, mark=*, line width=1pt] table [x=SNR, y=FER, col sep=comma] {res/generic/bp_20433484.csv}; \end{axis} \end{tikzpicture} \caption{$\left( 3, 6 \right)$-regular \ac{LDPC} code with $n=204, k=102$ \cite[\text{204.33.484}]{mackay_enc}} \end{subfigure}% \hfill% \begin{subfigure}[t]{0.48\textwidth} \centering \begin{tikzpicture} \begin{axis}[ grid=both, xlabel={$E_b / N_0$ (dB)}, ylabel={FER}, ymode=log, ymax=1.5, ymin=8e-5, width=\textwidth, height=0.75\textwidth, ] \addplot[RedOrange, line width=1pt, mark=*, solid] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/proximal/2d_ber_fer_dfr_20455187.csv}; \addplot[RedOrange, line width=1pt, mark=triangle, densely dashed] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/hybrid/2d_ber_fer_dfr_20455187.csv}; \addplot[Turquoise, line width=1pt, mark=*] table [x=SNR, y=FER, col sep=comma, discard if not={mu}{3.0}] {res/admm/ber_2d_20455187.csv}; \addplot [RoyalPurple, mark=*, line width=1pt, discard if gt={SNR}{5}] table [x=SNR, y=FER, col sep=comma] {res/generic/bp_20455187.csv}; \end{axis} \end{tikzpicture} \caption{$\left( 5, 10 \right)$-regular \ac{LDPC} code with $n=204, k=102$ \cite[\text{204.55.187}]{mackay_enc}} \end{subfigure}% \vspace{3mm} \begin{subfigure}[t]{0.48\textwidth} \centering \begin{tikzpicture} \begin{axis}[ grid=both, xlabel={$E_b / N_0$ (dB)}, ylabel={FER}, ymode=log, ymax=1.5, ymin=8e-5, width=\textwidth, height=0.75\textwidth, ] \addplot[RedOrange, line width=1pt, mark=*, solid] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/proximal/2d_ber_fer_dfr_40833844.csv}; \addplot[RedOrange, line width=1pt, mark=triangle, densely dashed] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/hybrid/2d_ber_fer_dfr_40833844.csv}; \addplot[Turquoise, line width=1pt, mark=*] table [x=SNR, y=FER, col sep=comma, discard if not={mu}{3.0}] {res/admm/ber_2d_40833844.csv}; \addplot [RoyalPurple, mark=*, line width=1pt, discard if gt={SNR}{3}] table [x=SNR, y=FER, col sep=comma] {res/generic/bp_40833844.csv}; \end{axis} \end{tikzpicture} \caption{$\left( 3, 6 \right)$-regular \ac{LDPC} code with $n=204, k=102$ \cite[\text{204.33.484}]{mackay_enc}} \end{subfigure}% \hfill% \begin{subfigure}[t]{0.48\textwidth} \centering \begin{tikzpicture} \begin{axis}[ grid=both, xlabel={$E_b / N_0$ (dB)}, ylabel={FER}, ymode=log, ymax=1.5, ymin=8e-5, width=\textwidth, height=0.75\textwidth, ] \addplot[RedOrange, line width=1pt, mark=*, solid] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/proximal/2d_ber_fer_dfr_pegreg252x504.csv}; \addplot[RedOrange, line width=1pt, mark=triangle, densely dashed] table [x=SNR, y=FER, col sep=comma, discard if not={gamma}{0.05}] {res/hybrid/2d_ber_fer_dfr_pegreg252x504.csv}; \addplot[Turquoise, line width=1pt, mark=*] table [x=SNR, y=FER, col sep=comma, discard if not={mu}{3.0}] {res/admm/ber_2d_pegreg252x504.csv}; \addplot [RoyalPurple, mark=*, line width=1pt] table [x=SNR, y=FER, col sep=comma, discard if gt={SNR}{3}] {res/generic/bp_pegreg252x504.csv}; \end{axis} \end{tikzpicture} \caption{LDPC code (progressive edge growth construction) with $n=504, k=252$ \cite[\text{PEGReg252x504}]{mackay_enc}} \end{subfigure}% \vspace{5mm} \begin{subfigure}[t]{\textwidth} \centering \begin{tikzpicture} \begin{axis}[hide axis, xmin=10, xmax=50, ymin=0, ymax=0.4, legend columns=1, legend cell align={left}, legend style={draw=white!15!black}] \addlegendimage{RedOrange, line width=1pt, mark=*, solid} \addlegendentry{Proximal decoding} \addlegendimage{RedOrange, line width=1pt, mark=triangle, densely dashed} \addlegendentry{Improved proximal decoding} \addlegendimage{Turquoise, line width=1pt, mark=*} \addlegendentry{\acs{LP} decoding using \acs{ADMM}} \addlegendimage{RoyalPurple, line width=1pt, mark=*, solid} \addlegendentry{\acs{BP} (20 iterations)} \addlegendimage{Black, line width=1pt, mark=*, solid} \addlegendentry{\acs{ML} decoding} \end{axis} \end{tikzpicture} \end{subfigure} \caption{Comparison of decoding performance between proximal decoding and \ac{LP} decoding using \ac{ADMM}} \label{fig:comp:prox_admm_dec} \end{figure}