From 1a3e9ffbe4741e6d01dca117233ca63b9cabecbf Mon Sep 17 00:00:00 2001 From: Andreas Tsouchlos Date: Sat, 18 Feb 2023 15:58:15 +0100 Subject: [PATCH] Rotated cube; continued writing lp decoding --- latex/thesis/chapters/decoding_techniques.tex | 143 ++++++++++++------ 1 file changed, 96 insertions(+), 47 deletions(-) diff --git a/latex/thesis/chapters/decoding_techniques.tex b/latex/thesis/chapters/decoding_techniques.tex index 375e301..7303ab6 100644 --- a/latex/thesis/chapters/decoding_techniques.tex +++ b/latex/thesis/chapters/decoding_techniques.tex @@ -90,36 +90,58 @@ which minimizes the objective function $f$ (as shown in figure \ref{fig:dec:spac \tikzstyle{codeword} = [color=KITblue, fill=KITblue, draw, circle, inner sep=0pt, minimum size=4pt] - \tdplotsetmaincoords{60}{245} + \tdplotsetmaincoords{60}{25} \begin{tikzpicture}[scale=1, transform shape, tdplot_main_coords] % Cube - \draw[dashed] (0, 0, 0) -- (2, 0, 0); - \draw[dashed] (2, 0, 0) -- (2, 0, 2); - \draw[] (2, 0, 2) -- (0, 0, 2); - \draw[] (0, 0, 2) -- (0, 0, 0); + \coordinate (p000) at (0, 0, 0); + \coordinate (p001) at (0, 0, 2); + \coordinate (p010) at (0, 2, 0); + \coordinate (p011) at (0, 2, 2); + \coordinate (p100) at (2, 0, 0); + \coordinate (p101) at (2, 0, 2); + \coordinate (p110) at (2, 2, 0); + \coordinate (p111) at (2, 2, 2); - \draw[] (0, 2, 0) -- (2, 2, 0); - \draw[] (2, 2, 0) -- (2, 2, 2); - \draw[] (2, 2, 2) -- (0, 2, 2); - \draw[] (0, 2, 2) -- (0, 2, 0); + \draw[] (p000) -- (p100); + \draw[] (p100) -- (p101); + \draw[] (p101) -- (p001); + \draw[] (p001) -- (p000); - \draw[] (0, 0, 0) -- (0, 2, 0); - \draw[dashed] (2, 0, 0) -- (2, 2, 0); - \draw[] (2, 0, 2) -- (2, 2, 2); - \draw[] (0, 0, 2) -- (0, 2, 2); + \draw[dashed] (p010) -- (p110); + \draw[] (p110) -- (p111); + \draw[] (p111) -- (p011); + \draw[dashed] (p011) -- (p010); + + \draw[dashed] (p000) -- (p010); + \draw[] (p100) -- (p110); + \draw[] (p101) -- (p111); + \draw[] (p001) -- (p011); + + % Polytope Vertices + + \node[codeword] (c000) at (p000) {}; + \node[codeword] (c101) at (p101) {}; + \node[codeword] (c110) at (p110) {}; + \node[codeword] (c011) at (p011) {}; + + % Polytope Edges + +% \draw[line width=1pt, color=KITblue] (c000) -- (c101); +% \draw[line width=1pt, color=KITblue] (c000) -- (c110); +% \draw[line width=1pt, color=KITblue] (c000) -- (c011); +% +% \draw[line width=1pt, color=KITblue] (c101) -- (c110); +% \draw[line width=1pt, color=KITblue] (c101) -- (c011); +% +% \draw[line width=1pt, color=KITblue] (c011) -- (c110); % Polytope Annotations - \node[codeword] (c000) at (0, 0, 0) {};% {$\left( 0, 0, 0 \right) $}; - \node[codeword] (c101) at (2, 0, 2) {};% {$\left( 1, 0, 1 \right) $}; - \node[codeword] (c110) at (2, 2, 0) {};% {$\left( 1, 1, 0 \right) $}; - \node[codeword] (c011) at (0, 2, 2) {};% {$\left( 0, 1, 1 \right) $}; - - \node[color=KITblue, right=0cm of c000] {$\left( 0, 0, 0 \right) $}; - \node[color=KITblue, above=0cm of c101] {$\left( 1, 0, 1 \right) $}; - \node[color=KITblue, left=0cm of c110] {$\left( 1, 1, 0 \right) $}; - \node[color=KITblue, left=-0.1cm of c011] {$\left( 0, 1, 1 \right) $}; + \node[color=KITblue, below=0cm of c000] {$\left( 0, 0, 0 \right) $}; + \node[color=KITblue, right=0.17cm of c101] {$\left( 1, 0, 1 \right) $}; + \node[color=KITblue, right=0cm of c110] {$\left( 1, 1, 0 \right) $}; + \node[color=KITblue, above=0cm of c011] {$\left( 0, 1, 1 \right) $}; % x @@ -146,15 +168,16 @@ which minimizes the objective function $f$ (as shown in figure \ref{fig:dec:spac \cite{feldman_paper}. They reframe the decoding problem as an \textit{integer linear program} and subsequently present two relaxations into \textit{linear programs}, one representing a formulation of exact \ac{LP} -decoding and an approximation with a more manageable representation. +decoding and one, which is an approximation with a more manageable +representation. To solve the resulting linear program, various optimization methods can be -used. -The one examined in this work is \ac{ADMM}. +used; +the one examined in this work is \ac{ADMM}. \todo{Why chose ADMM?} Feldman at al. begin by looking at the \ac{ML} decoding problem% \footnote{They assume that all codewords are equally likely to be transmitted, -making the \ac{ML} and \ac{MAP} decoding problems essentially equivalent.}% +making the \ac{ML} and \ac{MAP} decoding problems equivalent.}% % \begin{align*} \hat{\boldsymbol{c}} = \argmax_{\boldsymbol{c} \in \mathcal{C}} @@ -193,8 +216,8 @@ decoding is the following:% As solving integer linear programs is generally NP-hard, this decoding problem has to be approximated by one with looser constraints. -A technique called \textit{relaxation} is applied, -essentially modifying the constraints in order to broaden the considered +A technique called \textit{relaxation} is applied: +modifying the constraints in order to broaden the considered domain (e.g. by lifting the integer requirement). First, the authors present an equivalent \ac{LP} formulation of exact \ac{ML} decoding, redefining the constraints in terms of the \text{codeword polytope} @@ -207,11 +230,29 @@ decoding, redefining the constraints in terms of the \text{codeword polytope} ,\end{align*} % % which represents the \textit{convex hull} of all possible codewords, -i.e. the set of convex linear combinations of all codewords -(visualized in figure \ref{fig:dec:poly}). -However, since the number of constraints needed to characterize this codeword +i.e. the set of convex linear combinations of all codewords. +However, since the number of constraints needed to characterize the codeword polytope is exponential in the code length, this formulation is relaxed futher. +By observing that each check-node defines its own local single parity-check +code, and thus its own \textit{local codeword polytope}, +the \textit{relaxed codeword polytope} $Q$ is defined as the intersection of all +local codeword polytopes. +This consideration leads to the following constraints:% +% +\begin{align*} + \ldots +.\end{align*} +In figure \ref{fig:dec:poly} the two relaxations are compared based on an +example code. +Figure \ref{fig:dec:poly:exact} shows the codeword polytope +$\text{poly}\left( \mathcal{C} \right) $, i.e. the constraints for the +equivalent linear program to exact \ac{ML} decoding - only valid codewords are +feasible solutions. +Figures \ref{fig:dec:poly:local1} and \ref{fig:dec:poly:local2} show the local +codeword polytopes of each check node. +Their intersection, the relaxed codeword polytope $Q$, is shown in figure +\ref{fig:dec:poly:relaxed}. % % Codeword polytope visualization figure @@ -251,7 +292,7 @@ polytope is exponential in the code length, this formulation is relaxed futher. \end{align*} \caption{Definition of the visualized code} - \label{fig:} + \label{fig:dec:poly:code_def} \end{subfigure} \\[7em] \begin{subfigure}{\textwidth} \centering @@ -302,8 +343,8 @@ polytope is exponential in the code length, this formulation is relaxed futher. \node[color=KITblue, above=0cm of c011] {$\left( 0, 1, 1 \right) $}; \end{tikzpicture} - \caption{Codeword polytope} - \label{fig:} + \caption{Codeword polytope $\text{poly}\left( \mathcal{C} \right) $} + \label{fig:dec:poly:exact} \end{subfigure} \end{subfigure} \hfill% % @@ -374,9 +415,9 @@ polytope is exponential in the code length, this formulation is relaxed futher. \node[color=KITblue, above=0cm of c011] {$\left( 0, 1, 1 \right) $}; \end{tikzpicture} - \caption{Local codeword polytope of parity-check - $\begin{bmatrix} 1 & 1 & 1 \end{bmatrix}$} - \label{fig:} + \caption{Local codeword polytope of check node\\ $j=1$ + $\left( c_1 + c_2 + c_3 = 0 \right)$} + \label{fig:dec:poly:local1} \end{subfigure} \\[1em] \begin{subfigure}{\textwidth} \centering @@ -434,8 +475,9 @@ polytope is exponential in the code length, this formulation is relaxed futher. \node[color=KITblue, above=0cm of c111] {$\left( 1, 1, 1 \right) $}; \end{tikzpicture} - \caption{Local codeword polytope of parity-check - $\begin{bmatrix} 0 & 1 & 1 \end{bmatrix}$} + \caption{Local codeword polytope of check node\\ $j=2$ + $\left( c_2 + c_3 = 0\right)$} + \label{fig:dec:poly:local2} \end{subfigure}\\[1em] \begin{subfigure}{\textwidth} \centering @@ -493,8 +535,8 @@ polytope is exponential in the code length, this formulation is relaxed futher. {$\left( 1, \frac{1}{2}, \frac{1}{2} \right) $}; \end{tikzpicture} - \caption{Relaxed codeword polytope} - \label{fig:} + \caption{Relaxed codeword polytope $Q$} + \label{fig:dec:poly:relaxed} \end{subfigure} \end{subfigure} @@ -503,10 +545,17 @@ polytope is exponential in the code length, this formulation is relaxed futher. \label{fig:dec:poly} \end{figure} +\noindent% +It can be seen, that the relaxed codeword polytope $Q$ introduces fractional +vertices; +these represent erroneous non-codeword solutions to the linear program and +correspond to the so-called \textit{pseudocodewords} introduced in +\cite{feldman_paper}. +However, since for \ac{LDPC} codes $Q$ scales linearly with $n$, it is a lot +more tractable for practical applications. + \begin{itemize} - \item Equivalent \ac{ML} optimization problem - \item \Ac{LP} relaxation - \item \Ac{ADMM} as a solver + \item TODO: \Ac{ADMM} as a solver \end{itemize} @@ -595,7 +644,7 @@ $f\left( \boldsymbol{x} \right)$ to be minimized is% .\end{align} For the solution of the approximalte \ac{MAP} decoding problem, the two parts -of equation \ref{eq:prox:approx_map_problem} are considered separately: +of \ref{eq:prox:objective_function} are considered separately: the minimization of the objective function occurs in an alternating manner, switching between the minimization of the negative log-likelihood $L\left( \boldsymbol{y} \mid \boldsymbol{x} \right) $ and the scaled @@ -637,7 +686,7 @@ theoretically becomes better with larger $\gamma$, the constraint that $\gamma$ be small is important, as it keeps the effect of $h\left( \boldsymbol{x} \right) $ on the landscape of the objective function small. -Otherwise, unwanted stationary points, including local minima are introduced. +Otherwise, unwanted stationary points, including local minima, are introduced. The authors say that in practice, the value of $\gamma$ should be adjusted according to the decoding performance. The iterative decoding process \todo{projection with $\eta$} resulting from this considreation is shown in @@ -687,7 +736,7 @@ is% % Thus, the gradient of the negative log-likelihood becomes% \footnote{For the minimization, constants can be disregarded. For this reason, -it suffices to consider only the proportionality instead of the equality}% +it suffices to consider only the proportionality instead of the equality.}% % \begin{align*} \nabla L \left( \boldsymbol{y} \mid \boldsymbol{x} \right)