an.tsouchlos/ba-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Rotated cube; continued writing lp decoding

Browse Source
This commit is contained in:
Andreas Tsouchlos 2023-02-18 15:58:15 +01:00
parent 033cdb5197
commit 1a3e9ffbe4
1 changed files with 96 additions and 47 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

143
latex/thesis/chapters/decoding_techniques.tex
View File

@ -90,36 +90,58 @@ which minimizes the objective function $f$ (as shown in figure \ref{fig:dec:spac
\tikzstyle{codeword} = [color=KITblue, fill=KITblue, \tikzstyle{codeword} = [color=KITblue, fill=KITblue,
draw, circle, inner sep=0pt, minimum size=4pt] draw, circle, inner sep=0pt, minimum size=4pt]
\tdplotsetmaincoords{60}{245} \tdplotsetmaincoords{60}{25}
\begin{tikzpicture}[scale=1, transform shape, tdplot_main_coords] \begin{tikzpicture}[scale=1, transform shape, tdplot_main_coords]
% Cube % Cube
\draw[dashed] (0, 0, 0) -- (2, 0, 0); \coordinate (p000) at (0, 0, 0);
\draw[dashed] (2, 0, 0) -- (2, 0, 2); \coordinate (p001) at (0, 0, 2);
\draw[] (2, 0, 2) -- (0, 0, 2); \coordinate (p010) at (0, 2, 0);
\draw[] (0, 0, 2) -- (0, 0, 0); \coordinate (p011) at (0, 2, 2);
\coordinate (p100) at (2, 0, 0);
\coordinate (p101) at (2, 0, 2);
\coordinate (p110) at (2, 2, 0);
\coordinate (p111) at (2, 2, 2);
\draw[] (0, 2, 0) -- (2, 2, 0); \draw[] (p000) -- (p100);
\draw[] (2, 2, 0) -- (2, 2, 2); \draw[] (p100) -- (p101);
\draw[] (2, 2, 2) -- (0, 2, 2); \draw[] (p101) -- (p001);
\draw[] (0, 2, 2) -- (0, 2, 0); \draw[] (p001) -- (p000);
\draw[] (0, 0, 0) -- (0, 2, 0); \draw[dashed] (p010) -- (p110);
\draw[dashed] (2, 0, 0) -- (2, 2, 0); \draw[] (p110) -- (p111);
\draw[] (2, 0, 2) -- (2, 2, 2); \draw[] (p111) -- (p011);
\draw[] (0, 0, 2) -- (0, 2, 2); \draw[dashed] (p011) -- (p010);
\draw[dashed] (p000) -- (p010);
\draw[] (p100) -- (p110);
\draw[] (p101) -- (p111);
\draw[] (p001) -- (p011);
% Polytope Vertices
\node[codeword] (c000) at (p000) {};
\node[codeword] (c101) at (p101) {};
\node[codeword] (c110) at (p110) {};
\node[codeword] (c011) at (p011) {};
% Polytope Edges
% \draw[line width=1pt, color=KITblue] (c000) -- (c101);
% \draw[line width=1pt, color=KITblue] (c000) -- (c110);
% \draw[line width=1pt, color=KITblue] (c000) -- (c011);
%
% \draw[line width=1pt, color=KITblue] (c101) -- (c110);
% \draw[line width=1pt, color=KITblue] (c101) -- (c011);
%
% \draw[line width=1pt, color=KITblue] (c011) -- (c110);
% Polytope Annotations % Polytope Annotations
\node[codeword] (c000) at (0, 0, 0) {};% {$\left( 0, 0, 0 \right) $}; \node[color=KITblue, below=0cm of c000] {$\left( 0, 0, 0 \right) $};
\node[codeword] (c101) at (2, 0, 2) {};% {$\left( 1, 0, 1 \right) $}; \node[color=KITblue, right=0.17cm of c101] {$\left( 1, 0, 1 \right) $};
\node[codeword] (c110) at (2, 2, 0) {};% {$\left( 1, 1, 0 \right) $}; \node[color=KITblue, right=0cm of c110] {$\left( 1, 1, 0 \right) $};
\node[codeword] (c011) at (0, 2, 2) {};% {$\left( 0, 1, 1 \right) $}; \node[color=KITblue, above=0cm of c011] {$\left( 0, 1, 1 \right) $};
\node[color=KITblue, right=0cm of c000] {$\left( 0, 0, 0 \right) $};
\node[color=KITblue, above=0cm of c101] {$\left( 1, 0, 1 \right) $};
\node[color=KITblue, left=0cm of c110] {$\left( 1, 1, 0 \right) $};
\node[color=KITblue, left=-0.1cm of c011] {$\left( 0, 1, 1 \right) $};
% x % x
@ -146,15 +168,16 @@ which minimizes the objective function $f$ (as shown in figure \ref{fig:dec:spac
\cite{feldman_paper}. They reframe the decoding problem as an \cite{feldman_paper}. They reframe the decoding problem as an
\textit{integer linear program} and subsequently present two relaxations into \textit{integer linear program} and subsequently present two relaxations into
\textit{linear programs}, one representing a formulation of exact \ac{LP} \textit{linear programs}, one representing a formulation of exact \ac{LP}
decoding and an approximation with a more manageable representation. decoding and one, which is an approximation with a more manageable
representation.
To solve the resulting linear program, various optimization methods can be To solve the resulting linear program, various optimization methods can be
used. used;
The one examined in this work is \ac{ADMM}. the one examined in this work is \ac{ADMM}.
\todo{Why chose ADMM?} \todo{Why chose ADMM?}
Feldman at al. begin by looking at the \ac{ML} decoding problem% Feldman at al. begin by looking at the \ac{ML} decoding problem%
\footnote{They assume that all codewords are equally likely to be transmitted, \footnote{They assume that all codewords are equally likely to be transmitted,
making the \ac{ML} and \ac{MAP} decoding problems essentially equivalent.}% making the \ac{ML} and \ac{MAP} decoding problems equivalent.}%
% %
\begin{align*} \begin{align*}
\hat{\boldsymbol{c}} = \argmax_{\boldsymbol{c} \in \mathcal{C}} \hat{\boldsymbol{c}} = \argmax_{\boldsymbol{c} \in \mathcal{C}}
@ -193,8 +216,8 @@ decoding is the following:%
As solving integer linear programs is generally NP-hard, this decoding problem As solving integer linear programs is generally NP-hard, this decoding problem
has to be approximated by one with looser constraints. has to be approximated by one with looser constraints.
A technique called \textit{relaxation} is applied, A technique called \textit{relaxation} is applied:
essentially modifying the constraints in order to broaden the considered modifying the constraints in order to broaden the considered
domain (e.g. by lifting the integer requirement). domain (e.g. by lifting the integer requirement).
First, the authors present an equivalent \ac{LP} formulation of exact \ac{ML} First, the authors present an equivalent \ac{LP} formulation of exact \ac{ML}
decoding, redefining the constraints in terms of the \text{codeword polytope} decoding, redefining the constraints in terms of the \text{codeword polytope}
@ -207,11 +230,29 @@ decoding, redefining the constraints in terms of the \text{codeword polytope}
,\end{align*} % ,\end{align*} %
% %
which represents the \textit{convex hull} of all possible codewords, which represents the \textit{convex hull} of all possible codewords,
i.e. the set of convex linear combinations of all codewords i.e. the set of convex linear combinations of all codewords.
(visualized in figure \ref{fig:dec:poly}). However, since the number of constraints needed to characterize the codeword
However, since the number of constraints needed to characterize this codeword
polytope is exponential in the code length, this formulation is relaxed futher. polytope is exponential in the code length, this formulation is relaxed futher.
By observing that each check-node defines its own local single parity-check
code, and thus its own \textit{local codeword polytope},
the \textit{relaxed codeword polytope} $Q$ is defined as the intersection of all
local codeword polytopes.
This consideration leads to the following constraints:%
%
\begin{align*}
\ldots
.\end{align*}
In figure \ref{fig:dec:poly} the two relaxations are compared based on an
example code.
Figure \ref{fig:dec:poly:exact} shows the codeword polytope
$\text{poly}\left( \mathcal{C} \right) $, i.e. the constraints for the
equivalent linear program to exact \ac{ML} decoding - only valid codewords are
feasible solutions.
Figures \ref{fig:dec:poly:local1} and \ref{fig:dec:poly:local2} show the local
codeword polytopes of each check node.
Their intersection, the relaxed codeword polytope $Q$, is shown in figure
\ref{fig:dec:poly:relaxed}.
% %
% Codeword polytope visualization figure % Codeword polytope visualization figure
@ -251,7 +292,7 @@ polytope is exponential in the code length, this formulation is relaxed futher.
\end{align*} \end{align*}
\caption{Definition of the visualized code} \caption{Definition of the visualized code}
\label{fig:} \label{fig:dec:poly:code_def}
\end{subfigure} \\[7em] \end{subfigure} \\[7em]
\begin{subfigure}{\textwidth} \begin{subfigure}{\textwidth}
\centering \centering
@ -302,8 +343,8 @@ polytope is exponential in the code length, this formulation is relaxed futher.
\node[color=KITblue, above=0cm of c011] {$\left( 0, 1, 1 \right) $}; \node[color=KITblue, above=0cm of c011] {$\left( 0, 1, 1 \right) $};
\end{tikzpicture} \end{tikzpicture}
\caption{Codeword polytope} \caption{Codeword polytope $\text{poly}\left( \mathcal{C} \right) $}
\label{fig:} \label{fig:dec:poly:exact}
\end{subfigure} \end{subfigure}
\end{subfigure} \hfill% \end{subfigure} \hfill%
% %
@ -374,9 +415,9 @@ polytope is exponential in the code length, this formulation is relaxed futher.
\node[color=KITblue, above=0cm of c011] {$\left( 0, 1, 1 \right) $}; \node[color=KITblue, above=0cm of c011] {$\left( 0, 1, 1 \right) $};
\end{tikzpicture} \end{tikzpicture}
\caption{Local codeword polytope of parity-check \caption{Local codeword polytope of check node\\ $j=1$
$\begin{bmatrix} 1 & 1 & 1 \end{bmatrix}$} $\left( c_1 + c_2 + c_3 = 0 \right)$}
\label{fig:} \label{fig:dec:poly:local1}
\end{subfigure} \\[1em] \end{subfigure} \\[1em]
\begin{subfigure}{\textwidth} \begin{subfigure}{\textwidth}
\centering \centering
@ -434,8 +475,9 @@ polytope is exponential in the code length, this formulation is relaxed futher.
\node[color=KITblue, above=0cm of c111] {$\left( 1, 1, 1 \right) $}; \node[color=KITblue, above=0cm of c111] {$\left( 1, 1, 1 \right) $};
\end{tikzpicture} \end{tikzpicture}
\caption{Local codeword polytope of parity-check \caption{Local codeword polytope of check node\\ $j=2$
$\begin{bmatrix} 0 & 1 & 1 \end{bmatrix}$} $\left( c_2 + c_3 = 0\right)$}
\label{fig:dec:poly:local2}
\end{subfigure}\\[1em] \end{subfigure}\\[1em]
\begin{subfigure}{\textwidth} \begin{subfigure}{\textwidth}
\centering \centering
@ -493,8 +535,8 @@ polytope is exponential in the code length, this formulation is relaxed futher.
{$\left( 1, \frac{1}{2}, \frac{1}{2} \right) $}; {$\left( 1, \frac{1}{2}, \frac{1}{2} \right) $};
\end{tikzpicture} \end{tikzpicture}
\caption{Relaxed codeword polytope} \caption{Relaxed codeword polytope $Q$}
\label{fig:} \label{fig:dec:poly:relaxed}
\end{subfigure} \end{subfigure}
\end{subfigure} \end{subfigure}
@ -503,10 +545,17 @@ polytope is exponential in the code length, this formulation is relaxed futher.
\label{fig:dec:poly} \label{fig:dec:poly}
\end{figure} \end{figure}
\noindent%
It can be seen, that the relaxed codeword polytope $Q$ introduces fractional
vertices;
these represent erroneous non-codeword solutions to the linear program and
correspond to the so-called \textit{pseudocodewords} introduced in
\cite{feldman_paper}.
However, since for \ac{LDPC} codes $Q$ scales linearly with $n$, it is a lot
more tractable for practical applications.
\begin{itemize} \begin{itemize}
\item Equivalent \ac{ML} optimization problem \item TODO: \Ac{ADMM} as a solver
\item \Ac{LP} relaxation
\item \Ac{ADMM} as a solver
\end{itemize} \end{itemize}
@ -595,7 +644,7 @@ $f\left( \boldsymbol{x} \right)$ to be minimized is%
.\end{align} .\end{align}
For the solution of the approximalte \ac{MAP} decoding problem, the two parts For the solution of the approximalte \ac{MAP} decoding problem, the two parts
of equation \ref{eq:prox:approx_map_problem} are considered separately: of \ref{eq:prox:objective_function} are considered separately:
the minimization of the objective function occurs in an alternating the minimization of the objective function occurs in an alternating
manner, switching between the minimization of the negative log-likelihood manner, switching between the minimization of the negative log-likelihood
$L\left( \boldsymbol{y} \mid \boldsymbol{x} \right) $ and the scaled $L\left( \boldsymbol{y} \mid \boldsymbol{x} \right) $ and the scaled
@ -637,7 +686,7 @@ theoretically becomes better
with larger $\gamma$, the constraint that $\gamma$ be small is important, with larger $\gamma$, the constraint that $\gamma$ be small is important,
as it keeps the effect of $h\left( \boldsymbol{x} \right) $ on the landscape as it keeps the effect of $h\left( \boldsymbol{x} \right) $ on the landscape
of the objective function small. of the objective function small.
Otherwise, unwanted stationary points, including local minima are introduced. Otherwise, unwanted stationary points, including local minima, are introduced.
The authors say that in practice, the value of $\gamma$ should be adjusted The authors say that in practice, the value of $\gamma$ should be adjusted
according to the decoding performance. according to the decoding performance.
The iterative decoding process \todo{projection with $\eta$} resulting from this considreation is shown in The iterative decoding process \todo{projection with $\eta$} resulting from this considreation is shown in
@ -687,7 +736,7 @@ is%
% %
Thus, the gradient of the negative log-likelihood becomes% Thus, the gradient of the negative log-likelihood becomes%
\footnote{For the minimization, constants can be disregarded. For this reason, \footnote{For the minimization, constants can be disregarded. For this reason,
it suffices to consider only the proportionality instead of the equality}% it suffices to consider only the proportionality instead of the equality.}%
% %
\begin{align*} \begin{align*}
\nabla L \left( \boldsymbol{y} \mid \boldsymbol{x} \right) \nabla L \left( \boldsymbol{y} \mid \boldsymbol{x} \right)