Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Low-density parity-check codes: tracking non-stationary channel noise using sequential variational Bayesian estimates

Low-density parity-check codes: tracking non-stationary channel noise using sequential... We present a sequential Bayesian learning method for tracking non-stationary signal-to-noise ratios in low-density parity- check (LDPC) codes by way of probabilistic graphical models. We represent the LDPC code as a cluster graph using a general purpose cluster graph construction algorithm called the layered trees running intersection property (LTRIP) algorithm. The channel noise estimator is a global gamma cluster, which we extend to allow for Bayesian tracking of non-stationary noise variation. We evaluate our proposed model on real-world 5G drive-test data. Our results show that our model can track non- stationary channel noise accurately while adding performance benefits to the LDPC code, which outperforms an LDPC code with a fixed stationary knowledge of the actual channel noise. Keywords LDPC codes · Bayesian sequential learning · Variational inference · Cluster graphs 1 Introduction cally via the physical uplink shared channel (PUSCH) [1]. Instantaneous estimates of the channel noise should ideally In wireless communication systems, channel noise interferes be acquired at the same rate as the channel changes, which with radio transmissions between a transmitter and receiver. require frequent use of pilot signals. The downside of doing The nature of the noise is assumed to be non-stationary, since so is an additional communication overhead, since the time- it can change over time and vary according to environmental frequency resources could be utilised to carry payload data dynamics. instead [2]. Knowledge of channel noise is useful in a communica- Our interest is in finding a way to estimate the chan- tion system to adapt certain radio parameters, ultimately nel noise variance applicable to the additive white Gaussian optimising the reliability and overall quality of communica- noise (AWGN) channel without relying on scheduled pilot tion. These optimisation methods include, inter alia, adaptive signals. The aim of our work is not to omit the broader chan- modulation and coding (AMC), interference suppression, nel state information (CSI) system in its entirety, since it and rate-matching. also provides information on other channel properties out- Communication systems such as LTE and 5G rely on pre- side the scope of this work. Instead, our focus is on reliably defined reference signals, i.e. pilot signals, which are used and dynamically estimating the channel noise variance only, to estimate the channel noise. These pilot signals are gen- which we see as a step towards substituting certain elements erated in the physical layer and scheduled periodically via of the CSI system. We propose to combine received signal the physical uplink control channel (PUCCH), or aperiodi- values from the physical layer with a channel-coding scheme such as a low-density parity-check (LDPC) code. By doing so, we modify a standard LDPC code’s decoding algorithm, B J. du Toit which can learn the channel noise on-the-fly and simultane- [email protected] ously benefit from the channel-noise estimates. We are not J. du Preez attempting to design new LDPC codes, but rather explore [email protected] modifications to the existing LDPC decoding algorithm so R. Wolhuter that our methods can be applied to generic LDPC codes. [email protected] LDPC codes were first introduced by Gallager in 1962 Department of Electrical and Electronic Engineering, and rediscovered in 1995 by MacKay [3, 4]. These codes are Stellenbosch University, Stellenbosch, South Africa 123 J. du Toit et al. a family of block codes with good theoretical and practical no significant increase in computational complexity. In [11, properties. They have found extensive application in vari- 16, 17] SNRs are assumed stationary within fixed indepen- ous wireless communication systems, including the 5G New dent sections of the LDPC packet with a focus on finding a Radio (NR) technology standard, which will be used in our low complexity noise estimator. These studies do not model study. the sequential dynamics of channel noise variance, e.g. the While knowledge of the channel noise is useful for possible correlations between inferred instances of SNRs. aspects of the communication system discussed before, It is reasonable to assume that the same noise energy LDPC decoders also benefit from this information. LDPC can influence an entire LDPC packet over the duration of codes require knowledge of the channel noise variance when a packet due to the transmission speed at which communi- the communication channel is assumed to be an AWGN chan- cation takes place. The basic transmission time interval in nel [5]. Channel noise variance corresponds to the channel LTE systems, for example, is 1 ms [18]. Furthermore, a suc- signal-to-noise ratio (SNR), which we will use interchange- ceeding LDPC packet’s noise estimate can also depend on ably in this paper to refer to the channel noise. If the noise the noise induced on previous LDPC packets. We introduce variance is over- or understated, the performance of the a dependency between LDPC packets that gradually “for- LDPC decoder can suffer [6–8]. gets” previous channel noise estimates. This allows an LDPC decoder to track time-varying SNRs as packets arrive at the Alternative solutions, including the focus of our work, aimtoestimateSNRsinthe forward error correction (FEC) receiver. layer while concurrently decoding received messages. The Initially, we used a standard factor graph representation rationale is that through the use of LDPC codes, the channel with an additional continuous random variable to model the uncertainty can be made explicit in the LDPC decoder rather SNR at a packet level. The larger parity-check factors from than assuming it to have a fixed “confident” a priori value. the LDPC code created computational challenges, which led By making the decoder “aware” of its uncertainty (i.e. allow- us to review other options for representing the problem and ing it to learn the channel constraints), it may also improve doing inference. To address this, we (1) use a more general the error-correcting capability of the LDPC code. We use a graphical structure called cluster graphs, (2) use a variant of Bayesian approach that takes into account a statistical distri- LBP called loopy belief update (LBU), (3) create a message bution over the SNR, which is also a much better realisation order that relaxes repeated marginalisation from large parity- of the channel in terms of its stochastic time-varying nature. check factors, and (4) introduce a cluster-stopping criterion Iterative message passing algorithms used in LDPC codes that turns off uninformative clusters during message passing. such as bit-flipping use binary (hard-decision) messages Cluster graphs have been shown to outperform factor graphs between variable nodes and check nodes to update the in some inference tasks in terms of accuracy and convergence received bit values in a factor graph (also known as a Tanner properties [10, 19, 20]. graph). Whereas this decoding method is fast, it does not pro- Our contribution: We view this problem more generally vide probabilistic information about the received bits, which as a PGM and introduce a sequential Bayesian learning tech- is required to estimate an SNR statistic in AWGN channels. nique capable of tracking non-stationary SNRs over time. The sum-product algorithm uses probabilistic (soft-decision) We represent the PGM as a cluster graph compiled by means messages and is equivalent to the loopy belief propagation of a general purpose algorithm called layered trees running (LBP) algorithm used for performing generic inference tasks intersection property (LTRIP), developed in [20]. We believe on probabilistic graphical models (PGMs) [9, 10]. However, this is the first work that represents LDPC codes as a cluster as will become clear in Sect. 2, modelling SNRs in AWGN graph, which may be due to a lack of available algorithms channels results in the use of conditional Gaussian distri- that can construct valid cluster graphs. We demonstrate: (1) butions that during subsequent message passing morphs into tracking of non-stationary channel noise over time, and (2) mixture distributions—these have problematic properties for performance benefits of our approach compared to an LDPC inference that force us to employ approximation techniques. code with stationary knowledge of the channel as well as an As such, studies in [11–17] have proposed hybrid methods LDPC code with perfect knowledge of the channel. for jointly modelling the SNRs and the transmitted bit mes- This paper is structured as follows: In Sect. 2, we explain sages. These methods are all based on factor graphs with how LDPC codes are represented in a more general PGM some graph extension that uses an inference scheme dif- framework and extended with the channel noise estima- ferent from LBP for estimating the SNRs. A study in [14] tor. Our message passing approach and schedule are also presents a comparison between a variational message pass- explained. Section 3 describes how the PGM is updated ing (VMP) based estimator and an expectation maximisation sequentially, which allows the model to track non-stationary (EM) based estimator for stationary SNRs. The VMP-based channel noise. The results are shown in Sect. 4 and in Sect. 5 estimator demonstrated superior performance over the EM- we present a supplementary investigation. Finally, our con- based estimator and achieved a lower frame error rate, with clusions and future work are discussed in Sect. 6. 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… 2 LDPC codes with non-stationary SNR In the work here we do this by using the VMP approach— estimation as PGMs thereby replacing the sufficient statistics of γ and b random variables with their expected values. This leaves Eq. 2 in its In this section, we compile a cluster graph of an LDPC code conditional form [22, Section 4.3]—Sect. 2.3.1 provides the and introduce additional random variables that are used to details about this. estimate the channel noise. This is extended further to enable The channel noise is assumed independent and identically the tracking of non-stationary channel noise. We also discuss distributed (i.i.d.) over the length N of an LDPC packet (the our hybrid inference approach and message passing schedule. codeword length). The transmission duration of a packet is around 1ms [18], which we assume provides sufficient sam- ples for estimating the noise, and is a short enough time frame 2.1 Channel SNR estimation with LDPC codes for non-stationarity to not be a problem. The channel-noise precision can be translated to a rate-compensated SNR given We assume an AWGN channel model (without fading), E γ by SNR = 10 log ( ), where R is the code rate [9, dB which adds zero mean random Gaussian noise to a transmit- 10 2R Section 11.1], and γ is the distribution over the channel- ted signal. This can be defined by the following input–output noise precision. Note that we may use the terms precision relationship: and SNR interchangeably. The next section presents LDPC codes with SNR estimation using the gamma prior and con- X = B + W , (1) ditional Gaussian random variables discussed here. where X is a random variable representing the received sig- 2.2 Representation of LDPC codes with SNR nal at the receiver, B is a random variable representing the estimation transmit signal at the transmitter, and W ∼ N (0,γ ) is the added Gaussian distributed noise. See [9, Section 11.1] for a During the course of this study, we noted that cluster graph more formal definition. representations of LDPC codes are not addressed in the avail- The strength of the noise depends on the Gaussian pre- able channel coding literature. Researchers in the channel cision γ (the inverse of the variance). We denote the LDPC coding domain may be more familiar with the factor graph code’s bit sequence as b ,..., b , where N is the length 0 N −1 (or Tanner graph) representation of LDPC codes. While our of the codeword. We use BPSK signal modulation with study focuses on channel noise estimation, we address the unit energy per bit E = 1 (i.e. a normalised signal). The novelty and performance advantages of cluster graph LDPC channel-noise precision is an unknown quantity for which codes compared to factor graph LDPC codes in a subsequent we assign a gamma prior distribution—a Bayesian way of study [23]. Nonetheless, in the interest of readability, we offer treating unknown variables. The gamma distribution is the a brief summary of cluster graphs here without diminishing conjugate prior for the precision of a Gaussian distribution the importance of our other study. and is part of the exponential family [21, Section 2.3.6]. A cluster graph is an undirected graph consisting of two With these assumptions, the observed received signal x is types of nodes. A cluster node (ellipse) is a set of random vari- modelled using a conditional Gaussian likelihood function ables and a sepset (short for “separation set”) node (square) f (x | b ,μ,γ ) with known means μ ∈ {μ =− E = n n 0 b √ is a set of random variables shared between a pair of clus- −1,μ = E = 1} that represent the modulated binary 1 b ters. In most interesting cases (which also include LDPC data (i.e. the two different phases of the carrier wave). Since decoding), this graph will not be a tree structure, but will the same noise influences both states of b the received sig- contain loops. Inference on such a “loopy” system requires nal’s likelihood function is simplified to: the so-called running intersection property (RIP) [10, Sec- tion 11.3.2]. This specifies that any two clusters containing γ(x +1) 2 a shared variable, must be connected via a unique sequence e when b = 0, f (x | b ,γ ) =  (2) n n of sepsets all containing that particular variable, i.e. no loops γ(x −1) ⎩ γ e when b = 1. 2 are allowed for any particular variable. Our study uses a general purpose cluster graph construc- After multiplying in the prior distributions over b and γ , tion algorithm that produces a cluster graph of the LDPC the presence of the b terms in the joint changes this to a code. This algorithm is termed the layered trees running mixture distribution which does not form part of the expo- intersection property (LTRIP) algorithm developed in [20]. nential family. This implies that its marginals towards the The LTRIP algorithm proceeds by processing layers, with γ and b variables will not be conjugate towards their prior each layer dedicated to an individual variable. For each forms. To rectify this requires some form of approximation variable, it determines an optimal tree structure over all clus- that forces these marginals to the required conjugate forms. ters. The final sepsets are formed by merging the sepsets 123 J. du Toit et al. between pairs of clusters across all layers. The resulting smaller parity-check clusters φ , φ , φ , φ , φ , and φ , lead- 1 2 4 5 6 7 cluster graph satisfies the RIP. Compared to factor graphs, ing to considerable computational savings. cluster graphs offer the benefits of more precise inference and faster convergence. We refer the reader to [20]for more 2.3 Message passing approach detail regarding the LTRIP algorithm and [23] for a com- parison study between cluster graph and factor graph LDPC This section describes variational message passing (VMP) codes. between the gamma and conditional Gaussian nodes, loopy For LDPC codes, each cluster contains a parity-check fac- belief update (LBU) between the parity-check nodes and a tor equivalent to a parity-check constraint in the original hybrid message passing approach between the conditional parity-check matrix. To illustrate this, we use an irregular Gaussian nodes and parity-check nodes. All the required (16,8) LDPC code with H matrix given in Eq. 3. Note that update equations and rules are discussed. We also discuss this LDPC code is for illustrative purposes only, we use larger the message passing schedule mentioned previously, which standardised LDPC codes for our simulations. We denote the alleviates expensive parity-check computation. message bit sequence as b ,..., b , the parity-check bits as 0 7 b ,..., b , and the parity-check factors as φ ,...,φ .The 2.3.1 Variational message passing 8 15 0 7 cluster graph of the (16,8) LDPC code is shown in Fig. 1 in plate notation. This section concerns the message passing between the var- ious conditional Gaussian clusters and the gamma cluster. ⎡ ⎤ VMP makes a distinction between parent-to-child mes- ⎢ ⎥ sages and child-to-parent messages [22]. We deal with the ⎢ ⎥ ⎢ ⎥ parent-to-child messages that update the conditional Gaus- ⎢ ⎥ ⎢ ⎥ 1000111100010000 sian clusters first. ⎢ ⎥ H = (3) ⎢ ⎥ The gamma distribution and its expected moments are ⎢ ⎥ ⎢ ⎥ required for the parent-to-child messages from the γ cluster ⎢ ⎥ ⎣ ⎦ to the conditional Gaussian clusters θ . We use the following parameterisation of the gamma distribution: The gamma prior belief is captured by the global clus- 2 ( ) γ 2ω −1 − 2 2ω Gam(γ | ω, ν) = γ e , (4) ter ζ(γ ) (outside the plate) that is fully connected to ( ) all the observed conditional Gaussian clusters θ (x | 0 0 b ,γ ),...,θ (x | b ,γ ) via the γ sepsets. The condi- 0 15 15 15 where ν is the degrees of freedom (equivalent to the num- tional Gaussian clusters connects to the first layer of parity- ber of observations that convey a “confidence” in the mean check clusters of the LDPC code via sepsets b ,..., b .The 0 15 value), ω is a scaled precision, and  is the gamma function. structure inside the plate repeats as LDPC packets p ∈ P In exponential family form transformed to the log-domain arrive at the receiver. this is given as: Our study focuses on irregular LDPC codes as adopted by the 5G new radio (NR) standard by the 3rd Genera- γ ν 2ω log(Gam(γ | ω, ν)) = − (log(( )) tion Partnership Project (3GPP). Irregular LDPC codes are ν − 1 log(γ ) also known to outperform regular LDPC codes [5, 9]. With + log(2ω)). (5) regular LDPC codes, all parity-check factors have equal car- dinality, which is not the case for irregular LDPC codes. Ideally, the larger parity-check factors in irregular LDPC In this form, the left column vector is the natural parameters codes should have minimal connectivity, which helps reduce of the distribution and the right column vector is the sufficient expensive computation during inference. This is not possible statistics of the distribution. The expected moment vector of to do in a factor graph representation, as the number of edge the sufficient statistics is [21, Appendix B]: connections is equal to the cardinality of each parity-check factor. Instead, we bundle the larger parity-check clusters νω γ , log(γ ) = , (6) ν 1 away from the observed conditional Gaussian clusters using ψ( ) − log( ) 2 2ω a message passing schedule. The message passing sched- ule is described in Sect. 2.3.4. Note that no connections are where ψ is the digamma-function. present between observed conditional Gaussian clusters and the large parity-check clusters φ and φ as showninFig. 1. 0 3 An alternative parameterisation for the gamma distribution is: α The observed conditional Gaussian clusters are linked to the (shape), and β (rate), which translates to ν = 2α,and w = . 2β 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… Fig. 1 A PGM of an irregular (16,8) LDPC code with a global gamma prior and conditional Gaussian clusters linked to the smaller parity-check clusters. Note its loopy structure The other parent-to-child messages required to update + b = i (log(γ ) − γμ − log(2π )). the conditional Gaussian clusters are from the parity-check i (9) clusters. To understand these, we first need the categorical distribution in its exponential family form transformed to the log-domain given by [24, Section 2.3.1]: The x terms are the observed signal values and the Gaussian means μ are known and can be replaced by their fixed values μ =−1 and μ = 1. 0 1 log(π ) b = 0 log(p(b | π ,π )) = −log( π ), 0 1 i During the parent-to-child update from the gamma cluster, log(π ) b = 1 i =0,1 the γ and log(γ ) terms in Eq. 9 will be replaced with their (7) expected values given by Eq. 6. This is shown by: where π are the unnormalised bit probabilities. The expected μ =[γ  , log(γ )] , (10) ζ,θ moment vector of the sufficient statistics is: n b = i (γ  μ ) n i n T i −γ θ 2 π x i b = i ( ) i =0,1 n i 2 b = 0 , b = 1 = . (8) i =0,1 2 + b = i (log(γ ) − γ  μ − log(2π )). We choose the gamma distribution prior parameters by (11) considering its mean value, given by ν × ω, and we think of ν as the number of observations that its mean value is based on. Since the gamma distribution is conjugate to the Gaussian Similarly, during the parent-to-child update from a parity- distribution, the conditional Gaussian likelihood from Eq. 2 check cluster, the Iverson bracket b = i  in Eq. 9, will be can be rewritten into a similar form. The Gaussian precision replaced with its expected value. is common to both cases of b and is therefore updated as a The required expected value from a parity-check cluster single entity (unlike the standard Gaussian mixture model). The exponential family form of the conditional Gaussian dis- reduces to estimating the bit probabilities P(b = i ).We tribution transformed to the log-domain is given by: discuss the replacement of the Iverson bracket with expected values from parity-check clusters in Sect. 2.3.3 which deals b = i (γ μ ) x n i n log(p(x | b ,γ )) = n n −γ with the hybrid message passing. b = i ( ) x n n i 2 123 J. du Toit et al. The expected values that replace the γ and b = i  terms in Eq. 9 are based on the latest beliefs from the gamma cluster distribution and the parity-check cluster distribution. After the expected values are installed, Eq. 9 forms a mixture of two gamma distributions since x is observed and μ are n i known value. Equation 9 is kept in a conditional form rather than a higher dimensional joint form. Messages from conditional Gaussian clusters to parity- check clusters are child-to-parent messages. We also post- Fig. 2 A subsection of Fig. 1 to illustrate VMP message passing between the gamma cluster and conditional Gaussian clusters pone discussing these message updates to Sect. 2.3.3 (that deals with hybrid message passing). Messages from conditional Gaussian clusters to the gamma cluster are child-to-parent messages that require Eq. 9 to be marginalised towards obtaining gamma distribution parameters. The terms of Eq. 9 are re-arranged to obtain the appropriate parameters given by: Fig. 3 Example cluster graph to illustrate LBP and LBU message updat- 1 2 b = i (− (x − μ ) ) γ n n i ing rules log( p(x | b ,γ )) = n n log(γ ) b = i ( ) − b = i (log(2π )). (12) its parameters given by the natural parameter vector in Eq. 12. The message is absorbed into cluster ζ(γ ) by adding its nat- ural parameters to those of the prior (as given in Eq. 5). We note in Eq. 12 that for child-to-parent messages, the separation of the sufficient statistic vector during marginal- 2.3.2 LBU message passing isation removes the previous parent-to-child message from the updated natural parameter vector. With b = i  replaced This section concerns the message passing between the vari- by its expected value, x replaced with the observed signal ous parity-check clusters-here we make use of the Lauritzen– value, and μ =−1 and μ = 1, the left column gives the 0 1 Spiegelhalter message passing algorithm [25], also known as incremental update with which the prior gamma parameters the loopy belief update (LBU) algorithm. However, the fun- need to increase to form the posterior. This is shown by: damental concepts are easier to define via the well known Shafer–Shenoy algorithm [26], also known as the sum- 1 2 2 b = i  (− (x − 2x μ + μ ) product or loopy belief propagation (LBP) algorithm. Hybrid n n i i =0,1 2 i μ = , θ ,ζ 1 message passing, linking the VMP and the LBU sections b = i i =0,1 of the graph, also draws on understanding the relationship (13) between LBP and LBU. We therefore provide a brief sum- mary of both LBP and LBU here-a fuller version is available 2ω = , (14) prior ν − 1 in the definitive handbook by Koller & Friedman [10, Sec- tions 10.2, 10.3 and 11.3]. = + μ . (15) ζ prior θ ,ζ We use Fig. 3 to illustrate the various relevant concepts. As discussed in Sect. 2.2, cluster graphs contain cluster nodes The expected values of the updated gamma cluster can and sepsets, both of which are used during message passing to now be recalculated using Eq. 6 to be used for updating the update nodes. The cluster internal factor functions are given conditional Gaussian clusters in the next round. by φ and φ where a and b identifies the particular cluster a b With this background, we demonstrate how VMP is (in our application these functions are conditional distribu- applied (using a sub-graph extracted from Fig. 1)inFig. 2. tions enforcing even parity over all the variables involved in Cluster θ (x | b ,γ ) is updated with information from each parity-check cluster). We will at times somewhat abuse 0 0 0 cluster ζ(γ ) using a VMP parent-to-child message μ . the notation by also using these factor functions directly to ζ,θ This message is the expected values of log(Gam(γ | ω, ν)) identify their particular clusters. given by Eq. 6. The expected values are installed at the The sepset connecting the two clusters a and b is denoted corresponding γ terms in Eq. 9. The message from clus- as S and comprises the collection of random variables a,b ter θ (x | b ,γ ) to cluster ζ(γ ) is a VMP child-to-parent about which the two clusters will exchange information. The 0 0 0 message μ . This requires cluster θ (x | b ,γ ) to be notation \a is used to indicate the set of all cluster nodes θ ,ζ 0 0 0 marginalised in obtaining a distribution over sepset γ with excluding cluster node a. 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… Fig. 4 A subsection of Fig. 1 to illustrate LBU message passing between parity-check clusters. The dotted lines denote links to other adjacent clusters For LBP message passing, the message passed from clus- for these quantities (see [10, Section 10.3] for the deriva- ter a to cluster b is denoted as μ . The product of all tions). The sepset belief update is given by: a,b other messages incoming to cluster node a-excluding mes- sage μ -is denoted by μ . If anything changes in the b,a \b,a ψ = . (19) a,b a messages μ , we can propagate this towards cluster b via \b,a \S a,b the update equation: Note that this is computationally more efficient since it avoids μ = μ φ , (16) the repeated re-combination of the various messages present \b,a a a,b \S a,b in the μ term in Eq. 16 when considering different target \b,a clusters b. Using this we can now incrementally update the where the marginalisation sum over the set \S removes a,b cluster belief using: all variables not present in the sepset S . Note that this a,b implies that the larger a sepset, the cheaper the required a,b = . (20) marginalisation will be—another benefit of cluster graphs a,b over factor graphs which always have only single variables in their sepsets. With this background, we can now turn to Fig. 4 (a sub- Also note that, especially with clusters sharing links with graph extracted from Fig. 1) to illustrate how LBU applies to many others, the LBP formulation can be quite expensive the parity-check clusters. Cluster node φ is updated using due to the redundancy implied by the various message com- LBU message passing. Using Eq. 19, the updated sepset binations present in the μ term. \b,a belief is calculated by marginalising the cluster belief In a loopy system (such as we will typically have with to obtain the required sepset b . The previous sepset belief LDPC codes) these μ messages will have to be passed is cancelled (divided) as per the LBU update rules given by iteratively according to some schedule until they have all Eq. 20. This gives the update equations as: converged. At that point, we can then get an estimate for the marginal distribution of all the random variables present in a ψ = , (21) particular cluster—this is known as the cluster belief and is φ ,φ 1 1 4 b ,b ,b 0 4 11 given by: φ ,φ 1 4 = . (22) φ 4 = φ μ μ , (17) a a b,a \b,a ψ φ ,φ 1 4 i.e. it is the product of the cluster internal factor with all Unlike VMP, the form of the update rules remains the same incoming messages. Similarly, we can calculate the belief regardless of the message direction. We therefore simply over the sepset variables S as the product of the two oppos- iterate these messages until convergence using the message a,b ing messages passing through that sepset: passing schedule of Sect. 2.3.4. ψ = μ μ . (18) a,b a,b b,a 2.3.3 Hybrid message passing Note that the sepset beliefs, being the product of the two μ This section describes our hybrid message passing between a,b and μ messages, are intrinsically directionless. the VMP-based conditional Gaussian cluster nodes and the b,a In contrast to LBP, LBU message passing is expressed LBU-based parity-check nodes connected to it. Although our fully in terms of only cluster beliefs and sepset beliefs.For treatment is general and applies to any {b }, we will specif- this we use two alternative (although equivalent) expressions ically focus on the μ and μ messages running over θ ,φ φ ,θ 0 1 1 0 123 J. du Toit et al. Due to that (explicit) division, we refer to this as the post-cancellation message, i.e. the message we derive by marginalising the cluster belief and then removing the opposite direction message via division. But this strongly reminds us of the child-to-parent message of Eq. 23 which also (implicitly) accomplishes removing the influence of its parent. We therefore set our updated μ to beaa θ ,φ 0 1 categorical message with natural parameters given by the corresponding lefthand column of Eq. 23. Next, we need to determine how to incrementally update the cluster belief with such a post-cancellation message. We do this via Eq. 17 (but now with in mind) to get the cluster belief update equation: = φ μ μ , b \a,b b a,b Fig. 5 A subsection of Fig. 1 to illustrate hybrid message passing and updates between conditional Gaussian clusters and parity-check clusters a,b = . (25) a,b In short, we can directly update an LBU cluster belief by the {b } sepset in Fig. 5 (which in its turn is a sub-graph dividing the old post-cancellation message out and multiply- extracted from Fig. 1). ing the new one in. This gives us: We first consider the child-to-parent message μ run- θ ,φ 0 1 ning from the θ (x | b ,γ ) cluster to the φ (b , b , b , b ) 0 0 0 1 0 4 9 11 θ ,φ 0 1 cluster. According to VMP message passing we determine = . (26) φ 1 its natural parameters by re-arranging Eq. 9 so that the b θ ,φ 0 0 1 related terms move to the sufficient statistics column: The child-to-parent update rules for the general case that ⎡ ⎤ 2 2 γ x (log(γ )−γμ −log(2π)) 0 0 connects a conditional Gaussian cluster θ to a parity-check γμ x − + n 0 0 ⎣ 2 2 ⎦ log( p(x | b ,γ )) = 0 0 2 2 γ x (log(γ )−γμ −log(2π)) cluster φ are: 0 1 γμ x − + 1 0 2 2 b = 0 0 T . (23) γ x (log(γ )−γ μ −log(2π)) n 0 b = 1 γ  μ x − + 0 0 n 2 2 μ = , θ ,φ 2 n j (log(γ )−γ μ −log(2π)) γ x n 1 γ  μ x − + 1 n 2 2 From our model definition, the values of the means are (27) known as μ =−1 and μ = 1. The expected values of 0 1 θ ,φ n j the γ terms are known via Eq. 6. The natural parameters col- = . (28) φ j umn (the left-hand one) effectively measure the heights of θ ,φ n j the two Gaussians corresponding to b being either 0 or 1. Similar to what we saw before with Eq. 12, we note that Lastly, we consider the parent-to-child message μ φ ,θ 1 0 for child-to-parent messages, the separation of the sufficient running from the φ (b , b , b , b ) cluster to the θ (x | 1 0 4 9 11 0 0 statistic vector during marginalisation effectively removes b ,γ ) cluster. From the VMP viewpoint, our message μ φ ,θ 1 0 the previous parent-to-child message from the updated natu- simply is the expected values: ral parameter vector. The question now is how this is to be made compatible μ = b = i  with i ∈{0, 1}. φ ,θ 1 0 (b ,b ,b ,b ) φ 0 4 9 11 with the corresponding φ cluster which, of course, will also be updated via sepset beliefs from its other side (connecting to The first step is to find the distribution over b by marginal- other parity-check clusters using Eq. 20). To reconcile these ising: worlds, we start by comparing Eqs. 16 and 17 and noticing that we can also determine the (original) update message ψ (b ) = (b , b , b , b ). (29) θ ,φ 0 φ 0 4 9 11 0 1 1 using: b ,b ,b 4 9 11 μ = /μ . (24) a b,a From Eq. 19 this is the sepset belief—we also term it the pre- a,b cancellation message. \S a,b 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… The required expectations are the probabilities for b as found The idea is to keep the expensive clusters at the final layer in this sepset belief—we simply install them as the required so that the smaller (less expensive) parity clusters, in preced- parameters in the θ cluster. ing layers, can resolve most of the uncertainty about the even In general, where we have a message describing b run- parity states. When the larger parity clusters get updated, ning from the parent cluster φ to a child cluster θ ,wefirst some of the even parity states in their discrete tables may j n find the sepset belief: have zero probability, which are removed due to our software implementation. This further reduces a large parity cluster’s computational footprint. Our layered message passing sched- ψ (b ) = , (30) n φ θ ,φ j n j ule is detailed in Algorithm 1. \b and then replace the two probabilities for b found in this Algorithm 1 Layered message passing schedule sepset belief as the required expected sufficient statistics 1: S ← set initialised to large cluster IDs b = i  in the θ cluster: n n 2: A ← set initialised to all cluster IDs 3: M ← empty vector of pairs ⎡   ⎤ T 4: while A is not empty do b = i  (γ  μ ) n i 5: next Layer ← empty set of integers φ x j n ⎣   ⎦ θ −γ  2 6: for s in S do b = i  ( ) x n n 7: A.erase(s) 8: N ← all neighbours of s 9: for n in N do + b = i  ( log(γ ) − γ μ − log(2π )). 10: if n ∈ A then 11: M.push_back(pair(s, n)) (31) 12: if n ∈ / S then 13: next Layer.insert(n) 2.3.4 Message passing schedule 14: end if 15: end if 16: end for A message passing schedule is important for loopy graphs as 17: end for the message order can influence convergence speed, accu- 18: S ← next Layer racy, and the computational cost of inference. Although 19: end while 20: return M not empirically verified here, these feedback loops (or cycles) may reinforce inaccurate cluster beliefs causing self- fulfilling belief updates, which affect the LDPC decoder’s The observed conditional Gaussian clusters are coupled performance. This problem is more prominent in LDPC to the parity-check clusters in the layer furthest away from codes with small feedback loops as described in [5]. Tak- the initial isolated group of large clusters. We refer to this ing this into consideration, our message passing schedule (1) layer as the first parity-check layer. The smaller parity-check uses a structured schedule with a fixed computational cost, clusters in this layer are given priority in terms of their con- (2) aims to minimise the effect of loops, and (3) aims to nectivity to the conditional Gaussian clusters, which saves minimise the computational cost of inference. computation. If the first layer of parity-check clusters does The message passing schedule is determined by first iden- not have all the unique bits, the following layers are utilised tifying the larger parity-check clusters in the graph. We select until all conditional Gaussian clusters are connected. the larger clusters φ (with cardinality 7) and φ (with car- 0 3 The message passing order for the entire PGM starts at the dinality 6). The message schedule starts with the selected gamma cluster and flows down to the conditional Gaussian clusters as initial sources S and proceeds by visiting all its clusters through all the parity cluster layers until it reaches neighbouring clusters N , which become the immediate next the bottom layer. We refer to this as the forward sweep. The layer of clusters. A set of available clusters A is kept to backward sweep returns in the opposite direction, which con- make sure that clusters from previous layers are not revis- cludes one iteration of message passing. Pseudo-code for our ited, which helps minimise the effect of loops. We repeat message passing is shown in Algorithm 2. this procedure to add subsequent layers of clusters until all clusters are included. The source-destination pairs are stored in a message schedule M. This procedure isolates the ini- tially selected large parity-check clusters from the rest of the clusters as shown in Fig. 1. 123 J. du Toit et al. Algorithm 2 Message passing • a cluster is deactivated during message passing when 1: ψ ← initialised to uniform sepset beliefs messages entering it have not changed significantly. This 2: M ← initialised to source-destination pairs from message schedule 3 is determined by a symmetrical Kullback–Leibler diver- 3: S initialised to sepset variable sets between clusters gence measure between the newest and immediately 4: ← initialised to gamma prior parameters prior preceding sepset beliefs. 5: ← initialised to parity check factors 6: max I ter ← initialised to maximum number of sweeps • the stopping criterion for inference is when a valid code- 7: while iter < max I ter and not converged do word was detected (also known as a syndrome check) 8: // ————– Forward sweep ————– after all parity-check clusters “agree” on their shared 9: //VMP messages from cluster ζ to clusters θ bit values or when a maximum number of iterations is 10: for each conditional Gaussian cluster n do 11: μ ← from Equation 10 reached, ζ,θ 12: ← from Equation 11 • all discrete table factors support sparse representations to 13: end for reduce memory resources, 14: //Hybrid messages from θ to first layer parity-check clusters φ n j • zero probability states in discrete tables are removed dur- 15: for each conditional Gaussian cluster n do ing inference. 16: μ ← from Equation 27 θ ,φ n j 17: ← from Equation 28 The next section describes how the LDPC code tracks 18: ← j φ non-stationary SNRs using a Bayesian sequential learning 19: end for technique. 20: // Continues below 21: //LBU messages from first layer to final layer parity-check clus- ters 22: //s is parity-check source cluster, d is parity-check destination 3 Bayesian sequential learning for cluster 23: for pair(s, d)in M do non-stationary SNR estimation 24: ψ ← from Equation 19 φ ,φ s d 25: ← from Equation 20 Channel noise in a wireless communication system can 26: ← change over time, especially when the end user is mobile. d φ 27: ψ ← ψ This means the statistical properties of the received sig- φ ,φ s d φ ,φ s d 28: end for nal should be treated as non-stationary. We adapt Eq. 1 to 29: // ————– Backward sweep ————– define the following input–output relationship for the non- 30: //LBU messages from final layer towards first layer clusters φ stationary case: 31: for pair(d, s) in Reverse(M) do 32: ψ ← from Equation 19 φ ,φ s d X = B + W , (32) 33: ← from Equation 20 t t t 34: ← d φ where X is a random variable representing the received sig- 35: ψ ← ψ φ ,φ s d φ ,φ s d 36: end for nal at the receiver, B is a random variable representing the 37: //Hybrid messages from first layer parity-check clusters φ to θ j n transmit signal at the transmitter, and W ∼ N (0,γ ) is the t t 38: for each conditional Gaussian cluster n do added Gaussian distributed noise. In this case, the channel 39: ψ (b ) ← from Equation 30 θ ,φ n j noise precision γ is not constant for all time t. 40: ← from Equation 31 For the PGM to remain representative of the varying chan- 41: end for nel conditions, the parameters of the gamma distribution need 42: ← set to uniform prior 43: //VMP messages from θ to ζ n to adapt accordingly as LDPC packets arrive at the receiver. 44: for each conditional Gaussian cluster n do This can be achieved with Bayesian sequential learning also 45: μ ← from Equation 13 θ ,ζ known as Bayesian sequential filtering. Each time an LDPC 46: += μ ζ θ ,ζ packet is decoded, the parameters of the obtained posterior 47: end for 48: = + (see Equation 15) ζ ζ gamma distribution are stored and used as the prior dis- prior 49: iter ++ tribution when decoding the next LDPC packet. However, 50: if clusters φ calibrated and codeword valid then as an increased number of LDPC packets are decoded, our 51: converged stationarity assumptions cause an ever-increasing certainty 52: end if 53: end while around the gamma distribution’s mean and the PGM strug- gles to respond accurately to the changing noise. Tracking the The average divergence from both directions is used, taking into con- The following settings and software implementations sideration the mode-seeking and moment-seeking behaviours of the D (P||Q)+D (Q||P) KL KL apply to our inference approach: metric given by . 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… noise becomes restricted due to the strong underlying i.i.d. graph is compiled from the parity-check factors using the assumption across all noise estimates (from the observed LTRIP algorithm, and the message schedule is initialised by data), which will follow the average channel noise instead clusters with cardinality 8 and 10 (the largest factors), which of the evolving channel noise. form the bottom layer of the PGM (see Sect. 2.3.4). We use To remedy this, we introduce a time constant T that rep- BPSK modulation over an AWGN channel and assume a resents a period of time in which we assume the stationarity transmitted bit has unit energy. assumption holds. For example, if we assume no drastic changes in channel noise over a period of T seconds, our sta- 4.1 Purpose of stationary noise experiment T ×1000 tionarity assumption should be valid for S = number of LDPC packets (assuming a transmission time of 1ms per The stationary noise experiment is presented as a BER vs LDPC packet). SNR curve as typically would be the case for determining the We “relax” the i.i.d. assumption by accumulating the behaviour and performance of error correction codes over a posterior gamma distribution’s parameters as we normally range of SNR values. The purpose of the experiment is to would, but only up to S number of LDPC packets. After compare the BER performance between our proposed PGM the S number of packets are decoded, the posterior gamma and a PGM with perfect knowledge of the noise across a distribution’s parameters (the natural parameter vector) are range of SNR values. The selected SNR range consists of 6 S×N continuously being re-scaled with a scaling factor as equidistant points from 0 to 4.45 dB (inclusive). A random −1 bit message is encoded and Gaussian noise is added to each follows: bit, which repeats for 50k packets per SNR value. The same received bit values are presented to both PGMs to ensure a 2ω , − 1 ≤ S × N like-for-like comparison. We set the maximum number of ⎨ ν 2 − 1 =   (33) message passing iterations to 20 for both PGMs. Our chan- ζ T ⎪ − ⎪ S×N ν nel estimation PGM is parameterised with S = 10, which 2ω , − 1 > S × N . ν −1 2 − 1 assumes stationarity over 10 LDPC packets (equivalent to a period of 10ms). By scaling the gamma distribution’s parameters, the num- ber of observations ν that make up the posterior distribution 4.1.1 Results and interpretation reaches a ceiling value, which is not equal to the theoreti- cal maximum of the full i.i.d. assumption. This allows for The results of the stationary noise experiment are shown in variance around the gamma distribution’s mean so that the Fig. 6. The BER performance (shown on the left) of our pro- estimate does not become too “confident” in the data, which posed PGM closely follows the performance of a PGM with makes the gamma distribution responsive to the evolving perfect knowledge of the noise precision (the error bars reflect channel noise. Scaling the natural parameter vector in this the 95% confidence intervals). Similarly, the number of iter- way makes contributions from previous estimates progres- ations required to reach convergence (shown on the right) is sively less important compared to more recent estimates— almost identical between the two PGMs. allowing the PGM to “forget” historic channel noise. Note that this is not a windowing approach or a Kalman filter 4.2 Purpose of non-stationary noise experiment type approach where noise is added between estimations. The posterior gamma distribution remains informed by the The purpose of the experiment is to test whether our proposed entire sequence of received data, but the contributions from PGM is capable of tracking non-stationary channel noise, past data decay exponentially as packets are received and and if it benefits from the estimated channel noise informa- decoded. tion. We compare three scenarios: (1) a PGM that has perfect (instantaneous) knowledge of the channel noise precision, (2) a PGM using a fixed value for the channel noise precision, 4 Experimental investigation and (3) our proposed PGM that sequentially updates its chan- nel noise precision estimation. Note that the assumed channel This section describes the results obtained from stationary noise in (2) is the last precision mean obtained by running noise and non-stationary noise experiments. our proposed PGM over the test data once with S =∞. The LDPC code used is constructed from the 5G new radio While (2) assumes stationarity, its fixed value of the chan- (NR) standard [27]. We use base graph 2 with size (42, 52) nel was accumulated from the entire test sequence and is and expansion factor 11. The resultant H matrix is shortened advantageous, since it allows the PGM to “anticipate” future to a codeword length of N = 220 bits, of which K = 110 values of the channel noise. We set the maximum number of are message bits (producing a code rate R = 0.5). A cluster message passing iterations to 20 for all scenarios and param- 123 J. du Toit et al. Fig. 6 BER performance comparison between our proposed channel noise estimation PGM and a PGM with perfect knowledge of the channel noise. The BER and the number of iterations till convergence between the two systems are nearly identical eterise (3) with S = 10, which assumes stationarity over 10 LDPC packets (equivalent to a period of 10ms). 4.2.1 Drive-test data We test our model using actual drive test measurements obtained from Vodacom Group Limited, a mobile communi- cations company based in South Africa. Drive tests capture key network performance indicators that allow the mobile network operator to troubleshoot and optimise the network. We use the signal-to-interference-plus-noise ratio (SINR) measurements from a 5G device while driving approximately 10 km along a coastal route. The captured data includes han- dovers between cells as the user equipment travels in and out of reception range. The start and end times of the test along with the SINR measurements and GPS locations are shown in Fig. 7. The drive-test data had missing measurements and irregu- lar time intervals. The missing values were removed and the Fig. 7 Location of the drive test with the captured SINR measurements. sequence was re-indexed to create a time series with regu- Note the changes in signal-to-noise power as the user equipment moves lar intervals. It was upsampled using linear interpolation to through certain areas match a basic transmission time interval of 1ms. Note that our final test data may reflect more extreme channel variations precision values are used to add zero-mean Gaussian noise due to some discontinuities introduced as a consequence of to the transmitted signal, which produces the received signal. the described data cleaning. We generate random bit mes- Note that the same precision value is used to add noise to all sages that are encoded using the H matrix to produce LDPC signal values from the same LDPC packet. packets (or codewords). Bit values in an LDPC packet are The dataset used during the current study is not publicly modulated using BPSK modulation, and random Gaussian available due to the organisation’s data privacy policy, but noise is added to the resultant signal values using the SINR can be made available from the corresponding author on rea- drive-test data. The SINR data are converted from their loga- sonable request. rithmic form to a linear form to obtain precision values. The 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… 4.2.2 Results and interpretation Table 1 A summary of the overall performance comparison. Note that the expected behaviour of our PGM is similar to the PGM with perfect knowledge of the channel noise, and outperforms the PGM with fixed As stated earlier, we use the Gaussian distribution’s precision value of the average channel noise in terms of the BER to model the channel noise. Results from our experiment are BER (mean) Iterations (mean) shown in Fig. 8. The gamma posterior distribution is capable of tracking the actual noise precision (shown by the mean and True precision 0.003034 2.49 one standard deviation in the top part of the figure). The BER Fixed precision 0.005013 2.54 shown in the bottom part of the figure is a moving average Estimated precision 0.003336 2.49 consisting of 10000 packets centred around the current point in time; this helps to reveal the underlying trend. We use Dirichlet smoothing to avoid reporting BER values in the −7 range 0 < BER < 9 × 10 , which is smaller than the 5 Supplementary investigation possible BER. The BER shown in our results is calculated (a+bit errors in a packet) using: BER = , where a = In some instances, the BER performance of our proposed 10,000 (K +a) 0.005 and K is the number of message bits. PGM (estimating the SNR) is better than the PGM with We observe an improvement in the BER compared to a knowledge of the actual SNR. We noted this happening when PGM using a fixed value (a precision of 8.76 or 9.42 dB the estimated SNR is lower than the actual SNR, which is SNR) of the average noise precision (shown in the bottom counter-intuitive. part of the figure). A study presented in [8] investigates the effect of chan- The BER performance of our proposed PGM closely nel noise mismatch in LDPC codes. It found that the follows the BER performance of the PGM with perfect LDPC code’s performance degrades more quickly when the knowledge of the noise precision. This is due to the small assumed SNR is overestimated, but is less sensitive to degra- difference between the actual noise precision and the esti- dation when underestimated up to around 1 dB below the mated noise precision of our proposed PGM. actual SNR. What is also interesting is that the optimal BER In some experiments, we observed instances where the is at an assumed SNR lower than the actual SNR. estimated precision was much lower than the actual precision. We reproduce the same experiment to establish whether These were instances where the LDPC code failed to fix all bit this behaviour is similar for a cluster graph representation of errors, which the PGM then interpreted as worse than actual LDPC codes. Our channel noise mismatch experiment is pre- channel conditions. Another observation is that the gamma sented as a BER vs model SNR curve where the model SNR posterior has a wider standard deviation at higher precision is the assumed channel noise while the actual channel noise means, and a narrower standard deviation at lower precision is fixed at 1.22 dB (similar to [8]). The purpose of the exper- means. This is due to the characteristics of the gamma distri- iment is to (1) understand the impact on BER performance bution. We view this as a beneficial artifact, since the PGM when the actual channel noise is under- and over-estimated makes provision for more rapid changes in the channel con- by the model, and (2) determine where the optimal BER is. ditions at lower channel noise. (This can also be regarded as a A random bit message is encoded and Gaussian noise is "sceptical" type of behaviour.) When channel conditions are added using a fixed precision of 1.32 (equivalent to a SNR bad, the PGM’s channel estimation more confidently reports of 1.22 dB) to each bit, which repeats for 10k packets per that they are bad. model SNR value. The same received bit values are used for A summary of the overall results appears in Table 1.The each model SNR value to ensure a like-for-like comparison. average BER of our PGM is slightly higher than the PGM We set the maximum number of message passing iterations with perfect knowledge of the channel, and outperforms the to 20. The results of the channel noise mismatch experiment PGM with fixed value of the channel (at approximately 1.5 is shown in Fig. 9. times better BER on average). Our PGM requires approx- This result is similar to [8] and suggests that the LDPC imately 1.02 times fewer message passing iterations on code’s optimal BER is not necessarily at the point where average compared to the PGM with a fixed value of the chan- the model SNR is equal to the actual SNR, but somewhere nel, and the same number of iterations compared to the PGM slightly below the actual SNR (in this case at 0.84 dB). with perfect knowledge of the channel. A conservative estimation of the channel noise seems to be Note that the averages presented in this table are more beneficial, since the LDPC decoder is more forgiving in the influenced by the high SNR instances where bit errors are region below the actual noise. Using the drive-test data we infrequent. Figure 8 illustrates the performance advantage of run the same experiment as described in Sect. 4.2. However, our proposed PGM more clearly where bit errors occur more after each packet is decoded we adjust the estimated precision frequently at lower SNRs. value to be −0.1 dB below the posterior gamma mean value. We found that an adjustment larger than this yielded perfor- 123 J. du Toit et al. Fig. 8 Results of the three scenarios using the 5G device drive-test data. Note that our PGM tracks the changes in precision and maintains a BER similar to that of a PGM with perfect knowledge of the channel. It also outperforms a PGM employing a fixed average precision mance degradation. The result of this adjustment is shown in LDPC codes. We demonstrated the idea by employing a more Table 2. general probabilistic framework called cluster graphs, and A slight improvement in the BER can be seen, however, evaluated our proposed model using real-world data. The there is a trade-off between the number of iterations and BER results show that the performance of our proposed model is performance. The decoder requires slightly more iterations nearly indistinguishable from a model with perfect knowl- on average when the adjustment to the posterior gamma is edge of the channel noise. made. Apart from the performance advantages shown in our We emphasise that the purpose of the supplementary results, the approach embeds well within an LDPC decoder investigation is not to propose that an adjustment be made and does not require stand-alone machine learning tech- to the estimated SNR in practice, but rather to show the niques, pre-training, or management of large data sets. It is counter-intuitive behaviour of the LDPC code that we found capable of learning the channel noise accurately on-the-fly interesting. The benefit of adjusting the estimated SNR seems while decoding LDPC packets without compromise. minimal. The implications of this method with respect to the com- In our study, we simply selected the average SNR as the munication system are that (1) the scheduling and use of static value knowing that the true SNR will be better and pilot signals dedicated to channel noise estimation may at times will be worse. We refrain from making any asser- become redundant, (2) LDPC codes coupled with our pro- tions about the ideal static SNR value, recognising that it can posed channel-noise estimator can be used to estimate significantly vary based on specific circumstances. Instead, non-stationary channel noise on-the-fly with no need for our approach is to dynamically adjust to the real-time SNR scheduling, and (3) channel-noise information is now inher- conditions prevailing at any given moment. ent in the LDPC code and provides performance advantages to the code. Our current approach relies on parameterising a time con- stant, which is a prior assumption about the speed at which 6 Conclusion and future work channel noise will vary. Future work will focus on apply- ing real-time Bayesian model selection, allowing the LDPC This paper contributes a simple sequential Bayesian learn- decoder to choose intelligently between multiple assump- ing method for tracking non-stationary channel noise using 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… Fig. 9 The BER performance of a cluster graph LDPC code with the actual SNR fixed at 1.22 dB when the model SNR is varied. The error bars reflect the 95% confidence intervals Table 2 A summary of the overall performance comparison between of the more conservative approach performs better than our initial PGM the PGM that estimates the channel noise and the PGM that more conser- only in terms of the BER vatively estimates the channel noise. Note that the expected behaviour BER (mean) Iterations (mean) Estimated precision 0.003336 2.49 Estimated precision (−0.1 dB) 0.003329 2.49 tions (or combine multiple assumptions) about the speed at cate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, which the channel varies. Such work relates to Bayesian unless indicated otherwise in a credit line to the material. If material change-point detection, which accounts for changes in the is not included in the article’s Creative Commons licence and your underlying data-generating process while estimating param- intended use is not permitted by statutory regulation or exceeds the eters. While this study focuses on channel-noise estimation permitted use, you will need to obtain permission directly from the copy- right holder. To view a copy of this licence, visit http://creativecomm alone, the methodology could also be expanded to estimate ons.org/licenses/by/4.0/. other channel properties such as phase and complex chan- nel gain, which will depend on the modulation scheme and channel model discussed in Sect. 2. Acknowledgements The authors would like to thank Bonnie Xaba for References providing the drive-test data. She is a senior specialist in network bench- marking and E2E network performance quality at Vodacom Group 1. Rumney, M. (2013). LTE and the evolution to 4G wireless: Design LimitedinSouthAfrica. and measurement challenges (2nd ed.). Hoboken: Wiley. https:// doi.org/10.1002/9781118799475 Funding Open access funding provided by Stellenbosch University. No 2. Björnson, E., Hoydis, J., & Sanguinetti, L. (2017). Massive mimo funding was received for conducting this study. networks: Spectral, energy, and hardware efficiency. Foundations and Trends®in Signal Processing, 11(3–4), 154–655. https://doi. org/10.1561/2000000093 Declarations 3. Gallager, R. (1962). Low-density parity-check codes. IRE Trans- actions on Information Theory, 8(1), 21–28. Competing interests The authors declare no competing interests. 4. Davey, M. C., & MacKay, D. (1998). Low-density parity check codes over gf(q). IEEE Communications Letters, 2(6), 165–167. Open Access This article is licensed under a Creative Commons https://doi.org/10.1109/4234.681360 Attribution 4.0 International License, which permits use, sharing, adap- 5. Johnson, S. J. (2006). Introducing Low-density parity-check codes. tation, distribution and reproduction in any medium or format, as University of Newcastle, Australia, 1, 2006. long as you give appropriate credit to the original author(s) and the 6. Saeedi, H., & Banihashemi, A. H. (2007). Performance of belief source, provide a link to the Creative Commons licence, and indi- propagation for decoding ldpc codes in the presence of channel 123 J. du Toit et al. estimation error. IEEE Transactions on Communications, 55(1), conference—GLOBECOM (pp. 3743–3747). https://doi.org/10. 83–89. https://doi.org/10.1109/TCOMM.2006.887488 1109/GLOCOM.2012.6503699 7. Tan, W., & Cruz, J. R. (2004). Signal-to-noise ratio mismatch 18. Dahlman, E., Parkvall, S., & Skold, J. (2011). 4G LTE/LTE- for Low-density parity-check coded magnetic recording channels. Advanced for Mobile Broadband (1st ed., pp. 301–321). Oxford: IEEE Transactions on Magnetics, 40(2), 498–506. https://doi.org/ Academic Press. https://doi.org/10.1016/B978-0-12-385489-6. 10.1109/TMAG.2004.824130 00014-X 8. MacKay, D. J., & Hesketh, C. P. (2003). Performance of Low den- 19. Yedidia, J. S., Freeman, W. T., & Weiss, Y. (2005). Construct- sity parity check codes as a function of actual and assumed noise ing free-energy approximations and generalized belief propagation levels. Electronic Notes in Theoretical Computer Science, 74, 89– algorithms. IEEE Transactions on Information Theory, 51(7), 96. 2282–2312. https://doi.org/10.1109/TIT.2005.850085 9. MacKay, D. J. (2003). Information theory, inference and learning 20. Streicher, S., & Du Preez, J. (2017). Graph coloring: Comparing algorithms. Cambridge: Cambridge University Press. cluster graphs to factor graphs. Proceedings of the ACM multimedia 10. Koller, D., & Friedman, N. (2009). Probabilistic graphical models: 2017 workshop on South African Academic Participation (pp. 35– Principles and techniques (1st ed.). London: MIT Press. 42). https://doi.org/10.1145/3132711.3132717 11. Wang, S., Cui, L., Cheng, S., Zhai, Y., Yeary, M., & Wu, Q. 21. Bishop, C. (2006). Pattern Recognition and Machine Learning (2011). Noise adaptive LDPC decoding using particle filtering. (Vol. 4). London: Springer. IEEE Transactions on Communications, 59(4), 913–916. https:// 22. Winn, J., & Bishop, C. M. (2005). Variational message passing. doi.org/10.1109/CISS.2009.5054686 Journal of Machine Learning Research, 6, 661–694. 12. Jakubisin, D. J., Buehrer, R. M., & da Silva, C. R. (2016). 23. Du Toit, J., du Preez, J., & Wolhuter, R. (2022). LDPC Probabilistic receiver architecture combining bp, mf, and ep for codes: Comparing cluster graphs to factor graphs. arXiv preprint multi-signal detection. arXiv preprint arXiv:1604.04834. arXiv:2204.06350. 13. Wu, S., Kuang, L., Ni, Z., Lu, J., David Huang, D., & Guo, Q. 24. Taylor, R. M. (2023, March). Approximate message passing algo- (2014). Expectation propagation approach to joint channel estima- rithms for latent Dirichlet allocation. PhD thesis, Stellenbosch tion and decoding for ofdm systems. In 2014 IEEE international University conference on acoustics, speech and signal processing (ICASSP) 25. Lauritzen, S. L., & Spiegelhalter, D. J. (1988). Local computations (pp. 1941–1945). https://doi.org/10.1109/ICASSP.2014.6853937 with probabilities on graphical structures and their application to 14. Senst, M., & Ascheid, G. (2012). A message passing approach expert systems. Journal of the Royal Statistical Society: Series B to iterative Bayesian snr estimation. In 2012 International sympo- (Methodological), 50(2), 157–194. sium on signals, systems, and electronics (ISSSE) (pp. 1–6) (2012). 26. Shafer, G., & Shenoy, P. P. (1990). Probability propagation. Annals https://doi.org/10.1109/ISSSE.2012.6374328 of Mathematics and Artificial Intelligence, 2, 327–351. https://doi. 15. Nissila, M., & Pasupathy, S. (2009). Adaptive iterative detectors for org/10.1007/BF01531015 phase-uncertain channels via variational bounding. IEEE Trans- 27. Bae, J. H., Abotabl, A., Lin, H.-P., Song, K.-B., & Lee, J. (2019). actions on Communications, 57(3), 716–725. https://doi.org/10. An overview of channel coding for 5g nr cellular communications. 1109/TCOMM.2009.03.070068 APSIPA Transactions on Signal and Information Processing, 8, 16. Wang, S., Cui, L., & Cheng, S. (2011). Noise adaptive ldpc e17. decoding using expectation propagation. In 2011 IEEE Global Telecommunications Conference - GLOBECOM (pp. 1–5). https:// doi.org/10.1109/GLOCOM.2011.6133838 Publisher’s Note Springer Nature remains neutral with regard to juris- 17. Cui, L., Wang, S., & Cheng, S. (2012). Online snr statis- dictional claims in published maps and institutional affiliations. tic estimation for ldpc decoding over awgn channel using Laplace propagation. In 2012 IEEE global communications http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Telecommunication Systems Springer Journals

Low-density parity-check codes: tracking non-stationary channel noise using sequential variational Bayesian estimates

Download PDF
16 pages

Loading next page...
 
/lp/springer-journals/low-density-parity-check-codes-tracking-non-stationary-channel-noise-WqZY2BTUuV

References (21)

Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2023
ISSN
1018-4864
eISSN
1572-9451
DOI
10.1007/s11235-023-01083-5
Publisher site
See Article on Publisher Site

Abstract

We present a sequential Bayesian learning method for tracking non-stationary signal-to-noise ratios in low-density parity- check (LDPC) codes by way of probabilistic graphical models. We represent the LDPC code as a cluster graph using a general purpose cluster graph construction algorithm called the layered trees running intersection property (LTRIP) algorithm. The channel noise estimator is a global gamma cluster, which we extend to allow for Bayesian tracking of non-stationary noise variation. We evaluate our proposed model on real-world 5G drive-test data. Our results show that our model can track non- stationary channel noise accurately while adding performance benefits to the LDPC code, which outperforms an LDPC code with a fixed stationary knowledge of the actual channel noise. Keywords LDPC codes · Bayesian sequential learning · Variational inference · Cluster graphs 1 Introduction cally via the physical uplink shared channel (PUSCH) [1]. Instantaneous estimates of the channel noise should ideally In wireless communication systems, channel noise interferes be acquired at the same rate as the channel changes, which with radio transmissions between a transmitter and receiver. require frequent use of pilot signals. The downside of doing The nature of the noise is assumed to be non-stationary, since so is an additional communication overhead, since the time- it can change over time and vary according to environmental frequency resources could be utilised to carry payload data dynamics. instead [2]. Knowledge of channel noise is useful in a communica- Our interest is in finding a way to estimate the chan- tion system to adapt certain radio parameters, ultimately nel noise variance applicable to the additive white Gaussian optimising the reliability and overall quality of communica- noise (AWGN) channel without relying on scheduled pilot tion. These optimisation methods include, inter alia, adaptive signals. The aim of our work is not to omit the broader chan- modulation and coding (AMC), interference suppression, nel state information (CSI) system in its entirety, since it and rate-matching. also provides information on other channel properties out- Communication systems such as LTE and 5G rely on pre- side the scope of this work. Instead, our focus is on reliably defined reference signals, i.e. pilot signals, which are used and dynamically estimating the channel noise variance only, to estimate the channel noise. These pilot signals are gen- which we see as a step towards substituting certain elements erated in the physical layer and scheduled periodically via of the CSI system. We propose to combine received signal the physical uplink control channel (PUCCH), or aperiodi- values from the physical layer with a channel-coding scheme such as a low-density parity-check (LDPC) code. By doing so, we modify a standard LDPC code’s decoding algorithm, B J. du Toit which can learn the channel noise on-the-fly and simultane- [email protected] ously benefit from the channel-noise estimates. We are not J. du Preez attempting to design new LDPC codes, but rather explore [email protected] modifications to the existing LDPC decoding algorithm so R. Wolhuter that our methods can be applied to generic LDPC codes. [email protected] LDPC codes were first introduced by Gallager in 1962 Department of Electrical and Electronic Engineering, and rediscovered in 1995 by MacKay [3, 4]. These codes are Stellenbosch University, Stellenbosch, South Africa 123 J. du Toit et al. a family of block codes with good theoretical and practical no significant increase in computational complexity. In [11, properties. They have found extensive application in vari- 16, 17] SNRs are assumed stationary within fixed indepen- ous wireless communication systems, including the 5G New dent sections of the LDPC packet with a focus on finding a Radio (NR) technology standard, which will be used in our low complexity noise estimator. These studies do not model study. the sequential dynamics of channel noise variance, e.g. the While knowledge of the channel noise is useful for possible correlations between inferred instances of SNRs. aspects of the communication system discussed before, It is reasonable to assume that the same noise energy LDPC decoders also benefit from this information. LDPC can influence an entire LDPC packet over the duration of codes require knowledge of the channel noise variance when a packet due to the transmission speed at which communi- the communication channel is assumed to be an AWGN chan- cation takes place. The basic transmission time interval in nel [5]. Channel noise variance corresponds to the channel LTE systems, for example, is 1 ms [18]. Furthermore, a suc- signal-to-noise ratio (SNR), which we will use interchange- ceeding LDPC packet’s noise estimate can also depend on ably in this paper to refer to the channel noise. If the noise the noise induced on previous LDPC packets. We introduce variance is over- or understated, the performance of the a dependency between LDPC packets that gradually “for- LDPC decoder can suffer [6–8]. gets” previous channel noise estimates. This allows an LDPC decoder to track time-varying SNRs as packets arrive at the Alternative solutions, including the focus of our work, aimtoestimateSNRsinthe forward error correction (FEC) receiver. layer while concurrently decoding received messages. The Initially, we used a standard factor graph representation rationale is that through the use of LDPC codes, the channel with an additional continuous random variable to model the uncertainty can be made explicit in the LDPC decoder rather SNR at a packet level. The larger parity-check factors from than assuming it to have a fixed “confident” a priori value. the LDPC code created computational challenges, which led By making the decoder “aware” of its uncertainty (i.e. allow- us to review other options for representing the problem and ing it to learn the channel constraints), it may also improve doing inference. To address this, we (1) use a more general the error-correcting capability of the LDPC code. We use a graphical structure called cluster graphs, (2) use a variant of Bayesian approach that takes into account a statistical distri- LBP called loopy belief update (LBU), (3) create a message bution over the SNR, which is also a much better realisation order that relaxes repeated marginalisation from large parity- of the channel in terms of its stochastic time-varying nature. check factors, and (4) introduce a cluster-stopping criterion Iterative message passing algorithms used in LDPC codes that turns off uninformative clusters during message passing. such as bit-flipping use binary (hard-decision) messages Cluster graphs have been shown to outperform factor graphs between variable nodes and check nodes to update the in some inference tasks in terms of accuracy and convergence received bit values in a factor graph (also known as a Tanner properties [10, 19, 20]. graph). Whereas this decoding method is fast, it does not pro- Our contribution: We view this problem more generally vide probabilistic information about the received bits, which as a PGM and introduce a sequential Bayesian learning tech- is required to estimate an SNR statistic in AWGN channels. nique capable of tracking non-stationary SNRs over time. The sum-product algorithm uses probabilistic (soft-decision) We represent the PGM as a cluster graph compiled by means messages and is equivalent to the loopy belief propagation of a general purpose algorithm called layered trees running (LBP) algorithm used for performing generic inference tasks intersection property (LTRIP), developed in [20]. We believe on probabilistic graphical models (PGMs) [9, 10]. However, this is the first work that represents LDPC codes as a cluster as will become clear in Sect. 2, modelling SNRs in AWGN graph, which may be due to a lack of available algorithms channels results in the use of conditional Gaussian distri- that can construct valid cluster graphs. We demonstrate: (1) butions that during subsequent message passing morphs into tracking of non-stationary channel noise over time, and (2) mixture distributions—these have problematic properties for performance benefits of our approach compared to an LDPC inference that force us to employ approximation techniques. code with stationary knowledge of the channel as well as an As such, studies in [11–17] have proposed hybrid methods LDPC code with perfect knowledge of the channel. for jointly modelling the SNRs and the transmitted bit mes- This paper is structured as follows: In Sect. 2, we explain sages. These methods are all based on factor graphs with how LDPC codes are represented in a more general PGM some graph extension that uses an inference scheme dif- framework and extended with the channel noise estima- ferent from LBP for estimating the SNRs. A study in [14] tor. Our message passing approach and schedule are also presents a comparison between a variational message pass- explained. Section 3 describes how the PGM is updated ing (VMP) based estimator and an expectation maximisation sequentially, which allows the model to track non-stationary (EM) based estimator for stationary SNRs. The VMP-based channel noise. The results are shown in Sect. 4 and in Sect. 5 estimator demonstrated superior performance over the EM- we present a supplementary investigation. Finally, our con- based estimator and achieved a lower frame error rate, with clusions and future work are discussed in Sect. 6. 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… 2 LDPC codes with non-stationary SNR In the work here we do this by using the VMP approach— estimation as PGMs thereby replacing the sufficient statistics of γ and b random variables with their expected values. This leaves Eq. 2 in its In this section, we compile a cluster graph of an LDPC code conditional form [22, Section 4.3]—Sect. 2.3.1 provides the and introduce additional random variables that are used to details about this. estimate the channel noise. This is extended further to enable The channel noise is assumed independent and identically the tracking of non-stationary channel noise. We also discuss distributed (i.i.d.) over the length N of an LDPC packet (the our hybrid inference approach and message passing schedule. codeword length). The transmission duration of a packet is around 1ms [18], which we assume provides sufficient sam- ples for estimating the noise, and is a short enough time frame 2.1 Channel SNR estimation with LDPC codes for non-stationarity to not be a problem. The channel-noise precision can be translated to a rate-compensated SNR given We assume an AWGN channel model (without fading), E γ by SNR = 10 log ( ), where R is the code rate [9, dB which adds zero mean random Gaussian noise to a transmit- 10 2R Section 11.1], and γ is the distribution over the channel- ted signal. This can be defined by the following input–output noise precision. Note that we may use the terms precision relationship: and SNR interchangeably. The next section presents LDPC codes with SNR estimation using the gamma prior and con- X = B + W , (1) ditional Gaussian random variables discussed here. where X is a random variable representing the received sig- 2.2 Representation of LDPC codes with SNR nal at the receiver, B is a random variable representing the estimation transmit signal at the transmitter, and W ∼ N (0,γ ) is the added Gaussian distributed noise. See [9, Section 11.1] for a During the course of this study, we noted that cluster graph more formal definition. representations of LDPC codes are not addressed in the avail- The strength of the noise depends on the Gaussian pre- able channel coding literature. Researchers in the channel cision γ (the inverse of the variance). We denote the LDPC coding domain may be more familiar with the factor graph code’s bit sequence as b ,..., b , where N is the length 0 N −1 (or Tanner graph) representation of LDPC codes. While our of the codeword. We use BPSK signal modulation with study focuses on channel noise estimation, we address the unit energy per bit E = 1 (i.e. a normalised signal). The novelty and performance advantages of cluster graph LDPC channel-noise precision is an unknown quantity for which codes compared to factor graph LDPC codes in a subsequent we assign a gamma prior distribution—a Bayesian way of study [23]. Nonetheless, in the interest of readability, we offer treating unknown variables. The gamma distribution is the a brief summary of cluster graphs here without diminishing conjugate prior for the precision of a Gaussian distribution the importance of our other study. and is part of the exponential family [21, Section 2.3.6]. A cluster graph is an undirected graph consisting of two With these assumptions, the observed received signal x is types of nodes. A cluster node (ellipse) is a set of random vari- modelled using a conditional Gaussian likelihood function ables and a sepset (short for “separation set”) node (square) f (x | b ,μ,γ ) with known means μ ∈ {μ =− E = n n 0 b √ is a set of random variables shared between a pair of clus- −1,μ = E = 1} that represent the modulated binary 1 b ters. In most interesting cases (which also include LDPC data (i.e. the two different phases of the carrier wave). Since decoding), this graph will not be a tree structure, but will the same noise influences both states of b the received sig- contain loops. Inference on such a “loopy” system requires nal’s likelihood function is simplified to: the so-called running intersection property (RIP) [10, Sec- tion 11.3.2]. This specifies that any two clusters containing γ(x +1) 2 a shared variable, must be connected via a unique sequence e when b = 0, f (x | b ,γ ) =  (2) n n of sepsets all containing that particular variable, i.e. no loops γ(x −1) ⎩ γ e when b = 1. 2 are allowed for any particular variable. Our study uses a general purpose cluster graph construc- After multiplying in the prior distributions over b and γ , tion algorithm that produces a cluster graph of the LDPC the presence of the b terms in the joint changes this to a code. This algorithm is termed the layered trees running mixture distribution which does not form part of the expo- intersection property (LTRIP) algorithm developed in [20]. nential family. This implies that its marginals towards the The LTRIP algorithm proceeds by processing layers, with γ and b variables will not be conjugate towards their prior each layer dedicated to an individual variable. For each forms. To rectify this requires some form of approximation variable, it determines an optimal tree structure over all clus- that forces these marginals to the required conjugate forms. ters. The final sepsets are formed by merging the sepsets 123 J. du Toit et al. between pairs of clusters across all layers. The resulting smaller parity-check clusters φ , φ , φ , φ , φ , and φ , lead- 1 2 4 5 6 7 cluster graph satisfies the RIP. Compared to factor graphs, ing to considerable computational savings. cluster graphs offer the benefits of more precise inference and faster convergence. We refer the reader to [20]for more 2.3 Message passing approach detail regarding the LTRIP algorithm and [23] for a com- parison study between cluster graph and factor graph LDPC This section describes variational message passing (VMP) codes. between the gamma and conditional Gaussian nodes, loopy For LDPC codes, each cluster contains a parity-check fac- belief update (LBU) between the parity-check nodes and a tor equivalent to a parity-check constraint in the original hybrid message passing approach between the conditional parity-check matrix. To illustrate this, we use an irregular Gaussian nodes and parity-check nodes. All the required (16,8) LDPC code with H matrix given in Eq. 3. Note that update equations and rules are discussed. We also discuss this LDPC code is for illustrative purposes only, we use larger the message passing schedule mentioned previously, which standardised LDPC codes for our simulations. We denote the alleviates expensive parity-check computation. message bit sequence as b ,..., b , the parity-check bits as 0 7 b ,..., b , and the parity-check factors as φ ,...,φ .The 2.3.1 Variational message passing 8 15 0 7 cluster graph of the (16,8) LDPC code is shown in Fig. 1 in plate notation. This section concerns the message passing between the var- ious conditional Gaussian clusters and the gamma cluster. ⎡ ⎤ VMP makes a distinction between parent-to-child mes- ⎢ ⎥ sages and child-to-parent messages [22]. We deal with the ⎢ ⎥ ⎢ ⎥ parent-to-child messages that update the conditional Gaus- ⎢ ⎥ ⎢ ⎥ 1000111100010000 sian clusters first. ⎢ ⎥ H = (3) ⎢ ⎥ The gamma distribution and its expected moments are ⎢ ⎥ ⎢ ⎥ required for the parent-to-child messages from the γ cluster ⎢ ⎥ ⎣ ⎦ to the conditional Gaussian clusters θ . We use the following parameterisation of the gamma distribution: The gamma prior belief is captured by the global clus- 2 ( ) γ 2ω −1 − 2 2ω Gam(γ | ω, ν) = γ e , (4) ter ζ(γ ) (outside the plate) that is fully connected to ( ) all the observed conditional Gaussian clusters θ (x | 0 0 b ,γ ),...,θ (x | b ,γ ) via the γ sepsets. The condi- 0 15 15 15 where ν is the degrees of freedom (equivalent to the num- tional Gaussian clusters connects to the first layer of parity- ber of observations that convey a “confidence” in the mean check clusters of the LDPC code via sepsets b ,..., b .The 0 15 value), ω is a scaled precision, and  is the gamma function. structure inside the plate repeats as LDPC packets p ∈ P In exponential family form transformed to the log-domain arrive at the receiver. this is given as: Our study focuses on irregular LDPC codes as adopted by the 5G new radio (NR) standard by the 3rd Genera- γ ν 2ω log(Gam(γ | ω, ν)) = − (log(( )) tion Partnership Project (3GPP). Irregular LDPC codes are ν − 1 log(γ ) also known to outperform regular LDPC codes [5, 9]. With + log(2ω)). (5) regular LDPC codes, all parity-check factors have equal car- dinality, which is not the case for irregular LDPC codes. Ideally, the larger parity-check factors in irregular LDPC In this form, the left column vector is the natural parameters codes should have minimal connectivity, which helps reduce of the distribution and the right column vector is the sufficient expensive computation during inference. This is not possible statistics of the distribution. The expected moment vector of to do in a factor graph representation, as the number of edge the sufficient statistics is [21, Appendix B]: connections is equal to the cardinality of each parity-check factor. Instead, we bundle the larger parity-check clusters νω γ , log(γ ) = , (6) ν 1 away from the observed conditional Gaussian clusters using ψ( ) − log( ) 2 2ω a message passing schedule. The message passing sched- ule is described in Sect. 2.3.4. Note that no connections are where ψ is the digamma-function. present between observed conditional Gaussian clusters and the large parity-check clusters φ and φ as showninFig. 1. 0 3 An alternative parameterisation for the gamma distribution is: α The observed conditional Gaussian clusters are linked to the (shape), and β (rate), which translates to ν = 2α,and w = . 2β 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… Fig. 1 A PGM of an irregular (16,8) LDPC code with a global gamma prior and conditional Gaussian clusters linked to the smaller parity-check clusters. Note its loopy structure The other parent-to-child messages required to update + b = i (log(γ ) − γμ − log(2π )). the conditional Gaussian clusters are from the parity-check i (9) clusters. To understand these, we first need the categorical distribution in its exponential family form transformed to the log-domain given by [24, Section 2.3.1]: The x terms are the observed signal values and the Gaussian means μ are known and can be replaced by their fixed values μ =−1 and μ = 1. 0 1 log(π ) b = 0 log(p(b | π ,π )) = −log( π ), 0 1 i During the parent-to-child update from the gamma cluster, log(π ) b = 1 i =0,1 the γ and log(γ ) terms in Eq. 9 will be replaced with their (7) expected values given by Eq. 6. This is shown by: where π are the unnormalised bit probabilities. The expected μ =[γ  , log(γ )] , (10) ζ,θ moment vector of the sufficient statistics is: n b = i (γ  μ ) n i n T i −γ θ 2 π x i b = i ( ) i =0,1 n i 2 b = 0 , b = 1 = . (8) i =0,1 2 + b = i (log(γ ) − γ  μ − log(2π )). We choose the gamma distribution prior parameters by (11) considering its mean value, given by ν × ω, and we think of ν as the number of observations that its mean value is based on. Since the gamma distribution is conjugate to the Gaussian Similarly, during the parent-to-child update from a parity- distribution, the conditional Gaussian likelihood from Eq. 2 check cluster, the Iverson bracket b = i  in Eq. 9, will be can be rewritten into a similar form. The Gaussian precision replaced with its expected value. is common to both cases of b and is therefore updated as a The required expected value from a parity-check cluster single entity (unlike the standard Gaussian mixture model). The exponential family form of the conditional Gaussian dis- reduces to estimating the bit probabilities P(b = i ).We tribution transformed to the log-domain is given by: discuss the replacement of the Iverson bracket with expected values from parity-check clusters in Sect. 2.3.3 which deals b = i (γ μ ) x n i n log(p(x | b ,γ )) = n n −γ with the hybrid message passing. b = i ( ) x n n i 2 123 J. du Toit et al. The expected values that replace the γ and b = i  terms in Eq. 9 are based on the latest beliefs from the gamma cluster distribution and the parity-check cluster distribution. After the expected values are installed, Eq. 9 forms a mixture of two gamma distributions since x is observed and μ are n i known value. Equation 9 is kept in a conditional form rather than a higher dimensional joint form. Messages from conditional Gaussian clusters to parity- check clusters are child-to-parent messages. We also post- Fig. 2 A subsection of Fig. 1 to illustrate VMP message passing between the gamma cluster and conditional Gaussian clusters pone discussing these message updates to Sect. 2.3.3 (that deals with hybrid message passing). Messages from conditional Gaussian clusters to the gamma cluster are child-to-parent messages that require Eq. 9 to be marginalised towards obtaining gamma distribution parameters. The terms of Eq. 9 are re-arranged to obtain the appropriate parameters given by: Fig. 3 Example cluster graph to illustrate LBP and LBU message updat- 1 2 b = i (− (x − μ ) ) γ n n i ing rules log( p(x | b ,γ )) = n n log(γ ) b = i ( ) − b = i (log(2π )). (12) its parameters given by the natural parameter vector in Eq. 12. The message is absorbed into cluster ζ(γ ) by adding its nat- ural parameters to those of the prior (as given in Eq. 5). We note in Eq. 12 that for child-to-parent messages, the separation of the sufficient statistic vector during marginal- 2.3.2 LBU message passing isation removes the previous parent-to-child message from the updated natural parameter vector. With b = i  replaced This section concerns the message passing between the vari- by its expected value, x replaced with the observed signal ous parity-check clusters-here we make use of the Lauritzen– value, and μ =−1 and μ = 1, the left column gives the 0 1 Spiegelhalter message passing algorithm [25], also known as incremental update with which the prior gamma parameters the loopy belief update (LBU) algorithm. However, the fun- need to increase to form the posterior. This is shown by: damental concepts are easier to define via the well known Shafer–Shenoy algorithm [26], also known as the sum- 1 2 2 b = i  (− (x − 2x μ + μ ) product or loopy belief propagation (LBP) algorithm. Hybrid n n i i =0,1 2 i μ = , θ ,ζ 1 message passing, linking the VMP and the LBU sections b = i i =0,1 of the graph, also draws on understanding the relationship (13) between LBP and LBU. We therefore provide a brief sum- mary of both LBP and LBU here-a fuller version is available 2ω = , (14) prior ν − 1 in the definitive handbook by Koller & Friedman [10, Sec- tions 10.2, 10.3 and 11.3]. = + μ . (15) ζ prior θ ,ζ We use Fig. 3 to illustrate the various relevant concepts. As discussed in Sect. 2.2, cluster graphs contain cluster nodes The expected values of the updated gamma cluster can and sepsets, both of which are used during message passing to now be recalculated using Eq. 6 to be used for updating the update nodes. The cluster internal factor functions are given conditional Gaussian clusters in the next round. by φ and φ where a and b identifies the particular cluster a b With this background, we demonstrate how VMP is (in our application these functions are conditional distribu- applied (using a sub-graph extracted from Fig. 1)inFig. 2. tions enforcing even parity over all the variables involved in Cluster θ (x | b ,γ ) is updated with information from each parity-check cluster). We will at times somewhat abuse 0 0 0 cluster ζ(γ ) using a VMP parent-to-child message μ . the notation by also using these factor functions directly to ζ,θ This message is the expected values of log(Gam(γ | ω, ν)) identify their particular clusters. given by Eq. 6. The expected values are installed at the The sepset connecting the two clusters a and b is denoted corresponding γ terms in Eq. 9. The message from clus- as S and comprises the collection of random variables a,b ter θ (x | b ,γ ) to cluster ζ(γ ) is a VMP child-to-parent about which the two clusters will exchange information. The 0 0 0 message μ . This requires cluster θ (x | b ,γ ) to be notation \a is used to indicate the set of all cluster nodes θ ,ζ 0 0 0 marginalised in obtaining a distribution over sepset γ with excluding cluster node a. 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… Fig. 4 A subsection of Fig. 1 to illustrate LBU message passing between parity-check clusters. The dotted lines denote links to other adjacent clusters For LBP message passing, the message passed from clus- for these quantities (see [10, Section 10.3] for the deriva- ter a to cluster b is denoted as μ . The product of all tions). The sepset belief update is given by: a,b other messages incoming to cluster node a-excluding mes- sage μ -is denoted by μ . If anything changes in the b,a \b,a ψ = . (19) a,b a messages μ , we can propagate this towards cluster b via \b,a \S a,b the update equation: Note that this is computationally more efficient since it avoids μ = μ φ , (16) the repeated re-combination of the various messages present \b,a a a,b \S a,b in the μ term in Eq. 16 when considering different target \b,a clusters b. Using this we can now incrementally update the where the marginalisation sum over the set \S removes a,b cluster belief using: all variables not present in the sepset S . Note that this a,b implies that the larger a sepset, the cheaper the required a,b = . (20) marginalisation will be—another benefit of cluster graphs a,b over factor graphs which always have only single variables in their sepsets. With this background, we can now turn to Fig. 4 (a sub- Also note that, especially with clusters sharing links with graph extracted from Fig. 1) to illustrate how LBU applies to many others, the LBP formulation can be quite expensive the parity-check clusters. Cluster node φ is updated using due to the redundancy implied by the various message com- LBU message passing. Using Eq. 19, the updated sepset binations present in the μ term. \b,a belief is calculated by marginalising the cluster belief In a loopy system (such as we will typically have with to obtain the required sepset b . The previous sepset belief LDPC codes) these μ messages will have to be passed is cancelled (divided) as per the LBU update rules given by iteratively according to some schedule until they have all Eq. 20. This gives the update equations as: converged. At that point, we can then get an estimate for the marginal distribution of all the random variables present in a ψ = , (21) particular cluster—this is known as the cluster belief and is φ ,φ 1 1 4 b ,b ,b 0 4 11 given by: φ ,φ 1 4 = . (22) φ 4 = φ μ μ , (17) a a b,a \b,a ψ φ ,φ 1 4 i.e. it is the product of the cluster internal factor with all Unlike VMP, the form of the update rules remains the same incoming messages. Similarly, we can calculate the belief regardless of the message direction. We therefore simply over the sepset variables S as the product of the two oppos- iterate these messages until convergence using the message a,b ing messages passing through that sepset: passing schedule of Sect. 2.3.4. ψ = μ μ . (18) a,b a,b b,a 2.3.3 Hybrid message passing Note that the sepset beliefs, being the product of the two μ This section describes our hybrid message passing between a,b and μ messages, are intrinsically directionless. the VMP-based conditional Gaussian cluster nodes and the b,a In contrast to LBP, LBU message passing is expressed LBU-based parity-check nodes connected to it. Although our fully in terms of only cluster beliefs and sepset beliefs.For treatment is general and applies to any {b }, we will specif- this we use two alternative (although equivalent) expressions ically focus on the μ and μ messages running over θ ,φ φ ,θ 0 1 1 0 123 J. du Toit et al. Due to that (explicit) division, we refer to this as the post-cancellation message, i.e. the message we derive by marginalising the cluster belief and then removing the opposite direction message via division. But this strongly reminds us of the child-to-parent message of Eq. 23 which also (implicitly) accomplishes removing the influence of its parent. We therefore set our updated μ to beaa θ ,φ 0 1 categorical message with natural parameters given by the corresponding lefthand column of Eq. 23. Next, we need to determine how to incrementally update the cluster belief with such a post-cancellation message. We do this via Eq. 17 (but now with in mind) to get the cluster belief update equation: = φ μ μ , b \a,b b a,b Fig. 5 A subsection of Fig. 1 to illustrate hybrid message passing and updates between conditional Gaussian clusters and parity-check clusters a,b = . (25) a,b In short, we can directly update an LBU cluster belief by the {b } sepset in Fig. 5 (which in its turn is a sub-graph dividing the old post-cancellation message out and multiply- extracted from Fig. 1). ing the new one in. This gives us: We first consider the child-to-parent message μ run- θ ,φ 0 1 ning from the θ (x | b ,γ ) cluster to the φ (b , b , b , b ) 0 0 0 1 0 4 9 11 θ ,φ 0 1 cluster. According to VMP message passing we determine = . (26) φ 1 its natural parameters by re-arranging Eq. 9 so that the b θ ,φ 0 0 1 related terms move to the sufficient statistics column: The child-to-parent update rules for the general case that ⎡ ⎤ 2 2 γ x (log(γ )−γμ −log(2π)) 0 0 connects a conditional Gaussian cluster θ to a parity-check γμ x − + n 0 0 ⎣ 2 2 ⎦ log( p(x | b ,γ )) = 0 0 2 2 γ x (log(γ )−γμ −log(2π)) cluster φ are: 0 1 γμ x − + 1 0 2 2 b = 0 0 T . (23) γ x (log(γ )−γ μ −log(2π)) n 0 b = 1 γ  μ x − + 0 0 n 2 2 μ = , θ ,φ 2 n j (log(γ )−γ μ −log(2π)) γ x n 1 γ  μ x − + 1 n 2 2 From our model definition, the values of the means are (27) known as μ =−1 and μ = 1. The expected values of 0 1 θ ,φ n j the γ terms are known via Eq. 6. The natural parameters col- = . (28) φ j umn (the left-hand one) effectively measure the heights of θ ,φ n j the two Gaussians corresponding to b being either 0 or 1. Similar to what we saw before with Eq. 12, we note that Lastly, we consider the parent-to-child message μ φ ,θ 1 0 for child-to-parent messages, the separation of the sufficient running from the φ (b , b , b , b ) cluster to the θ (x | 1 0 4 9 11 0 0 statistic vector during marginalisation effectively removes b ,γ ) cluster. From the VMP viewpoint, our message μ φ ,θ 1 0 the previous parent-to-child message from the updated natu- simply is the expected values: ral parameter vector. The question now is how this is to be made compatible μ = b = i  with i ∈{0, 1}. φ ,θ 1 0 (b ,b ,b ,b ) φ 0 4 9 11 with the corresponding φ cluster which, of course, will also be updated via sepset beliefs from its other side (connecting to The first step is to find the distribution over b by marginal- other parity-check clusters using Eq. 20). To reconcile these ising: worlds, we start by comparing Eqs. 16 and 17 and noticing that we can also determine the (original) update message ψ (b ) = (b , b , b , b ). (29) θ ,φ 0 φ 0 4 9 11 0 1 1 using: b ,b ,b 4 9 11 μ = /μ . (24) a b,a From Eq. 19 this is the sepset belief—we also term it the pre- a,b cancellation message. \S a,b 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… The required expectations are the probabilities for b as found The idea is to keep the expensive clusters at the final layer in this sepset belief—we simply install them as the required so that the smaller (less expensive) parity clusters, in preced- parameters in the θ cluster. ing layers, can resolve most of the uncertainty about the even In general, where we have a message describing b run- parity states. When the larger parity clusters get updated, ning from the parent cluster φ to a child cluster θ ,wefirst some of the even parity states in their discrete tables may j n find the sepset belief: have zero probability, which are removed due to our software implementation. This further reduces a large parity cluster’s computational footprint. Our layered message passing sched- ψ (b ) = , (30) n φ θ ,φ j n j ule is detailed in Algorithm 1. \b and then replace the two probabilities for b found in this Algorithm 1 Layered message passing schedule sepset belief as the required expected sufficient statistics 1: S ← set initialised to large cluster IDs b = i  in the θ cluster: n n 2: A ← set initialised to all cluster IDs 3: M ← empty vector of pairs ⎡   ⎤ T 4: while A is not empty do b = i  (γ  μ ) n i 5: next Layer ← empty set of integers φ x j n ⎣   ⎦ θ −γ  2 6: for s in S do b = i  ( ) x n n 7: A.erase(s) 8: N ← all neighbours of s 9: for n in N do + b = i  ( log(γ ) − γ μ − log(2π )). 10: if n ∈ A then 11: M.push_back(pair(s, n)) (31) 12: if n ∈ / S then 13: next Layer.insert(n) 2.3.4 Message passing schedule 14: end if 15: end if 16: end for A message passing schedule is important for loopy graphs as 17: end for the message order can influence convergence speed, accu- 18: S ← next Layer racy, and the computational cost of inference. Although 19: end while 20: return M not empirically verified here, these feedback loops (or cycles) may reinforce inaccurate cluster beliefs causing self- fulfilling belief updates, which affect the LDPC decoder’s The observed conditional Gaussian clusters are coupled performance. This problem is more prominent in LDPC to the parity-check clusters in the layer furthest away from codes with small feedback loops as described in [5]. Tak- the initial isolated group of large clusters. We refer to this ing this into consideration, our message passing schedule (1) layer as the first parity-check layer. The smaller parity-check uses a structured schedule with a fixed computational cost, clusters in this layer are given priority in terms of their con- (2) aims to minimise the effect of loops, and (3) aims to nectivity to the conditional Gaussian clusters, which saves minimise the computational cost of inference. computation. If the first layer of parity-check clusters does The message passing schedule is determined by first iden- not have all the unique bits, the following layers are utilised tifying the larger parity-check clusters in the graph. We select until all conditional Gaussian clusters are connected. the larger clusters φ (with cardinality 7) and φ (with car- 0 3 The message passing order for the entire PGM starts at the dinality 6). The message schedule starts with the selected gamma cluster and flows down to the conditional Gaussian clusters as initial sources S and proceeds by visiting all its clusters through all the parity cluster layers until it reaches neighbouring clusters N , which become the immediate next the bottom layer. We refer to this as the forward sweep. The layer of clusters. A set of available clusters A is kept to backward sweep returns in the opposite direction, which con- make sure that clusters from previous layers are not revis- cludes one iteration of message passing. Pseudo-code for our ited, which helps minimise the effect of loops. We repeat message passing is shown in Algorithm 2. this procedure to add subsequent layers of clusters until all clusters are included. The source-destination pairs are stored in a message schedule M. This procedure isolates the ini- tially selected large parity-check clusters from the rest of the clusters as shown in Fig. 1. 123 J. du Toit et al. Algorithm 2 Message passing • a cluster is deactivated during message passing when 1: ψ ← initialised to uniform sepset beliefs messages entering it have not changed significantly. This 2: M ← initialised to source-destination pairs from message schedule 3 is determined by a symmetrical Kullback–Leibler diver- 3: S initialised to sepset variable sets between clusters gence measure between the newest and immediately 4: ← initialised to gamma prior parameters prior preceding sepset beliefs. 5: ← initialised to parity check factors 6: max I ter ← initialised to maximum number of sweeps • the stopping criterion for inference is when a valid code- 7: while iter < max I ter and not converged do word was detected (also known as a syndrome check) 8: // ————– Forward sweep ————– after all parity-check clusters “agree” on their shared 9: //VMP messages from cluster ζ to clusters θ bit values or when a maximum number of iterations is 10: for each conditional Gaussian cluster n do 11: μ ← from Equation 10 reached, ζ,θ 12: ← from Equation 11 • all discrete table factors support sparse representations to 13: end for reduce memory resources, 14: //Hybrid messages from θ to first layer parity-check clusters φ n j • zero probability states in discrete tables are removed dur- 15: for each conditional Gaussian cluster n do ing inference. 16: μ ← from Equation 27 θ ,φ n j 17: ← from Equation 28 The next section describes how the LDPC code tracks 18: ← j φ non-stationary SNRs using a Bayesian sequential learning 19: end for technique. 20: // Continues below 21: //LBU messages from first layer to final layer parity-check clus- ters 22: //s is parity-check source cluster, d is parity-check destination 3 Bayesian sequential learning for cluster 23: for pair(s, d)in M do non-stationary SNR estimation 24: ψ ← from Equation 19 φ ,φ s d 25: ← from Equation 20 Channel noise in a wireless communication system can 26: ← change over time, especially when the end user is mobile. d φ 27: ψ ← ψ This means the statistical properties of the received sig- φ ,φ s d φ ,φ s d 28: end for nal should be treated as non-stationary. We adapt Eq. 1 to 29: // ————– Backward sweep ————– define the following input–output relationship for the non- 30: //LBU messages from final layer towards first layer clusters φ stationary case: 31: for pair(d, s) in Reverse(M) do 32: ψ ← from Equation 19 φ ,φ s d X = B + W , (32) 33: ← from Equation 20 t t t 34: ← d φ where X is a random variable representing the received sig- 35: ψ ← ψ φ ,φ s d φ ,φ s d 36: end for nal at the receiver, B is a random variable representing the 37: //Hybrid messages from first layer parity-check clusters φ to θ j n transmit signal at the transmitter, and W ∼ N (0,γ ) is the t t 38: for each conditional Gaussian cluster n do added Gaussian distributed noise. In this case, the channel 39: ψ (b ) ← from Equation 30 θ ,φ n j noise precision γ is not constant for all time t. 40: ← from Equation 31 For the PGM to remain representative of the varying chan- 41: end for nel conditions, the parameters of the gamma distribution need 42: ← set to uniform prior 43: //VMP messages from θ to ζ n to adapt accordingly as LDPC packets arrive at the receiver. 44: for each conditional Gaussian cluster n do This can be achieved with Bayesian sequential learning also 45: μ ← from Equation 13 θ ,ζ known as Bayesian sequential filtering. Each time an LDPC 46: += μ ζ θ ,ζ packet is decoded, the parameters of the obtained posterior 47: end for 48: = + (see Equation 15) ζ ζ gamma distribution are stored and used as the prior dis- prior 49: iter ++ tribution when decoding the next LDPC packet. However, 50: if clusters φ calibrated and codeword valid then as an increased number of LDPC packets are decoded, our 51: converged stationarity assumptions cause an ever-increasing certainty 52: end if 53: end while around the gamma distribution’s mean and the PGM strug- gles to respond accurately to the changing noise. Tracking the The average divergence from both directions is used, taking into con- The following settings and software implementations sideration the mode-seeking and moment-seeking behaviours of the D (P||Q)+D (Q||P) KL KL apply to our inference approach: metric given by . 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… noise becomes restricted due to the strong underlying i.i.d. graph is compiled from the parity-check factors using the assumption across all noise estimates (from the observed LTRIP algorithm, and the message schedule is initialised by data), which will follow the average channel noise instead clusters with cardinality 8 and 10 (the largest factors), which of the evolving channel noise. form the bottom layer of the PGM (see Sect. 2.3.4). We use To remedy this, we introduce a time constant T that rep- BPSK modulation over an AWGN channel and assume a resents a period of time in which we assume the stationarity transmitted bit has unit energy. assumption holds. For example, if we assume no drastic changes in channel noise over a period of T seconds, our sta- 4.1 Purpose of stationary noise experiment T ×1000 tionarity assumption should be valid for S = number of LDPC packets (assuming a transmission time of 1ms per The stationary noise experiment is presented as a BER vs LDPC packet). SNR curve as typically would be the case for determining the We “relax” the i.i.d. assumption by accumulating the behaviour and performance of error correction codes over a posterior gamma distribution’s parameters as we normally range of SNR values. The purpose of the experiment is to would, but only up to S number of LDPC packets. After compare the BER performance between our proposed PGM the S number of packets are decoded, the posterior gamma and a PGM with perfect knowledge of the noise across a distribution’s parameters (the natural parameter vector) are range of SNR values. The selected SNR range consists of 6 S×N continuously being re-scaled with a scaling factor as equidistant points from 0 to 4.45 dB (inclusive). A random −1 bit message is encoded and Gaussian noise is added to each follows: bit, which repeats for 50k packets per SNR value. The same received bit values are presented to both PGMs to ensure a 2ω , − 1 ≤ S × N like-for-like comparison. We set the maximum number of ⎨ ν 2 − 1 =   (33) message passing iterations to 20 for both PGMs. Our chan- ζ T ⎪ − ⎪ S×N ν nel estimation PGM is parameterised with S = 10, which 2ω , − 1 > S × N . ν −1 2 − 1 assumes stationarity over 10 LDPC packets (equivalent to a period of 10ms). By scaling the gamma distribution’s parameters, the num- ber of observations ν that make up the posterior distribution 4.1.1 Results and interpretation reaches a ceiling value, which is not equal to the theoreti- cal maximum of the full i.i.d. assumption. This allows for The results of the stationary noise experiment are shown in variance around the gamma distribution’s mean so that the Fig. 6. The BER performance (shown on the left) of our pro- estimate does not become too “confident” in the data, which posed PGM closely follows the performance of a PGM with makes the gamma distribution responsive to the evolving perfect knowledge of the noise precision (the error bars reflect channel noise. Scaling the natural parameter vector in this the 95% confidence intervals). Similarly, the number of iter- way makes contributions from previous estimates progres- ations required to reach convergence (shown on the right) is sively less important compared to more recent estimates— almost identical between the two PGMs. allowing the PGM to “forget” historic channel noise. Note that this is not a windowing approach or a Kalman filter 4.2 Purpose of non-stationary noise experiment type approach where noise is added between estimations. The posterior gamma distribution remains informed by the The purpose of the experiment is to test whether our proposed entire sequence of received data, but the contributions from PGM is capable of tracking non-stationary channel noise, past data decay exponentially as packets are received and and if it benefits from the estimated channel noise informa- decoded. tion. We compare three scenarios: (1) a PGM that has perfect (instantaneous) knowledge of the channel noise precision, (2) a PGM using a fixed value for the channel noise precision, 4 Experimental investigation and (3) our proposed PGM that sequentially updates its chan- nel noise precision estimation. Note that the assumed channel This section describes the results obtained from stationary noise in (2) is the last precision mean obtained by running noise and non-stationary noise experiments. our proposed PGM over the test data once with S =∞. The LDPC code used is constructed from the 5G new radio While (2) assumes stationarity, its fixed value of the chan- (NR) standard [27]. We use base graph 2 with size (42, 52) nel was accumulated from the entire test sequence and is and expansion factor 11. The resultant H matrix is shortened advantageous, since it allows the PGM to “anticipate” future to a codeword length of N = 220 bits, of which K = 110 values of the channel noise. We set the maximum number of are message bits (producing a code rate R = 0.5). A cluster message passing iterations to 20 for all scenarios and param- 123 J. du Toit et al. Fig. 6 BER performance comparison between our proposed channel noise estimation PGM and a PGM with perfect knowledge of the channel noise. The BER and the number of iterations till convergence between the two systems are nearly identical eterise (3) with S = 10, which assumes stationarity over 10 LDPC packets (equivalent to a period of 10ms). 4.2.1 Drive-test data We test our model using actual drive test measurements obtained from Vodacom Group Limited, a mobile communi- cations company based in South Africa. Drive tests capture key network performance indicators that allow the mobile network operator to troubleshoot and optimise the network. We use the signal-to-interference-plus-noise ratio (SINR) measurements from a 5G device while driving approximately 10 km along a coastal route. The captured data includes han- dovers between cells as the user equipment travels in and out of reception range. The start and end times of the test along with the SINR measurements and GPS locations are shown in Fig. 7. The drive-test data had missing measurements and irregu- lar time intervals. The missing values were removed and the Fig. 7 Location of the drive test with the captured SINR measurements. sequence was re-indexed to create a time series with regu- Note the changes in signal-to-noise power as the user equipment moves lar intervals. It was upsampled using linear interpolation to through certain areas match a basic transmission time interval of 1ms. Note that our final test data may reflect more extreme channel variations precision values are used to add zero-mean Gaussian noise due to some discontinuities introduced as a consequence of to the transmitted signal, which produces the received signal. the described data cleaning. We generate random bit mes- Note that the same precision value is used to add noise to all sages that are encoded using the H matrix to produce LDPC signal values from the same LDPC packet. packets (or codewords). Bit values in an LDPC packet are The dataset used during the current study is not publicly modulated using BPSK modulation, and random Gaussian available due to the organisation’s data privacy policy, but noise is added to the resultant signal values using the SINR can be made available from the corresponding author on rea- drive-test data. The SINR data are converted from their loga- sonable request. rithmic form to a linear form to obtain precision values. The 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… 4.2.2 Results and interpretation Table 1 A summary of the overall performance comparison. Note that the expected behaviour of our PGM is similar to the PGM with perfect knowledge of the channel noise, and outperforms the PGM with fixed As stated earlier, we use the Gaussian distribution’s precision value of the average channel noise in terms of the BER to model the channel noise. Results from our experiment are BER (mean) Iterations (mean) shown in Fig. 8. The gamma posterior distribution is capable of tracking the actual noise precision (shown by the mean and True precision 0.003034 2.49 one standard deviation in the top part of the figure). The BER Fixed precision 0.005013 2.54 shown in the bottom part of the figure is a moving average Estimated precision 0.003336 2.49 consisting of 10000 packets centred around the current point in time; this helps to reveal the underlying trend. We use Dirichlet smoothing to avoid reporting BER values in the −7 range 0 < BER < 9 × 10 , which is smaller than the 5 Supplementary investigation possible BER. The BER shown in our results is calculated (a+bit errors in a packet) using: BER = , where a = In some instances, the BER performance of our proposed 10,000 (K +a) 0.005 and K is the number of message bits. PGM (estimating the SNR) is better than the PGM with We observe an improvement in the BER compared to a knowledge of the actual SNR. We noted this happening when PGM using a fixed value (a precision of 8.76 or 9.42 dB the estimated SNR is lower than the actual SNR, which is SNR) of the average noise precision (shown in the bottom counter-intuitive. part of the figure). A study presented in [8] investigates the effect of chan- The BER performance of our proposed PGM closely nel noise mismatch in LDPC codes. It found that the follows the BER performance of the PGM with perfect LDPC code’s performance degrades more quickly when the knowledge of the noise precision. This is due to the small assumed SNR is overestimated, but is less sensitive to degra- difference between the actual noise precision and the esti- dation when underestimated up to around 1 dB below the mated noise precision of our proposed PGM. actual SNR. What is also interesting is that the optimal BER In some experiments, we observed instances where the is at an assumed SNR lower than the actual SNR. estimated precision was much lower than the actual precision. We reproduce the same experiment to establish whether These were instances where the LDPC code failed to fix all bit this behaviour is similar for a cluster graph representation of errors, which the PGM then interpreted as worse than actual LDPC codes. Our channel noise mismatch experiment is pre- channel conditions. Another observation is that the gamma sented as a BER vs model SNR curve where the model SNR posterior has a wider standard deviation at higher precision is the assumed channel noise while the actual channel noise means, and a narrower standard deviation at lower precision is fixed at 1.22 dB (similar to [8]). The purpose of the exper- means. This is due to the characteristics of the gamma distri- iment is to (1) understand the impact on BER performance bution. We view this as a beneficial artifact, since the PGM when the actual channel noise is under- and over-estimated makes provision for more rapid changes in the channel con- by the model, and (2) determine where the optimal BER is. ditions at lower channel noise. (This can also be regarded as a A random bit message is encoded and Gaussian noise is "sceptical" type of behaviour.) When channel conditions are added using a fixed precision of 1.32 (equivalent to a SNR bad, the PGM’s channel estimation more confidently reports of 1.22 dB) to each bit, which repeats for 10k packets per that they are bad. model SNR value. The same received bit values are used for A summary of the overall results appears in Table 1.The each model SNR value to ensure a like-for-like comparison. average BER of our PGM is slightly higher than the PGM We set the maximum number of message passing iterations with perfect knowledge of the channel, and outperforms the to 20. The results of the channel noise mismatch experiment PGM with fixed value of the channel (at approximately 1.5 is shown in Fig. 9. times better BER on average). Our PGM requires approx- This result is similar to [8] and suggests that the LDPC imately 1.02 times fewer message passing iterations on code’s optimal BER is not necessarily at the point where average compared to the PGM with a fixed value of the chan- the model SNR is equal to the actual SNR, but somewhere nel, and the same number of iterations compared to the PGM slightly below the actual SNR (in this case at 0.84 dB). with perfect knowledge of the channel. A conservative estimation of the channel noise seems to be Note that the averages presented in this table are more beneficial, since the LDPC decoder is more forgiving in the influenced by the high SNR instances where bit errors are region below the actual noise. Using the drive-test data we infrequent. Figure 8 illustrates the performance advantage of run the same experiment as described in Sect. 4.2. However, our proposed PGM more clearly where bit errors occur more after each packet is decoded we adjust the estimated precision frequently at lower SNRs. value to be −0.1 dB below the posterior gamma mean value. We found that an adjustment larger than this yielded perfor- 123 J. du Toit et al. Fig. 8 Results of the three scenarios using the 5G device drive-test data. Note that our PGM tracks the changes in precision and maintains a BER similar to that of a PGM with perfect knowledge of the channel. It also outperforms a PGM employing a fixed average precision mance degradation. The result of this adjustment is shown in LDPC codes. We demonstrated the idea by employing a more Table 2. general probabilistic framework called cluster graphs, and A slight improvement in the BER can be seen, however, evaluated our proposed model using real-world data. The there is a trade-off between the number of iterations and BER results show that the performance of our proposed model is performance. The decoder requires slightly more iterations nearly indistinguishable from a model with perfect knowl- on average when the adjustment to the posterior gamma is edge of the channel noise. made. Apart from the performance advantages shown in our We emphasise that the purpose of the supplementary results, the approach embeds well within an LDPC decoder investigation is not to propose that an adjustment be made and does not require stand-alone machine learning tech- to the estimated SNR in practice, but rather to show the niques, pre-training, or management of large data sets. It is counter-intuitive behaviour of the LDPC code that we found capable of learning the channel noise accurately on-the-fly interesting. The benefit of adjusting the estimated SNR seems while decoding LDPC packets without compromise. minimal. The implications of this method with respect to the com- In our study, we simply selected the average SNR as the munication system are that (1) the scheduling and use of static value knowing that the true SNR will be better and pilot signals dedicated to channel noise estimation may at times will be worse. We refrain from making any asser- become redundant, (2) LDPC codes coupled with our pro- tions about the ideal static SNR value, recognising that it can posed channel-noise estimator can be used to estimate significantly vary based on specific circumstances. Instead, non-stationary channel noise on-the-fly with no need for our approach is to dynamically adjust to the real-time SNR scheduling, and (3) channel-noise information is now inher- conditions prevailing at any given moment. ent in the LDPC code and provides performance advantages to the code. Our current approach relies on parameterising a time con- stant, which is a prior assumption about the speed at which 6 Conclusion and future work channel noise will vary. Future work will focus on apply- ing real-time Bayesian model selection, allowing the LDPC This paper contributes a simple sequential Bayesian learn- decoder to choose intelligently between multiple assump- ing method for tracking non-stationary channel noise using 123 Low-density parity-check codes: tracking non-stationary channel noise using sequential… Fig. 9 The BER performance of a cluster graph LDPC code with the actual SNR fixed at 1.22 dB when the model SNR is varied. The error bars reflect the 95% confidence intervals Table 2 A summary of the overall performance comparison between of the more conservative approach performs better than our initial PGM the PGM that estimates the channel noise and the PGM that more conser- only in terms of the BER vatively estimates the channel noise. Note that the expected behaviour BER (mean) Iterations (mean) Estimated precision 0.003336 2.49 Estimated precision (−0.1 dB) 0.003329 2.49 tions (or combine multiple assumptions) about the speed at cate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, which the channel varies. Such work relates to Bayesian unless indicated otherwise in a credit line to the material. If material change-point detection, which accounts for changes in the is not included in the article’s Creative Commons licence and your underlying data-generating process while estimating param- intended use is not permitted by statutory regulation or exceeds the eters. While this study focuses on channel-noise estimation permitted use, you will need to obtain permission directly from the copy- right holder. To view a copy of this licence, visit http://creativecomm alone, the methodology could also be expanded to estimate ons.org/licenses/by/4.0/. other channel properties such as phase and complex chan- nel gain, which will depend on the modulation scheme and channel model discussed in Sect. 2. Acknowledgements The authors would like to thank Bonnie Xaba for References providing the drive-test data. She is a senior specialist in network bench- marking and E2E network performance quality at Vodacom Group 1. Rumney, M. (2013). LTE and the evolution to 4G wireless: Design LimitedinSouthAfrica. and measurement challenges (2nd ed.). Hoboken: Wiley. https:// doi.org/10.1002/9781118799475 Funding Open access funding provided by Stellenbosch University. No 2. Björnson, E., Hoydis, J., & Sanguinetti, L. (2017). Massive mimo funding was received for conducting this study. networks: Spectral, energy, and hardware efficiency. Foundations and Trends®in Signal Processing, 11(3–4), 154–655. https://doi. org/10.1561/2000000093 Declarations 3. Gallager, R. (1962). Low-density parity-check codes. IRE Trans- actions on Information Theory, 8(1), 21–28. Competing interests The authors declare no competing interests. 4. Davey, M. C., & MacKay, D. (1998). Low-density parity check codes over gf(q). IEEE Communications Letters, 2(6), 165–167. Open Access This article is licensed under a Creative Commons https://doi.org/10.1109/4234.681360 Attribution 4.0 International License, which permits use, sharing, adap- 5. Johnson, S. J. (2006). Introducing Low-density parity-check codes. tation, distribution and reproduction in any medium or format, as University of Newcastle, Australia, 1, 2006. long as you give appropriate credit to the original author(s) and the 6. Saeedi, H., & Banihashemi, A. H. (2007). Performance of belief source, provide a link to the Creative Commons licence, and indi- propagation for decoding ldpc codes in the presence of channel 123 J. du Toit et al. estimation error. IEEE Transactions on Communications, 55(1), conference—GLOBECOM (pp. 3743–3747). https://doi.org/10. 83–89. https://doi.org/10.1109/TCOMM.2006.887488 1109/GLOCOM.2012.6503699 7. Tan, W., & Cruz, J. R. (2004). Signal-to-noise ratio mismatch 18. Dahlman, E., Parkvall, S., & Skold, J. (2011). 4G LTE/LTE- for Low-density parity-check coded magnetic recording channels. Advanced for Mobile Broadband (1st ed., pp. 301–321). Oxford: IEEE Transactions on Magnetics, 40(2), 498–506. https://doi.org/ Academic Press. https://doi.org/10.1016/B978-0-12-385489-6. 10.1109/TMAG.2004.824130 00014-X 8. MacKay, D. J., & Hesketh, C. P. (2003). Performance of Low den- 19. Yedidia, J. S., Freeman, W. T., & Weiss, Y. (2005). Construct- sity parity check codes as a function of actual and assumed noise ing free-energy approximations and generalized belief propagation levels. Electronic Notes in Theoretical Computer Science, 74, 89– algorithms. IEEE Transactions on Information Theory, 51(7), 96. 2282–2312. https://doi.org/10.1109/TIT.2005.850085 9. MacKay, D. J. (2003). Information theory, inference and learning 20. Streicher, S., & Du Preez, J. (2017). Graph coloring: Comparing algorithms. Cambridge: Cambridge University Press. cluster graphs to factor graphs. Proceedings of the ACM multimedia 10. Koller, D., & Friedman, N. (2009). Probabilistic graphical models: 2017 workshop on South African Academic Participation (pp. 35– Principles and techniques (1st ed.). London: MIT Press. 42). https://doi.org/10.1145/3132711.3132717 11. Wang, S., Cui, L., Cheng, S., Zhai, Y., Yeary, M., & Wu, Q. 21. Bishop, C. (2006). Pattern Recognition and Machine Learning (2011). Noise adaptive LDPC decoding using particle filtering. (Vol. 4). London: Springer. IEEE Transactions on Communications, 59(4), 913–916. https:// 22. Winn, J., & Bishop, C. M. (2005). Variational message passing. doi.org/10.1109/CISS.2009.5054686 Journal of Machine Learning Research, 6, 661–694. 12. Jakubisin, D. J., Buehrer, R. M., & da Silva, C. R. (2016). 23. Du Toit, J., du Preez, J., & Wolhuter, R. (2022). LDPC Probabilistic receiver architecture combining bp, mf, and ep for codes: Comparing cluster graphs to factor graphs. arXiv preprint multi-signal detection. arXiv preprint arXiv:1604.04834. arXiv:2204.06350. 13. Wu, S., Kuang, L., Ni, Z., Lu, J., David Huang, D., & Guo, Q. 24. Taylor, R. M. (2023, March). Approximate message passing algo- (2014). Expectation propagation approach to joint channel estima- rithms for latent Dirichlet allocation. PhD thesis, Stellenbosch tion and decoding for ofdm systems. In 2014 IEEE international University conference on acoustics, speech and signal processing (ICASSP) 25. Lauritzen, S. L., & Spiegelhalter, D. J. (1988). Local computations (pp. 1941–1945). https://doi.org/10.1109/ICASSP.2014.6853937 with probabilities on graphical structures and their application to 14. Senst, M., & Ascheid, G. (2012). A message passing approach expert systems. Journal of the Royal Statistical Society: Series B to iterative Bayesian snr estimation. In 2012 International sympo- (Methodological), 50(2), 157–194. sium on signals, systems, and electronics (ISSSE) (pp. 1–6) (2012). 26. Shafer, G., & Shenoy, P. P. (1990). Probability propagation. Annals https://doi.org/10.1109/ISSSE.2012.6374328 of Mathematics and Artificial Intelligence, 2, 327–351. https://doi. 15. Nissila, M., & Pasupathy, S. (2009). Adaptive iterative detectors for org/10.1007/BF01531015 phase-uncertain channels via variational bounding. IEEE Trans- 27. Bae, J. H., Abotabl, A., Lin, H.-P., Song, K.-B., & Lee, J. (2019). actions on Communications, 57(3), 716–725. https://doi.org/10. An overview of channel coding for 5g nr cellular communications. 1109/TCOMM.2009.03.070068 APSIPA Transactions on Signal and Information Processing, 8, 16. Wang, S., Cui, L., & Cheng, S. (2011). Noise adaptive ldpc e17. decoding using expectation propagation. In 2011 IEEE Global Telecommunications Conference - GLOBECOM (pp. 1–5). https:// doi.org/10.1109/GLOCOM.2011.6133838 Publisher’s Note Springer Nature remains neutral with regard to juris- 17. Cui, L., Wang, S., & Cheng, S. (2012). Online snr statis- dictional claims in published maps and institutional affiliations. tic estimation for ldpc decoding over awgn channel using Laplace propagation. In 2012 IEEE global communications

Journal

Telecommunication Systems – Springer Journals

Published: Feb 1, 2024

Keywords: LDPC codes; Bayesian sequential learning; Variational inference; Cluster graphs

There are no references for this article.