Artificial Intelligence and Quantum Computing for Advanced Wireless Networks. Savo G. Glisic

problem of fitting straight lines to patterns of data. In a linear regression model, the variable of interest (the so‐called “dependent” variable) is predicted from k other variables (the so‐called “independent” variables) using a linear equation. If Y denotes the dependent variable and X₁, …, X_k, are the independent variables, then the assumption is that the value of Y at time t is determined by the linear equation

(2.1)

      The corresponding equation for predicting Y_t from the corresponding values of the X’s is therefore

(2.2)

      where the b’s are estimates of the betas obtained by least squares, that is, minimizing the squared prediction error within the sample. This is about the simplest possible model for predicting one variable from a group of others, and it rests on the assumption that the expected value of Y is a linear function of the X variables. More precisely, the following is assumed:

      1 The expected value of Y is a linear function of the X variables. This means: (i) If Xi changes by an amount ∆Xi, holding other variables fixed, then the expected value of Y changes by a proportional amount βi∆Xi, for some constant βi (which in general could be a positive or negative number). (ii) The value of βi is always the same, regardless of values of the other X’s. (iii) The total effect of the X’s on the expected value of Y is the sum of their separate effects.

      2 The unexplained variations of Y are independent random variables (in particular, not “autocorrelated” if the variables are time series).

      3 They all have the same variance (“homoscedasticity”).

      4 They are normally distributed.

      These assumptions will never be exactly satisfied by real data, but you hope that they are not badly wrong. For proper regression modeling, we need to collect data that are relevant and informative with respect to our decision problem, and then define the variables and construct the model in such a way that the assumptions listed above are plausible, at least as a first‐order approximation to reality.

      If we normalize the values of Y and X as

      with the correlation function defined as

      the phenomenon that Galton noted was that the regression line for predicting Y* from X* passes through the origin and has a slope equal to the correlation between Y and X; that is, the regression equation in normalized units is

Figures 2.1 and 2.2 illustrate this equation [1]. When the units of X and Y are standardized and both are also normally distributed, their values are distributed in an elliptical pattern that is symmetric around the 45° line, which has a slope equal to 1.

      However, the regression line for predicting Y* from X* is not the 45° line. Rather, it is a line passing through the origin whose slope is r_XY, the dashed gray line in the picture below, which is tilted toward the horizontal because the correlation is less than 1 in magnitude. In other words, it is a line that “regresses” (i.e. moves backward) toward the X‐axis.

      2.1.2 Logistic Regression

      Logistic regression analysis studies the association between a categorical dependent variable and a set of independent (explanatory) variables. The term logistic regression is used when the dependent variable has only two values, such as 0 and 1, or Yes and No. Suppose the numerical values of 0 and 1 are assigned to the two outcomes of a binary variable. Often, 0 represents a negative response, and 1 represents a positive response. The mean of this variable will be the proportion of positive responses. If p is the proportion of observations with an outcome of 1, then 1 − p is the probability of a outcome of 0. The ratio p/(1 − p) is called the odds and the logit is the logarithm of the odds, or just log odds. Formally, the logit transformation is written as l = logit (p) = ln (p/(1 − p)). Note that while p ranges between 0 and 1, the logit ranges between minus and plus infinity. Also, note that the zero logit occurs when p is 0.50. The logistic transformation is the inverse of the logit transformation. It is written as p = logistic(l) = e^l/(1 + e^l)

      The difference between two log odds can be used to compare two proportions, such as that of boys versus girls. Formally, this difference is written as

      (2.3)

Figure 2.1 If X and Y are two jointly normally distributed random variables, then in standardized units (X*, Y*) their values are scattered in an elliptical pattern that is symmetric around the 45° line.

      Source: Modified from Introduction to linear regression analysis [50]. Available at https://people.duke.edu/~rnau/regintro.htm.

This difference is often referred to as the log odds ratio. The odds ratio is often used to compare