Artificial Intelligence and Quantum Computing for Advanced Wireless Networks. Savo G. Glisic

concept can be extend and, instead of making a hard class assignment, a fuzzy class assignment can be used by allowing 0 ≤ u_χ(x) ≤ 1 and requiring for all x. This is another vector quantization algorithm called fuzzy k ‐means. The k‐means algorithm is based on a quadratic objective function, which is known to be strongly affected by outliers. This drawback can be alleviated by taking the l₁ norm of the approximation errors and modifying the problem to J_K‐medians = subject to U^t U = I and u_ij ∈ {0, 1}. A different approach can be used to find data representatives less affected by outliers, which we may call robust vector quantization, , where Φ(x) is a function less sensitive to outliers than Φ(x) = x, for instance, Φ(x) = x^α with α about 0.5.

      Principal component analysis (PCA): Introduced in Section 2.1, is by far one of the most popular algorithms for dimensionality reduction [39–42]. Given a set of observations x, with dimension M (they lie in ℝ^M), PCA is the standard technique for finding the single best (in the sense of least‐square error) subspace of a given dimension, m. Without loss of generality, we may assume the data is zero‐mean and the subspace to fit is a linear subspace (passing through the origin).

      This algorithm is based on the search for orthogonal directions explaining as much variance of the data as possible. In terms of dimensionality reduction, it can be formulated [43] as the problem of finding the m orthonormal directions w_i minimizing the representation error . In this objective function, the reduced vectors are the projections χ = (〈w₁, x〉, …, 〈w_m, x〉)^t This can be much more compactly written as χ = W^tx, where W is a M × m matrix whose columns are the orthonormal directions w_i {or equivalently W^t W = I). The approximation to the original vectors is given by x〉w_i, or equivalently, W χ .

In Figure 2.15, we have shown a graphical representation of a PCA transformation in only two dimensions (x ∈ ℝ²) with a slightly different notation (χ represented by z). As can be seen from Figure 2.15, the variance of the data in the original data space is best captured in the rotated space given by vectors =W^tx. χ₁ is the first principal component, and it goes in the direction of most variance; χ₂ is the second principal component, and is orthogonal to the first direction and goes in the second direction with the most variance (in ℝ² there is not much choice, but in the general case, ℝ^M, there is). Observe that without loss of generality the data is centered about the origin of the output space. We can rewrite the objective function as

      Note that the class membership matrix (U in vector quantization) has been substituted in this case by W^t X, which in general can take any positive or negative value. It, thus, has lost its membership meaning and simply constitutes the weights of the linear combination of the column vectors of W that better approximate each input x. Finally, the PCA objective function can also be written as

      J_PCA = Tr{W^t ∑_X W} [44], where is the covariance matrix of the observed data. The PCA formulation has also been extended to complex‐valued input vectors [45]; the method is called non‐circular PCA.

      The matrix projection of the input vectors onto a lower‐dimensional space ( χ = W^tx) is a widespread technique in dimensionality reduction. As an illustration, let us look at the following example [46]:

      Design Example 2.5

      Assume that we are analyzing scientific articles related to a specific domain. Each article will be represented by a vector x of word frequencies; that is, we choose a set of M words representative of our scientific area, and we annotate how many times each word appears in each article. Each vector x is then orthogonally projected onto the new subspace defined by the vectors w_i. Each vector w_i has dimension M, and it can be understood as a “topic” (i.e. a topic is characterized by the relative frequencies of the M different words; two different topics will differ in the relative frequencies of the M words). The projection of x onto each w_i gives an idea of how important topic w_i is for representing the article. Important topics have large projection values and, therefore, large values in the corresponding component of χ.

      It can be shown [43, 47], as already indicated in Section 2.1, that when the input vectors, x, are zero‐mean (if they are not, we can transform the input data simply by subtracting the sample average vector), then the solution of the minimization of J_PCA is given by the m eigenvectors associated to the largest m eigenvalues of the covariance matrix of x {, note that the covariance matrix of x is a M × M matrix with M eigenvalues). If the eigenvalue decomposition of the input covariance matrix is Скачать книгу