Friday, July 22, 2016

Lemma - Dyadic (XOR) Convolution Theorem


Note1: This lemma is used to prove the Dyadic Convolution Theorem.

Note2: To understand this proof, you should have a good understanding of Bit-wise operators. The first answer of this question has a nice explanation in case you need it.

Lemma Let $n(k)$ count the number of $1$s in the binary expansion of the non-negative integer $k$. We then claim that $n(i)+n(j)$ and $n(i \oplus j)$ have the same parity. Here again $\oplus$ denote the BitXOR operator.

(In the following proof, we use $'\&'$ to denote BitAND and $'|'$ to denote BitOR)

Proof It suffices to prove that $n(i)+n(j)-n(i\oplus j)$ is even. To do this, we find $n(i|j)$ in two ways. First, $i|j$ has a 0 if both bits of $i$ and $j$ in a position are 0, while otherwise it has a 1. Therefore,

$n(i|j)=n(i \oplus j)+n(i\&j)$

Here, $n(i \oplus j)$ counts the number of 1's in positions that has a 1 in exactly one of two numbers $i$ and $j$, while $n(i\&j)$ counts the number of 1's in positions that has a 1 in both the numbers.

On the other hand, we can simply use the inclusion-exclusion principle to compute $n(i|j)$. Here we have,

$n(i|j)=n(i)+n(j)-n(i\&j)$

Equating the two we get, $n(i)+n(j)-n(i\oplus j)=2\text{ }n(i\&j)$. $\blacksquare$

Dyadic (XOR) Convolution theorem

The Dyadic (XOR) convolution of the sequences $a$ and $b$ is the sequence $c=a*b$ defined by

$c_k=\displaystyle\sum_{i\oplus j = k}a_ib_j=\sum_{i}a_ib_{i\oplus k}$

where the symbol $\oplus$ stands for bit-wise XOR operator. All the three sequences must of the same length that is a power of 2 (we could pad them with zeroes if they are not).

The Dyadic (XOR) convolution theorem states that for two sequences $a$ and $b$,

$c=a*b \implies H_mc=H_ma\cdot H_mb$

where $\cdot$ deontes pointwise multiplication and $H_m$ is the Hadamard matrix. We omit the normalization of the matrix throughout this post.

Though all these are available in Wikipedia and other webpages, nowhere was I able to find a proof of the Dyadic (XOR) convolution theorem. So that is the point of this post. Let's attack the theorem head-on.

Before we proceed to the proof of the convolution theorem, we notice a property of the Hadamard matrices. Indexing the rows and colomns of the Hadamard matrices from 0, we can directly have the $(i,j)$ of entry of the matrix using the following formula.

$(H_m)_{(i,j)}=(-1)^{n(i\&j)}$

where $n(k)$ counts the number of 1s in the binary expansion of the non-negative ineger $k$.This definition plays a crucial role in our proof. We now have all that we need to prove the Dyadic (XOR) convolution theorem. For simplicity, we drop the suffix in the Hadamard matrix, $H_m$. 

Proof of the Convolution Theorem Notice that the $k$th element of the vector $Ha$ is given by

$(Ha)_k=(-1)^{n(0\&k)}a_0+(-1)^{n(1\&k)}a_1+(-1)^{n(2\&k)}a_2+\cdots$

We get this by directly multiplying the $k$th row of the Hadamard matrix with the $a$ vector. Similar expression holds for $Hb$. Therefore, the $k$the entry of the pointwise product is

$\begin{align}
(Ha)_k\cdot(Hb)_k&=\displaystyle\left(\sum_i (-1)^{n(i\&k)}a_i\right)\left(\sum_j (-1)^{n(j\&k)}a_j\right)\\
&=\bigl((-1)^{n(0\&k)}a_0+(-1)^{n(1\&k)}a_1+(-1)^{n(2\&k)}a_2+\cdots\bigl)\bigl((-1)^{n(0\&k)}b_0+(-1)^{n(1\&k)}b_1+(-1)^{n(2\&k)}b_2+\cdots\bigl)
\end{align}$

Therefore,

$[a_ib_j](Ha)_k\cdot(Hb)_k=(-1)^{n(i\&k)+n(j\&k)}$

where $[]$ denotes the Coefficient extracting operator.

Now the $k$th element of $Hc$ is

$(Hc)_k=(-1)^{n(0\&k)}c_0+(-1)^{n(1\&k)}c_1+\cdots$

Therefore, $[c_r](Hc)_k=(-1)^{n(r\&k)}$ from which we get

$[a_ib_{i \oplus r}](Hc)_k=(-1)^{n(r\&k)}$ (Using $c_r=a_0b_{0 \oplus r}+a_1b_{1 \oplus r}+a_2b_{2 \oplus r}+\cdots$)

Plugging in $r=i \oplus j$ gives

$[a_ib_j](Hc)_k=(-1)^{n((i \oplus j)\&k)}=(-1)^{n(i\&k\text{ } \oplus \text{ } j\&k)}$

For the Dyadic Convolution theorem to hold, we have to show that the coefficients are equal. That means, the exponents have the same parity. Take, $i'=i\&k$ and $j'=j\&k$. Using the lemma, we see that this is indeed true. $\blacksquare$

Friday, July 15, 2016

An application of Bivariate Ordinary Generating functions

It's interesting to note how each type of Generating function has its own advantages and disadvantages. For example, we saw how, given an OGF or an EGF, partial sums are readily available with certain simple manipulations. But in case of DGFs, we had to rely on Dirichlet's Hyperbola method to evaluate the partial sums.

On the other hand, if you are able to calculate the number of numbers satisfying a 'property' via DGF, then it's not too hard to also calculate the sum of numbers satisfying the same 'property' with a simple change in the DGF. However, it is not the case with OGFs.

Consider the question of finding the number of numbers with atmost $n$ digits whose digit-sum is $k$. We can easily construct its OGF as follows.

$f(x) = (1+x+x^2+\cdots+x^9)^n$

The above expression simply says that each digit can take any value from 0 to 9. Expand $f(x)$ and the coefficient of $[x^k]$ will give you the required answer.

Here we consider the question of finding the sum of numbers with atomst $n$ digits with a digit-sum of $k$. I'm not saying that this cannot be done with an one-variable OGF, but certainly using a bivariate OGF makes things simpler. In addition to noting how much each digit adds to the sum, we must also make a note of each of the digit with its place value. We define some functions as follows:

$f_n(x,y) = 1+xy^{10^{n-1}}+x^2y^{2.10^{n-1}}+\cdots+x^9y^{9.10^{n-1}}$

Each function $f_n(x,y)$ encodes the information about the $n$th digit, making a note of what it adds to the digit sum and digit with its place value. Defining a new function

$f(x,y)=f_1(x,y).f_2(x,y)\cdots.f_n(x,y)$

gives what we need. Differentiating the above generating function w.r.t $y$ and substituting $y=1$ (If you wanna know what this does, refer 'Multivariate Generating Functions' in 'Analytic Combinatorics'), tells us how much each digit contributes to the overall 'sum of numbers with atmost $n$ digits'. . We now let,

$g(x)=\dfrac{d}{dy}f(x,y)\biggl\vert_{y=1}$

This Generating function we are left with is a single variable OGF in which the coefficient of $x^k$ gives the sum of numbers whose digit-sum is $k$. Play around with numbers with certain other 'properties' and you will really see what this post is trying to convey.

I think my next post will be on Multiset Cycle Indices, though am not sure where and/or how to start. C ya then.



Until then
Yours Aye,
Me

Sunday, November 29, 2015

C-code for the PrimePi function

This code uses the algorithm described here. The same algorithm has been used for SumOfPrimes function here.


#include<stdio.h>
#include<math.h>

#define M 1000000
#define Mod 1000000000

int Lower[M+1], Higher[M+1];
char Used[M+1];

long long SumNat(long long n)
{
    long long p = n, q = n+1;
    if(p%2)
        q /= 2;
    else
        p /= 2;
    return (p % Mod) * (q % Mod) % Mod;
}

long long PrimePi(long long n)
{
    long long v = sqrt(n), p, temp, q, j, end, i, d, t;
    Higher[1] = n % Mod - 1;
    for(p = 2; p <= v; p++) {
        Lower[p] = p - 1;
        Higher[p] = (n/p) % Mod - 1;
    }
    for(p = 2; p <= v; p++) {
        if(Lower[p] == Lower[p-1])
            continue;
        temp = Lower[p-1];
        q = p * p;
        Higher[1] -= Higher[p] - temp;
        if(Higher[1] < 0)
            Higher[1] += Mod;
        j = 1 + (p & 1);
        end = (v <= n/q) ? v : n/q;
        for(i = p + j; i <= 1 + end; i += j) {
            if(Used[i])
                continue;
            d = i * p;
            if(d <= v)
                Higher[i] -= Higher[d] - temp;
            else {
                t = n/d;
                Higher[i] -= Lower[t] - temp;
            }
            if(Higher[i] < 0)
                Higher[i] += Mod;
        }
        if(q <= v)
            for(i = q; i <= end; i += p*j)
                Used[i] = 1;
        for(i = v; i >= q; i--) {
            t = i/p;
            Lower[i] -= Lower[t] - temp;
        }
    }
    return Higher[1] % Mod;
}

int main() {
    printf("%lld", PrimePi(1000000000000LL));
    return 0;
}

Well, I conclude the discussion on the PrimePi functions with this post, I'll think of something to continue blogging which may take a while.



Until then
Yours Aye
Me

C-code for the SumOfPrimes function


We saw the algorithm for the generalized Prime counting function in the previous post. Here is the C code for the same, with some slight optimizations.


#include<stdio.h>
#include<math.h>

#define M 1000000
#define Mod 1000000000

long long Lower[M+1], Higher[M+1];
char Used[M+1];

long long SumNat(long long n)
{
    long long p = n, q = n+1;
    if(p%2)
        q /= 2;
    else
        p /= 2;
    return (p % Mod) * (q % Mod) % Mod;
}

long long SumPrimePi(long long n)
{
    long long v = sqrt(n), p, temp, q, j, end, i, d, t;
    Higher[1] = SumNat(n) - 1;
    for(p = 2; p <= v; p++) {
        Lower[p] = SumNat(p) - 1;
        Higher[p] = SumNat(n/p) - 1;
    }
    for(p = 2; p <= v; p++) {
        if(Lower[p] == Lower[p-1])
            continue;
        temp = Lower[p-1];
        q = p * p;
        Higher[1] -= (Higher[p] - temp) % Mod * p % Mod;
        if(Higher[1] < 0)
            Higher[1] += Mod;
        j = 1 + (p & 1);
        end = (v <= n/q) ? v : n/q;
        for(i = p + j; i <= 1 + end; i += j) {
            if(Used[i])
                continue;
            d = i * p;
            if(d <= v)
                Higher[i] -= (Higher[d] - temp) % Mod * p % Mod;
            else {
                t = n/d;
                Higher[i] -= (Lower[t] - temp) % Mod * p % Mod;
            }
            if(Higher[i] < 0)
                Higher[i] += Mod;
        }
        if(q <= v)
            for(i = q; i <= end; i += p*j)
                Used[i] = 1;
        for(i = v; i >= q; i--) {
            t = i/p;
            Lower[i] -= (Lower[t] - temp) % Mod * p % Mod;
            if(Lower[i] < 0)
                Lower[i] += Mod;
        }
    }
    return Higher[1] % Mod;
}

int main() {
    printf("%lld", SumPrimePi(1000000000000LL));
    return 0;
}


The code computes the value of $\pi_1(10^{12})$ in about 6 seconds which is way, way better than Mathematica's computation time. If you are looking for the C-code for $\pi(x)$, you can find it in this post.

Saturday, November 28, 2015

Algorithm for the Prime counting function


In Counting Primes - Prime Counting function - Part III, we saw the function used to calculate the sum of all primes less than a given value and it's Mathematica code. Well, the Mathematica code is good enough but we can do better with C. But if we try to convert the code given into C, we run into a problem. In the given code, we have relied heavily upon Mathematica's in-built functions. But we can sort the problem easily by using the some of the properties $\varphi_k(x,\pi(\sqrt{x}))$. First note that,

Result 1 : $\pi_k(x)=\varphi_k(x,\pi(\sqrt{x}))=\varphi_k(x,\sqrt{x})=\varphi_k(x,x-2)=\varphi_k(x,x-1)$, since $\pi(x) \leq x$.

Secondly, $\pi_k(x)$ increases only at prime values of $x$. This can be used to test the primality of $x$ in our C-code. Or casting it in different terms, we can say

Result 2 : $x$ is prime, if and only if $\pi_k(x) > \pi_k(x-1)$ (or) $\varphi_k(x,x-2) > \varphi_k(x-1,x-2)$.



The Algorithm to find $\pi_k(n)$ will now proceed as follows.

Input : Get the value of $n$.

Step 1: Create a table with 2 rows. Mark the first row as $m$ and fill it as $1,2,\cdots,\lfloor\sqrt{n}\rfloor, \lfloor n/\lfloor\sqrt{n}\rfloor\rfloor,\cdots,\lfloor n/2\rfloor,\lfloor n/1 \rfloor$. Mark the second row as $\varphi_k(m)$ and the fill the corresponding $\varphi_k(m,0)$ values.
Step 2 : Initiate $r=2$.
Step 3 : Check whether $r$ prime or not. To do this, compare the corresponding $\varphi$ entries of $r$ and $r-1$ (making use of Result 2).
Step 4 : If they are same, skip to Step 6. Else, conclude $r$ to be a prime.
Step 5 : For each $m\geq r^2$ , starting from the last entry and moving left, update the values of $\varphi_1$ using the recurrence, $\varphi_k(m)=\varphi_k(m)-r^k*(\varphi_k(\lfloor m/r \rfloor)-\varphi_k(r-1))$
Step 6 : Increment $r$.
Step 7: If $r$ equals $\lfloor\sqrt{n}\rfloor$, output the rightmost entry of the second row and Stop. Else, goto Step 3.



For example, the tables while computing $\pi_1(20)$ are given below.

TABLE 1 :: $\varphi_1(m,0)$
$
\begin{matrix}
m&1&2&3&4&5(=\lfloor20/4\rfloor)&6(=\lfloor20/3\rfloor)&10(=\lfloor20/2\rfloor)&20(=\lfloor20/1\rfloor)\\
\varphi_1(m)&0&2&5&9&14&20&54&209\\
\end{matrix}
$

TABLE 2 :: $\varphi_1(m,1)$
$
\begin{matrix}
m&1&2&3&4&5&6&10&20\\
\varphi_1(m)&0&2&5&5&10&10&26&101\\
\end{matrix}
$

TABLE 3 :: $\varphi_1(m,2)$
$
\begin{matrix}
m&1&2&3&4&5&6&10&20\\
\varphi_1(m)&0&2&5&5&10&10&17&77\\
\end{matrix}
$

Note a couple of things here. The table is modified $\pi(\lfloor\sqrt{n}\rfloor)$ times after the initialization. As a byproduct of the algorithm, we get the SumOfPrimes for all the entries in the first row. Also, we generate the list of primes $\leq \lfloor\sqrt{n}\rfloor$.

After I wrote this, I found that no text can really equal a cute demonstration in terms of clarity. So I found this, which will visualizes the algorithm's execution. Hope that clarifies things. I'll post the C-code for this algorithm in the next post.


Until then
Yours Aye
Me

Friday, November 27, 2015

Pólya's Enumeration Theorem with Applications - Part II

This post continues what we discussed here.

Pólya's Enumeration theorem allows us to do much more. In all of the examples above, we did not care about the number of balls we had for each color. In fact, that information was not needed to answer the questions we asked ourselves. Also, in both the necklace and the urn problems, we allowed only one ball at a given position. What if there can be more than one ball at one particular position? Consider the following question.

Example 5: 8 indistinguishable Urns arranged in a circle. What is the number of ways to distribute $n$ blue balls into these urns discounting rotational symmetry?

To make the question clear, number the urns from 1 to 8 in clockwise direction. Let's say we have 16 blue balls. Then according to the question, the distributions {1,2,3,4,5,1,0,0} and {5,1,0,0,1,2,3,4} are identical because one can be transformed to other by rotations. To solve this problem with Pólya's Enumeration Theorem we first find the cycle index of Cyclic group of degree 8.

$Z_{C_8}(x_1,x_2,...,x_8)=\frac{1}{8}(x_1^8+x_2^4+2x_4^2+4x_8)$

Pólya's Enumeration theorem says the answer we are looking for is given by

$\frac{1}{8}((X_1^8)_n+(X_2^4)_n+2(X_4^2)_n+4(X_8)_n)$

where $X_a^b=\frac{1}{(1-z^a)^b}$ and $(F(z))_n$ is the coefficient of $z^n$ in the power series of $F(z)$. With Mathematica's help we find the answer to be $30667$ when $n=16$.

The same procedure can be applied for all the permutations groups and each will give the required answer according to their nature. One thing of interest here is that when there are $n$ balls of the same color, $Z_{S_r}$ counts the number of partitions of $n$ into $r$ parts ($0$ included) and $Z_{-S_r}$ counts the number of partitions of $n$ into $r$ distinct parts ($0$ included). This interpretation has an interesting application in the multiplicative partition of $n$ which will be final piece of our discussion.

Now, the previous problem had balls of the same color. What if there are $n_i$ balls of color $c_i$. This is the second main point that I wished to convey in this post. Let's look at the next problem.

Example 6: 4 indistinguishable Urns are arranged in a circle. What is the number of ways to distribute $m$ different colored balls, with $n_i$ balls of color $c_i$, into these urns discounting rotational symmetry?

The answer to this question proceeds in pretty much the same way as the previous answer. We find the cycle index of Cyclic group of degree 4. The important distinction comes in the definition of $(F(z))_n$. Write $n=(n_1,n_2,..,n_m)$ and define,

$(F(z))_n=\Pi_{j=1}^m(F(z))_{n_j}$ where $F(z)$ will be of the form $\Pi_{i=1}^rX_i^\alpha$.

In case of 4 Urns with 1 white balls and 1 black balls discounting rotational symmetry, we use

$Z_{C_4}(x_1,x_2,x_3,x_4)=\frac{1}{4}(x_1^4+x_2^2+2x_4)$

and the final answer will be $\frac{1}{4}((X_1^4)_{(1,1)}+(X_2^2)_{(1,1)}+2(X_4)_{(1,1)}) = 4$

because {bg,$0$,$0$,$0$}, {b,$0$,g,$0$}, {b,g,$0$,$0$} and {g,b,$0$,$0$} are the possible arrangements.

Example 7: What is the number of ways to distribute 2 white balls and 1 black balls into 3 indistinguishable Urns without taking order into account?

This question can be interpreted in at least two (equivalent) ways. First, it is the number of ways of partitioning the 2-partite number (2,1) into three 2-partite numbers. Second, it is the number of multiplicative partitions of $12(=2^2\times3^1)$. That's easy. $12 = 1\times 2\times 6 = 1\times 3\times 4 = 2\times 2\times 3 = 1\times 1\times 12$. Let's check whether Pólya's Enumeration Theorem gives us the same answer.

$Z_{S_3}=\frac{1}{6}(x_1^3+3x_1x_2+2x_3)$

Following the same procedure as above we see the answer is $\frac{1}{6}((X_1^3)_{(2,1)}+3(X_1X_2)_{(2,1)}+2(X_3)_{(2,1)})$.  Let's just evaluate one of these terms, preferably $(X_1X_2)_{(2,1)}$.

$(X_1X_2)_{(2,1)}=[z^2]\frac{1}{(1-z)(1-z^2)}\times[z^1]\frac{1}{(1-z)(1-z^2)}=2\times1=2$.

The final answer is $\frac{1}{6}(6\times3+3\times2\times1+2\times0\times0)=4$ which we saw already. In terms of balls and Urns the distribution would look like {$0$,w,wb}, {$0$,b,ww}, {w,w,b} and {$0$,$0$,wwb}.

Example 8: What is the answer to the same question if we want no two urns to have the same pattern of balls?

In this case it asks for the multiplicative partition of 12 with distinct numbers in which case the last two multiplicative partitions listed above will be eliminated. In terms of Urns and balls, {w,w,b} will be eliminated because two urns have one white ball each and {$0$,$0$,wwb} will be eliminated because two urns are empty. To get the answer via Pólya's Enumeration theorem, we'll be using $Z_{-S_3}$ instead of $Z_{S_3}$.

$Z_{-S_3}=\frac{1}{6}(x_1^3-3x_1x_2+2x_3)$

Hence the answer will be  $\frac{1}{6}((X_1^3)_{(2,1)}-3(X_1X_2)_{(2,1)}+2(X_3)_{(2,1)})=2$.

Well, that's it from my end. I'm quite impressed with the way I progressed but it's still upto a first time reader to really judge how good I was in explaining what I intended to explain. Any suggestions and corrections are welcome. It feels extremely good to have come up with this post and I enjoyed it. Hope you'll do too. C ya later.

Until then
Yours Aye,
Me