Let's say we are given a sequence of $2n$ terms and were told that these are generated by a linear recurrence of order $n$ with the first $n$ terms of the sequence being the initial terms. More specifically, given $n$ terms, we want a recurrence that generates the last $\lfloor \frac{n}{2}\rfloor$ terms with the first $\lceil \frac{n}{2} \rceil$ terms.
The problem of finding the recurrence given the terms can be solved by Berlekamp Massey (BM) Algorithm. Though there are not many sources that I could find for solving recurrences with BM, I found the CodeForces entry that explains much of the details. Yet another source that I found very interesting is Berlekamp Massey algorithm.
To make the algorithm more clear than the quoted page, let's take an example. Let the sequence of terms be $[1,2,4,10,24,50,124]$.
Looking at (only) the first three terms, we make a first guess that
$\text{Guess 1:}\quad a_n=2a_{n-1}$ (or) $a_n-2a_{n-1}=0$
But when we include the fourth term, we run into a problem. We have a discrepancy.
$\Delta_4=\text{Actual}-a_4=10-2a_3=10-8=2$
We stare into the (only) first four terms of the sequence and come with another guess.
$\text{Guess 2:}\quad a_n=2a_{n-1}+a_{n-2}$ (or) $a_n-2a_{n-1}-2a_{n-2}=0$
And this correctly generates the fourth term.
(two tiny details at this point for the curious minded - starts here...)
First, someone could've guessed $a_n=2a_{n-1}+2a_{n-3}$. That works too. Continuing with the BM algorithm, we may end up with different answers but both recurrences will correctly generate the last three terms using the first four terms. Try it!
Second, though our second guess correctly predicts $a_4$, it fails for $a_3$. This is not a problem as Flawed guesses, especially for the first terms in the first half, gets 'course-corrected' at later stage of the algorithm. In the final algorithm, we only be making guesses, that too trivial guesses, at the initial stage.
(... and ends here)
We try it for the next term and we see the recurrence correctly generates fifth term. We're good. We try it for the next term and we run into problem. We have a discrepancy of $8(=58-50)$.
But this time we don't have to guess anything more. Note that we re-write our 'Guess 2' as
$\text{Guess 2:}\quad a_n=2a_{n-1}+a_{n-2}+k(a_{n-2}-2_{n-3})$
for any real $k$.
Where did the terms in the bracket come from? It our first guess shifted by two terms.
Why have we shifted it by two terms? Because we are currently finding a discrepancy at the sixth term and our last guess created a discrepancy at fourth term.
Will this not affect the previous terms? That is the key of BM algorithm. The terms in the bracket will become zero when we evaluate it for earlier terms.
For example, when we evaluate Guess 2 for $a_5$, the terms in the bracket will be $a_3-2a_2$ which is actually zero. In fact, only because it was zero, we made it our first guess.
Now that we have an additional factor of $k$ in our 'Guess 2' now, we can use it to our advantage. Choosing $k=-4$, we remove the discrepancy at the sixth position.
From now on, we can continue using the same idea. Anytime we face a discrepancy, we can choose an right multiple (based on the current and previous deviation) of our previous guess (shifted by the right amount) to correct that discrepancy. And this continues for all elements in the given sequence, thus completing the algorithm.
This type of problem comes every now and then in Competitive programming. For a given problem, we can quickly code a brute force method that can churn out the first few terms and we would be interested in finding a linear recurrence if there is one.
If we have the recurrence, finding the $n$th term (even for $n$ of order $10^{10^7}$) of the recurrence can be found in a matter of seconds. For example, we have this wonderful Codechef Editorial that explains how to do that.
Here is an (over)commented Python code of the BM algorithm based on what I wrote above.
from timeit import default_timer
start = default_timer()
def berlmassey(l0, p0):
# Input: list of length 2n and a prime p
if len(l) <= 1:
return l
if l[0] != 1: # WLOG, first element of the list should be 1
return []
old_rec = [] # there is no recurrence to start with
v = 1 # deviation of the (empty) recurrence that occurs in the first element
old_pos = 0 # trivially the list was fine until the first element
curr_rec = [l[1] % p] # we guess something that's trivially true
curr_pos = 1 # starting with the second element of the list
while curr_pos < len(l):
temp = 0
for i in range(len(curr_rec)): # check what the current recurrence gives for the next element
temp += curr_rec[i] * l[curr_pos - i - 1]
temp %= p
temp = (l[curr_pos] - temp) % p # is there a deviation?
if temp == 0: # If no deviation, current recurrence is still good and hence moving on..
curr_pos += 1
continue
temp_curr_rec = curr_rec[:] # just a temporary storage
v = temp * pow(v, -1, p) % p # normalizing factor based on current and last deviation
old_rec = [p - x for x in old_rec] # re-strucuturing the last successful recurrence
old_rec = [1] + old_rec
old_rec = [0] * (curr_pos - old_pos - 1) + old_rec
if len(old_rec) > len(curr_rec): # making last successful and current recurrences the same length
curr_rec = curr_rec + [0] * (len(old_rec) - len(curr_rec))
else:
old_rec = old_rec + [0] * (len(curr_rec) - len(old_rec))
curr_rec = [(curr_rec[i] + v * old_rec[i]) % p for i in range(len(curr_rec))] # updating the current recurrence with the normalizing constant and the (old) last successful recurrence
old_rec = temp_curr_rec[:] # the recurrence we started with becomes the last successful recurrence
v = temp # the deviation that occured at this position
old_pos = curr_pos # this position becomes the last deviated position
curr_pos += 1 # On to the next iteration..
return curr_rec
print(berlmassey([1, 2, 4, 10, 24, 50, 124], 1000000007))
print(default_timer() - start)
Finally, a couple of points regarding the code. As there are divisions involved, I have used a prime and made everything with modulo operations. We can also use exact divisions if there is a need. Hope you enjoyed it.
Until then
Yours Aye
Me
The problem of finding the recurrence given the terms can be solved by Berlekamp Massey (BM) Algorithm. Though there are not many sources that I could find for solving recurrences with BM, I found the CodeForces entry that explains much of the details. Yet another source that I found very interesting is Berlekamp Massey algorithm.
To make the algorithm more clear than the quoted page, let's take an example. Let the sequence of terms be $[1,2,4,10,24,50,124]$.
Looking at (only) the first three terms, we make a first guess that
$\text{Guess 1:}\quad a_n=2a_{n-1}$ (or) $a_n-2a_{n-1}=0$
But when we include the fourth term, we run into a problem. We have a discrepancy.
$\Delta_4=\text{Actual}-a_4=10-2a_3=10-8=2$
We stare into the (only) first four terms of the sequence and come with another guess.
$\text{Guess 2:}\quad a_n=2a_{n-1}+a_{n-2}$ (or) $a_n-2a_{n-1}-2a_{n-2}=0$
And this correctly generates the fourth term.
(two tiny details at this point for the curious minded - starts here...)
First, someone could've guessed $a_n=2a_{n-1}+2a_{n-3}$. That works too. Continuing with the BM algorithm, we may end up with different answers but both recurrences will correctly generate the last three terms using the first four terms. Try it!
Second, though our second guess correctly predicts $a_4$, it fails for $a_3$. This is not a problem as Flawed guesses, especially for the first terms in the first half, gets 'course-corrected' at later stage of the algorithm. In the final algorithm, we only be making guesses, that too trivial guesses, at the initial stage.
(... and ends here)
We try it for the next term and we see the recurrence correctly generates fifth term. We're good. We try it for the next term and we run into problem. We have a discrepancy of $8(=58-50)$.
But this time we don't have to guess anything more. Note that we re-write our 'Guess 2' as
$\text{Guess 2:}\quad a_n=2a_{n-1}+a_{n-2}+k(a_{n-2}-2_{n-3})$
for any real $k$.
Where did the terms in the bracket come from? It our first guess shifted by two terms.
Why have we shifted it by two terms? Because we are currently finding a discrepancy at the sixth term and our last guess created a discrepancy at fourth term.
Will this not affect the previous terms? That is the key of BM algorithm. The terms in the bracket will become zero when we evaluate it for earlier terms.
For example, when we evaluate Guess 2 for $a_5$, the terms in the bracket will be $a_3-2a_2$ which is actually zero. In fact, only because it was zero, we made it our first guess.
Now that we have an additional factor of $k$ in our 'Guess 2' now, we can use it to our advantage. Choosing $k=-4$, we remove the discrepancy at the sixth position.
From now on, we can continue using the same idea. Anytime we face a discrepancy, we can choose an right multiple (based on the current and previous deviation) of our previous guess (shifted by the right amount) to correct that discrepancy. And this continues for all elements in the given sequence, thus completing the algorithm.
This type of problem comes every now and then in Competitive programming. For a given problem, we can quickly code a brute force method that can churn out the first few terms and we would be interested in finding a linear recurrence if there is one.
If we have the recurrence, finding the $n$th term (even for $n$ of order $10^{10^7}$) of the recurrence can be found in a matter of seconds. For example, we have this wonderful Codechef Editorial that explains how to do that.
Here is an (over)commented Python code of the BM algorithm based on what I wrote above.
from timeit import default_timer
start = default_timer()
def berlmassey(l0, p0):
# Input: list of length 2n and a prime p
# Output: Recurrence of length n that takes the first n terms as initial values and generates the next n (and possibly the further) terms
l, p = l0[:], p0if len(l) <= 1:
return l
if l[0] != 1: # WLOG, first element of the list should be 1
return []
old_rec = [] # there is no recurrence to start with
v = 1 # deviation of the (empty) recurrence that occurs in the first element
old_pos = 0 # trivially the list was fine until the first element
curr_rec = [l[1] % p] # we guess something that's trivially true
curr_pos = 1 # starting with the second element of the list
while curr_pos < len(l):
temp = 0
for i in range(len(curr_rec)): # check what the current recurrence gives for the next element
temp += curr_rec[i] * l[curr_pos - i - 1]
temp %= p
temp = (l[curr_pos] - temp) % p # is there a deviation?
if temp == 0: # If no deviation, current recurrence is still good and hence moving on..
curr_pos += 1
continue
temp_curr_rec = curr_rec[:] # just a temporary storage
v = temp * pow(v, -1, p) % p # normalizing factor based on current and last deviation
old_rec = [p - x for x in old_rec] # re-strucuturing the last successful recurrence
old_rec = [1] + old_rec
old_rec = [0] * (curr_pos - old_pos - 1) + old_rec
if len(old_rec) > len(curr_rec): # making last successful and current recurrences the same length
curr_rec = curr_rec + [0] * (len(old_rec) - len(curr_rec))
else:
old_rec = old_rec + [0] * (len(curr_rec) - len(old_rec))
curr_rec = [(curr_rec[i] + v * old_rec[i]) % p for i in range(len(curr_rec))] # updating the current recurrence with the normalizing constant and the (old) last successful recurrence
old_rec = temp_curr_rec[:] # the recurrence we started with becomes the last successful recurrence
v = temp # the deviation that occured at this position
old_pos = curr_pos # this position becomes the last deviated position
curr_pos += 1 # On to the next iteration..
return curr_rec
print(berlmassey([1, 2, 4, 10, 24, 50, 124], 1000000007))
print(default_timer() - start)
Finally, a couple of points regarding the code. As there are divisions involved, I have used a prime and made everything with modulo operations. We can also use exact divisions if there is a need. Hope you enjoyed it.
Until then
Yours Aye
Me
No comments:
Post a Comment