Python for Bioinformatics: Sending data into a generator function

Thursday, January 28, 2010

Sending data into a generator function

Generators and iterators are relatively new to Python. Here is a nice discussion of functional programming that has a lot of good information about them, as well as other topics.

There is some technical distinction between the terms. An interator is something you can invoke next on, and a generator function or expression is something that returns an iterator. At least that much, I think I understand. So this first example is a simple generator function for the Fibonacci series:

>>> def fib():
...     a, b = 1,1
...     while True:
...         yield a
...         a, b = b, a+b
... 
>>> it = fib()
>>> import itertools
>>> list(itertools.islice(it, 10))
[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

We use islice from itertools to get a slice from it. This setup has the disadvantage that we are computing two values ahead of the one we actually return, but it allows me to do the yield in only one place.

One great thing about iterators is that they allow you to deal with sequences that go on forever (like the Fibonacci sequence does) in a compact way. One can make a finite iterator like this:

>>> def f():
...     for i in range(3):
...         yield 2**i
...     raise StopIteration
... 
>>> it = f()
>>> for n in it:  print n
... 
1
2
4
>>> it = f()
>>> list(it)
[1, 2, 4]

The next example uses an "advanced" feature of iterators. It is possible to code it so that a value can be passed into the generator function by use of the send method. See here. In this contrived example, we simply advance the generator a given number of steps:

>>> def fib():
...     a, b = 1,1
...     while True:
...         value = (yield a)
...         try:
...             if value:
...                 value = int(value)
...         except ValueError:
...             value = None
...         if value:
...             for i in range(value):
...                  a, b = b, a+b
...         else:
...             a, b = b, a+b
... 
>>> it = fib()
>>> it.next()
1
>>> it.send(5)
8
>>> import itertools
>>> list(itertools.islice(it, 10))
[13, 21, 34, 55, 89, 144, 233, 377, 610, 987]

The call to send does a yield thing.

A more serious use of send is described by Alex Martelli here (in the first answer to the question).