Python Iterators and Tricks By albro

Iterators have many uses in programming languages. Programmers frequently use iterators when using ready-made data structures. In simpler words, with iterators, we can traverse a list, array, or other specific data structure and check each of its elements in a loop.

What is an iterator?

Maybe understanding the concept of iterator is a bit difficult and confusing!

Let's see the concept and situation of using iterators with a very simple example in the Python programming language.

Suppose you have a list of some numbers and you want to traverse on this list. As you know, this is possible simply by using a for loop.

my_list = [1, 2, 3, 4]

for x in my_list:
    print(x) #or do something ...

But what happens behind the performance of the above loop?

In a very simple case, suppose we have a pointer variable to the first element of this list (in the Python list data structure). When we use the list in a for loop, in the first execution of the loop, the first element is checked and traversed.

In the second iteration of the loop, we must move one element forward, that is, our pointer must point to the second element (the element after the current element).

The said process is repeated for the number of elements in the list so that we don't have any more elements to replace with the previous element; In this case, our loop is completed. simply!

The concept of iterator in an object is as simple as it was said!!

In the picture below, the general and sequential process of iterators is drawn.

When we are creating an arbitrary data structure in Python, if we want our data structure to have the feature of being iterated, we must implement two methods related to iterators in our class.

__iter__ function

This function is called when an object of our data structure is placed in a location for navigation. (for example in the for loop)

The __iter__ method actually returns an object that has a method named __next__.

__next__ function

In the __next__ method, a process is performed that the navigation pointer of our elements moves one step forward, points to the next element and finally outputs the current element.

An example to understand the structure of iterators in Python

To better understand the structure of iterators, suppose we have a class with a variable named current and set its initial value to 1.

class Numbers:
    def __init__(self):
        self.current = 1

Now, if we create an example from our test class and try to use it in a simple for loop, we will encounter an error!

obj = Numbers()

for x in obj:
      print(x)


# Run Result:
# Traceback (most recent call last):
#   File "", line 1, in 
#     for x in obj:
# TypeError: 'Numbers' object is not iterable

As it is clear from the last line of the given error, the objects of the Numbers class cannot be iterated, or in other words, they're not iterable.

As mentioned before, we need two more helper functions to make the desired class iterable. Next, we will add two necessary methods to our class. That is, our class will be something like the code below.

class Numbers:
    def __init__(self):
        self.current = 1

    def __iter__(self):
        return self

    def __next__(self): #have infinit end!
        output = self.current
        self.current += 1
        return output

Now, if we run the same loop as before, we will see the number 1 printed in the output at the beginning, and the number 2 in the next line, and this process continues to infinity.

Usually, there is a certain limitation in the data structure we have; For example, there is a countable number of elements in it, or in our educational example (Numbers class), it can be considered that the iteration will advance to a certain ceiling.

In order to take the expected ceiling as input, we change the constructor of the class and we will take the two values low and high as input when creating the object.

Also, to control the non-overflow of the expected limit, a condition should be placed in the __next__ method to check that the expected limit has been reached. If we reach our ceiling, we can use the StopIteration statement to announce to the for loop that the iteration is over.

class Numbers:
    def __init__(self, low, high):
        self.current = low
        self.limit = high

    def __iter__(self):
        return self

    def __next__(self):
        if self.current > self.limit:
            raise StopIteration
        output = self.current
        self.current += 1
        return output

As a result of executing the following test code, we will have the numbers 1 to 10 in the output.

obj = Numbers(1, 10)

for x in obj:
    print(x)

Now if we use the same object obj in another loop, you will see that no result will be obtained!

The reason for this is that the variable related to current has reached its maximum once the iteration operation has been executed, and in other times the iteration, because it has reached the limit, terminates the loop at the beginning of the execution by declaring StopIteration.

To avoid this, we can return the variables used as pointers (iterations) to their initial value in the __iter__ method.

Here only the value of the current variable changes and our problem is exactly this variable.

The problem can be easily solved by keeping the low value when creating an object from the class and assigning it to the current variable when executing the __iter__ method.

class Numbers:
    def __init__(self, low, high):
        self.low = low
        self.limit = high

    def __iter__(self):
        self.current = self.low
        return self

    def __next__(self):
        if self.current > self.limit:
            raise StopIteration
        output = self.current
        self.current += 1
        return output

Sometimes, due to the complexity of the process of maintaining the current element and jumping to its next element in each iteration step, the iterable class is defined as a separate class and only the __iter__ method is used in the main class.

If __iter__ is called, an object of the auxiliary class is created and returned; It should be noted that the auxiliary class must have two functions __iter__ and __next__.