fpl?It's a Python wrapper for the Fantasy Premier League, which is a fantasy football game - "a game in which participants assemble an imaginary team of real life footballers and score points based on those players' actual statistical performance or their perceived contribution on the field of play. " At the time of writing this post there are more than 6 million teams entered, which includes loads of people on Steem who are participating in Steem's very own league (see here).
Alongside working on FPL Plus I've also had some other ideas for FPL related projects. For example, one of my Reddit bots, which has been running for ~3 years now, is closed source because I'm so embarrassed by the code. While also having a few bugs, it is also dependent on other sources other than the FPL API, which isn't great, as sometimes they mess up and in turn it messes my bot up. Because of this I've wanted to rewrite it and add some new features, however before doing this I wanted to make fpl asynchronous. The biggest reason for this is that when using synchronous requests it takes a really long while to update, for example, all the players (I'll show a comparison further down) - for some of the features I want to add to the bot this is unacceptably slow. So I set out to convert everything (apart from the CLI) to use asynchronous requests using aiohttp!
aiohttp instead of requests for asynchronous HTTP requestsreturn_json to return dict instead of objectpytest-aiohttp for async supporthttps://github.com/amosbastian/fpl/pull/19
aiohttp and asyncioI asked a couple of people if they had experience writing asynchronous Python packages, but most people didn't. In the end I found a Python package called aiohttp, which seemed to be exactly what I needed. Before I was using the requests package, while also passing the responsibility of sending requests to the classes themselves, as shown in the image below:
An example of the FPL class in version 0.5.5
As you can see the _get_information() function is called and then used to assign values to the class' properties. I didn't think it was a good idea to have this, especially when I decided to add the option to return a dict instead of a class, since I was assigning the values to properties of a different name (e.g. first_name != player_first_name) which would cause inconsistency.
So, for the asynchronous version of fpl I decided to send a GET request from the functions in the FPL class and pass it to e.g. the User class instead.
An example of the FPL class in version 0.6.0
I also refactored some functions so that they use other functions of the FPL class, with asyncio, to speed up the requests significantly. For example, the get_players() function now doesn't send any requests itself, but creates a list of get_player() coroutines and uses asyncio.ensure_future() and asyncio.gather().
The updated functions get_player() and get_players()
To show the speedup I created two code snippets, one using the synchronous version and one using the asynchronous version, that both do the same thing. I timed both of them, and as you can see, the speedup is crazy!
| fpl 0.5.5 | fpl 0.6.0 |
|---|---|
| 1m35.663s | 0m1.313s |
Other than just making it everything asynchronous a lot of work has gone into refactoring everything to, in my opinion, make fpl more intuitive and accessible. I don't really want to go too deep into this, as the most important feature of this update is its asynchronisity, but I hope that people who have used fpl before this update will notice this!
I've also decided to remove the MongoDB support, because after thinking about it I felt it didn't really belong in the package and was too opinionated. Furthermore it was only really there to speed up the package itself by saving information to a database, but with the new asynchronous version it's already fast enough and this isn't needed anymore.
When I first started with converting everything from synchronous to asynchronous I thought I could just keep the unit tests I had already written. I was also not passing the session as a class parameter, which was working fine at first, but then I found out it was giving warning messages that had to do with unclosed client sessions. Once I started passing it as a parameter the unit tests broke and I didn't really know what to do. After many hours trying to fix this, I resorted to asking on StackOverflow.
Thankfully someone responded, but unfortunately it meant rewriting most of the unit tests to work with pytest-aiohttp (a plugin for pytest) instead of unittest, which I thought would be much more difficult than it ended up being. Instead of having a setUp() function in each test class I ended up using pytest's fixtures like so:
which allows you to access e.g. the fpl variable from your tests like so:
which works great! In the end I'm really happy I ended up switching to pytest as it works great, looks much more clean and seems easier to use. I would definitely recommend switching to it for anyone who still uses unittest!
You can simply install it with pip:
pip install fpl
From version 0.6.0 it requires Python 3.6+, so make sure you are using an up to date version of Python!
I will be updating the documentation and adding contributor guidelines soon, but if you play FPL (or don't) and know Python, then don't hesitate to contact me on Discord (Amos#4622)!