Introducing Apscheduler
Apscheduler is a job manager (similar to cron) to run sections of code on a schedule. I use it frequently to manage jobs that need to run at a set interval.
Apscheduler can be used to schedule code execution in three different ways:
- Cron style scheduling
- Interval system scheduling
- One-off execution
Job Persistence
If you want to use a database to store jobs, you can use any of these supported databases:
Apscheduler can be used to save jobs in a database to survive restarts and reboots. The job scheduler can persist as well as run jobs that should have run when it was offline.
- Any supported SQLAlchemy DB
- MongoDB
- Redis
- RethinkDB
- ZooKeeper
Installing Apscheduler
I recommend using pip or better yet pipenv
pip install apscheduler
How to use Apscheduler
I am not going to cover all use cases as Apscheduler is a very powerful module. In most cases, I use the BackgroundScheduler but there are seven different schedulers depending on your needs. For example, there is a blocking scheduler if you want to force jobs to be executed in a single thread. There are customized schedulers for Qt, Twisted, Tornado, and Async.
Setup
You have import the correct scheduler you want to use.
from apscheduler.schedulers.background import BackgroundScheduler
You also have to instantiate the scheduler.
scheduler = BackgroundScheduler()
This setup will store the job queue in memory and will not persist across restarts and reboots. You can use a database if you need this functionality.
One of the great things about Apscheduler is that it allows you to easily multi-thread your application without dealing with threads directly.
Adding a job
Once you have the boilerplate finished, you need to add a job. You have two choices here, use add_job() or decorate a function with scheduled_job()
I am going to use add_job() as it is the most straightforward to use.
scheduler.add_job(load_blacklist, 'interval', minutes=10, id='load_blacklist')
In some of my scripts that don't yet support the Global Blacklist API, I load a local copy of my blacklist. I do this using an interval based job schedule every 10 minutes.
Starting the scheduler
Once you have added your jobs, you can start the scheduler.
scheduler.start()
You can also stop and pause the scheduler. You can and list, modify, and remove jobs.
Advanced Usage
There is a lot of advanced functionality in Apscheduler that allows for a ton of customization.
Examples
max_instances- By default, Apscheduler supports 10 threads but there is a limit of one instance of any specific job. This allows you to run multiple instances of a specific job at once.misfire_grace_time- This is used to run jobs while the scheduler was unable to run a job because it was offline or busy.
Logging
Apscheduler supports the logging module and has its own logger named apscheduler.
Example logging boiler plate
import logging
logging.basicConfig()
logging.getLogger('apscheduler').setLevel(logging.INFO)
This should give you enough information to see the value of Apscheduler and how to start using it for your Python projects. More information can be found in the Online Documentation and you can find examples in the examples folder.
Apscheduler Github Repo
https://github.com/agronholm/apscheduler