This post is from a low-reputation account and contains an unverified outbound link. Be cautious before clicking external links.
Seen - A lightweight crawling/spider framework for everyone
Words
87
Reading
1 min
Listen
Play
8y
Seen
A lightweight crawling/spider framework for everyone
Screenshots
Hunter's comment
"Seen is a lightweight web crawling framework for everyone."
This is used to crawl your webpages very fast.
No need of Robot.txt
Pros
Supports JavaScript
Crawls webpages Fast
Simple to use
Open Source
Note:Python 3.5+ is Needed to install
Syntax for Installation
pip install seen
Link
https://github.com/HuberTRoy/Seen
Contributors
Hunter: 
This is posted on Steemhunt - A place where you can dig products and earn STEEM.
View on Steemhunt.com