Application to scrape game review data set for Kaggle.
Scraper employs fuzzy throttling and semi-shuffled URL request sequences.
p
MCGameScraper
public
1 stars
0 forks
0 issues
Commits
List of commits on branch main.Unverified
6e0849527df5f75dc8411d023e1ea0d8cd0c97f7feature(export): app.export.make_language_table uses language detection to assign ISO 693-1 codes to each review; app.export.makedb_simple_binary() produces sqlite3 db with a postive and a negative table for reviews with grade > 6 and grade < 4, respectively
pplan-do-break-fix committed 4 years ago
Unverified
db8b738b6fd79a3a0c08c93e133d03344ab0717breadme
pplan-do-break-fix committed 4 years ago
Unverified
b1f0fc9a77bedde705f4f2d17d619e617a20a499feature - fetch now handles all 5xx status codes the same as a 504
pplan-do-break-fix committed 4 years ago
Unverified
7060afa96c3f23c96d65b12e1e1c2db3eef2fb4afix - bad anonymous arguments in new_user_review() SQL statement
pplan-do-break-fix committed 4 years ago
Unverified
b5fd1988811770b01faf790d23121c409fddd3ddfix - improves handling of None type for main review elements
pplan-do-break-fix committed 4 years ago
Unverified
80ba19477208b2a7d3455e3e9d5510647bf20f86feature - scrape_critic_reviews now handles reviews with no associated date
pplan-do-break-fix committed 4 years ago