I have a lot of people asking me how I get my data for game statistics I am posting reports on. Hopefully this post answers most of the questions.
Skip this paragraph if you know how google works. As you know google and other search engines use applications capped crawlers that aimlessly scout the internet and download everything they can find usefully and store it for the search engine application to process. The data is then indexed by indexing applications and stored into a specialized database. When you search using a keyword google goes into the database and gets what you might be looking for.
How I get data is similar. Instead of indexing and looking for data aimlessly my tool indexes very specific pages and knows exactly what its looking for making it very efficient. Efficiency saves my computer as well as web server resources.
Where do I get data from?
Anywhere there is valuable data. Currently only lolbase and riot ranking ladders are indexed. Unfortunately riotstats.com is still under heavy development and has bugs and issues as well as missing a lot of valuable statistics.
I would recommend using JaxUploader from riotstats.com just because its uploading data to BOTH lolbase and riotstats.
Got more questions? Let me know.
No comments:
Post a Comment