I am building a video blog page using Joomla 3.0 + k2 hosted on my own Ubuntu 14.04 server, the videos will be uploaded to sites like youtube or vimeo too. I want to show the viewed times of the videos on various sites on my own page. I've thought of a way to do this, but I need some suggestions on some details(I haven't completed my studying of how to make extensions, so basically I will have to use some more brute forces to do this):
First of all, I am going to use casperJs to scrape the video pages and get the item view times. So far I have managed to print the right number on Ubuntu's command line. And because one of the video sites is in China, with low speed connection, it usually takes 20 sec or more to scrape the page and get the item viewed number, so I will have to create a cron job to run the casperjs script every few hours instead of running the script when user requests my page.
Next, I have two options: I can either store the data in a .json file, or store them in Joomla's database. I think using .json is much easier, but if I want to use the database, is it doable? If I write some script to directly create/update/read tables of the joomla database, will it do hard to the system? And I haven't figured out how to store the data I get from casperjs to the database, if you have information about it, please share some light.
There is going to be a lot of work, so before I dive in, I'd like to ask for some suggestions. Overall, am I in the right track? Is there any steps that deserver refine?