2

In the middle of the course, I am working on/ thinking about the final project. Part of it is getting and storing short news messages from a news site. Since there is no RSS-feed, I've made a php-program that screen scrapes the messages and the headlines, and puts them in a MySQL-tabel. This works fine if I request the php-file in the browser.

The code puts the list of headlines in an associative array (called $index). After at most an hour, I'd like to check if there are any new messages to scrape. I created a loop (while (true)) using sleep(), to compare the new $index-array with the former $index. Only new messages will be inserted in MySQL.

But this means that the php-file needs to execute forever.

Is there a different (smarter) way to make this php-file run say once every 15 minutes. It should run independently of users visiting the site and requesting this php-file.

Thanks for any replies/advices.

Hans

1
  • You can try it out with a cron job. search it on the Cpanel. if you are using shared hosting, you can schedule a cron job once in 15 minutes only. if you are using VPS or Dedi you can schedule it once a minute. May 26 at 11:35
1

You need something like Cron, which is a time based job scheduler.

6
  • Thanks, what i see online about Cron is promising. And I assume I'll make the php-script check if a headline is 'new news' by performing a MySQL-search (for efficiency in the most recent 30 rows)? Because the $oldindex - $newindex comparison won't work without looping. Jul 25 '15 at 9:01
  • RSS would have been the easiest way to go as screen scraping can be difficult, especially when the website is changed and your code can't find the keywords anymore. Also in this case you'd have to watch out for old news being rehashed with different phrasing. But if you're ok with that, then your idea should work, loading the headlines into an associative array and using something like array_diff_assoc() to check for differences between new $index and old $index.
    – ronga
    Jul 25 '15 at 13:27
  • Thanks Ronga. for me there is one difficulty left: In between the cronjobs the $oldindex will have disappeared from memory, because the php-program finishes after every run. So that means I need to store the old index in a different way, I think Jul 25 '15 at 18:12
  • Are you not storing it on the jharvard server? You're using phpmyadmin when you run your mysql query, right? Alternatively, you could declare $index as a static variable so it will retain its value even after the function exits.
    – ronga
    Jul 26 '15 at 6:56
  • Yes, jharvard, but I'm also trying a cheap server to make the project last after the course. Phpmyadmin and Mysql are working fine. The thing is: in my first php-script I didn't need a Mysql-query to check for 'new' news, the oldindex would stay intact and update on every run during the everlasting while-loop. Now I learn about Cron (so no loop but a single run every 15 minutes). I think now I need to add a Mysql-query to see if headlines are already in the table or need to be inserted. My point of doubt was: What happens to values of php-variables (oldindex) after the script finishes in Cron? Jul 26 '15 at 12:20
0

ZOMBIE QUESTION.

The correct answer is to use cron as suggested in comments.

If your server supports ssh access and cron, you can edit your cron table with crontab -e and list it with crontab -l.

Setting up PHP scripts to run in cron has a few extra steps. It's not enough just to include a "shebang line" like #!/usr/bin/env php at the top of the file and set chmod +x, because cron doesn't execute programs in the user's shell environment.

To run a script every 15 minutes, you'll need a crontab entry something like this:

*/15 * * * * /path/to/php/interpreter /path/to/script.php

Replace the example paths above with the actual full pathnames to your PHP binary and your script. Full pathnames are necessary because of cron's oddball execution environment.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .