f00j00
Posts: 4
Joined: Sat Jul 22, 2017 12:05 am

Website changes monitor and notify

Sat Jul 22, 2017 12:12 am

Hello,
Im wondering if anyone was able to have a script running 24/7 on a rbp that will check periodically for any website changes and notify via email if found any..

Thanks for your help

User avatar
CarlRJ
Posts: 599
Joined: Thu Feb 20, 2014 4:00 am
Location: San Diego, California

Re: Website changes monitor and notify

Sat Jul 22, 2017 7:45 am

f00j00 wrote:Im wondering if anyone was able to have a script running 24/7 on a rbp that will check periodically for any website changes and notify via email if found any.
You say, "any website changes" - do you mean any website on the entire internet, or any change to any page on a particular website, or any change to a particular page on a particular website? The latter is certainly doable. I have a number of Raspberry Pi's running 24/7, with scripts that periodically either send data to my websites, or collect data from other websites.

I use Python3 with the requests library, and the html.parser library included with Python. I don't know of a tool specifically designed to do such things, but one (or more) may exist.

If your question is, "is it possible/reasonable to monitor specific pages on one or more websites, using an always-on Pi", the answer is yes, lots of people are doing this kind of thing. If you're looking for a tool that does it all automatically, I don't know of one; someone else here might.

f00j00
Posts: 4
Joined: Sat Jul 22, 2017 12:05 am

Re: Website changes monitor and notify

Sat Jul 22, 2017 9:50 am

Monitor any changes for given urls and notify if true..

CarlRJ wrote:
f00j00 wrote:Im wondering if anyone was able to have a script running 24/7 on a rbp that will check periodically for any website changes and notify via email if found any.
You say, "any website changes" - do you mean any website on the entire internet, or any change to any page on a particular website, or any change to a particular page on a particular website? The latter is certainly doable. I have a number of Raspberry Pi's running 24/7, with scripts that periodically either send data to my websites, or collect data from other websites.

I use Python3 with the requests library, and the html.parser library included with Python. I don't know of a tool specifically designed to do such things, but one (or more) may exist.

If your question is, "is it possible/reasonable to monitor specific pages on one or more websites, using an always-on Pi", the answer is yes, lots of people are doing this kind of thing. If you're looking for a tool that does it all automatically, I don't know of one; someone else here might.

redtyger
Posts: 1
Joined: Thu Jan 18, 2018 12:37 am

Re: Website changes monitor and notify

Thu Jan 18, 2018 12:44 am

> Im wondering if anyone was able to have a script running 24/7 on a rbp that will check periodically for any website changes and notify via email if found any..

There's an open source project that does what you're looking for: https://github.com/DocNow/diffengine.

Another option is to use https://versionista.com which is a SaaS product and offers a free plan.

emp00
Posts: 5
Joined: Sun Mar 18, 2018 11:29 am

Re: Website changes monitor and notify

Sun Mar 18, 2018 11:37 am

In the meantime is there some software available doing this "website change monitoring" job on a rbp machine? There are indeed some web-services such as "changedetection.com" or "visualping.io" but they all cost subscription money. How can I set up the same functionality on a rbp with free software? Thanks for helping out!

f00j00
Posts: 4
Joined: Sat Jul 22, 2017 12:05 am

Re: Website changes monitor and notify

Tue May 15, 2018 12:01 pm

I still didnt find a free solution/script to run on my rbp, any help please?

droleary
Posts: 174
Joined: Fri Feb 09, 2018 3:45 am
Location: Minneapolis, MN USA
Contact: Website Skype

Re: Website changes monitor and notify

Tue May 15, 2018 1:16 pm

So script it yourself. It's easy enough to wget or curl a bunch of URLs, put them in a timestamped archive, and diff them or compare their checksum (cksum/md5sum/shasum/etc.). Drop it in your crontab and you're done.

If you need more help than that, post to one of the programming subtopics. Or maybe "Off topic discussion", because very little of this has to do with the RPi itself.

raspberryboy
Posts: 1
Joined: Thu Apr 04, 2019 3:50 am

Re: Website changes monitor and notify

Thu Apr 04, 2019 3:53 am

I built a python website change monitoring tool called sitestalker. You may find it useful. http://github.com/gitb0y/phishcracker/b ... _README.md

Return to “General discussion”