Monitor Website Changes with PowerShell

Long ago, I began to use a service, ChangeDetection.com to monitor the USPSA classifier page for changes of published classifiers (in fact I still do despite not having shot for several years). I run across a reason earlier this week for the need to monitor several pages since Trend Micro doesn’t have a notification service for SMB security product updates. Going to the ChangeDetection website, it now redirects to Visualping. I’m sure that Visualping is a good service but if you want to monitor more than 2 pages once per day in a month’s time, you’re going to be out of luck without forking out some cash.

Hello, PowerShell!

After contemplating whether to do this in Bash, PHP, or something else, the ease of importing data through CSV and piping it into a foreach loop had me jumping straight to PowerShell. As the list is always evolving, a CSV seems the easiest non-database way to handle this. I doubt the inefficiencies of CSV will ever hinder the script to the point of needing something like SQLite although that’s always an easy option if you don’t mind importing extra modules.

What this script does:

  • Defines file path for CSV input file and output files for comparison
  • Downloads text content of the URL given, output is saved as HTML with a .new file extension using the ID presented in the CSV for future tracking
  • Comparison is done as to whether the page changed or not, if so, an email is sent to the user
  • New file is copied to the old one for future comparison.

Caveat:

  • The first time the script is run as well as the first time you run it after adding a new entry to the CSV, you’ll likely recieve an error because the .old file doesn’t exist. Simply run the script again and they’ll be generated and no further errors will occur.

Instructions:

  • Modify the script for email information and file paths
  • All lines needing changed are prefixed with # CHANGE
  • Setup a scheduled task
  • Eat popcorn

CSV:

  • Normal CSV file with header fields
    • ID
    • Title
    • URL
  • ID is used to track the page in the cache. The files downloaded when the script is ran is copied to a .old filename for future comparison.

Download check-url.ps1 (as a ZIP file)