Raspberry Pi Web Scraping and Push Notifications

Had the Raspberry Pi for a while now and haven’t really done much with it so I decided to build something semi-useful. Basically we’ll be scraping the web using:-
Python (programming language)
BeautifulSoup (Python library to assist with screen scraping).
Pushbullet (allow for notifications to Android phone – both SMS and push notifications)
Firstly, lets get the web page we’re interested in the live departure table.


Live Departure Boards - National Rail Enquiries - Google Chrome

Using BeautifulSoup we can get a handle on the td elements for “Due, Destination, Status, Platform and Details”and as there’s 5 on a row (but we don’t know how many rows will be present) we can divide the tds returned by 5 to get us reach of the rows elements. So if there’s 6 rows we’ll have 30 elements.

play.py - CDevPythonpython_projplay.py (3.5.1)_7

We only care about the 3 element in each row (the “Status”) and if it’s not on time.  Again each row comprises of 5 elements so we can (crudely?) use the use a condition on the modulus operator and check that if it’s text isn’t “On time”.  If it’s not then send a PushBullet notification to my phone with the train’s original time (back 2 elements and the status that’s not “On time” (i.e late again thanks Northern Rail!).  Notifications to the Android phone of late trains…..


All that remains now is to set a cron job on the Raspberry Pi so that the script kicks in at the times I want it (7am and 4.30).  Stick a condition on the code so that it checks the right journey depending on the time and away you go.

play.py - CDevPythonpython_projplay.py (3.5.1)_6

Leave a Reply

%d bloggers like this: