Simple Selenium

It might be quite tricky sometimes just to get selenium up and running. First, the commands in terminal to install and get the necessary…

Simple Selenium

It might be quite tricky sometimes just to get selenium up and running. First, the commands in terminal to install and get the necessary drivers.

brew install wget
wget https://github.com/mozilla/geckodriver/releases/download/v0.21.0/geckodriver-v0.21.0-linux64.tar.gz
tar xvfz geckodriver-v0.21.0-linux64.tar.gz
mv geckodriver ~/.local/bin

Here, I have used version 21.0 of geckodriver for Mozilla, and version 61.0.2 for Firefox. Sometimes, errors just occur because of mismatching versions, so a little trial and error may be required.

A good application of Selenium would be to use it to create a simple Instagram bot.

We first import the main library in selenium and import Firefox, which basically allows us to control the Firefox browser. We could also use Chrome, but that would also mean downloading a different geckodriver (and finding the right combination of Chrome and geckodriver that would work nice together).

from selenium.webdriver import Firefox

And some of the other utilities from the selenium library

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys

And just one more library ‘time’ to build in some delays.

import time

We first start a browser session with Firefox. You will see the Firefox browser pop up.

Head ON

Usually one does this headless i.e. without the browser appearing, but to allow you to visualise what is happening, let’s do one with the head on first.

fieryfox = Firefox()

Now, go where you want to go on the interwebs, and wait a while before entering the username and password.

fieryfox.get('https://www.instagram.com/accounts/login/')
print(fieryfox.title)
login_wait = WebDriverWait(fieryfox, 10)
elem = login_wait.until(EC.visibility_of_element_located((By.XPATH, ".//input[@name='username']")))
elem.send_keys("enter_your_username")
elem = login_wait.until(EC.visibility_of_element_located((By.XPATH, ".//input[@name='password']")))
elem.send_keys("enter_your_password")

Now click on the login button

fieryfox.find_element_by_xpath("//button[contains(.,'Log in')]").click()

Now you are on the main page after login. It’s simple to do a quick check.

print(fieryfox.title)

Look for the search bar and search for anything

search = WebDriverWait(fieryfox, 10).until(
	EC.visibility_of_element_located(
		(By.XPATH, "//input[@placeholder='Search']")
	)
)
search.clear()
search.send_keys('#singapore')
time.sleep(3)
search.send_keys(Keys.ENTER)
time.sleep(1)
search.send_keys(Keys.ENTER)

print(fieryfox.title)

We click on an image by looking for the elements with a class name ‘v1Nh3’, and click on the first item we find.

The image pops up, and we look for the button to like the image and click it.

image_links = fieryfox.find_elements_by_class_name('v1Nh3')
image_links[0].click()

like_element  = fieryfox.find_element_by_xpath("//button/span[@aria-label='Like']")
like_element.click()

Headless

Now we do the exact same thing, but headless. Almost exactly the same, but we just need to set the Options first.

from selenium.webdriver.firefox.options import Options

opts = Options()
opts.set_headless()
assert opts.headless
fieryfoxy = Firefox(options=opts)

#navigate to the page and log in.
fieryfoxy.get('https://www.instagram.com/accounts/login/')

The rest of the steps are almost identical.

The Jupyter notebook with the code is here