Introduction: In this article, i'll walk you through a Python script that automates a Google search using the Selenium WebDriver library. You'll see each step of the code, which is designed to open a web browser, navigate to Google's homepage, perform a search, and scroll through the search results.
Selenium is a powerful tool for automating web interactions and testing web applications. It allows you to control a web browser programmatically, making it ideal for tasks like web scraping, automated testing, and more.
Step 1: Import Required Libraries
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service as ChromeService
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
It is mandatory to import the libraries you will be using and you start out by doing this so your code doesn't throw errors.
Selenium and Time are imported for this program.
Step 2: Set Up Variables
# Set Chrome Driver path
chrome_driver_path = "C:/Users/ebube/Downloads/chromedriver.exe"
# Optional - Keep the browser open if the script crashes.
chrome_options = webdriver.ChromeOptions()
chrome_options.add_experimental_option("detach", True)
# Set service and driver variables
service = ChromeService(executable_path=chrome_driver_path)
driver = webdriver.Chrome(service=service, options=chrome_options)
Step 3: Navigating to Google's Homepage
# Navigate to the Google homepage
driver.get("https://www.google.com")
Step 3: Handling Google Sign-In Pop-Up (Optional)
# Switch to iFrame Pop-Up and Continue as guest
time.sleep(2)
WebDriverWait(driver, 3).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[name=callout]")))
WebDriverWait(driver, 3).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-label='Stay signed out']"))).click()
# Switch back to web content
driver.switch_to.default_content()
driver.implicitly_wait(2) # Adjust the wait time as needed
Step 4: Performing a Google Search
# Find the search input element
search_input = driver.find_element(by=By.NAME, value="q")
# Enter the search term "Travel" into the search input element
search_input.send_keys("Travel")
search_input.send_keys(Keys.ENTER)
Step 5: Scrolling Through Search Results
# Scroll search result page slowly
total_height = int(driver.execute_script("return document.body.scrollHeight"))
for i in range(30, total_height, 1):
driver.execute_script("window.scrollTo(0, {});".format(i))
Step 6: Closing the WebDriver Instance
# Wait and Close the WebDriver
time.sleep(3)
driver.quit()
With that, your automation is set. Here is a demo of what you've done so far.
NB: I had issues with achieving the exact scroll pattern but what I did was close to what I wanted.
View the Demo and Source Code on Github: https://github.com/ebube-tech/automation-projects.git
Conclusion: In this tutorial, we've demonstrated how to automate a Google search using Python and Selenium WebDriver. You can adapt this script for various web automation tasks, web scraping, or interactions with web pages. Customize wait times and parameters to suit your specific use case, and always respect website terms of service and policies when automating interactions with websites.