Roberthalf Scrap
Roberthalf Scrap
Project overview :
This project aims to scrape job listing information from a specific website, https://www.roberthalf.ca/en/jobs, using web automation with Selenium in Python. The project involves navigating through multiple pages of job listings, extracting relevant details from each job listing, and saving this data to a CSV file for further analysis.
What we did for this project :
1. Imports: The project starts with importing necessary libraries, including Selenium for web automation, ChromeDriverManager for managing the Chrome WebDriver, time for controlling timing, and CSV for handling data storage.
2. get_driver() Function: A function named get_driver() is defined to set up a Selenium WebDriver. This function configures the WebDriver with specific options and opens a given URL.
3. scrap() Function: The scrap() function is created to extract job details from the current web page. It uses XPath expressions to locate and retrieve the following information:
• Position
• Job Type
• Location
• Salary
• Job Description
• Job Requirements
• Qualifications
4. save_csv() Function: Another function, save_csv(), is defined to save the scraped data to a CSV file. It appends data to the CSV file, adding a header row if the file is empty or creating a new file if it doesn’t exist.
5. Main Script: The main part of the script utilizes the WebDriver to navigate through multiple pages of job listings and scrape data from each listing. The script uses a loop to iterate through the job listing pages. It collects job details using the scrap() function, then saves this data to a CSV file using the save_csv() function. The total count of scraped job listings is maintained for further analytics.
Project results :
The project successfully scrapes job listings from the specified website and saves the job details into a CSV file. It navigates through multiple pages of job listings, extracts various attributes, and ensures data is correctly appended to the CSV file.