Parameterized Testing with Pytest and Selenium

Introduction

Automated testing is an integral aspect of modern software development as it saves time, improves software quality, faster development cycles, and increases confidence in a software release.

As the software needs to expand, so does the software testing need, and in most cases, numerous sets of inputs are required to ensure the functionality of the code written executes as it should.

In a case like this, testing every unit of the software in bits does not do the magic; therefore, there is a need for parameterized testing.

Parameterized testing is a software testing technique in which the same test case runs multiple input data sets. Each input data here is referred to as a parameterized test case. In this tutorial, we will perform parameterization testing using Pytest and Selenium.

Understanding Parameterized Testing

As defined earlier, parameterized testing is a software testing approach that enables testers to run multiple input data sets using the same test case. Rather than write separate test cases for each input scenario, you only need to create a single test case that accepts different input values, making your testing process more robust and scalable.

Parameterized testing is of extreme advantage in various real-world scenarios across several industries. For example, the e-commerce niche can test the website's checkout process by parameterizing the test to cover different payment methods(PayPal, bank transfer, credit card) and shipping options. It can also test a website’s login functionality by parameterizing the test to validate different login credentials ( valid and invalid usernames and passwords).

Parameterized testing is effective for the following reasons:

• Increased Test Coverage: It enables testers to cover a wide range of possible test scenarios with different input data, thus leading to comprehensive testing.

• Efficiency and Maintainability: It reduces code duplication and increases scalability, and in times of change, it requires changes in only one place, the test function’s data inputs, making maintenance effortless and less error-prone.

• Enhanced Debugging and Reporting: In cases of failure, it is usually easier to pinpoint the inputs that caused the failure, and more often, a clear and informative output indicating which input values caused test failures is provided.

Overview of Pytest and Selenium

Pytest is a popular framework for testing Python scripts. It simplifies writing small readable tests and can support complex functional testing for applications and libraries. Some common features of pytest that make it stand out for developers and testers include:

• Parameterized testing

• Powerful assertions

• Concise test syntax 

• Parallel test execution 

• Fixture parameterization

In this tutorial, pytest is used alongside selenium for effective parameterized testing.

Prerequisites

The following tools are required to follow up on this tutorial

• Download and install the latest version of Python.

• Install Selenium WebDriver using this command in the command line pip install selenium

• Download and install Pycharm or any other IDE of your choice.

• Install pytest using this command in your command line pip install -U pytest

Creating a Sample Test Scenario

In this tutorial, we will create a new account on thiscodeworks.com. This is a breakdown of this task:

• Enter the username field

• Enter the password field

• Fill in our email ID

• Click on Create Account to submit

Writing Basic Selenium Testcase Without Parametrization

In this section, we will perform the above task with just Selenium.

Importing the modules

The first step to carrying out this task successfully is to import the necessary modules and classes.

from selenium import webdriver

from selenium.webdriver.chrome.service import Service as ChromeService

from webdriver_manager.chrome import ChromeDriverManager

from selenium.webdriver.common.by import By

The code above does the following:

• from selenium import webdriver: This code line imports the webdriver module from the selenium library as it is the core functionality for automating web browsers.

• from selenium.webdriver.chrome.service import Service as ChromeService: This line imports Service class from the selenium.webdriver.chrome.service module and renames it as ChromeService. This Service class controls the behavior of the WebDriver service.

• from webdriver_manager.chrome import ChromeDriverManager: In this line, we import the ChromeDriverManager class from the webdriver_manager.chrome module. The ChromeDriverManager is a utility that helps manage and download your system's appropriate Chrome WebDriver executable.

• from selenium.webdriver.common.by import By: This line imports the By class from the selenium.webdriver.common.by module.

Initializing the Selenium webdriver instance

The next step is to initialize a Selenium WebDriver instance for Google Chrome, maximize the window for proper view, and instruct the webdriver to navigate to the appropriate URL. The lines of code below achieve these tasks.

driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))

driver.maximize_window()

driver.get("https://www.thiscodeworks.com/extension/signup")

Filling out the form fields

In this section, we will be focused on filling out the form fields.

To fill out the username field, right-click the box that reads ‘Enter username’. A list of options will pop up, scroll down and click ‘Inspect’. This will highlight the elements of that field. The image below shows you a screenshot of the expected result.

Next, we locate the element using the find_element and the By.ID module. Enter the username we want using the send.keys() class.

username = driver.find_element(By.ID, "username-login")

username.send_keys("fname")

Using this step, we will do the same for the password and email fields. We now have

password = driver.find_element(By.ID, "password-login")

password.send_keys("password1234")

email = driver.find_element(By.ID, "email-login")

email.send_keys("[email protected]")

Lastly, we will submit the form. To do this, right-click the 'Create an account' button and follow the steps for locating the element. Rather than using send.keys() class, here we use the .click() to click the button and submit the form.

We have successfully automated the filling out of this form. What do we do when we need to test different data on this form to ensure it works as it should?

We will likely not depend on these steps above as we must erase the previous data, enter the new data, and test. In such a situation, we can parameterize the test using pytest.

Parameterizing Tests with Pytest

In this tutorial, we will use the Pytest parameterization feature to enter different data sets into the user account creation form indicated in our test scenario.

To do this, we will import all the necessary selenium modules and classes we used in the last step, and we will also import the pytest package. We also want to ensure we assert that the success title appears on the next page. We will import a webdriver wait class to ensure the browser stays open until a certain condition is met.

import pytest
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Next, we use Pytest's @pytest.mark.parametrize decorator to define parameterized test cases. We also provide a set of test data for three parameters (username, password, and email) as indicated on the account creation form. Pytest will execute the associated test function multiple times, once for each data set, allowing you to run the same test with different input values to ensure its correctness across various scenarios.

@pytest.mark.parametrize("username, password, email",
                        [
                            ("Denjanny29", "monkey12345", "[email protected]"),
                            ("mothalice21", "alice2345", "[email protected]"),
                            ("mrkay", "alice2345", "[email protected]"),
                            ("madave34", "alice2345", "[email protected]"),
                            ("jenniferkays", "alice2345", "[email protected]"),
                            ("davelambo", "alice2345", "[email protected]")
                            # You can add more data sets here
                        ]
                        )

Next, we create a test function named test_submit_login_form. This function tests the submission of the login form and asserts if the expected success message is present in the title.

def test_submit_login_form(username, password, email):
   driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))
   driver.maximize_window()
   driver.get("https://www.thiscodeworks.com/extension/signup")


   # Fill out the form using the provided values
   username_field = driver.find_element(By.ID, "username-login")
   password_field = driver.find_element(By.ID, "password-login")
   email_field = driver.find_element(By.ID, "email-login")


   username_field.send_keys(username)  # Use the username argument
   password_field.send_keys(password)  # Use the password argument
   email_field.send_keys(email)        # Use the email argument


   #Submit the form
   submit = driver.find_element(By.CSS_SELECTOR, "button[type='submit']")
   submit.click()


   # Wait for success message to appear in the title
   WebDriverWait(driver, 10).until(EC.title_contains("Success"))


   #Check for success message in title
   title = driver.title


   # Check if the expected success message is present in the title
   assert "Success" in title, "'Success' is not in the title"
   driver.quit()

PS: As a rule for pytest, any pytest file must start with test_ or end with _test. The pytest method names should start with test_ for example def test_login.

You can run the code from your IDE or with the command line. In this tutorial, we run this code using the Pychram IDE; however, you can also use the command line. To run code using the command line, enter the code below:

py.test -v -s

This code ensures that the code is run and all necessary information is displayed and any necessary output is printed out in the console.

From the image above, we can see that after we ran the code, 1 test failed and 5 tests passed. Pycharm also gives us more information about the code that failed.

Data Sources for Parameterization

Data used in parameterization can be obtained from several sources. You can source it from various places, such as CSV files, Excel spreadsheets, databases, or even directly from(like our examples above). Choose the method that best suits your project’s needs.

Loading Test Data from Other Sources (CSV files and Excel sheets)

This tutorial will examine loading test data from CSV files and Excel sheets.

CSV Files

To load test data from CSV files, follow the simple steps below.

• Import the csv module.

CSV is a module that comes with the standard Python package, so you do not need to install anything rather, import the module directly into your script using the code below.

import csv.

• Create a CSV file.

Create a file that contains your test data. Each row should represent a set of data for a test case, and each column should represent a different parameter. In the tutorial, we will be using the details below. The title of the CSV file below is test_details

Ensure your CSV file is in the same folder as your test script.

• Load Data from CSV

You can use the csv module to load data from the CSV file in your Python script:

Import csv
# Define a function to read data from the CSV file
def read_test_data_from_csv(file_path):
   test_data = []
   with open(file_path, newline='') as csvfile:
       reader = csv.DictReader(csvfile)
       for row in reader:
           test_data.append(row)
   return test_data


# Path to your CSV file (change this to your CSV file's path)
csv_file_path = 'test_details.csv'

# Use pytest's parametrize decorator to run the test with different data from the CSV
@pytest.mark.parametrize("test_data", read_test_data_from_csv(csv_file_path))

The complete code for this section is below and can also be found in this repo

# Import necessary libraries (no changes needed here)
import pytest
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import csv

# Define a function to read data from the CSV file
def read_test_data_from_csv(file_path):
   test_data = []
   with open(file_path, newline='') as csvfile:
       reader = csv.DictReader(csvfile)
       for row in reader:
           test_data.append(row)
   return test_data

# Path to your CSV file (change this to your CSV file's path)
csv_file_path = 'test_details.csv'

# Use pytest's parametrize decorator to run the test with different data from the CSV
@pytest.mark.parametrize("test_data", read_test_data_from_csv(csv_file_path))
def test_submit_login_form(test_data):
   # Set up the WebDriver (no changes needed here)
   driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))
   driver.maximize_window()
   driver.get("https://www.thiscodeworks.com/extension/signup")

   # Fill out the form using the data from the CSV file

   # Input your own information here:
   # Update the field names to match the columns in your CSV file

   username_field = driver.find_element(By.ID, "username-login")
   username_field.send_keys(test_data['username'])  # Use the correct column name

   password_field = driver.find_element(By.ID, "password-login")
   password_field.send_keys(test_data['password'])  # Use the correct column name

   email_field = driver.find_element(By.ID, "email-login")
   email_field.send_keys(test_data['email'])  # Use the correct column name

   # Submit the form (no changes needed here)
   submit = driver.find_element(By.CSS_SELECTOR, "button[type='submit']")
   submit.click()

We can see the result when the script above is run.

Excel files

Using Excel files is another way to load test data. To do this, follow the steps below:

• Install the openpyxl library.

To install this library, enter pip install openpyxl into your command line.

• Create an Excel file.

Create an Excel spreadsheet that contains your test data. Each sheet should represent a different data set, and each column should represent a different parameter. Ensure your Excel file is saved in the same folder as your test file. We will use the test data in the image below in this tutorial.

• Load the data from the Excel sheet.

import openpyxl

# Function to read data from Excel file

def read_test_data_from_excel(excel_file):

   workbook = openpyxl.load_workbook(excel_file)

   sheet = workbook.active

   test_data = []

   for row in sheet.iter_rows(min_row=2, values_only=True):

       username, password, email = row

       test_data.append((username, password, email))

   return test_data

# Get the test data from the Excel file

test_data = read_test_data_from_excel('test_new.xlsx')

@pytest.mark.parametrize("username, password, email", test_data)

The complete code for the section is below and can also be found in this repo. It contains scripts for all necessary imports, loading the data from the Excel sheet, and scripts to enter user details.

import pytest
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import openpyxl

# Function to read data from Excel file
def read_test_data_from_excel(excel_file):
   workbook = openpyxl.load_workbook(excel_file)
   sheet = workbook.active
   test_data = []

   for row in sheet.iter_rows(min_row=2, values_only=True):
       username, password, email = row
       test_data.append((username, password, email))

   return test_data

# Get the test data from the Excel file
test_data = read_test_data_from_excel('test_new.xlsx')

@pytest.mark.parametrize("username, password, email", test_data)
def test_submit_login_form(username, password, email):
   driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))
   driver.maximize_window()
   driver.get("https://www.thiscodeworks.com/extension/signup")

   # Fill out the form using the provided values
   username_field = driver.find_element(By.ID, "username-login")
   password_field = driver.find_element(By.ID, "password-login")
   email_field = driver.find_element(By.ID, "email-login")

   username_field.send_keys(username)  # Use the username argument
   password_field.send_keys(password)  # Use the password argument
   email_field.send_keys(email)        # Use the email argument

   # Submit the form
   submit = driver.find_element(By.CSS_SELECTOR, "button[type='submit']")
   submit.click()

   # Wait for success message to appear in the title
   WebDriverWait(driver, 10).until(EC.title_contains("Success"))

   # Check for success message in title
   title = driver.title

   # Check if the expected success message is present in the title
   assert "Success" in title, "'Success' is not in the title"
   driver.quit()

When the code is run, we have the test output below:

Advanced Parametrization

Using fixtures with parameterization

Using fixtures in combination with @pytest.mark.parameterize is an advanced parameterization method in Pytest that allows you to parameterize your test functions with a more complex setup and teardown logic. In this section, we will examine the code below to understand the use of fixtures with parameterization further.

import pytest

# Define a fixture to provide the sets of two numbers and their expected results
@pytest.fixture(params=[(3, 2, 1), (5, 7, -2), (10, 5, 5), (0, 0, 0), (-1, -3, 2)])
def number_sets(request):
  return request.param

# Use the @pytest.mark.parametrize decorator to run the test with multiple parameterized values
@pytest.mark.parametrize("number_sets", [(3, 2, 1), (5, 7, -2), (10, 5, 5), (0, 0, 0), (-1, -3, 2) ])
def test_subtraction_with_parametrize(number_sets):
  num1, num2, expected_result = number_sets
  result = num1 - num2
  assert result == expected_result

In the code above:

1. We create a number_sets fixture using the @pytest.fixture decorator. This fixture takes a list of tuples where each tuple contains two numbers and their expected result after subtraction.

2. We use the @pytest.mark.parameterize decorator to create a test function; test_subtraction_with_parameterizew also uses the number_sets fixture. We subtract the two numbers inside the test function and assert that the result equals the expected result. The decorator allows us to run the same test with multiple parameterized values.

When this test is run, it will test for each set of numbers, checking if the subtraction result matches the expected output.

Conditional Parameterization

Conditional parameterization is a testing approach that selectively enables or disables specific test cases based on certain conditions or criteria. This allows you to specify the rules or conditions that control which tests are executed and which ones are skipped. To understand this advanced parameterization technique, let us examine the code below.

import pytest
# Define a fixture to provide the sets of two numbers and their expected results
@pytest.fixture(params=[(3, 2, 1), (5, 7, -2), (10, 5, 5), (0, 0, 0), (-1, -3, 2)])
def number_sets(request):
   return request.param

# Define the test function with conditional parameterization
def test_subtraction_with_condition(number_sets):
   num1, num2, expected_result = number_sets

   # Skip the test if any of the numbers are not positive
   if num1 <= 0 or num2 <= 0:
       pytest.skip("Skipping test for non-positive numbers.")

   result = num1 - num2
   assert result == expected_result

# Use the @pytest.mark.parametrize decorator to run the test with multiple parameterized values
@pytest.mark.parametrize("number_sets", [(3, 2, 1), (5, 7, -2), (10, 5, 5), (0, 0, 0), (-1, -3, 2)])
def test_subtraction_with_parametrize(number_sets):
   num1, num2, expected_result = number_sets
   result = num1 - num2
   assert result == expected_result 

In the code above, the test_subtraction_with_condition test function introduces conditional parameterization. It checks whether either of the numbers in a test case is not positive and skips the test for such cases using pytest.skip. When the code is run, we have the output below:

Best Practices for Organizing and Structuring Parameterized Tests with Pytest

To effectively organize and structure parameterized tests while avoiding pitfalls, the under-listed best practices should be followed:

1. Separate test logic from test data

Keep test data separate from the test logic. You can do this by defining the test data in a fixture or an external data source such as an Excel sheet or CSV file.

2. Use fixtures for setup and teardown

Leveraging Pytest fixtures to set up the test environment and perform cleanup after the test can help maintain a clean and organized test structure.

3. Keep tests independent

Each test function should be independent of other tests. Avoid relying on previous tests' states, and ensure tests can be executed in any order.

4. Provide informative test names.

Use descriptive and informative test names to make it easier to identify which test cases fail and why. You can include parameterized data in the test name to make failures more meaningful.

5. Avoid overusing parameterization

Excessive use of parametrization can make test cases hard to maintain and slow test execution. As such, it should be avoided.

Conclusion

Parameterized testing, which allows us to use the same test case to run multiple sets of data, is one very integral function of pytest and it also creates ease and scalability when carrying out automated testing. 

In this tutorial, we have stated the importance of parameterized testing. We discussed and illustrated how to carry out parameterized testing with selenium and pytest, load data from different sources for your test, and the various advanced parameterization techniques.

Parameterized testing is a software testing approach that enables testers to run multiple input data sets using the same test case. Rather than write separate test cases for each input scenario, you only need to create a single test case that accepts different input values, making your testing process more robust and scalable.