TestNG Framework with Java – A Complete Guide for QA Automation Engineers

 TestNG (Test Next Generation) is one of the most widely used testing frameworks in Java, especially in automation testing. It provides powerful features such as test grouping, assertions, data-driven testing, parallel execution, dependency injection, reporting, and integration with Selenium and Appium.

1. Introduction to TestNG

TestNG is a testing framework inspired by JUnit and NUnit but designed with additional features that make it ideal for large-scale automation:

✔ Flexible test configuration

✔ Parallel execution

✔ In-built reporting

✔ Groups and prioritization

✔ Data-driven testing

It is heavily used with Selenium WebDriver and Appium.

2. Why TestNG? Key Features

Feature	Description
Annotations	Clear structure (@Test, @BeforeSuite, etc.)
Parallel Testing	Run tests faster using multi-threading
Data Providers	Enables data-driven testing
Test Grouping	Smoke, regression, sanity, etc.
Dependency Tests	Control test execution flow
Reporting	Generates HTML reports automatically
Compatibility	Works smoothly with Maven, Gradle, Jenkins

3. Installing & Setting Up TestNG

✔ In Eclipse:

1. Go to Help → Eclipse Marketplace

2. Search for TestNG

3. Click Install

✔ In Maven Project (Recommended):

Add TestNG dependency to pom.xml:

<dependency>

    <groupId>org.testng</groupId>

    <artifactId>testng</artifactId>

    <version>7.9.0</version>

    <scope>test</scope>

</dependency>

4. Creating Your First TestNG Test Case

import org.testng.annotations.Test;

public class HelloTestNG {

    @Test

    public void firstTest() {

        System.out.println("Hello TestNG!");

    }

}

Run this test via:

Right-click → Run As → TestNG Test

Or using testng.xml

5. Understanding TestNG Annotations

Annotation	Purpose
@BeforeSuite	Runs once before entire suite
@AfterSuite	Runs once after suite
@BeforeClass	Runs before first method of class
@AfterClass	Runs after all methods of class
@BeforeMethod	Runs before every @Test method
@AfterMethod	Runs after every @Test method
@Test	Marks a test case

6. Assertions in TestNG

Common Assertions:

• Assert.assertTrue()
Assert.assertFalse()
Assert.assertEquals()
Assert.assertNotEquals()

• Assert.fail()

7. TestNG XML Suite – The Heart of Execution

A sample testng.xml:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="AutomationSuite">
    <test name="RegressionTests">
        <classes>
            <class name="tests.LoginTest"/>
            <class name="tests.ProductTest"/>
        </classes>
    </test>
</suite>

Benefits:
✔ Run test groups
✔ Add parallel settings
✔ Pass parameters
✔ Control test execution

8. Data-Driven Testing with @DataProvider

public Object[][] getData() {
return new Object[][] {
{"user1", "pass1"},
{"user2", "pass2"}
};
}
@Test(dataProvider = "loginData")
public void loginTest(String user, String pass) {
System.out.println("Login with: " + user + " / " + pass);
}
Supports reading from:
Excel
JSON
CSV
Database

 9.Test Groups (Smoke, Regression, Sanity)

public void smokeTest() {}
@Test(groups = "regression")
public void regressionTest() {}
Run only smoke tests:
<run>
<include name="smoke"/>
</run>
</groups>

10. Parallel Execution in TestNG

Example:
Options:
parallel="tests"
parallel="classes"
parallel="methods"
Benefits:
✔ Faster execution
✔ Useful for Selenium grid / cloud execution (BrowserStack, Selenium Grid)

11. Dependency Tests

public void launchApp() {
System.out.println("App launched");
}
@Test(dependsOnMethods = "launchApp")
public void login() {
System.out.println("Login successful");
}
Useful when the order is required.

12. Passing Parameters from testng.xml

@Test
public void runOnBrowser(String browserName) {
System.out.println("Running on browser: " + browserName);
}

13. Reporting in TestNG

TestNG generates:
For advanced reporting:
Extent Reports
Allure Reports
Example (Extent Reports integration):
ExtentTest test = extent.createTest("MyTest");
test.pass("Step passed!");
extent.flush();

14. Integrating TestNG with Maven

To run TestNG tests via Maven:
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.2.5</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
Run:

15. Best Practices for a Professional TestNG Framework

✔ Use Page Object Model (POM)
Keep your test logic separate from UI elements.
✔ Use DataProviders for test data
Avoid hardcoding data inside tests.
✔ Use groups (smoke, regression, sanity)
Allows selective execution.
✔ Use dependency injection only when required
Don’t overuse dependsOnMethods.
✔ Add Retry Logic for flaky tests
Use IRetryAnalyzer.
✔ Maintain a clear folder structure:
/tests
/pages
/utils
testng.xml
pom.xml
✔ Integrate with CI/CD (Jenkins, GitHub Actions)
Run tests automatically on commit.

Advanced TestNG Framework Structure

src

 └── test

     ├── java

     │    ├── tests        # Test classes

     │    ├── pages        # Page Object classes

     │    └── utils        # Utilities (AppiumUtils, DriverFactory, etc.)

     └── resources

          ├── testng.xml   # Suite definition

          ├── browserstack.yml

          └── data.properties

Base Test Class

Handles driver setup for both local and BrowserStack:

java

public class AndroidBaseTest extends AppiumUtils {
    public AndroidDriver driver;
    public FormPage formPage;

    @BeforeClass(alwaysRun = true)
    @Parameters({"executionEnv"})
    public void setUp(@Optional("local") String executionEnv) throws Exception {
        if (executionEnv.equalsIgnoreCase("local")) {
            UiAutomator2Options options = new UiAutomator2Options();
            options.setDeviceName("emulator-5554");
            options.setApp(System.getProperty("user.dir") + "/src/test/resources/General-Store.apk");
            driver = new AndroidDriver(new URL("http://127.0.0.1:4723"), options);
        } else if (executionEnv.equalsIgnoreCase("browserstack")) {
            UiAutomator2Options options = new UiAutomator2Options();
            // BrowserStack SDK will inject capabilities from browserstack.yml
            driver = new AndroidDriver(new URL("https://hub.browserstack.com/wd/hub"), options);
        }
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10));
        formPage = new FormPage(driver);
    }

    @AfterClass(alwaysRun = true)
    public void tearDown() {
        if (driver != null) driver.quit();
    }
}

3. TestNG XML Suite

Define multiple suites (smoke, regression, parallel):

xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="AutomationSuite" parallel="classes" thread-count="3">
    <test name="RegressionTests">
        <parameter name="executionEnv" value="browserstack"/>
        <classes>
            <class name="tests.LoginTest"/>
            <class name="tests.ProductTest"/>
        </classes>
    </test>
</suite>

4. Data-Driven Testing

Use @DataProvider for flexible test data:

java

@DataProvider(name = "loginData")
public Object[][] getLoginData() {
    return new Object[][] {
        {"user1", "pass1"},
        {"user2", "pass2"}
    };
}

@Test(dataProvider = "loginData", groups = "smoke")
public void loginTest(String user, String pass) {
    formPage.login(user, pass);
    Assert.assertTrue(formPage.isLoginSuccessful());
}

5. Parallel Execution

Enable parallel runs in testng.xml:

xml

<suite name="Suite" parallel="methods" thread-count="5">

This is especially useful when running on BrowserStack cloud devices.

6. Reporting

Integrate Extent Reports for rich HTML reports:

java

ExtentReports extent = new ExtentReports();
ExtentTest test = extent.createTest("Login Test");
test.pass("Login successful!");
extent.flush();

Reports are generated in test-output/index.html.

7. Maven Integration

Run suites via Maven profiles:

xml

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>3.2.5</version>
  <configuration>
    <suiteXmlFiles>
      <suiteXmlFile>testng.xml</suiteXmlFile>
    </suiteXmlFiles>
  </configuration>
</plugin>

Run:

bash

mvn test -PRegression

⭐ Best Practices for Scaling

• Page Object Model (POM): Keep locators & actions separate from test logic.

• Groups: Use @Test(groups="regression") for selective runs.

• Retry Logic: Implement IRetryAnalyzer for flaky tests.

• CI/CD: Integrate with Jenkins/GitHub Actions for automated runs.

• Cloud Execution: Use BrowserStack/Selenium Grid for parallel device/browser runs.

✅ With this setup, you’ll have a robust TestNG framework that runs locally and on BrowserStack, supports data-driven tests, parallel execution, and generates professional reports.