Implementation of Percy Visual Testing Tool for Android Apps

Percy Visual Testing • Dec 30, 2024

At Halodoc, we are constantly striving to innovate and improve the user experience of our mobile applications through robust quality assurance (QA) processes. However, we encountered significant limitations with traditional testing methods when it came to detecting visual discrepancies in our Android app. Despite relying on manual inspections and functional test automation using the Appium framework, these approaches often failed to catch subtle visual regressions that emerged after each new deployment.

In this blog, we'll explore how we integrated Percy into our testing workflow, and how it empowered us to effectively address our visual regression challenges.

Importance of Visual Testing

Visual testing is the process of verifying the visual aspects of a user interface (UI) to ensure it appears as intended, focusing on layout, design, colors, fonts, and overall aesthetics. This type of testing is crucial for catching visual discrepancies that may arise from code changes or responsive design issues. As applications evolve rapidly with frequent updates and feature releases, visual regressions such as UI elements shifting position, layout changes, or design inconsistencies can easily go unnoticed during functional testing, potentially impacting the user experience.

To address this, visual regression testing is essential before deploying new requirements to production, as design tools like Figma provide detailed UI designs for specific features, but changes in requirements can unintentionally affect other pages within the application. Percy is a visual testing tool designed to help teams automate this process by capturing screenshots of the application and comparing them to baseline images, which quickly identify visual changes or regressions and ensure that their applications look and behave as intended.

Getting Started with Integrating Percy into your Appium

Install prerequisites

Sign up for a Percy account at Percy Dashboard . If you already have an account, just log in to the Percy dashboard.
After logging into your Percy account, navigate to your dashboard and create a project.
After you’ve created the project, you’ll be see a PERCY_TOKEN, please refer below screenshot.

Set this token as an environment variable and Percy will use the PERCY_TOKEN to know which project to upload the screenshots to. You will need this PERCY_TOKEN in next steps.
export PERCY_TOKEN=your_generated_token
To verify whether Percy is installed in our system, execute the command. This command will print the PERCY_TOKEN to the terminal. If the variable is not set, you'll see nothing or an empty output.
echo $PERCY_TOKEN
Install Percy CLI by running the following command in local default location.
npm install --save-dev @percy/cli
Add Percy Dependency in the build.gradle.kt file.
testImplementation("io.percy:percy-appium-app:1.1.1")

Integration of Percy library into our test automation framework

At Halodoc, for Android app test automation, we use a Appium with Cucumber written on Java language, so the below integration steps.

Create a Percy object in BasePage.
Inject the appium driver while creating instance of percy.

Adding method for capturing screenshots in BasePage which can be access through all test targets where invoking percy.screenshot().

Create a Cucumber feature file that contains the scenarios for visual testing.

Cucumber is a popular tool for Behaviour Driven Development (BDD), which allows you to write tests in a natural language format, such as Gherkin syntax. Tags are used to categorise and organise your scenarios and features, allowing you to run specific subsets of tests based on certain criteria.

The tests are executed through step definitions that map to application actions or validations.
Here's an example feature file:

Now, write the step definitions for the Cucumber steps in a Java class.

By using the below method in Page Class, Percy will capture the screenshot with the name of “Health Store Home Screen" and the same screenshot will be appeared on Percy website.

Run Percy Visual Testing

Start your Percy Server by running the following command.
npx @percy/cli exec:start
Run the runner file with desired scenarios.

For the first run, Percy will create a new build, takes snapshots and uploads them to your project. After build being approved, this will be consider as baseline snapshot.

When after running the next build, Percy takes new snapshots. Then we can see the comparisons between the two runs on the new build and check the differences between the current snapshot and the baseline snapshot.
Verify Results on Percy Dashboard.

Navigate to the Percy Dashboard to view the captured screenshots. Percy will show visual differences (if any) between the current snapshot and the baseline snapshot.
If there is no difference between baseline and new screenshot, then it will show unchanged as below. Percy will mark as "No visual changes" because nothing has changed in between screenshots.

If visual differences are detected, Percy will highlight them has seen in the below screenshot. Left side is the baseline screenshot and right side is the new screenshot captured during the test. Areas where the images don't match will be highlighted (usually in a red color), showing exactly where the UI changed.

Percy will flag this difference and put it up for review, Reviewer can confirm this difference as a visual regression bug. If this change is intended, the build can be approved, ignoring this change.

Conclusion

Percy is a robust tool for automated visual testing, making it an excellent addition to any testing strategy, especially for teams focused on user interface consistency. By catching visual bugs early in the development process, Percy ensures visual consistency across the application and enhances the quality of deliverables. During our integration of new requirement changes, Percy helped us identify and improve the UI/UX inconsistencies, leading to a noticeably enhanced user experience.

However, Percy cannot replace functional testing tools like Appium, XCUITest, or others, as it does not validate the functionality or behavior of the application. To deliver the highest quality product, combining Percy (for visual testing) with functional testing tools (for behavior validation) is essential, ensuring both visual and functional excellence.

References

Join us

Scalability, reliability and maintainability are the three pillars that govern what we build at Halodoc Tech. We are actively looking for engineers at all levels and if solving hard problems with challenging requirements is your forte, please reach out to us with your resumé at careers.india@halodoc.com.

About Halodoc

Halodoc is the number 1 all around Healthcare application in Indonesia. Our mission is to simplify and bring quality healthcare across Indonesia, from Sabang to Merauke. We connect 20,000+ doctors with patients in need through our Tele-consultation service. We partner with 3500+ pharmacies in 100+ cities to bring medicine to your doorstep. We've also partnered with Indonesia's largest lab provider to provide lab home services, and to top it off we have recently launched a premium appointment service that partners with 500+ hospitals that allow patients to book a doctor appointment inside our application. We are extremely fortunate to be trusted by our investors, such as the Bill & Melinda Gates Foundation, Singtel, UOB Ventures, Allianz, GoJek, Astra, Temasek, and many more. We recently closed our Series D round and In total have raised around USD $100+ million for our mission. Our team works tirelessly to make sure that we create the best healthcare solution personalised for all of our patient's needs, and are continuously on a path to simplify healthcare for Indonesia.