Playwright and Selenium are the two big choices when choosing a browser user-interface test programming tool. Selenium has been around a long time. Playwright is the new kid in town.

I’ve used Selenium before. It is a pita but it mostly works. Unfortunately, even 99% “mostly works” is a problem when you are running hundreds of tests. That means something is always failing. So you wind up writing all your code with “wait-until-something-is true” and “try-this-multiple-times-until-it-succeeds” features. And then it still fails once in a while, so you just have to repeat the whole test and then it works. The bigger the project, the worse this problem becomes.

In short, we use Selenium because we have to, not because we like it.

I had the opportunity to start a new project so I tried Playwright. Still a learning curve. Still requires a lot of work. But they took all the “wait-until” stuff and moved it “behind the scenes”. It still has to be done, but you, the programmer, don’t have to handle it yourself. unless you want to. Much better.

After 2 weeks working with Playwright, I was still impressed. Progress was slow but steady.  The hardest part was that I am using Vaadin as the front-end development tool and they make serious use of the shadow DOM, so each new element type took trial and error to get working. This would have been the same amount of work in Playwright and Selenium.

I was also fighting against the “best-practices” of Playwright. I like to use “id” attributes whenever I am selecting something. And I really like to use XPath. Yes, XPath can be brittle, but don’t kid yourself: UI testing is always going to be brittle. Now, Playwright doesn’t support XPath inside the shadow-dom, and I was constantly running into that problem. Eventually, everything I was doing with XPath was easily handled using CSS. For example, select the element with type “vaadin-form-layout” and attributes class=”user=prefs” and id=”userPrefsId”. So I was writing “xpath-ish” and it was easily translated into CSS “vaadin-form-layout[user=”prefs”][id=userPrefsId”]” and so forth. Anyway, personal preferences and nothing to do with the subject of Playwright vs Selenium.

And then I read this guy and he scared me:

His arguments were reasonable but not sufficiently detailed to be definitively persuasive. To summarize at a very high level, the best arguments were:

1. Playwright did waiting wrong.
2. Selenium is the web-standard and google/facebook will make sure it is up to date. Playwright could get left behind.

Ok, both of these are serious accusations. And I don’t feel qualified to comment on their validity.

Emotionally, the author, Zhimin Zhan seemed a bit cranky. Certainly seems like an expert, but sometimes experts get cranky when their favorite technology gets left behind. Either possibility seemed plausible.

So I decided I would redo the last 2 weeks work in Playwright in Selenium. It only took a few hours.

As soon as I ran the same tests in Selenium, I remembered why it was always so frustrating:

The first problem was with the Save-Menu. On startup it is inactive (disabled=”true”) and my tests assert that. However, the Selenium method isEnabled() returned true. Google “selenium isenabled not working”. My God! How many years now and that is STILL broken?

We shouldn’t all have to write this crappy code:

public static boolean isEnabled(WebElement element) {
   boolean enabled1 = element.isEnabled(); //this can be wrong
   String disabled = element.getAttribute("disabled"); //this is reliable
   boolean disabled2 = disabled != null && Boolean.parseBoolean(disabled);
   if (enabled1 == disabled2) {
      System.err.println("discrepancy in isEnabled");
      enabled1 = !disabled2;
   }
   return enabled1;
}

The next problem was when I clicked on a VaadinSideNavItem. I got this error:

org.openqa.selenium.ElementClickInterceptedException: element click intercepted: Element <vaadin-side-nav-item path="domain/person" 
id="CometPersonInit-peopleNav" role="listitem" has-children="">...</vaadin-side-nav-item> is not clickable at point (127, 92).
Other element would receive the click: <html lang="en" theme="dark">...</html>

The ‘theme=”dark”‘ element is adjacent to the side-nav button. So something was wrong with the point location calculation.

Setting an implicit wait period did not work. An explicit wait period did not work either. One thing that really sucks about wait code is that it swallows the exception so you don’t see the actual problem. (You are ignoring the exception and not logging it.) So when it fails, all you know is that it did not work for X seconds; not why.

The third issue is that the Selenium isVisible() method checks the element values but does not actually check to see if the element has been scrolled into view. Playwright does it correctly (my interpretation of “isVisible” is literal). Playwright also automatically scrolls elements into view when you try and act upon them. Very nice.

So I had 3 immediate frustrations with Selenium that Playwright took care of.

I sat back and googled some more. I found a video I liked where the speaker asserted that there was no comparison between the two.

At this point I was inclined to agree with him. And I’d invested a day to verify I was making the best decision. Out with Selenium. In with Playwright.

Now I am not saying Playwright is without difficulties. The codegen tool is a miss more than a hit when I use it to try and auto-code the Locators. And I believe I have found a bug where it just fails to work correctly with chromium. That was a huge frustration that cost a week of stoppage hell until I ran out of ideas and tried firefox instead of chromium. Firefox worked, and I was able to start moving again.