Webdriver StaleElementReferenceException

January 26, 2012

Our end-to-end tests were quite flaky for a while because of a StaleElementReferenceException (*). We have a lot of tests and we run them very often (we have about 180 scenarios and run the tests about 100 times a day). We encountered the “Element is no longer attached to the DOM” exceptions in 5 different cases:

1 - When we didn’t wait (or waited for the wrong thing)
2 - Waiting fails
3 - @FindBy injected elements
4 - findElement on WebElements
5 - Bad luck
I’ll go into details for each case and show how we fixed it.

1 - When we didn’t wait (or for the wrong thing)

This is by far the most common case. Here is an example:
(When you click the “make some attendees optional” in Google Calendar, an icon appears next to each participant.)
  public void clickMakeSomeAttendeesOptional() {
    makeSomeAttendeesOptionalLink.click();
  }
  public void makeOptional(String name) {
    driver.findElement(String.format(REQUIRED_ATTENDEE_ICON, name))
.click(); // StaleElementReferenceException here
  }
Fix: Even though clicking on “make some attendees optional” is very fast in replacing the attendees list and clicking on the required ATTENDEE_ICON works most of the time (and always when you’re trying it out or debugging), that’s not good enough.
The fix is easy, on the action prior to the action that throws the exception, you’ll have to wait.
  public void clickMakeSomeAttendeesOptional() {
    makeSomeAttendeesOptionalLink.click();
    waitUntilVisible(optionalAttendeesLegend);
  }

2 - Waiting fails

This one is a bit more annoying. Every once in a while, the exception gets thrown while waiting for an element to be visible. Example:
WebDriverWait wait = new WebDriverWait(driver, timeout);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath(String.format(ATTENDEE, names.get(0)))); // StaleElementReferenceException here
Fix: The reason this happens is that WebDriverWait treats NoSuchElementException different than StaleElementReferenceException. So what we do is wrap the ExpectedCondition with one that throws NoSuchElementException.
// In concrete Page object
  public void waitUntilMonthViewIsDisplayed() {
    waitUntilVisible(By.id("mvEventContainer"));
  }
// In abstract parent
  protected RenderedWebElement waitUntilVisible(By locator) {
    WebDriverWait wait = new WebDriverWait(driver, 20);
    return (RenderedWebElement) wait.until(refreshed(ExpectedConditions.visibilityOfElementLocated(locator)));
  }
  private <T> ExpectedCondition<T> refreshed(final Function<WebDriver, T> originalFunction) {
    return new ExpectedCondition<T>() {
      @Override
      public T apply(WebDriver webdriver) {
        try {
          return originalFunction.apply(webdriver);
        } catch (StaleElementReferenceException sere) {
          throw new NoSuchElementException("Element stale.", sere);
        }
      }
    };
  }

3 - @FindBy injected elements

Now it gets really tricky. In some (rare) occasions the element we get by @FindBy is unusable (i.e. if you use it, it’s stale). This happens even though the page object was only created one line before.
@FindBy(id = "tgCol0")
private RenderedWebElement gridFirstColumn;
  public EventBubble clickGridFirstColumn() {
    gridFirstColumn.click(); // StaleElementReferenceException here
    return newEventBubble();
  }
Fix:
We solve that by wrapping web elements with our own project specific element which knows how to find itself again. We already used our own ElementLocatorFactory for custom @FindBys:
protected Page(WebDriver driver) {
OurElementLocatorFactory factory = new OurElementLocatorFactory(driver);
PageFactory.initElements(factory, this);
}
In OurElementLocator we return the special OurWebElement (or OurRenderedWebElement) that knows how to locate itself again in case of a StaleElementReferenceException:
The methods from WebElement are all overridden in OurWebElement using the same pattern:
  @Override
  public void click() {
    try {
      underlyingElement.click();
    } catch (StaleElementReferenceException sere) {
      againLocate();
      click();
    }
  }
  protected void againLocate() {
    underlyingElement = locator.locate();
  }

4 - findElement on WebElement

It can happen that you use an element you already found and you want to have one of its kids. This may work (i.e. the element is found), but when you want to invoke the method on it, it’s stale.
WebElement element = driver.findElement(...);
WebElement e2 = element.findElement(...);
e2.click();  // StaleElementReferenceException here
Fix: We override the method findElement/findElements method in OurWebElement. If the element is found, we wrap it into a OurWebElement that knows how to locate itself again (see above). If the element is stale, we locate it again and then try again.
  protected OurWebElement(WebElement underlyingElement, Locator locator) {
    this.underlyingElement = underlyingElement;
    this.locator = locator;
  }
  @Override
  public WebElement findElement(By by) {
    try {
      return wrap(underlyingElement.findElement(by),
                  new FindElementLocator(this, by));
    } catch (StaleElementReferenceException sere) {
      againLocate();
     return findElement(by);
    }
  }
  protected void againLocate() {
    underlyingElement = locator.locate();
  }

5 - Bad luck

driver.findElement(By.xpath("//div[text()='" + text + "']")).click(); // StaleElementReferenceException here
Yes, this can happen. I know it looks like this should always work but give it enough chances (in javascript-heavy application) and it will fail.
Fix: The page objects aren’t supposed to use WebDriver directly anymore - even though they are passed an instance to WebDriver in the constructor, they are not supposed to keep a reference to it. Instead the parent class provides the functionality:
// In Concrete PageObject class
  public void toggleCalendar(String name) {
    findElement( "//div[@class='t23']//div[text()='" + name + "']").click();
  }
// In Abstract parent class
  protected WebElement findElement(By by) {
    return OurWebElement.wrap(driver.findElement(by), new OurWebElement.FindElementLocator(driver, by));
}

Conclusion


Most of the time the StaleElementReferenceException can be addressed by waiting for something at the right time and this should be your first line of thought. Currently it can be that this is not enough though. Cases 2 - 5 should really be handled by WebDriver IMO. Waiting should always work. Elements provided by @FindBy should always work (when I just created the page object). And the result of my tests should not depend on whether I’m lucky or not :-)

I hope these details can be of help to you when you run into a StaleElementReferenceException: Element is no longer attached to the DOM.
(*) This exception occurs, when you have a reference to an element and you want to call a method on it, but the underlying DOM has changed.

Posted by master, 0 Comments

Book review: The Mythical Man-Month (by Frederick P. Brooks)

January 8, 2011

The book is 35 years old and it shows in many places. What is fascinating in the book is not so much the solutions that they came up with at that time (obviously some technology hasn’t been around at that time), but that many of the problems they faced are still problems today.

For example in the chapter “Why did the tower of Babel fail?” he writes:

“So it is today. Schedule disaster, functional misfits, and system bugs all arise because the left hand doesn’t know what the right hand is doing. As work proceeds, the several teams slowly change the functions, sizes, and speed of their own programs, and they explicitly or implicitly change their assumptions about the inputs available and the uses to be made of outputs.”

His suggestion to using the telephone often, regular project meetings and a (physical) workbook (that needs to updated by each engineer) are outdated of course. However it is interesting to see, that he pointed his finger on a big problem that is still valid today. We’re still looking to solve that. In the meantime, a lot of things have addressed these concerns: Javadoc, Wikis, Design by contract, Unit tests etc.

So if you decide to read this book and expect to get solutions out of it, you will probably be disappointed. But you will be amazed, that many of the big problems from back then are actually still big problems today. Even more, they seem to be problems that still catch projects by surprise as they rediscover them. In that regard, the time spent reading this small book is time very well spent!

Posted by Jerome Mueller, 0 Comments

Go social!

March 30, 2010

Posted by master, 0 Comments

Nice Haircut!

January 22, 2010

Posted by Jerome Mueller, 0 Comments

Swiss Softball National Team Training

January 18, 2010

I used these kinds of pictures for the Trainingsday of the Swiss National Softball Team to explain some common mistakes. All the pictures here [pdf], in case somebody wants to have a closer look at them later on.

Andi rolling hands over

Andi rolling hands over

Posted by master, 0 Comments

Some Christmas Cards

December 24, 2009

Merry Christmas to everybody.

Posted by Jerome Mueller, 0 Comments

Sweet child of mine - violin

October 5, 2009

Sometimes, one word is enough: Wow!!!

Posted by Jerome Mueller, 2 Comments

Meteor come - BOOM

September 9, 2009

Meteor come -> Boom | Agile come -> Boom

Meteor come -> Boom | Agile come -> Boom

No more defects. No more test phase. No more testers.

Posted by master, 0 Comments

And why did you become extinct?

September 4, 2009

There is no Test Phase in Agile projects. I imagine this conversation will take place sometime in the not so distant future:

And why did you guys become extinct?

And why did you guys become extinct?

Posted by master, 0 Comments

First Swiss Testing Night - Impressions

June 12, 2009

Here are some impressions from the first Swiss Testing Night:

Screenshot Swiss Testing Night Website

Screenshot Swiss Testing Night Website

The Swiss Testing Night Programme. Three quarters of an hour to torment QA people with ideas about Testing by a developer.

Intro by Adrian Zwingli

Intro by Adrian Zwingli

I think this was the moment I was the most nervous: Being introduced by Adrian Zwingli (Chief Organizer). Everybody is ready, waiting for you. Then you realize: There’s no way back - only one way to go. That’s when most of the nervousness goes away - kind of like in baseball when you take the mound.

Setting up the stage

Setting up the stage

Setting things up. The remote control didn’t work, probably because the infrared receiver was blocked by the podium. And no headset style microphone - the room was set up for a speaker talking from behind the podium. No chance I was going to do that though. So I turned the podium sideways and picked the micro up - Rockstar style ;-)

Apologizing to our sponsor (HP)

Me apologizing to our sponsor (HP)

I think that’s the moment I realized that saying “one of the reason for the existence of bugs are bug tracking tools” might not sit too well with our sponsor HP - who sells Quality Center and Winrunner among other things.

Oh well - sometimes you step on somebodys toe - life goes on. That plus HP has a mighty big toe.

One of my favorite slides - reference to Starship Troopers

One of my favorite slides

This is one of my favorite slides (introducing the second part of the speech). A reference to Starship Troopers (how could you leave out a reference to the best movie ever, if your topic is “Crusade against Bugs”)?

Not enough chairs guaranteed at least a partial standing ovation

Not enough chairs guaranteed at least a partial standing ovation at the end

A misunderstanding between the organizers and the location provided for about 20 chairs - the other 80 people had to stand the whole time. Before the speech I thought about having people rotate after every part of the speech. That way most people would get at least 15 min of sitting time. I’m afraid I forgot. Not that it was a good idea to begin with.

All the photos here were taken by Stephan Wiesner (@Stephan: If you’re reading - thanks a lot for letting me use them on my site).

Posted by Jerome Mueller, 3 Comments