Webdriver StaleElementReferenceException
January 26, 2012
Our end-to-end tests were quite flaky for a while because of a StaleElementReferenceException (*). We have a lot of tests and we run them very often (we have about 180 scenarios and run the tests about 100 times a day). We encountered the “Element is no longer attached to the DOM” exceptions in 5 different cases:
1 - When we didn’t wait (or waited for the wrong thing)
2 - Waiting fails
3 - @FindBy injected elements
4 - findElement on WebElements
5 - Bad luck
I’ll go into details for each case and show how we fixed it.
1 - When we didn’t wait (or for the wrong thing)
This is by far the most common case. Here is an example:
(When you click the “make some attendees optional” in Google Calendar, an icon appears next to each participant.)
public void clickMakeSomeAttendeesOptional() {makeSomeAttendeesOptionalLink.click();}public void makeOptional(String name) {driver.findElement(String.format(REQUIRED_ATTENDEE_ICON, name)).click(); // StaleElementReferenceException here}
Fix: Even though clicking on “make some attendees optional” is very fast in replacing the attendees list and clicking on the required ATTENDEE_ICON works most of the time (and always when you’re trying it out or debugging), that’s not good enough.
The fix is easy, on the action prior to the action that throws the exception, you’ll have to wait.
public void clickMakeSomeAttendeesOptional() {makeSomeAttendeesOptionalLink.click();waitUntilVisible(optionalAttendeesLegend);}
2 - Waiting fails
[UPDATE: 21.11.2012 (thanks loc) - This isn't needed anymore, because ExpectedConditions handles it now]
This one is a bit more annoying. Every once in a while, the exception gets thrown while waiting for an element to be visible. Example:
WebDriverWait wait = new WebDriverWait(driver, timeout);wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath(String.format(ATTENDEE, names.get(0)))); // StaleElementReferenceException here
Fix: The reason this happens is that WebDriverWait treats NoSuchElementException different than StaleElementReferenceException. So what we do is wrap the ExpectedCondition with one that throws NoSuchElementException.
// In concrete Page objectpublic void waitUntilMonthViewIsDisplayed() {waitUntilVisible(By.id("mvEventContainer"));}// In abstract parentprotected RenderedWebElement waitUntilVisible(By locator) {WebDriverWait wait = new WebDriverWait(driver, 20);return (RenderedWebElement) wait.until(refreshed(ExpectedConditions.visibilityOfElementLocated(locator)));}private <T> ExpectedCondition<T> refreshed(final Function<WebDriver, T> originalFunction) {return new ExpectedCondition<T>() {@Overridepublic T apply(WebDriver webdriver) {try {return originalFunction.apply(webdriver);} catch (StaleElementReferenceException sere) {throw new NoSuchElementException("Element stale.", sere);}}};}
3 - @FindBy injected elements
Now it gets really tricky. In some (rare) occasions the element we get by @FindBy is unusable (i.e. if you use it, it’s stale). This happens even though the page object was only created one line before.
@FindBy(id = "tgCol0")private RenderedWebElement gridFirstColumn;
public EventBubble clickGridFirstColumn() {gridFirstColumn.click(); // StaleElementReferenceException herereturn newEventBubble();}
Fix:
We solve that by wrapping web elements with our own project specific element which knows how to find itself again. We already used our own ElementLocatorFactory for custom @FindBys:
protected Page(WebDriver driver) {
OurElementLocatorFactory factory = new OurElementLocatorFactory(driver);
PageFactory.initElements(factory, this);
}
In OurElementLocator we return the special OurWebElement (or OurRenderedWebElement) that knows how to locate itself again in case of a StaleElementReferenceException:
The methods from WebElement are all overridden in OurWebElement using the same pattern:
@Overridepublic void click() {try {underlyingElement.click();} catch (StaleElementReferenceException sere) {againLocate();click();}}
protected void againLocate() {underlyingElement = locator.locate();}
4 - findElement on WebElement
It can happen that you use an element you already found and you want to have one of its kids. This may work (i.e. the element is found), but when you want to invoke the method on it, it’s stale.
WebElement element = driver.findElement(...);WebElement e2 = element.findElement(...);e2.click(); // StaleElementReferenceException here
Fix: We override the method findElement/findElements method in OurWebElement. If the element is found, we wrap it into a OurWebElement that knows how to locate itself again (see above). If the element is stale, we locate it again and then try again.
class OurWebElement implements WebElement {public static WebElement wrap(WebElement element, Locator locator) { return new OurWebElement(element, locator); }protected OurWebElement(WebElement underlyingElement, Locator locator) {this.underlyingElement = underlyingElement;this.locator = locator;}
@Overridepublic WebElement findElement(By by) {try {return wrap(underlyingElement.findElement(by),new FindElementLocator(this, by));} catch (StaleElementReferenceException sere) {againLocate();return findElement(by);}@Override public void click() { try { delay(); underlyingElement.click(); } catch (StaleElementReferenceException sere) { againLocate(); click(); } }}
protected void againLocate() {underlyingElement = locator.locate();}
5 - Bad luck
driver.findElement(By.xpath("//div[text()='" + text + "']")).click(); // StaleElementReferenceException here
Yes, this can happen. I know it looks like this should always work but give it enough chances (in javascript-heavy application) and it will fail.
Fix: The page objects aren’t supposed to use WebDriver directly anymore - even though they are passed an instance to WebDriver in the constructor, they are not supposed to keep a reference to it. Instead the parent class provides the functionality:
// In Concrete PageObject classpublic void toggleCalendar(String name) {findElement( "//div[@class='t23']//div[text()='" + name + "']").click();}// In Abstract parent classprotected WebElement findElement(By by) {return OurWebElement.wrap(driver.findElement(by), new OurWebElement.FindElementLocator(driver, by));}
Conclusion
Most of the time the StaleElementReferenceException can be addressed by waiting for something at the right time and this should be your first line of thought. Currently it can be that this is not enough though. Cases 2 - 5 should really be handled by WebDriver IMO. Waiting should always work. Elements provided by @FindBy should always work (when I just created the page object). And the result of my tests should not depend on whether I’m lucky or not 
I hope these details can be of help to you when you run into a StaleElementReferenceException: Element is no longer attached to the DOM.
(*) This exception occurs, when you have a reference to an element and you want to call a method on it, but the underlying DOM has changed.
Thank you for you post! I meet this frequently. However, I don’t understand how the case 2 can happen. See the code of
http://grepcode.com/file/repo1.maven.org/maven2/org.seleniumhq.selenium/selenium-support/2.19.0/org/openqa/selenium/support/ui/ExpectedConditions.java#ExpectedConditions.invisibilityOfElementLocated%28org.openqa.selenium.By%29
Secondly,the StaleElementReferenceException occurs on e2.click, why do we try and catch the wrap function?
Hi Loc
You’re right, this isn’t necessary anymore. The code I was dealing with at the time was:
public static ExpectedCondition visibilityOfElementLocated( () {
final By locator) {
return new ExpectedCondition
@Override
public WebElement apply(WebDriver driver) {
return elementIfVisible(driver.findElement(locator));
}
};
}
Thanks!
What about my second concern: why do we need the try…catch the wrap function when the StaleElementReferenceException occurs on *e2*. The try…catch means that the exception may occur also on *e*?
Not sure I understand your question correctly.
I assume you’re talking about case 4. The StaleElementReferenceException can be thrown on any of the first 3 lines (but what I’m trying to address is the case where the exception is noted).
We’re overriding all methods from WebElement, which includes click() as well.
class OurWebElement implements WebElement {
// Constructor see in case 4
@Override
public void click() {
try {
underlyingElement.click();
} catch (StaleElementReferenceException sere) {
againLocate();
click();
}
}
}
Since e2 is an instance of OurWebElement, it has the ability to locate itself again and then execute the click on the newly acquired one (which is itself an instance of OurWebElement).
I’m not sure which element you refer to by ‘e’. Do you mean the ‘element’ in
WebElement e2 = element.findElement(…)
(i.e. the second line in case 4)?
—
I’ve added the wrap method for clarity in the case above
Yes, ‘e’ is ‘element’
Because you said “The StaleElementReferenceException can be thrown on any of the first 3 lines”, everything is clear. The usage is
WebElement element = OurWebElement.wrap(driver.findElement(…));
WebElement e2 = element.findElement(…); // safe
e2.click(); // safe if OurWebElement also override click method.
Yes, exactly. Thanks for pointers to make this post better.
Hello,
In example 4, how did you implemented this method:
locate()
Thanks
Ah, here are the locator classes:
public static interface Locator {
WebElement locate();
}
public static class FindElementLocator implements Locator {
private SearchContext searchContext;
private By by;
public FindElementLocator(SearchContext searchContext, By by) {
this.searchContext = searchContext;
this.by = by;
}
@Override
public WebElement locate() {
return searchContext.findElement(by);
}
}
public static class FindElementsLocator implements Locator {
private SearchContext searchContext;
private By by;
private int index;
public FindElementsLocator(SearchContext searchContext, By by, int index) {
this.searchContext = searchContext;
this.by = by;
this.index = index;
}
@Override
public WebElement locate() {
return searchContext.findElements(by).get(index);
}
}
Thanks Jerome!
After your response I fixed my code and now it handles the problems you described in cases 4 and 5.
I still have 2 questions:
1. Regarding point 3, how should we return OurWebElement from class OurElementLocatorFactory(driver) ?
2. I have this particular case:
final WebElement webEl= wait.until(driver, ExpectedConditions.elementToBeClickable(By.id(….))); //safe
select = new Select(webEl); //StaleElementReferenceException
What do you suggest to do in this case?
In case 2, I think a good idea would be to use what you proposed in point 3 ( to use the safe PageFactory). Am I right?
Thanks a lot Jerome! Brilliant ideas divided in 5 points
Good work
For
final WebElement webEl= wait.until(driver, ExpectedConditions.elementToBeClickable(By.id(….))); //safe
select = new Select(webEl); //StaleElementReferenceException
Yes, the PageFactory should do the trick. In any case you have to make sure that webEl can refind itself (i.e. is of type OurWebElement).
Hi Jerome,
One last question: how did you implemented the ‘findElements’ method in OurWebElement ?
Until now I did not need it, but now I do
Thanks
We wrap every single element in the list that is found:
@Override findElements(By by) {
public List
try {
return wrap(underlyingElement.findElements(by), this, by);
} catch (StaleElementReferenceException sere) {
againLocate();
return findElements(by);
}
}
public static List wrap(List elements, ret = new ArrayList(elements.size());
SearchContext searchContext, By by) {
List
for (int i = 0; i < elements.size(); i++) {
WebElement element = elements.get(i);
ret.add(wrap(element,
new FindElementsLocator(searchContext, by, i)));
}
return ret;
}
I had a simpler version:
public List findElements(By by) {
try {
return underlyingElement.findElements(by);
} catch (StaleElementReferenceException sere) {
againLocate();
return findElements(by);
}
}
but I think your approach is better because the returned elements are also of type OurWebElement. Am I right?
Anyway it’s quite difficult to test the code unless you have an easy reproducible SERE when using WebElement.findElements(…)
Regards
Yes, you should also wrap the returned elements, because you might be using them at one point to call methods, and if they aren’t wrapped, they might throw the SERE again.
Hi Jerome,
I also encountered some problem with this method:
@Override
public String getTagName() {
String result = “”;
try {
result = underlyingElement.getTagName();
} catch (StaleElementReferenceException sere) {
againLocate();
getTagName();
}
return result;
}
It seems it has some problems… Did you implement it differently?
With this implementation when SERE occurs this part:
“return result” from the first call will remain on the stack. Although the underlying element was found again this function will return default value “” with this implementation … which is wrong…
No, we implemented it slightly differently.
@Override
public String getTagName() {
try {
return underlyingElement.getTagName();
} catch (StaleElementReferenceException sere) {
againLocate();
return getTagName();
}
}
If you want to go with your implementation, you’ll have to assign the value that you get from getTagName() to result:
catch(SERE sere) {
againLocate();
result = getTagName();
}
Otherwise, you’re running into what you just described.
Indeed…the first option proposed by you looks perfect. Thanks
Hi Jerome,
One quick question:
I got an IndexOutOfBoundsException at this line:
“List returnedElements = new ArrayList(elements.size());”
in wrapElementsList method.
private static List wrapElementsList(WebDriver driver, List elements, SearchContext searchContext, By byLoc)
{
List returnedElements = new ArrayList(elements.size());
for (int i = 0; i < elements.size(); i++) {
WebElement element = (WebElement)elements.get(i);
returnedElements.add(new CustomWebElement(driver, element, new FindElementsLocator(searchContext, byLoc, i)));
}
return returnedElements;
}
}
Did you see this error before?
Thanks
I investigated the issue and it seems that in my case in FindElementsLocator the index = 11 and in OurWebElement , method wrapElementsList elements.size() = 12.
Excerpt from the log:
FindElementsLocator. index = 11
elements.size() is 10
java.lang.IndexOutOfBoundsException: Index: 11, Size: 10…
Hi Jerome,
In FindElementLocator in locate() method I get an indexOutOfBounds. Line:
return (WebElement)this.searchContext.findElements(this.byLoc).get(this.index);
Could you please give me an idea regarding how to fix it, if you have any..?
Br
I know the cause of this nasty thing: let’s say I use the wrapper and I find 3 subelements using elem.findElements; when I want to use one of these 3 elements it is stale, and we try to find again the elements but this time instead of 3 elements we have only one (the page refreshed)!
To solve this we need to wait before searching the child elements(using findElements ) until the number of child elements is stable. I implemented something like this before wrapping:
waitUntilOneResultDisplayed(60);
Hi Jerome, here in above classes where you passing driver, I am getting NullPointerException.
Hi Prasad
I hope you get the idea of what you can do to avoid the StaleElementReferenceException. I’m afraid I can’t help you with the NullPointerException, since I don’t have your code. NullPointerExceptions are usually easy to debug, so I’m convinced you’ll find the problem yourself.
Do you mind if I quote a couple of your articles as long
as I provide credit and sources back to your blog? My blog
site is in the very same niche as yours and my visitors would really benefit from a lot of the information you present here.
Please let me know if this alright with you. Many thanks!
Sure, no problem.