Run Selenium (AJAX) tests in TestMaker

[fcohen]

PushToTest version 5.2 will feature Selenium integrated into the PushToTest test automation platform. Selenium users will create functional tests of Web and Ajax applications using the Selenium plug-in to Firefox. Selenium outputs Selenese files in HTML format.

Users will have two ways to play Selenium tests in the PushToTest distributed test environment:

1) ScriptRunner:

   <resources>
    <selenium path="BrewBizCoffeeTest.selenium"/>
  </resources>

  ...

 <run name="CoffeeTest" testclass="BrewBizCoffeeTest. selenium" 
                  method="selenium" langtype="selenium">
 </run>

Internally this uses the Denali contributed (http://tinyurl.com/6lzq9c) HTMLUnit/Rhino approach to operate the test.

2) Transformer for Selenese -> Jython:

PushToTest will contribute a Selenese-to-PushToTest transformer. The Transformer outputs a Jython script that uses HTMLUnit and Rhino to play-back tests. We will use the Denali Selenium-PushToTest transformer contribution. Dominique and Olivier of Denali contributed the transformer. Download the transformer at: http://tinyurl.com/6lzq9c. We will use this approach to write a Selenium ScriptRunner (langtype="selenium").

For both of the above options, Selenium tests receive dynamic operational data at runtime from PushToTest Data Production Libraries (DPLs.) For instance, a Selenium test that operates a sign-in page receives the account number and password from a DPL that accesses data from a comma-separated-value file (or a relational database or custom DPL.)

Additionally, Selenium tests produce results data that PushToTest Results Analysis engine renders into hundreds of charts. And, these charts correlate Selenium test operation to resource utilization (CPU, Network, Memory, Threads) in the back-end server.

3) Operate a Selenium test in a TestNode/Selenium? RC/Browser

This test operates a Selenese test using a TestNode, Selenium RC, and a browser. TestMaker operates the test and receives the results. TestMaker processes the results into a transaction log file and processed the TestMaker results engine.

[fcohen]

A TestMaker user offered to contribute a utility they wrote that runs Selenium tests in TestMaker. They still need to do testing on the module and we need determine a way to provide attribution to the individuals and their company. This is a very exciting development. -Frank

[fcohen]

Original comments from ticket 129 (now closed as a duplicate) showing the thoughts behind this ticket:

From an instant messanger chat between Frank Cohen and William Martinez:

Frank: My sense is that TG4W is not up to the task of recording Ajax apps, but I could be completely wrong about that. what do you think?

William: AJAX is not more than javascript executing in the browser. TG4W will not be able to reproduce a sub-window drag and drop, nor fireworks in the cursor as you move, but it will record the final values you enter in the form that gets posted. We may have problems if the application posts data not using the normal HTTP post command, but using the HTMLRequest to send XML data. That will not be caught by TG4W. And to reproduce in TM side, it depends on the javascript support that HTMLUnit has.

frank: Thanks, that is my understanding too. I am expecting that Luis Carlos will wind-up writing code to implement the unit tests of the petstore because HTMLRequest is used frequently. since there is no description language for Ajax, it seems like we need some sort of strange hybrid monster of MaxQ and TestGen?4Web and a transformation engine. selenium has the right user interface to record an Ajax app, but its playback expects the browser to be there. i suppose we could write a transformer to go from Selenium to Java or Jython with HTMLUnit. I need to do some investigation on this.

William: So, we can add the Selenium Wrapper for Java/Jython and indirectly execute the test. We are actually investigating that to be used in Avantica for the QA team to test web paegs.

Frank: Interesting

[ddewaleffe]

Our contribution will be based on (some work is still needed, but we start to use it):

translating selenium tests to a jython script that runs under testmaker and call a component relying on htmlunit to execute the selenium actions. This is just in the direction discussed by Frank.

This will require (at least) upgrading to more recent htmlunit in TM...

We will work on how to package this in the coming days...

[question: is there a way, when executing a test script (say in jython) to know if the script is being run as a functional/monitoring test or as a load test? Idea that we have with this is to avoid checking some/most assertXXX() when run as loadtest, but still use the same script source as functional/monitoring test. ] ]

[ddewaleffe]

We are progressing on this... Observation made is that running in this mode requires quite a lot of CPU on the testnodes to cope with running htmlunit/js. The impact is that one has to increase the # of testnodes to get realistic results.

[fcohen]

Received a contribution from Dominique and Olivier. Evaluating it over the next few days. -Frank

[fcohen]

Denali's contribution is available for download at:

http://www.denali.be/SharedPubli/WebObjects/SharedPubli.woa/wa/PDA/s;jsessionid=C2C7E5A26C60A8CCA6720D1074258120.sharedpubli?s=1006871&wosid=nbHGNhk9Yn0oFM6lOuZRYg

[root]

Milestone release deleted

[fcohen]

Adding a link to my discussion of Selenium support in TestMaker in the Selenium developer's forum: http://clearspace.openqa.org/thread/9866?tstart=0

[fcohen]

Interesting idea from the Selenium developer's forum is to build a Selenium RC driver for HTMLUnit. Selenium would run natively in HMTLUnit. -Frank

[fcohen]

Confirmed implemented. -Frank

[fcohen]

Luis Carlos wrote the following:

Selenium Unit

Documentation of changes made by “Push To Test” Community

The most important changes made to the Selenium Unit are:

1) Create a current page instead of using get enclosed page from current window.

private HtmlPage? currentPage; instead of getWebClient.getCurrentWindow().getEnclosedPage();

Reason: In Appcelerator’s calendar we found that in a pop-up window, get enclosed page got only that windows, not the full page. To fix this issue we maintain a current page, every functions that changes the page updates currentPage:

public void click(String locator) {

…

currentPage = (HtmlPage?)((ClickableElement?)elem).click();

… }

2) setSeep method implementation: setSpeed method sets a delay between steps in the denali run. This was our first approach to attack the problem of synchronization with Ajax system.

Implementation: setSpeed stores a sleep time. We stop momentarily the processing of the test (Thread.sleep) in all the functions that changes the page (like click). setSpeed is not currently implemented for get* methods or methods that don’t change the page.

3) clickAndWait implemented using click.

public void clickAndWait(String locator){

click(locator);

}

HTMLUnit is synchronous then clickAndWait can be mapped to click because click is currently synchronous. Should we change click to make it asynchronous?

4) Link=Locator system implemented

In order to avoid using user-extension.js (that has some problem with selenium IDE 1.0 beta) we implemented Link=Locator system using the next code

public HtmlElement? getElement ( String locator) {

// Fix selenium-ide's simplified 'id locators', where the locator // contains // a bare id and is not an actual XPath expression.

// 1. Try with an ID. if (locator.startsWith("link=")){

locator = locator.substring(5);

} String theLocator = locator; boolean bareLocator = false; if (!(theLocator.charAt(0) == '/')) {

bareLocator = true; theLocator = "//*[@id='" + locator + "']";

} HtmlElement? result =

(HtmlElement?) getPage().getFirstByXPath(theLocator);

if (bareLocator && result == null) {

// 2. No matching element with modified locator, try again with a // name-based locator.

theLocator = "//*[@name='" + locator + "']"; result = (HtmlElement?) getPage().getFirstByXPath(theLocator);

} if (bareLocator && result == null) {

// 2. No matching element with modified locator, try again with a // name-based locator.

theLocator = "//*[contains(text(),'"+locator+"')]"; result = (HtmlElement?) getPage().getFirstByXPath(theLocator);

} if (result == null) {

debugElementLocationFailed(locator); throw new RuntimeException?

("Failed to locate element with locator " + locator);

} return result;

}

5) Wait for condition system implemented

In order to avoid using setSpeed system that has some problem with Ajax testing (http://agiletesting.blogspot.com/2006/03/ajax-testing-with-selenium-using_21.html ), we implemented the wait for condition system, using the next code

public void waitForCondition(String script, String timeout) throws java.lang.Exception{

long theTimeout = Integer.parseInt(timeout); long initialTime = new Date().getTime(); while (true){

if (new Date().getTime()-initialTime > theTimeout)

break;

Object obj = webClient.getJavaScriptEngine().

execute(this.getPage(),script,"title",0);

System.out.println("Stript "+script); System.out.println("Result "+obj);

Thread.sleep(100); Thread.yield();

if (obj == null)

continue;

if (obj instanceof org.mozilla.javascript.Undefined)

continue;

if (obj instanceof Boolean)

if (((Boolean)obj).booleanValue() == false)

continue;

//not boolean but also not null

return;

} throw new java.lang.Exception

("waitForCondition: Timeout reached");

}

Are these changes correct? Please advice,

LC