Wednesday, December 28, 2011

How to integrate Sikuli In selenium

Lets start with some basics
Selenium is a Web Automation tool which we can integrate in java projects or any any other supported language products
Selenium is a open source tool but it has some limitations like it can not interact with things which are not part of browsers, browser pop ups, Flash Contents, Java Applets etc.

To solve this we can use another open source tool "SIKULI" and integrate it with our existing selenium automation framework.

We have other tools like AutoIt for same purpose but what's the difference, Why i choose SIKULI a new tool over AutoIT

Reasons are simple
1. I found SIKULI easier to understand as it needs only screenshots, as a Automation suite designer its my responsibility that who ever is using my Suite should find it easy to use and understand, Sikuli serves that purpose more than AUTOIT
2. Sikuli is easy to integrate with your existing  java based selenium framework as we can include sikuli jar in our framework same as including selenium jar.

How To Integrate in Existing framework
First we need to know how to use Sikuli in java


      How to use Sikuli Script in your JAVA programs

The core of Sikuli Script is written in Java, which means you can use Sikuli Script as a standard JAVA library in your program. This document lets you know how to do that.

     Get sikuli-script.jar from your Sikuli IDE installation path.

Sikuli Script is packed in a JAR file - sikuli-script.jar. Depending on the operating system you use, you can find the sikuli-script.jar in according places.
  • Windows, Linux: Sikuli-IDE/sikuli-script.jar
  • Mac OS X: Sikuli-IDE.app/Contents/Resources/Java/sikuli-script.jar

     2Install OpenCV.

Sikuli Script uses OpenCV as its computer vision engine. On Windows, you can simply add the path Sikuli-IDE/tmplib to the environment variable PATH, and then Windows can automatically find OpenCV's DLLs in there. However, it's more complicated on Mac and Linux. Therefore, our suggestion is to install OpenCV by yourself.
  • Mac OS X: you can install OpenCV using  Darwin Ports. Type "port install opencv" in Terminal.app once you have installed Darwin Ports.
  • Linux: OpenCV is often packed as a series of packages. For example, libcv4, libcvaux4, libhighgui4 on Ubuntu Linux.
== 3. Include sikuli-script.jar in the CLASSPATH of your Java project.==
We use Eclipse as an example. After adding sikuli-script.jar as a library into your project, the project hierarchy should look like this.

   4Import the Sikuli classes you need

You can simply "import org.sikuli.script.*" or import the classes you need. In most cases, you would need at least RegionScreen, and SikuliScript. (Note: the package name in 0.9 and 0.10 was edu.mit.csail.uid. If you are using Sikuli X, please use org.sikuli.script as the package name instead.)

  5. Write code!

Here is a hello world example on Mac. The program clicks on the spotlight icon on the screen, waits spotlight's input window appears, and then types "hello world" and hits ENTER.

import org.sikuli.script.*;

public class TestSikuli {

        public static void main(String[] args) {
                Screen s = new Screen();
                try{
                        s.click("imgs/spotlight.png", 0);
                        s.wait("imgs/spotlight-input.png");
                        s.type(null, "hello world\n", 0);
                }
                catch(FindFailed e){
                        e.printStackTrace();                    
                }

        }

}


After you have tested sikuli lets integrate it with our Existing SELENIUM framework

First Include in your test sheet a flag by which your framework knows that next commands in the sheet represent sikuli commands as selenium and sikuli both have same command set like click, type etc.

Also it gives one more advantage you can use java reflection for sikuli to minimise your coding efforts

Like you include a flag named 'sikuli' with values on and off, if value is on than use sikuli functions else use selenium. By this way you can easily manage both selenium and sikuli functions.

Using both SIKULI and SELENIUM you will be able to easily handle all windows that selenium is not able to handle.

Disadvantages of using SIKULI in Selnium
1. It limits the selenium RC, we can not use it remotely and it we can use selenium with sikuli as a standalone server
2. It decreases the speed of selenium as sikuli click takes much more time compared to selenium click as sikuli first searches and matches then click.
3. Sikuli takes more CPU resources than selenium



10 comments:

  1. Hi, thanks for the article. Didn't know about the OpenCV requirement. Also some comments:

    * one other advantage of this integration is that Sikuli is cross platform. AutoIt restricts automation to Windows only.

    * it is theoretically possible to use Sikuli Selenium in a distributed Selenium Grid config, however, that requires more work than the traditional integration approach. You need to add in additional tools like psexec.exe (or winexe for *nix) or custom XML-RPC servers, etc. See my blog post for some ideas:

    http://autumnator.wordpress.com/2011/12/22/autoit-sikuli-and-other-tools-with-selenium-grid/

    or do the harder way and build a Sikuli Grid from scratch, using NativeDriver, etc.

    http://code.google.com/p/nativedriver/

    ReplyDelete
  2. @mangaroo
    Thanks i missed cross platform, thanks for sharing it and also as you said we can integrate sikuli in grid but it requires much effort

    Thanks for blogPost links

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. Hello Friends,
    I am using Selenium but for UI content I am using Sikuli and i want to incorporate them with my Selenium script is there any way I can do that???

    Thanks in advance...

    ReplyDelete
    Replies
    1. It depends how you are using selenium, if you are using selenium IDE only then using sikuli and selenium together is not possible but if you are using Selenium RC in java then include SIKULI jar in your project and call SIKULI functions in same way you call selenium functions

      Delete
  5. Wanted to provide update since my last comment, you may be able to use Sikuli with Selenium remotely, just not in a Grid deployment.

    For a single node remote only deployment (whether Selenium RC or WebDriver) look at this Sikuli remote implementation: https://github.com/enix12enix/sikuli-remote-control

    ReplyDelete
  6. Hi,
    I am using Selenium using C# language. I wanted to know if we can integrate sikuli with our vsts.


    Thanks

    ReplyDelete
  7. interesting blog..i am on the same path now (i.e trying to complement my selenium scripts with Sikuli)
    I will be looking into the Sikuli remote project..
    tnx

    ReplyDelete
  8. can the data be read from the excel sheet using sikuli...??

    ReplyDelete
    Replies
    1. there is a text() function in SIKULI to extract text, read this http://doc.sikuli.org/region.html#extracting-text-from-a-region

      Delete