Wednesday, December 28, 2011

How to integrate Sikuli In selenium

This Blog is all about discussing integration of sikuli in selenium and how can we optimally use SIKULI with SELENIUM

Lets start with some basics
 Selenium is a Web Automation tool which we can integrate in java projects or any any other supported language products
Selenium is a open source tool but it has some limitations like it can not interact with things which are not part of browsers, browser pop ups, Flash Contents, Java Applets etc.

To solve this we can use another open source tool "SIKULI" and integrate it with our existing selenium automation framework.

We have other tools like AutoIt for same purpose but what's the difference, Why i choose SIKULI a new tool over AutoIT

Reasons are simple
1. I found SIKULI easier to understand as it needs only screenshots, as a Automation suite designer its my responsibility that who ever is using my Suite should find it easy to use and understand, Sikuli serves that purpose more than AUTOIT
2. Sikuli is easy to integrate with your existing  java based selenium framework as we can include sikuli jar in our framework same as including selenium jar.

How To Integrate in Existing framework
First we need to know how to use Sikuli in java

      How to use Sikuli Script in your JAVA programs

The core of Sikuli Script is written in Java, which means you can use Sikuli Script as a standard JAVA library in your program. This document lets you know how to do that.

     Get sikuli-script.jar from your Sikuli IDE installation path.

Sikuli Script is packed in a JAR file - sikuli-script.jar. Depending on the operating system you use, you can find the sikuli-script.jar in according places.
  • Windows, Linux: Sikuli-IDE/sikuli-script.jar
  • Mac OS X:

     2Install OpenCV.

Sikuli Script uses OpenCV as its computer vision engine. On Windows, you can simply add the path Sikuli-IDE/tmplib to the environment variable PATH, and then Windows can automatically find OpenCV's DLLs in there. However, it's more complicated on Mac and Linux. Therefore, our suggestion is to install OpenCV by yourself.
  • Mac OS X: you can install OpenCV using  Darwin Ports. Type "port install opencv" in once you have installed Darwin Ports.
  • Linux: OpenCV is often packed as a series of packages. For example, libcv4, libcvaux4, libhighgui4 on Ubuntu Linux.
== 3. Include sikuli-script.jar in the CLASSPATH of your Java project.==
We use Eclipse as an example. After adding sikuli-script.jar as a library into your project, the project hierarchy should look like this.

   4Import the Sikuli classes you need

You can simply "import org.sikuli.script.*" or import the classes you need. In most cases, you would need at least RegionScreen, and SikuliScript. (Note: the package name in 0.9 and 0.10 was If you are using Sikuli X, please use org.sikuli.script as the package name instead.)

  5. Write code!

Here is a hello world example on Mac. The program clicks on the spotlight icon on the screen, waits spotlight's input window appears, and then types "hello world" and hits ENTER.

import org.sikuli.script.*;

public class TestSikuli {

        public static void main(String[] args) {
                Screen s = new Screen();
              "imgs/spotlight.png", 0);
                        s.type(null, "hello world\n", 0);
                catch(FindFailed e){



After you have tested sikuli lets integrate it with our Existing SELENIUM framework

First Include in your test sheet a flag by which your framework knows that next commands in the sheet represent sikuli commands as selenium and sikuli both have same command set like click, type etc.

Also it gives one more advantage you can use java reflection for sikuli to minimise your coding efforts

Like you include a flag named 'sikuli' with values on and off, if value is on than use sikuli functions else use selenium. By this way you can easily manage both selenium and sikuli functions.

Using both SIKULI and SELENIUM you will be able to easily handle all windows that selenium is not able to handle.

Disadvantages of using SIKULI in Selnium
1. It limits the selenium RC, we can not use it remotely and it we can use selenium with sikuli as a standalone server
2. It decreases the speed of selenium as sikuli click takes much more time compared to selenium click as sikuli first searches and matches then click.
3. Sikuli takes more CPU resources than selenium