Enterprise Synthetic Documentation

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.

You may often need to monitor applications whose child controls cannot be recognized by the Recorder.  When creating scripts for monitoring such applications, you should bear mind these generic recommendations, that can also help with applications where controls can be recognized, but attaching to controls is not reliable or text cannot be captured.  

Note that these tips may be not applicable to every specific situation you may encounter.

Applications under consideration may range from single control applications, represented by a single frame window, to a complex mixture of recognized and unrecognized controls.  The examples of single control application would include Citrix XenDesktop and Windows Remote Services client.  The complex applications include, for instance, .NET applications with Infragistics controls and Java applications with SWT controls.


Environment Standardization 

Typically you need to run the same scripts on multiple robots.  Standardizing environment parameters helps to avoid the inconsistencies in the behavior of the monitored software that might otherwise cause consistent or intermittent problems.  We recommend standardizing the following across all robots:

  • Screen resolution.
  • Color depth.
  • Windows themes.
  • Application window sizing and positioning across your monitoring environment.  Script against maximized frame windows.
  • Font preferences and sizes.
  • Browser settings for Web applications, like scale (unless you have a reason to do it differently, standardize on 100%), date formats, and so on.
  • Versions and tunings of software involved (for instance, Windows, Internet Explorer, Java Runtime Environment, Common Language Runtime).

Optical Character Recognition

OCR is the recommended method to acquire text in your scripts for non-supported technologies and for unrecognized child controls.  It works well where text could not be supported obtained otherwise and can be used when you need to wait for text to appear or disappear.  The OCR technology allows using the text for waiting/synchronization purposes and for introducing related logic to the script.  The technology also provides the ability to get coordinates of the text; and this can be more scalable across different robots in cases when some of elements of environment standardization cannot be achieved. See TextMap and RelativeRectangle objects for advanced techniques.

For more information, see Optical Character Recognition.

Image Recognition

Likewise, using Recorder and Framework functions related to Image Maps also allows to achieve synchronization, script logic, and obtaining coordinates.  BitmapSelect function allows combining finding image and mouse click.

Expose Child Controls

Wherever possible allow child controls to be recognized.  For instance, Citrix XenApp is preferable to XenDesktop, because this allows pop-ups to be recognized.

Using Keyboard

Using the keyboard often allows improved reliability and portability of the scripts across different robots when environment standardization cannot be fully achieved.  Here are some considerations to take into account:

  • The keyboard focus may be immediately on the desired control; so instead of using the Type method against a GUI object, it is possible to use Keyboard.Type.
  • The keyboard shortcut for the needed action is preferable to a mouse click.
  • Tabs and navigation keys can be used to move focus and to make menu selections.  Once the focus or menu selection is established, Keyboard.Type "{Return}" would initiate the necessary action, or Keyboard.Type "<string>" would fill in the field.
  • Necessary selection in selection controls, including the Windows desktop, may be accomplished by typing ahead.
  • Framework provides KeyCopyAll constant.  By passing this constant as an argument to Type method, you can put all selectable text to the Clipboard object and then use it in the script.
  • Wherever necessary, pause the script to allow the application to process Windows messages; you may need to use several Type calls, with a pause in between.  Depending upon how slow the application is and whether status needs to be displayed, you can use DoEvents, Sleep, or SleepFor.

Using Mouse

When using the mouse with unrecognized controls you must know the coordinates for clicking, mouse moving and dragging.  If the environment standardization detailed above can be achieved, you can use hardcoded coordinates.  Otherwise, here are some considerations:


  • Use Optical Character Recognition (OCR) to get the coordinates of the text to calculate the point to:
    • click inside the text, for instance to click on a button,
    • click outside the text, for instance to click on a radio button or on a checkbox,
    • other uses of the mouse.
  • In some cases, coordinates of the screen, the framework window, or another recognized control can be used to calculate the coordinates needed for mouse operations.
  • Clicking may be used to set focus at some logically consistent point or to pop up/drop down a menu; this might enable the keyboard operations described above.

Using Clipboard

Sometimes it is helpful to select the text and put it onto the clipboard, like in the Framework WaitForSelectableText function.

Synchronizing Script with the Application

You need to use the appropriate wait functions to synchronize your script with the application.  When you need to synchronize with controls that cannot be recognized, usually your only choices are using images or using OCR.  In some cases, you can monitor the text that can be selected and put onto the clipboard using WaitForSelectableText or the monitor the contents of a file using WaitForTextInFile.


The scripting methods described above usually result in scripting code whose actions are not easy to understand, so detailed comments facilitate the maintenance of these scripts.