LoadRunner – The parameter file is stored in the script directory as .dat file extension.  It can be opened with a simple text editor like Notepad as it is stored in pure text format.  Therefore, you can manipulate the files via the Notepad.  You can also use office applications such as Microsoft Excel or OpenOffice.org Calc to work on the file.  This greatly the amount of work required for maintaining a long list of data.

Quick Test Professional – The parameter file is also stored in the script directory as an excel file, which will be an .xls file extension.  Being an excel file, it can be opened by office applications like Microsoft Excel.  Again, if you do not have money to purchase a licensed Microsoft product, you can always use open source products like OpenOffice.org Calc as an alternative tool as it is still able to work with .xls files.

We hope that by providing you such tips, you will be able to work efficiently with large list of data (that is especially prevalent in LoadRunner)!

• QTP scripts are developed in QTP not in LoadRunner Vugen – You won’t be able to record and replay a QTP script in Vugen.  All the recording and modification has to be done on QTP.  Once the modification of the QTP script is completed, it will be ported into LoadRunner Controller as GUI Vuser for load testing.
• QTP is required to install in the Load Generators (LG) – The load generators will be required to install QTP as they are used to run the scripts.
• Load Generators will be required to set the same resolution to the machine that developed the QTP script – As QTP is object sensitive, the resolution of the LG is required to be the same with the machine that was initially used to develop the QTP script to avoid any problems arising from missing objects (due to a different resolution)
• [1] GUI Vuser license is required for LoadRunner and QTP concurrent license is required for QTP – This is the 1^st setup approach and budgeting for your licenses.  The total amount of Vuser that you wanted to generate will be the same for both LoadRunner and QTP.  Meaning, if you want to run 50 concurrent users using GUI Vuser protocol, you will need 50 GUI Vuser license and 50 QTP Concurrent license.  In this setup, the LG will create 50 instances of QTP to run the GUI Vuser.  This is described in “What licenses are required to run a scenario with GUI Vusers” from HP Software Support.
• [2] GUI Vuser license is required for LoadRunner, QTP seat/concurrent license is required for QTP and Windows Terminal License is required for Windows – This is the 2^nd setup approach and budgeting for your licenses.   For a 50 concurrent user load test, you will need 50 GUI Vuser protocol, 50 Windows Terminal License and 1 QTP license.  In this setup, the QTP license is merely used to develop the script.  The LG will create 50 remote desktop connections and each of these connections will run 1 GUI Vuser.  This setup utilizes remote desktop functionality to emulate the virtual users.  I believed that HP Support does not really state about the licenses required from Windows Terminal and this is additional cost for you to factor if you are going though this mode.  Personally, I experimented using Windows Server 2003 (that is capable of 3 remote connections by default) with this approach and it works for me.  On the server, the remote connections are established and the QTP script launches the browser for testing.  (I wonder if this is permissible in the first place and like to know if anyone else is doing it.  If you got some input on this, please feel free to shout it here!)

With the above information, you should be slightly more familiar with Quick Test Professional (QTP) integrating with LoadRunner.  However, there may be more things that need to be considered and I may have left out.  Do feel free to feedback and experiences and anything that I missed out when using QTP in load testing!

1. Do you have sufficient hardware? – How may load generators are required for the entire load test?  Refer to this document from HP Support, “Is resource (cpu and memory) footprint information available to help in estimating the load generators required for load testing with LoadRunner?” for estimating the load generators required by multiplying the observations with the total number of virtual users generated in the test.   In addition to the processor utilization recommended in the document, do watch the memory and network consumption of the load generators.   The network interface card (NIC) can become a bottleneck for load generation.  A 1GB NIC will definitely generate and receive more load than a 100MB NIC.  Therefore, do a check on the machines assuming the load generator roles and make appropriate measures to prevent the NIC in becoming the bottleneck.   With the above, you will get a rough gauge of the required resource for your test.
2. Do you need spare hardware? – Do cater some spare machines if additional load generators are required, such as an additional of 5 load generators.  This is to stand by in the event of increasd virtual users and machine malfunction.
3. Remember that it’s a team effort! – Can you secure human resources for the load test?  You will need the web, application and database server teams to be around to observe the system behavior during the test or to assist in any reboot or refresh activities for the system.  Network and firewall/security team may be required to ensure that connectivity is allowed for the load test.    Therefore, in the planning, you have to get the relevant parties to commit the human resources!
4. Installation of LoadRunner – This require administrative or power users privileges in order for installation to be successful.  That raise another set of questions: Will you be able to install LoadRunner in the premises?  Who can you look for to install the LoadRunner program?  Do you need clearance for installation?  Is the system being hardened to prevent installation?
5. Offshore Installation – Is offshore installation required for the load test setup?  Can you perform the installation via remote connection or do you require someone from the site to assist in the installation?
6. Running of JVM/.NET Probes for Diagnostics – This will require installation and instrumentation on the application server, meaning it will be intrusive to the system as a whole.  Do you require additional permissions for such installation?
7. Post-installation – Do remember to test if the installation is complete and the programs are working before moving on to the next phase.  Remember to do a connectivity check between the load generators to the controller to ensure they communicate prior the load test.
8. Working in different environments – Are you required to work in different stages of a load test project?  Example, you are required to run the test in (A) the  data center followed by (B) the local client segment and lastly (C) the remote client segment?  In this context, how do you ensure the scripts worked from A to C?  It will be easy if it is a change of hostnames and URL addresses that is being used in these environments as it can be achieved through parameterization.  But what if there are additional authentication mechanism introduced in environment B and C?  Authentication mechanisms such as additional cookie in the clients are added into the URL or 2-factor authentication can render the scripts developed in environment A useless!  How do you go about it?  Should you cater more time to re-script for the worst scenario or is it better to disable the authentication mechansim during the load test?

These are some basic considerations we usually asked our ourselves and clients.  Hope it will be useful in your planning.  What are other considerations make for a load test for you?

In LoadRunner SAP monitor, it requires the SAP CCMS (Computer Center Management System) for monitoring to be effective.  This is unlike the typical monitoring mechanism that is used for Windows or Oracle (refer to “How does the monitoring work in LoadRunner?”), which is directly querying the monitor data from Windows Perfmon or Oracle V$ dynamic views.  LoadRunner SAP monitor requires an additional component, SiteScope (refer to “What’s SiteScope”) to act as an intermediary between SAP CCMS and LoadRunner.

LoadRunner itself does not have the capability to monitor SAP directly and solely depends on SiteScope to query/monitor SAP CCMS.  SAP CCMS is already a powerful component in SAP as it is able to monitor anything in the SAP system. This ranges from server resources (e.g. processor or memory utilization) to oracle databases (e.g. buffer efficiency).  You may not even require other tools to do monitoring in the first place!  No additional configuration needed for monitoring Windows resources (Windows Resources) as SAP CCMS handles it by a single channel through SiteScope.

For SiteScope to communicate to SAP CCMS, it will require SAP Java Connector (JCO) to be installed on the SiteScope box with some configurations to access the SAP system.  The detailed configuration can be found in Configuring Sitescope Monitors documentation (instead of LoadRunner).  To download SAP Java Connector, you can go to Connectors in SAP site.  You will need a valid SAP account to access the download page.

On the ports portion, you will need to have TCP32## between the SiteScope and the SAP system and TCP8888 between SiteScope and LoadRunner (Refer to “Implementing SiteScope with LoadRunner”).  TCP32## is the typical port number used by SAPGUI Front-end to talk to the SAP application servers while TCP8888 is used by LoadRunner to query the monitoring data displayed on the XML page of SiteScope.  Both port numbers are to be opened on a two-way traffic communication.
As SiteScope is querying monitoring data just like a normal user running the CCMS transaction from SAPGUI Front-end, a SAP account on the SAP system required.  The following information must be ready prior the setup of the monitoring.

• SAP Client
• System Number
• Authorization User Name
• Authorization Password
• SAP Router String (Optional)

Do take note on this!  With all things mentioned about SAP CCMS with LoadRunner-Sitescope, there’s a catch (at least to me).  SiteScope requires an optional license to monitor SAP CCMS.  Which means, that if you are just having LoadRunner, it’s not sufficient to setup the monitoring environment.

Performance testing recording and execution for SAPGUI relies on the SAP Scripting API and ActiveX the SAP R/3 application server and on all SAP GUI clients that are participating in the load test. As of date of this article, LoadRunner documentation does not have updated details on configuring SAPGUI for scripting such as supported versions.  However, the configurations are somewhat similar to other performance testing product which in this article, we refered to IBM instead.

The basic requirements for LoadRunner (or performance testing tools) to record SAPGUI is on two things: (a) the client machines to have the entire SAPGUI Client installed and (b) the SAP server to have scripting enabled.  (c) Other than the configurations, you may have to take note of the ports used by the SAPGUI, namely, TCP32## where ## represent the System Number.  This is especially important if the environment you are in is tight on opening of ports.

[1] Ensure client scripting enabled

• To ensure that scripting is available on the client-end, check that there is a Scripting directory located in the SAP GUI installation directory.  If this directory does not exist, then the SAP Scripting API is not installed and you must reinstall SAP GUI with the SAP Scripting API option.

[2] Ensure server scripting enabled

• Logon to SAP with a user with administrator privileges.  Administrator privileges is required to enable scripting on the server.
• In SAP, run the transaction rz11, and enter parameter name sapgui/user_scripting.
• If the parameter is not found, ensure that you have the correct support package level from SAP and contact SAP should you require further guidance on this.
• If the parameter exists, enable the parameter with the Current value changed to TRUE.  Save it and scripting will be enabled the next time you log on.

(Source: IBM, Configuring SAP R/3 for performance testing)

[3] Ensure communication between SAPGUI with SAP servers

Ensure that you have connectivity on TCP32##.  This is the port number used by SAPGUI to communicate to the dispatcher.

Also, before starting the scripting, for newbies, you may like to gather the following information as they will be used for the scripting work.  The user account used in the scripting (accessing the SAP system) should have sufficient privileges to the transactions.

• SAP Client
• System Number
• Authorization User Name
• Authorization Password
• SAP Router String (Optional)

In summary,

• Ensure client scripting enabled
• Ensure server scripting enabled
• Ensure communication between SAPGUI with SAP servers

Let’s go through an overview for implementing the Oracle DB. Basically, we are doing in this sequence:

1. Get an account in Oracle and access to the V$ tables that contains the monitoring data.
2. Install 32-bit Oracle client on the Controller machine.
3. Ensuring that proper configuration is done to connect to the DB using TNSNAMES.ora and defining the Oracle path.
4. Login to the DB using SQL*PLUS and run a query to see if you can collect the statistics from the V$ tables.
5. Launch LoadRunner Controller, configure the monitor and start your load test!

Ok, let’s dive into the details!

1. Request for an account and password to be created in the database instance (unless you are the DBA). Example, loadtester, and grant them access to read the following table. These tables consist of the statistic of the database instance.

• V$SESSTAT
• V$SYSSTAT
• V$STATNAME
• V$INSTANCE
• V$SESSION

2. Ensure that Oracle client libraries are installed on the Controller. Remembered that in order to monitor, you need local installation of the client in order for LoadRunner to query the monitoring data. If you do not have the client libraries, download it from Oracle Downloads. Download only the 32-bit Oracle client.

3. If you have the client installed, ensure the 32-bit Oracle client is installed on the Controller, not the 16-bit client.

4. Verify that %OracleHome%\bin is included in the path environment. This can be done by going to START > My Computer > Properties > “Advanced Tab” > Enviromental Settings.

5. Configure the TNSNAMES.ora file on the Controller. TNSNAMES.ora file is a SQL*Net configuration file that defines databases addresses for establishing connections to them. This file normally resides in the ORACLE_HOME\NETWORK\ADMIN directory. (Source: OraFaq)

Attached is an example of the TNSNAMES.ora file. Click here to download.

6. Ensure login successful with the created account (e.g. loadtester) with SQL*Plus from the Controller machine.

7. Ensure the privileges are given properly by typing the following command. If they don’t return results or return access rights issues, most probably the steps for prior to this one has not been configured properly. Therefore, approach the Database Administrator for assistance or go back to the previous steps to ensure they are properly configured.

• SELECT * FROM V$SESSTAT
• SELECT * FROM V$SYSSTAT
• SELECT * FROM V$STATNAME
• SELECT * FROM V$INSTANCE
• SELECT * FROM V$SESSION

8. Launch LoadRunner Controller as per norm and add the Oracle Database Monitor. When prompt, enter the account name (e.g. loadtester), it’s corresponding password and the destination server name. Statistics should be drawn from the V$ table and displayed in Controller.

With the above, you should be able to successfully configure the monitoring environment and the Controller with minimal difficulty.

Note:
You can change the sampling interval for the monitor using the vmon.cfg file located in {loadrunner-installed-dir}\monitors\vmon.cfg

To configure Sybase database monitoring, we will require ASE Monitoring Server (not the ASE) to be installed on the Controller (refer to “How does the monitoring work in LoadRunner?” for monitoring concepts). Furthermore, we will require an additional DLL in Controller for it to work. From the Monitor Reference, the DLL is named, libct.dll. That is for version 12. While in version 15, it’s named libsybct.dll. ASE in short for Adaptive Server Enterprise is a database product from Sybase. To get the latest copy of the ASE server can be obtained from the Sybase website.

• Downloading the installation requires the registration with Sybase. Registration process is hassle-free without any approval needed and you will be able to proceed with download once the registration completes.
• Upon full installation, all components are installed which includes the ASE database and the ASE Monitoring Server. Sample databases are provided in this installation if you required them.
• Both servers are not started by default. Therefore, you will need to configure them and start them up manually.
• To start the server go to START > Programs > Sybase > Adaptive Server Enterprise > Server Config.
• To start the ASE Monitoring Server, go to Service of Windows and find Sybase MONServer_<ASE Monitoring Server Name>.
• Alternatively, the ASE Monitoring Server can be booted from configuring the ASE Monitoring Server. Details for booting the ASEMS can be found in the additional resources.

NOTES

• Controller uses ASE Monitoring Server to monitor and not ASE and therefore, there is a need to ensure the ASE Monitoring Server is started, properly configured and accessible by the Controller.
• Login information to configure the ASE and ASE Monitoring Server are; Login: sa, Password: <blank>
• There maybe a difference of the filenames for the DLLs being used by ASE Open Client Connectivity. The filenames in v15 is libsybct.dll as compared to libct.dll in v12.
• Replacing the file libct.dll with a renamed libsybct.dll (libsybct.dll renamed to libct.dll) in the Controller does not have any impact to the monitoring.
• When configuring the ASE Monitoring Server in the Controller, in the Monitored Server Machines, put the machine name on the network (not the Sybase ASE server itself). In the Resource Measurements on the server machine, put the ASE Monitoring Server name (not the Sybase ASE server itself)

TROUBLESHOOTING

• After installing the Sybase ASE. Ensure that monitoring can be performed by the ASE Monitoring Server on the local machine. Use the Sybase Central console to perform the previous mentioned, by adding the “Monitor” folder. If the monitoring failed, it maybe the case that the ASE Monitoring Server is not started or configured properly. From here, troubleshoot accordingly.
• If monitoring was successful from ASE Monitoring Server locally, use dsedit tool from Sybase client to ensure connectivity.
• If dsedit tool locates the ASE and ASE Monitoring Server successfully, but connectivity fails when connecting via Controller, there might be a configuration problem on the Controller-side.

ADDITIONAL RESOURCES

• http://www.sypron.nl/main.html -Webpage that contains Sybase ASE information.
• http://www.sypron.nl/get_ase_soft.html – Download information for Sybase ASE
• http://www.sybase.com/ase_1500devel – Downloading Adaptive Server Enterprise version 15.0 Developer’s Edition
• http://www.isug.com/Sybase_FAQ/ASE/section1.6.html#scview – Information on ASE Monitoring Server
• http://manuals.sybase.com/onlinebooks/group-as/asg1250e/monbook – Monitoring Server; Information on methods of booting the ASE Monitoring Server.

We will go through the approaches in LoadRunner and briefly touch on the similar options available in SilkPerformer.

[1] Using Web (HTTP/HTML) protocol

The actual approach that was utilized in the load test. Back to the basics, the communication channel used by ArcGIS Desktop (ArcMap) is Web Services over HTTP, that’s SOAP/HTTP. The Web Services include finding the service provided by a ArcGIS Server, calculating and loading of the map. Therefore, normal means of using Web (HTTP/HTML) should suffix. For licensing concerns, Web (HTTP/HTML) protocol is grouped under the Web 2.0 Bundle license.

Further to this, (a) options for capturing the Content-Headers should be enabled and (b) the old recording engine in VUGen should be used. An article will be posted on the sample recording of ArcGIS Desktop (ArcMap) using Web (HTTP/HTML).

[2] Using Web Services protocol

As communcation is using Web Services, the most clear-cut protocol will be Web Services protocol which you can parameterize it with the Wizard. Refer to “Is it possible to load test Web Services without front-end?” . Understanding of the location of the WSDL file and parameters to passed in will be required for this approach. Of course, this requires the SOA Bundle license in order for load execution in Controller.

[3] Using .NET protocol

Using .NET protocol, you will need to know the assemblies and methods used by ArcMap. Therefore, go through the entire list of assemblies and methods that maybe involved in the communication between the client, ArcGIS Desktop (ArcMap) to the server, ArcGIS Server.

[4] Using Visual Studio AddIn

Using Visual Studio AddIn (after the installation of the plugin), you can create the user by calling methods from the ArcGIS SDK. Our evaluation of using ArcGIS SDK was using the C# language and making calls of zooming and panning activities. Unlike the conventional approach of knowing the communication between the client and server, and ignoring the client-activities, in this approach, we focus on the client-activities and ignore the communication used.

Similarly in SilkPerformer (refer to “What’s SilkPerformer” for a review of the product), there are four approaches to record and replay of an activity in ArcGIS Desktop (ArcMap) that are almost the same to LoadRunner with of course a different branding and licensing mechanism.

1] Using Web License

The basic function provided by SilkPerformer with same capabilitis as the Web (HTTP/HTML) protocol in LoadRunner.

[2] Using SOA Edition

Similar to SOA Bundle in LoadRunner, the SilkPerformer SOA Edition allows the reading of WSDL file and generation of the APIs based on the WSDL file.

[3] Using .NET Explorer

Similar to .NET protocol in LoadRunner, SilkPerformer will require the knowledge of the assemblies and methods being used by the user in a normal context. Using the .NET Explorer, SilkPerformer will generate the API based on the selected assemblies and methods.

[4] Using Visual Studio Plugin

Similar to Visual Studio Addin in LoadRunner, an additional plugin provided by SilkPerformer is installed into Visual Studio. And by using the desired langauges (Java, Visual Basic or C#) and the knowledge of ArcGIS SDK, you can create powerful scripts that emulates the activities of the real user.

If you have other approaches in load testing the application, please let us know and we can share it out with everyone here. Our understanding that there are other alternatives such as EggPlant Load Tester and using of RDP (Remote Desktop Protocol in LoadRunner 9.0 onwards) to load test ArcGIS.

When JMS is implemented via HTTP/XML, it’s simple here with the typical record and replay of the HTTP protocol. However, be aware that the asynchronous natural of messaging technology still applies. (LoadRunner have “partial” support on asynchronous application, in my humble opinion not really an accurate measure of the JMS load test). Therefore, if you are recording via HTTP protocol, our suggestion is to understand the interval that the message is being sent to the server and emulate it as lr_think_time in the script. When recording the application, you will be able to see the XML tags within the web_custom_request similar to Web Services.

If it’s implemented via RMI-Java, you will have to record using Java Record and Replay Vuser and this required understanding of the classes that communicates from the client to the server. The developer will be required to be involved in the recording as he/she can provide advise on the classes used. Furthermore, JMS implementation on RMI-Java requires a license for Java Record and Replay Vuser. Therefore, do take note of the availability of the licenses in your organization. If the JMS is implemented over T3 protocol in WebLogic, just have to take note that it’s BEA proprietary version of the RMI-Java implementation. Some information of T3 can be found in the following links to help you gain understanding of it:

• FAQs: Server-Related Questions, What is the function of T3 in WebLogic Server? (BEA WebLogic Server 8.1 Documentation)
• Using WebLogic RMI with T3 Protocol (BEA WebLogic Server 9.1 Documentation)

Both of the above leads to the same “depends” as it is dependent on the underlying protocol used. To sum up, before the load test, conduct an assessment on the communication mechanism (HTTP/XML or RMI-Java). Use WinSock recording or WireShark Network Analyzer to determine the communication. In a HTTP implementation, we will see conventional HTTP headers and XML tags. While in RMI-Java implementation, it will be a load of binaries with identified variable values. Thereafter, proposed the appropriate recording solution to it.

What’s JMX in the first place!

JMX (Java Management Extensions) technology provides a simple, standard way of managing resources such as applications, devices, and services. Because JMX technology is dynamic, you can use it to monitor and manage resources as they are created, installed and implemented. (Source: Sun Microsystems, “Ch1 : Introduction to JMX Technology”)

A simple illustration of how the JMX architecture can be found in Wiki, “Java Management Extensions”. This will give you an overview of the different levels and how the communication work between them.

Additional references can be found in Sun Microsystems which we recommend browsing through to get an general understanding of this technology.

• “Overview of Monitoring and Management”
• “Java Management Extensions (JMX) Technology”
• “JDK 6 Java Management Extensions (JMX)”

Ok, now we’ve got an understanding on JMX. In the context of WebLogic, JMX is in place to monitor the resources usage of the server.

How does WebLogic (JMX) monitoring work in LoadRunner?

JMX specification is designed to monitor Java applications from another Java application. Therefore, in order for LoadRunner to monitor WebLogic resources via JMX, it will be required that LoadRunner to be a Java-application to perform the monitoring and collect the data from the MBeans. This is achieved by installing JDK of the same version as the WebLogic server and running a mini version of application server in the LoadRunner machine to collect the monitoring data which is reason the installation of JDK as well as the copying of the weblogic.jar file is required.

What can you get from WebLogic (JMX) monitoring?

You will be able to monitor the runtime state of the WebLogic via JMX from the Runtime MBeans. The list of JMX counters can be found in “WebLogic Server MBean Reference”.

If you need to see how much time is spent or breakdown on individual methods, SQL calls or even the chain of calls, you are looking at the wrong tool. You may like to explore Diagnostics for a solution. Refer to “What’s Diagnostics?” for more information

We’ve put together a list of references that will you understand the JMX technology and also the fundamentals that WebLogic (JMX) working with LoadRunner. We hope this information is useful to you. If you have additional information with regards to the monitoring, please feel free to write in to us.