CONTINUOUS TESTINGSHIFT LEFTOpen Source Performance Testing: How Does LoadRunner Stack Up?

LoadRunner says they support open source performance testing with JMeter - but what does that actually mean? Is the support good, good enough, or not good at all? Let’s take a closer look.
srdjan nalis srdjan nalisFebruary 24, 20205 min

Written By Srdjan Nalis and Jacob Sharir

LoadRunner says they support open source performance testing with JMeter  – but what does that actually mean? Is the support good, good enough, or not good at all? Let’s take a closer look. 

LoadRunner only supports some JMeter capabilities

Loadrunner is limited to running only certain protocols in JMeter – primarily HTTP and HTML protocols. There is support for some plugins, but it’s not comprehensive and and versions may be outdated. If you want comprehensive JMeter support, for every JMeter add in and every protocol, LoadRunner will limit your plans. 

There are other open source limitations

LoadRunner’s test execution framework is proprietary. You’ll need to know ANSI C to use it. In contrast, open source test execution frameworks like Taurus allow developers to use whatever test framework and scripting language they prefer, and do so from their IDE of choice. The BlazeMeter team founded the Taurus open source project, and the BlazeMeter platform uses Taurus for test execution.  

Another LoadRunner limitation is Selenium support: it’s only supported in the cloud – not on prem, and with a very limited variety of frameworks (only JUnit and Mocha).

You have to port to parallelize 

Our friends published these handy images to illustrate how parallel testing can reduce testing time. Depending on your resources and execution framework – you can likely expect even better results. 

With LoadRunner – you can only run parallel tests in LoadRunner’s proprietary scripting language (ANSI C) – no JMeter parallel tests. Parallelization requires the proprietary path. 

Keep your data on prem and execute in the cloud? Not possible

LoadRunner doesn’t offer a true private cloud option to allow secure execution of cloud-based open source tests. This limitation may be especially notable for organizations who want the flexibility of open source and cloud, but with enterprise-level security.

Let’s do the math – LoadRunner is resource intensive 

To spin up a LoadRunner load generator for a few thousand machines, you’re looking at over 10 minutes. In contrast, it takes less than 5 minutes on BlazeMeter, regardless of which open source execution framework you choose to test with. Who has that kind of time to waste? Not you. 

Let’s look at machine capacity. On average, you’ll get… 

  • LoadRunner TruClient protocol  – 50 Virtual Users
  • JMeter executing on BlazeMeter – 1000 Virtual Users

Here’s the math: 

Memory Needed 

  • Single user in TruClient = approximately 80MB
  • Single user in JMeter & BlazeMeter – 2-3 MB 

Using LoadRunner is going to take a lot of machines. Who has that money to waste? Not you. 

Reporting is an MVP 

LoadRunner reporting for JMeter is an MVP (Minimum Viable Product). You get some basic stats. 

You’ll only get: 

  • JMeter Active threads
  • JMeter Hits
  • JMeter Throughput
  • JMeter Transactions
  • End User Experience Testing (JMeter & Selenium)

Contrast that with the other kind of MVP (more valuable) approach you’ll find with BlazeMeter. Here are more details about what you’re seeing below -check out the number of virtual users 😉 

 

In this report, you’re even seeing what is happening to your UI under load, using JMeter and Selenium. In one report, you see what your user sees at peak times, complete with application monitoring stats. Micro Focus doesn’t offer you anything like this. 

 

If open source is important to you (and you know it is) – LoadRunner is probably not the answer

At BlazeMeter and Broadcom – we’re committed to helping organizations run open source with enterprise scale, features, and security. We are dedicated to continuously contributing to the open-source community and fulfilling the technological gaps we identify when collaborating with our end-users.

We provide comprehensive support for JMeter – every protocol and every add in. In fact, we’re one of the biggest contributors to the JMeter project. We also contribute directly to many other open source projects, and we have dedicated resources for these initiatives. We run BlazeMeter Labs – an open source project dedicated to open source development and contributions and BlazeMeter University with multiple courses on open source technologies. If you need help running your open source tools on BlazeMeter – we’re here to help. 

Ready to go? Try the LoadRunner script converter and move your scripts to open source with a few clicks.  Want to learn more? Check out this webinar.

 

srdjan nalis

srdjan nalis

Customer Products Adoption Manager EMEA | Continuous Testing at Broadcom Inc.