Apache JMeter: Input from File, with HTTP Testing as An Example

Previous post covers how to set up basic HTTP testing with Apache JMeter. Many times we found ourselves need to read input from a file, say, a list of URLs for testing. Apache JMeter can be easily configured to do so. We’ll walk through a HTTP testing reading from URLs from an input text file.

0. Add Thread Group: Start JMeter, right click Test Plan > Add > Thread Group.

1. Add HTTP Request: Right click Thread Group > Add > Sampler > HTTP Request, specify the Path as ${PATH}, leave everything else as default. This is shown as below,
2. Add CSV Data Set Config: Right click Thread Group > Add > Config Element > CSV Data Set Config. Set the filename as the file contains the input URLs, Variables Names as “PATH,”,  Delimiter as “n”, so that JMeter will parse the URLs line by line and each line is fed to the PATH variable, which we used in HTTP Request at step 1. Another two important settings are Recycle on EOF and Stop Thread on EOF.

Recycle on EOF: The file is re-read from the beginning once it reaches EOF if set to true. In this  test, we set it to false.

Stop Thread on EOF: if Recycle on EOF is set to false, should the thread be stopped on EOF. In this test, we set it to true.

Below is a screenshot of what is the settings look like,

3. Add Listeners: Add listeners to show the testing results, we added Graph Results, View Results Tree and View Results in Table in our test.

4. Adjust Group Thread Settings: Adjust Thread Group settings. In our test, we set the settings as below,

In this case, JMeter will fire up 5 threads in 1 second and each thread will fetch 200 URLs from the input text file. Thread 1 fetches first line, thread 2 fetches second line etc. Since we set the Recycle on EOF at CVS config as false, the test will end either the input file reaches an end or 1000 URLs are fired.

If we set Recycle on EOF as true, the test will only end only 1000 URLs are fired as it can re-read the URLs from the beginning of the input file once it reaches EOF.

5. Test results: We can run the test, and view the results in the three listeners in different format.

References:
Apache JMeter user manual: http://jmeter.apache.org/usermanual/component_reference.html#CSV_Data_Set_Config

One Small Change in a Minute that Increases Daily App Download by 2000+

Recently I made a small change to my Video Converter Android app description. And the dialy download count has increased from 2500~3000 to 5000+, as shown below,

increase

The change only takes one minute. I added the descriptions in all languages Google Play allows. And it’s all done by Google translate automatically. Google Play developer console has make this really simple. It can be done in a minute.

I thought about the reason behind this. If users want to find some apps, they will probably search by the keywords. Many people use their mother tongue to search. The description is probably the most important information that the search engine is collecting about an app. The description, name etc. are analyzed to build the search index. If the description is in many languages, the search engine is going to have more info to build the index and have more keywords related to the app. Therefore the app can reach a broader range of users through search.

That’s it, since the small change in one minute, Video Converter Android has enjoyed 5000+ daily downloads.

Apache JMeter for HTTP Testing

Apache JMeter is a Java application used for various testing and performance evaluation. It is originally designed for web testing but has been extended to support other functions.

This post covers the basic steps of setting JMeter for HTTP testing on Ubuntu.

0. Install JMeter

At Linux Ubuntu, type the command below,

$ sudo apt-get install jmeter

Alternatively, one can go the official website to download the latest release. Once installed, start JMeter by command jmeter.

1. Right click Test Plan > Add > Thread Group. The Thread Group has several fields to allow us adjusting the testing behavior.

Figure 1. Thread Group

2. Right click Thread Group > Add > Config Element > HTTP Request Defaults. The settings in this configuration element will be applied to all HTTP requests we created later. We simply set the server name and leave the rest of the settings as default. This is shown as below,

Figure 2. HTTP Request Defaults

3. Right click Thread Group > Add > Sampler > HTTP Request. We leave all settings unchanged except for the path. We set path to the url we want to test, in this case, we want to test the YouTube About page. This is shown as below,

Figure 3. HTTP Request

We also add another HTTP Request to test Press&Blogs page.

4. Right click Thead Group > Add > Listener > View Results in Table. This will allow us to view all the testing results.

5. Save the test plan and run the test by clicking run > start.

6. Below is the testing results.

Figure 4. View Results in Table

7. We can add other listeners like View results in Tree, Graph Results etc. Each listener allows us to see the results in a different way.

References:
1. Apache JMeter website: http://jmeter.apache.org/

Specify the Main Class in a jar File for Hadoop

Hadoop support execution of jar file. For an executable jar file in normal java execution, one can specify the main class in the command line, as covered in my previous post: switch between main classes in a jar file.

However, the rules are a bit different for executable jar file running with hadoop. Basically the following rules hold (I tested on Hadoop 1.0.3),

  1. If a jar file contains a main class specified in its manifest file, hadoop will take the main class even if the command specify another main class. This is different from normal java execution where we can specify a main class to overwrite the one in the manifest file.
  2. If a jar file does not contain a main class in manifest file, hadoop allows us to specify the main class.

At eclipse, when one export a project as runnable jar file, it always ask for a main class at Launch configuration, shown as below,

tar

The main class selected will be put in the manifest file. Below is the content of the META-INF/MANIFEST.MF file in my helloworld project where the main class is set to HelloWorld.

Manifest-Version: 1.0
Class-Path: .
Main-Class: hello.HelloWorld

One can browse the jar file using a file extractor, open the manifest file using a file editor, and simply delete the last line to remove the main class configuration, and save the changes to the jar file when prompted. This will create a runnable jar file without main class.

The modified jar file can then be used in Hadoop with user supplied main class configuration, as shown in the sample command below,

$ hadoop jar hello.jar hello.HelloWorld

Video Converter Android > 500,000 Downloads

Video converter android has reached 500,000 downloads today. My previous estimation was that it will reach this milestone next month, but recently the daily download count has been stable around 5,000, that’s 2,000 more than the previous daily download count. So here we are, at 500, 269 total downloads on Jul 26 2012.

Video converter android is firstly released on Dec 17 2011. It takes less than 8 months to reach 500,000. I will continue to polish the app, make it more user friendly, and provide more features.

Below is screenshot of the statistics shown on my Google play developer console,

Figure 1. video converter android total downloads and active installs

And here is the recent daily download count jump from ~3000 to ~5000,

increase

Figure 2. the daily download count jump

I’ll write a separate post to explain why this jump occurs.

Cloud based Distributed Multimedia Processing

Multimedia processing is computing resources intensive, especially when it comes to videos. I will just use video conversion as an example.

People watch videos on mobile phones and most phones only play videos in specific format. Video conversion on mobile phone (e.g. converting video on android) can take a long time to finish when a video size is big, the phone is soon heat up, and the battery is drained quickly.

Can you do cloud-based video conversion? The idea is use distributed systems on the cloud to convert a user uploaded quickly and then allows users to download the converted video.

Let’s look at the conversion first. If we can divide a video into multiple chunks, convert them in parallel using distributed framework like Hadoop, and combine the converted chunks together. I guess this can be very fast regardless the video size, since we can use more nodes/computing resources to convert bigger videos.

But how about upload and download? It takes time and bandwidth. For upload, can we start converting while uploading? For download, can we combine the conversion with video streaming service? It allows users to watch the converted part while the conversion is in progress and save/combine the video at the same time. In addition, bandwidth is going to be faster and faster as time goes on.

An interesting tech problem in this idea is that how can we develop the video conversion tool as a map reduce job?

How to Import a Maven Project into Eclipse

Project managed by Maven cannot be imported to Eclipse by default. However the conversion step is really simple.

1. Convert the project

Issue the command below at the project directory where pom.xml is found.

$ mvn eclipse:eclipse

This will generate the eclipse files, including .project and .classpath.

2. Add maven local repository to classpath at eclipse.

  • At Eclipse IDE, Select Window > Preferences
  • Select Java > Build Path > Classpath Variables
  • Click New button > defined a new variable named M2_REPO that pointing to your local Maven repository. For Linux, it will be ~/.m2/repository by default; for windows, it will be C:Documents and Settingsusername.m2repository by default.
  • Click OK all the way to finish the setting.

3. Eclipse>File>Import>Existing Projects into Workspace, just like you do for a typical Eclipse project.

Switch Between Main Classes in a Jar File

Many developers used to write a main function for every class that can be tested separately. When we export it as a jar file, we usually specify a main class for execution of the jar file. Running this jar file is straightforward,

$ java -jar test.jar

But from time to time, some bugs are found at the system and we want to test a specific class. In other words, we want to run the main function in a class that is not set as main class.

1. Export the Jar File

At Eclipse, when exporting a runnable jar file, we’re given option to choose the main class at Launch configuration and three options of how to handle the libraries, as shown below,

In my project, I have used a library org.*.* and my package is like study.*.*

  • The first option unpack the libraries. The content of the jar file will contain the following

  • The second option put the libraries (probably in jar file) into the jar file. The content is like below,

  • The third option export the library to a separate folder, my exported jar file is named as test4.jar, and the library folder is named test4_lib

Interested readers can examine the META-INF/MANIFEST.MF to see how the library path and main class path is declared.

I figured out how to switch main class for the first and third option, but not the second. If anyone figured out how it works, please leave a comment. Thank you in advance for that!

2. Switch the Main Classes

At export, we can only specify one main class at launch configuration, say Main0. Now suppose we have another main class called Main1, and we want to run it. Then we’ll need to specify the classpath and the full path to the class.

$ java -classpath test.jar study.test.Main1

Note that we can even execute multiple main classes simultaneously in a single jar. For example,

$ java -classpath test.jar study.test.Main0 && java -classpath test.jar study.test.Main1

This will spawn two JVM to run the two processes.

How to Install Apache Cassandra on Ubuntu Linux

Apache Cassandra is a column oriented distributed database system. This post covers how to install Apache Cassandra on a single machine for development purpose.

Download
0. Make sure Java JDK 1.6 or higher is installed and configured properly.
1. Download Apache Cassandra from its website at http://cassandra.apache.org/download/.
2. Extract the downloaded file using the command below,

tar xvf apache-cassandra-x.x.x-bin.tar.gz

3. Append the following lines to ~/.bashrc

export PATH=$PATH:<path to cassandra directory>/bin

As an example, in my machine it is like,

export PATH=$PATH:/home/jicheng/Downloads/apache-cassandra-1.1.2/bin

Configure
1. The configuration files are located at conf folder of the cassandra root directory.

  • cassandra.yaml: cassandra storage configuration file
  • log4j-tools.properties: log configuration file
  • log4j-server.properties: cassandra server log configuration file
  • cassandra-evn.sh: cassandra running environment configuration file
  • commitlog_archiving.properties: commitlog archiving configuration file
  • cassandra-topology.properties: topology configuration file

2. Make sure the paths exist for data_file_directory, commitlog_directory and saved_caches_directory. If not, create the directories. In cassandra version 1.1.2, the path is /var/lib/cassandra, then we can use the command below to create the directory.

$ sudo mkdir -p /var/lib/cassandra
$ sudo chown -R `whoami` /var/lib/cassandra

3. Also ensure the log directory exist. The default log directory is /var/log/cassandra. Use the command below to create it.

$ sudo mkdir -p /var/log/cassandra
$ sudo chown -R `whoami` /var/log/cassandra

Run
Since we have set the path for cassandra, we can start a new terminal, and type the following command to run it:

$ cassandra –f

-f means to run cassandra in foreground mode, so we can stop it by pressing Ctrl+C.

Connect and Play with Cassandra-cli
Cassandra comes with a command line interface tool called cassandra-cli. We can connect to the Cassandra and play with it using cassandra-cli.

Figure 1.Connect to Cassandra with cassandra-cli

Video Converter Android 1.1.0 Released

If you simply want to download the app, go to go to https://play.google.com/store/apps/details?id=roman10.media.converter

It has been two and a half months since Video Converter Android 1.0.0 is released. For all the users who have sent feedback and suggestions to me, I apologize for the delay and I hope the new version can resolve issues by some of you.

The highlights of the new version are as below,

  • Reduce size profile: user specify a target video size, the converter will try to convert to about the size. It will even convert to multiple output files if a single file of target size is too small for the video.
  • Keep quality profile: lots of users feedback to me that the quality of conversion sucks. This profile allows you to keep the original quality.
  • Arbitrary resolution: previously users can only select from a few resolutions. Now you can enter the height and width as whatever value you want. Unreasonable values will fail.
  • UI polish: lots of UI improvement has been made. Below is a screenshot of the new UI. Hope you like it.

1_1_0_converting1_1_0_quality1_1_0_size

Figure 1. Screenshots of Video Converter Android 1.1.0

I’ll continue to work on the app. The features expected are as the following,

  • Extract audio and save as mp3 function
  • Extract video clip from original video
  • Extract a frame from original video

As always, bug reports, suggestions and feedback are appreciated! Please leave comments below.

For downloading the app, go to https://play.google.com/store/apps/details?id=roman10.media.converter

Side Note: Video Converter Android has been downloaded for more than 400,000 times and it’s expected to surpass 500K next month.