Friday, January 22, 2010

Remote Profiling of JBoss using VisualVM

I ran into an issue while trying to setup profiling (using VisualVM) of a remote JBoss server instance. It turns out that the default steps I have found on the internet, assume that your target server (Where JBoss is running on) is accessible via the hostname not only by IP address. Well, just in case you live in an environment where you can access your server via IP but not via hostname, this blog post may hopefully save you an hour googling the solution. In order to setup remote profiling I did:

To my run.conf file (JBoss/bin) I added:

JAVA_OPTS="$JAVA_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n"
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.port=6789"
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.ssl=false"
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.authenticate=false"
JAVA_OPTS="$JAVA_OPTS -Djava.rmi.server.hostname=IP_ADDRESS_OF_MY_JBOSS"

Particularly, the last line was very important. Without it I was able to telnet to port 6789, thinking everything is cool, however I was unable to connect to it via VisualVM. This missing line caused a few frustrations.

Anyway, once these lines were added, you need to configure jstatd. I created a policy file (I called it tools.policy) for the JVM containing:

grant {
  permission java.security.AllPermission;
};

Then, I would be able to start up jstatd:

jstatd -p 1099 -J-Djava.security.policy=tools.policy

And finally, I was able to startup JBoss and connect to my server from VisualVM.

5 comments:

Argonist said...

Thank you for blog. I followed your blog, but it dont be working. Maybe have you any idea?

jboss starts up: java -Dprogram.name=run.sh -Xms1024m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseTLAB -XX:TLABSize=64K -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=10.8.1.5 -Djava.rmi.server.useLocalHostname=true -Djava.net.preferIPv4Stack=true -Djava.endorsed.dirs=/home/testjboss/jboss-5.1.0.GA/lib/endorsed -classpath /home/testjboss/jboss-5.1.0.GA/bin/run.jar org.jboss.Main -c stage -b 10.8.1.5

jstatd is starting also....

I tried to connect to it via VisualVM, but there is a error message from VisualVM:
"Cannot connect to 10.8.1.5:9999 using service:jmx:jmi:///jndi/rmi://10.8.1.5:9999/jmxrmi"

Logfile from VisualVM:
INFO [null]: Total physical memory 4.137.959.424
Warning: use of system property netbeans.home in com.sun.tools.visualvm.modules.appui.AboutAction has been obsoleted in favor of InstalledFileLocator
INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi:///jndi/rmi://10.8.1.5:9999/jmxrmi)
java.io.EOFException: SSL peer shut down incorrectly
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789)
Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:808)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1120)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:623)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)

..
...


Thanks

Anonymous said...

Argonist ,I have the same problem. have you found what was wrong ???

Anonymous said...

Thank you very much. You helped my pain go away :-)

Did you find out why this is necessary - I only have on network interface anyway?

/René

Anonymous said...

can i profile my application running on jboss usingf visualvm ???? pleas help

Anonymous said...

Thank you Gunnar !!!!!!!!!!!!