No Fluff Just Stuff Symposium in Atlanta

This was my first time presenting at an NFJS event in Atlanta, and you can be certain I’ll be back. Earlier in the Summer, I had the opportunity to address the Atlanta Java Users Group (AJUG) and show off the flexibility of Git. For the NFJS Symposium, I talked about Hadoop, Git, Encryption on the JVM and Open Source Debugging, which are my “fun but informative talks” lineup for 2010.

Colorado Springs Open Source User Group

Gary Hessler runs a great user group in Denver’s sister city to the south. I had the opportunity to present Git, one of my favorite topics since it is accessible no matter what language you program in and what platform you use. My NFJS colleague, Tim Berglund presented his always well-received Decision Making talk. It is an interesting divergence from the typical programmer presentation and gives you techniques to deal with team dynamics.

JavaZone in Norway

The ever-excellent JavaBin User Group in Norway put on a stellar conference called JavaZone for 2000+ people. What a show! The diversity of talk formats, speakers, and topics is simply incredible. After the conference, about 25 of us experienced a once-in-a-lifetime event of hiking in the Norwegian countryside. The otherworldly photos and hike details will make you want to attend next year!

The sessions are recorded at JavaZone and two of mine are available online. The former was in the big room in a formal setting. The latter talk was in a smaller room at the end of the conference and had a more informal feel where questions could be asked of the audience and vice-versa.
Encryption Boot Camp on the JVM

Hadoop: Divide and Conquer Gigantic Datasets

StrangeLoop in St. Louis Missouri

After returning from JavaZone, I headed to Missouri for the super-technical event named StrangeLoop brewed by Alex Miller. The lineup of speakers was stellar and the non-profit atmosphere was very relaxing. Hilary Mason, Guy Steele, Yehuda Katz, Josh Bloch, Doug Crockford, and many others. I will be attending next year if the timing works out again.

No Fluff Just Stuff Symposium in Minneapolis

My home base of conferences, NFJS, brought me up to the always warm audiences of Minneapolis. This has to be one of my top-5 favorite stops on the tour due to the beautiful hotel and the technologically advanced attendees. Their questions are deep and I try to be as prepared as possible for them. I had the privilege of Brian Sletten and David Hussman sitting in my classes about Hadoop.

SpringOne

The sprint of conferences concluded with my attendance and helping out with the logistics of SpringOne2GX in Chicago. The hotel was spectacular and the attendance overwhelming. It seemed to be just shy of a 1000 people and double last year’s attendance. I attended some great sessions and had hallway chats with the likes of Hamlet D`Arcy, Hans Dockter, Paul King, Andres Almiray (our co-incidence rate at conferences is getting to be uncanny!), Peter Bell, and Chris Beams. I’ll try to put in some abstracts for next year and get invited to speak!

The news keeps pouring in day after day and week after week of significant company-damaging data breaches. No wonder; Only 23% of companies surveyed in a recent poll indicated that data encryption was even a priority. We should not be having, reading about and reacting to most of these. Such events are typically quite preventable. The JVM platform has some of the strongest and simplest support for encryption of any programming language. An easy to use Java API for encryption, as well as several high level open source libraries, are at developers’ beck and call. I’ve embarked on a mission this year to educate as many developers as possible about the basic vocabulary of encryption, the history of how these algorithms and techniques came about, and how to effectively implement the right use of encryption for business applications.

The Solution in the form of Education

I first brought this topic of Encryption on the JVM to user groups in my home town of Denver, Colorado, USA. It received a more than warm welcome and deep after-talk discussions. Next, I took this topic on the road with the No Fluff Just Stuff symposium series in the USA. Next, I’m excited to get to share these same concepts with audiences in Sweden at the always-cutting-edge Øredev conference, shortly followed by the equally esteemed Devoxx in Belgium.

Acting on the Need for Encryption

Discard the notion that encryption is too hard to learn. Embrace that encryption is quickly becoming a necessary skill of sought-after developers, the world over. This talk will get you up to speed and send you on your way to making your applications more secure, leverage encryption properly, and protect your valuable customer data from prying eyes. No longer just a notable stretch goal, this is the new responsible level of application engineering. I hope to see you at one of these exciting events!

The RefCard is intended to be a jumping off point and provides information about and links to the frameworks that now support GAE as well as external resources to help you dive deep if this new arena of cloud computing. Grab a copy and send me feedback.

<Unknown Application> (pid ###)

I have been successful in working around this by renaming the hsperfdata temp directory as noted in a separate thread on the Sun forums.

First, I would start a simple helloworld Java application.

java com.ambientideas.HelloWorldJava
Hello Java World!

Then I would get the PID:

jps
11554 HelloWorldJava
11573 Jps

Then I tried to jmap it, but got the following error

jmap -heap 11554
Attaching to process ID 11791, please wait...
attach: task_for_pid(11791) failed (5)
Error attaching to process: Error attaching to process, or no such process

So I thought, “These memory tools often require root access, so I’ll sudo it.” When I did, I got the following equally cryptic error:

sudo jmap -heap 11791
Attaching to process ID 11791, please wait...
Exception in thread main java.lang.RuntimeException: gHotSpotVMTypes was not initialized properly in the remote process; can not continue
at sun.jvm.hotspot.HotSpotTypeDataBase.readVMTypes(HotSpotTypeDataBase.java:111)
at sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:68)
at sun.jvm.hotspot.MacOSXTypeDataBase.<init>(MacOSXTypeDataBase.java:35)
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:560)
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:481)
at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:319)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:146)
at sun.jvm.hotspot.tools.JMap.main(JMap.java:128)

I though, “Hmmmm, this looks a lot like a java version mismatch. Let’s check which java version I’m running.” I had javac compiled the HelloWorld against Java 6, so was expecting a matching java. I made a quick stop into the Java Preferences pane. Yup. Java 6 set as the first-priority choice.

Let’s check the command line too…

java -version
java version 1.6.0_13
Java(TM) SE Runtime Environment (build 1.6.0_13-b03-211)
Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02-83, mixed mode)

Excellent. Just what I expected. Java 6. But a little doubt crept into my mind. What if the sudo’ed invocation were finding a different Java. That didn’t sound plausible, but yet was still worth checking out.

sudo java -version
java version 1.5.0_19
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_19-b02-304)
Java HotSpot(TM) Client VM (build 1.5.0_19-137, mixed mode, sharing)

Aha! There’s the culprit. So somehow, sudo’ed commands are running Java 5, even though I’ve set a preference for 6 in my profile. I searched the web high and low for a way to specify the Java version for “sudo” specifically, but could find no trace. I even tried the -E flag to maintain the caller’s environment variables, as is possible on Linux distributions. Not implemented on Mac.

I did a quick check of my path and a “which java”. Both ultimately routed to Java 6. I also had JAVA_HOME and JAVA_VERSION set just fine.

set | grep JAVA
JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
JAVA_VERSION=1.6

echo $PATH
/Applications/Dev/btrace/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:
/usr/X11/bin:./:
/Users/mccm06/scripts:/Applications/Dev/apache-ant/bin:
/Applications/Dev/apache-maven/bin

So I turned to harness the wisdom of one of the really smart guys (that uses a Mac) I’m privileged to hang out with in Denver, Fred Jean. Fred said, “Try adding the 1.6 java explicitly to the front of your path rather than letting Java Preferences pane be the sole controller.” I opened up ~/.bash_profile (alternatively you could edit ~/.bash_rc)and slammed in the following:

export PATH=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Commands:$PATH

I opened a new terminal to pick up the shell changes. Then I started checking if this fixed the issue in reverse order, starting with the sudo:

sudo java -version
java version 1.6.0_13
Java(TM) SE Runtime Environment (build 1.6.0_13-b03-211)
Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02-83, mixed mode)

Looking good! Now let’s get back to our core goal, jmap.

sudo jmap -heap 5951
Attaching to process ID 5951, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 11.3-b02-83
using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC
Heap Configuration:
    MinHeapFreeRatio = 40
    MaxHeapFreeRatio = 70
    MaxHeapSize = 88080384 (84.0MB)
    NewSize = 21757952 (20.75MB)
    MaxNewSize = 43581440 (41.5625MB)
    OldSize = 64159744 (61.1875MB)
    NewRatio = 7
    SurvivorRatio = 6
    PermSize = 21757952 (20.75MB)
    MaxPermSize = 88080384 (84.0MB)
Heap Usage:
New Generation (Eden + 1 Survivor Space):
    capacity = 19070976 (18.1875MB)
    used = 2351616 (2.24267578125MB)
    free = 16719360 (15.94482421875MB)
    12.330863402061855% used
Eden Space:
    capacity = 16384000 (15.625MB)
    used = 2351616 (2.24267578125MB)
    free = 14032384 (13.38232421875MB)
    14.353125% used
From Space:
    capacity = 2686976 (2.5625MB)
    used = 0 (0.0MB)
    free = 2686976 (2.5625MB)
    0.0% used
To Space:
    capacity = 2686976 (2.5625MB)
    used = 0 (0.0MB)
    free = 2686976 (2.5625MB)
    0.0% used
concurrent mark-sweep generation:
    capacity = 64159744 (61.1875MB)
    used = 0 (0.0MB)
    free = 64159744 (61.1875MB)
    0.0% used
Perm Generation:
    capacity = 21757952 (20.75MB)
    used = 4716880 (4.4983673095703125MB)
    free = 17041072 (16.251632690429688MB)
    21.678878600338855% used

Thanks to a combination of Ken Sipe, Fred Jean, a few good guesses and some investigative work, the problem is solved! I’m longwindedly calling this the “java version is different for sudo-initiated processes on Mac” so that the next person Googling for this using similar keywords will hopefully find this tutorial.

As a closing note, I often get asked how I set my environment variables for Java so that Spotlight-launched, Quicksilver-launched, Dock-launched, and Terminal-launched processes can all see the variables (yes, there’s some odd pitfalls about that on a Mac due to the process parents being different). The short answer is to create a text file called /private/etc/launchd.conf and put the following (applicable portions, based on your tool usage) contents in it:

setenv JAVA_VERSION 1.6
# A) Some process-launchers will cause their child to default to 1.5 if using /Library/Java/Home
# even if Java Preferences pane says 1.6 at the top
# setenv JAVA_HOME /Library/Java/Home
# B) Alternate: Force Java 1.6 as Java Home
setenv JAVA_HOME /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
setenv GROOVY_HOME /Applications/Dev/groovy
setenv GRAILS_HOME /Applications/Dev/grails
setenv NEXUS_HOME /Applications/Dev/nexus/nexus-webapp
setenv JRUBY_HOME /Applications/Dev/jruby
# Set up Ant
setenv ANT_HOME /Applications/Dev/apache-ant
setenv ANT_OPTS -Xmx512M
# Set up Maven
setenv MAVEN_OPTS -Xmx1024M
setenv M2_HOME /Applications/Dev/apache-maven

As we have learned the hard way, the only pitfall is that the java version specified in launchd.conf is not honored by sudo’ed java invocations.

Drop me an email if you’ve seen this issue and this tutorial helps you solve it.

The timing was great, as I did a Mastering Maven talk at the San Diego Java Users Group (SDJUG) on Tuesday night. A handful of the attendees came with printed copies of the RefCard in hand!

I’m always looking for ways to contribute to the Maven community, and this was by far the most fun I’ve had building materials to promote this unique Convention over Configuration build tool.

Jason van Zyl on Maven 3 from Sonatype on Vimeo.

Here are the takeaway points, highly distilled to 140 proof, for those without time to watch the video:

• Improved overarching performance (tools, engine, resolution, downloads, builds)
• Better tooling integration
• Tie-ins with OSGi
• Excellence in Eclipse integration
• Possibility of non-XML POMs
• Lifecycle extension points
• High-performance artifact resolution engine
• Documentation-hyperlinked error messages

If you want to get access to the Maven 3.0 code, you can view the list of SVN checkout URLs here (or here for the hardcore Git lovers), or just view the source in your browser here. If you want early access to the binaries, you can download them from Apache.

Just pass the useFile=false parameter on the command line or set it in the plugin config section of your pom.xml.

mvn test -Dsurefire.useFile=false

Before:

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.ambientideas.AppTest
Hello World! This is a JUnit test!
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec <<< FAILURE!Results :Failed tests:testApp(com.ambientideas.AppTest)Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

After:

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.ambientideas.AppTest
Hello World! This is a JUnit test!
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec <<< FAILURE!
testApp(com.ambientideas.AppTest)  Time elapsed: 0.014 sec  <<< FAILURE!
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at com.ambientideas.AppTest.testApp(AppTest.java:37)

Results :Failed tests:testApp(com.ambientideas.AppTest)Tests run: 1, Failures: 1, Errors: 0, Skipped: 0