A couple of tips you may find handy if you're doing some performance tuning/troubleshooting on Java/Linux:
- invest in decent APM for collecting trend data and mining it (A customer used New Relic for example)
- strace, sar, vmstat, iostat, netstat and top are your friends, among others
- A couple of good tips for tuning the Linux network stack
- JDK 7 has a nifty tool called Mission Control ported over from JRockit
- BTrace is an AOP oriented runtime analysis tool
- Gatling is a pretty good stress tool
- set up a process to measure, research issues/fix them and iterate over and over until you know what the limits are.
- your biggest gains are going to come from eliminating inefficiencies in code Throw all preconcieved notions out the door.
You really need decent tools, profilers, etc to be able to analyze, measure and hone in on what might be causing problems. This becomes especially true when you're dealing with enterprise software which might involve scaling multiple nodes out horizontally or vertically and cluster/system level interactions.