HDD / SSD Battle

The Problem

You know, the laptop I'm using for my daily work job is not the fastest one. In contrast, it's more than 5 years old and pretty slow. Yeah I know, hardware can never be fast enought, but it's really slow considering the things I have to work on.

For instance, we are using xtext modeling and hence usually have a couple of Eclipse instances running at the same time (outer & inner workbench), additionally to using m2eclipse to build the projects with Maven in Eclipse. Moreover, we have some quite big workspaces with tens of thousands of class files.

All of this is probably not unusual, but unfortunately too much for my poor old laptop. It takes minutes to start or end Eclipse, not to mention the times required for cleaning all projects. However, my company currently does not really like the idea to buy new laptops so we have to find ways to speed things up without spending too much money. I have blogged before about some ways to speeding up your system.

The Solution?

It's pretty clear that the bottleneck is the hard drive currently. We have proven this by some inspection tools, the drive is working hard all the time when executing some build, for instance. Now we managed to get a solid state drive (SSD) to test the performance improvements it would offer. Well, fasten your seatbelt...

We have measured some typical tasks with real data and projects on a developer's laptop – first with the built-in hard disk, then after installing the SSD and copying the harddrive content over. Note that we have tried to make a fair comparison, keeping the setup indentical in both scenarios. These are the results.

The Battle

Working With Eclipse:

• Start Eclipse 3.5.1 with an empty workspace until Welcome screen is displayed: 52 s → 12 s (factor 4.3)
• Start Eclipse with a medium-size workspace: 125 s → 30 s (factor 4.2)
• Clean all projects in that workspace: 445 s → 115 s (factor 3.9)
• Exit Eclipse and wait until workspace is saved: 28 s → 7 s (factor 4.0)

Working With Maven:

• Maven "clean install" in mdium-size project: 668 s → 336 s (factor 2.0)

Booting Windows:

• Turn on computer and wait for login screen: 62 s → 33 s (factor 1.9)
• After login, until Windows is ready (autostart applications are loaded): 135 s → 44 s (factor 3.1)

The Bottom Line

As you can see, the SSD is speeding up boot time by factor 2-3, which already is impressive. Maven build usually gets executed 2 times faster. Eclipse speed-up is even more, namely around factor 4. That's pretty cool! You really feel the performance difference!

Additionally, after some more weeks of testing, what we like most is that the whole system feels much more reactive; that is, when executing some big job like rebuild a huge workspace, you can switch context and nicely work in another instance of Eclipse, for instance – a single tasks is not blocking the whole system any more.

All in all, that's an incredible speed-up considering the prices of SSD! Now, go and tell your boss ;-)

Read More

Having Fun with Encoding!

Compiler Plugin

Recently, I edited our main company root POM to upgrade some plugins to new versions. Of course, we are following best practice to lock down the plugin version, so when a new version is available we only need to adjust the parent POM. Nearly all version updates were on the last build number digit, which is the z in x.y.z version string – so I didn't expect much difficulties.

However, for the compiler plugin, it was a jump from version 2.0.2 to 2.1, and indeed it turned out that some of the test cases failed compiling with strange encoding issues when using the new compiler plugin version.

Specify Encoding

We are following the suggestion to specify a POM property for source file encoding, for not being forced to configure encoding for all relevant plugins individually. Moreover, we were exactly using what's shown in the example:

<project>
  ...
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    ...
  </properties>
  ...
</project>

That is, we assumed our source files were UTF-8 encoded, which is the most widely used encoding for unicode characters. But, for some of the projects, that's actually not the case since we are using Eclipse with the default setting for text file encoding which is Cp1252 (Western European) on our german Windows.

Why didn't we ever notice that? Well, it happens that both the UTF-8 as well as Cp1252 encodings are backwards compatible with ASCII. We are coding most of the stuff in english (concerning package, class, method, attribute and parameter names, and even Javadoc comments), so the resulting byte stream will never be different for both encodings. However, some of the files used german umlauts in line comments which are exactly the files that can't be compiled any more with new compiler plugin version.

When looking at the debug output of compiler plugin 2.0.2 mojo configuration, you can see that the encoding is not explicitely set, probably meaning that the platform default encoding is used (which is again Cp1252 on all build machines):

[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile' -->
[DEBUG]   (f) basedir = ...
[DEBUG]   (f) buildDirectory = ...
[DEBUG]   (f) classpathElements = [...]
[DEBUG]   (f) compileSourceRoots = [...]
[DEBUG]   (f) compilerId = javac
[DEBUG]   (f) debug = true
[DEBUG]   (f) failOnError = true
[DEBUG]   (f) fork = false
[DEBUG]   (f) optimize = true
[DEBUG]   (f) outputDirectory = ...
[DEBUG]   (f) outputFileName = xxx-0.2.0-SNAPSHOT
[DEBUG]   (f) projectArtifact = xxx:jar:0.2.0-SNAPSHOT
[DEBUG]   (f) showDeprecation = false
[DEBUG]   (f) showWarnings = false
[DEBUG]   (f) source = 1.6
[DEBUG]   (f) staleMillis = 0
[DEBUG]   (f) target = 1.6
[DEBUG]   (f) verbose = false
[DEBUG] -- end configuration --

The new version 2.1 of compiler plugin is now considering what has been configured in project.build.sourceEncoding property, and hence tries to compile the Cp1252 coded source file with UTF-8 encoding which doesn't work when umlauts are used.

Specify Correct Encoding

Of course, the solution is to specify the correct encoding in project.build.sourceEncoding property, matching the encoding that is used in the development environment when writing the source files.

Oh, yes, Cp1252 is quite similar to ISO 8859-1 encoding (only some special characters on positions 0x80–0x9F are different which we don't use), so in fact we are using ISO 8859-1 now to allow builds on non-Windows platforms as well.

Certainly, it would be nice if the plugins had a history on their site where you can find this type of changes for new versions, without having to search in the Jira...

Read More

Speeding Up Your System

We did a lot of interesting stuff lately, including upgrading Eclipse to the new Galileo release, and in turn upgrading oAW (openArchitectureware) to the new versions of Xtext, Xpand, Xcheck etc. which are now part of the Eclipse Modeling project. I will blog about all this later...

But, what really started to hurt us was the performance of our Windows XP based development laptops. They aren't really brand new ones, but not that old either. Nevertheless, they seem to have an issue with all those java, class and jar files involved when starting Eclipse, doing a "Clean Project", when using Maven to build the software etc.

IT department was not willing to provide us with new hardware (disk especially) at this time, and no, using Linux is no option either. Hence, we had to find out other areas of improvement by tuning our system.

Here is what we did to speed things up for Windows XP. Note that things might or might not be different for Windows Vista or Windows 7.

1. Disable Indexing Service

By default, there is a Microsoft "Indexing Service" running in your Windows system. According to msdn, this is "a base service for Microsoft Windows 2000 or later that extracts content from files and constructs an indexed catalog to facilitate efficient and rapid searching."

Well, to be honest, I have never heard of that service before (and rarely use the search function of Windows), but it turned out to cause lots of harddisk traffic. So we decided to disable this service, which is recommended by some people.

Actually, there are several ways to do so (wihtout using Microsoft Management Console (MMC) with an appropriate snap-in):

• Disable the Indexing Service in your list of local services.
• In the Properties window of your local disk, remove option "Allow Indexing Service to index this disk for fast file searching".
• Remove the function via the Control Panel > Add or Remove Programs > Add/Remove Windows Components > Uncheck "Indexing Service".

2. Don't Virus-Scan Java Stuff

Our anti-virus tool was configured to scan literally everything, including java, class and jar files. This seems to be exaggerated from security point of view, but IT folks said they could not configure the anti-virus to ignore a particular set of file extensions (like *.jar).

Hence, what we did instead was to exclude two folders from being scanned on the fly: javatools (where everything Java related is put, including Eclipse releases) and projects (where all our projects reside). Instead, these folders are now scanned once a week, which is acceptable for IT guys.

Know what? That speeded up starting time of Eclipse by factor 5!

Surprisingly, excluding the Maven local repository (located in your personal settings folder) from being scanned on the fly did not make much difference, so we didn't handle this folder any special.

3. Optimize Subversion

Are you using Subversion? If so, are you using TortoiseSVN, the Windows Shell Extension for Subversion? Well, in that case you will know and probably like the little overlay icons that are used to indicate the state of files and folders. This feature is recursive, whereby overlay changes in lower level folders are propagated up through the folder hierarchy so that you don’t forget about changes you made deep in the tree.

Starting with release 1.2, a new TSVNCache program is used to maintain a cache of your working copy status, providing much faster access to this information. Not only does this prevent explorer from blocking while acquiring status, but it also makes recursive overlays workable.

This is all nice, but there is a major drawback: TSVNCache by default looks for changes on all drives and in all folders, killing disk performance with all the I/O it's doing.

You can enable a TSVNCacheWindow showing all the folders being crawled by TSVNCache. To do so, open the Registry Editor, and create a new DWORD at HKEY_CURRENT_USER\Software\TortoiseSVN\CacheTrayIcon with value of 1. After that you have to restart TSVNCache which is easiest done by just killing the process, it will be automatically restarted when you do any TortoiseSVN operation. Now there should be a small tortoise icon in Windows tray area which opens the TSVNCacheWindow. Watch how TortoiseSVN scans files and folders whenever you write to a file...

Time to fix that! That should be quite easy if you're keeping all of your working copies below one specific folder (or a small set of folders), like we do. All you have to do is to setup TortoiseSVN to only scan your sourcefolder paths:

1. Right-click in Explorer on any folder and select "TortoiseSVN > Settings...".
2. In the Settings window's tree, click on the "Icon Overlays" entry.
3. In the "Exclude Paths" input field, put C:\* to exclude the entire C drive. If you have more drives, exclude them all at the top level. Use newlines to separate the values.
4. In the "Include Paths" input field, list all of the locations where your working copies are stored, again separated by newlines.
5. Switch off "Network drives" option in "Drive Types" area.

All in all, your settings should now look like in the following screenshot. Thanks to Paraesthesia for this nice tip!

Epilogue

You won't believe what difference these three little tunings made to our system performance. The harddisk is not any more busy all the time, applications (like Eclipse) are starting much faster, and build time has decreased drastically. Not bad for not spending anything on new hardware! Well, next thing we will check is what effect a new (big, fast) hard disk will have... ;o)

Read More