Writing a client/server application communicating over sockets is pretty easy in Java. The server creates an instance of the aptly named ServerSocket class, which waits for incoming connections. The client creates a Socket instance, which then connects to the server (thus creating another Socket on the server side via the accept method of the ServerSocket), and that’s it. Although Sockets are normally blocking read operations, it is easy to emulate a single threaded non-blocking communication with simply checking the available bytes on the socket, and reading only that much.

Blocking secure socket communication, the problem

Switching to a secure socket connection is very easy: First, you need to create some keystores, then some initialization in the code and then you can use some factory classes to create the secure sockets instead of the original constructor calls. Everything else is almost the same.. except..

Getting the number of available bytes are not reliable any more. The reason is simple: As your security and application data arrives in the same stream, the security engine is not able to tell the amount of available application bytes without trying to decrypt everything, but that is a complex operation. Probably more complex and requires more effort than you would like to have in an infinite loop.

Because of this, our workaround for blocking communication is not usable anymore. For a simple application with just one or a handful of clients it is acceptable to have a separate thread for all of the connected clients on the server side, but we need another solution for high user counts.

SocketChannels

Java NIO has a nonblocking socket solution. SocketChannels are designed for this. Because of the fundamentally different usage and the additional classes, such as Selectors, most people never uses them, but they are not overly complex, and with a little practice can be used easily. For beginners, I’d recommend reading this article: Rox Java NIO Tutorial. It’s probably not the easiest to understand, but contains very useful tips and explains typical pitfalls.

Using this, we solve the issue with the blocking channels, but security is still lacking. The bad news is: It is pretty hard to make SocketChannels secure. This is your second chance to turn back, and use the blocking solution, they say a few thousand threads are not really a problem for a good server

Securing SocketChannels

Ok, you are desperate. You want ‘em non-blocking channels communicate securely. Here is what you need to add to the code:

1. You don’t need to do anything with the ServerSocketChannel, and the way it accepts connections.
2. I prefer doing the reading and writing in a separate class, which “owns” the connection and the SSLEngine. (See Below)
3. Initialize your security.
4. On the server side, for every new socketchannel, create a new SSLEngine.
5. On the client side, do the same for your one channel.
6. Initiate the handshake.
7. After the handshake, wrap everything you want to send and unwrap everything that you receive on both sides.

So far it is not too complex, right? Let’s dig deeper.

Initializing security:

private void setupSecurity() {
	SecureRandom secureRandom = new SecureRandom();
	secureRandom.nextInt();

	KeyStore clientKeyStore = KeyStore.getInstance("JKS");
	clientKeyStore.load(new FileInputStream("client.jks"), "KeyStorePassword".toCharArray()); 

	TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509");
	tmf.init(clientKeyStore);

	KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
	kmf.init(clientKeyStore, "KeyInKeystorePassword".toCharArray());

	sslContext = SSLContext.getInstance("TLS");
	sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), secureRandom);
}

Couldn’t be more straightforward. Creating an SSLEngine is just as easy:

SSLEngine engine = sslContext.createSSLEngine();

And now the handshake.. You can find a lot of explanations here (it’s a bit dry, but very useful). The scaffolding for the code can be found in the same article, but you are better off if you check the code here: [jdk home]\sample\nio\server\ChannelIOSecure.java

It is very complicated, and there is a hidden “flaw”: It’s meant to be used with blocking channels. (See below). Also, some structures are a bit more complex than I’d usually like it, for example the 3 levels deep switch constructs. Anyway, for now you should just try to copy it as closely as you can, and if you can make it work, then you can improve the code as much as you want.

The problems I faced

Obviously, it didn’t work for me the first time I tried. Here are some tips:

• Don’t give up. It’s complex and ugly, but it can be done.
• If the handshake doesn’t start, and/or the server complains about plaintext connection, insert the engine.beginHandshake() line before the handshake process. Sometimes the only issue that you see is that the clients sends the messages and the server simply absorbs them without any apparent effect.
• If you get repeated buffer underflow errors on the server side, and using non-blocking communication, it’s time to add a new check for zero length reads. Trying to unwrap zero length ByteBuffers can cause this issue.
• In case of unsupported record version problems or sequence violation (during the handshake) try to empty your buffers before you read/unwrap into them and after you have sent them to the other party. Leftover bytes can cause this issue. (Unsupported record version Unknown) (Handshake message sequence violation)
• If the server/client complains about invalid handshake mac, check your flushing code. You may loose some important bytes. (bad handshake record MAC)
• Google is your friend: There are a lot of people out there who solved the same issues and wrote an article about it.
• It’s a good idea to add some logging to the code to see what is happening. It helps you to see the sequence of events on both sides.
• There are some issues and solutions here I didn’t face. Still, you may see them.

I hope this will help some of you to develop some great, secure applications.

A minap, amikor a javas Annotation-ökről olvastam, belefutottam az általam korábban még soha (tudatosan) nem használt apt (Annotation Processing Tool) nevű eszközbe. Ahogy olvasgattam a képességeiről, felmerült bennem, hogy ezzel meg lehetne oldani azt, amit én statikus DI-nak és statikus AOP-nek hívok.

Az apt lényege, hogy – főként az annotációkat használva, de generálisan is – fordítás előtt átszalad az egész kódon, és meghív bizonyos, a felhasználó által specifikált feldolgozó egységeket, amik átalakíthatják a kódot, vagy hozzáadhatnak új osztályokat, etc. Gyakorlatilag beteszünk egy extra lépést a kód elkészítése és lefordítása közé.
Valami ilyesmi a statikus AOP lényege: Bár kód szinten elszeparáljuk a különböző dolgokat, például a logolást, és pusztán egy annotationnel jelöljük, hogy ez a metódus kellene, hogy logoljon (@LoggingMethod, pl.), a lefordított osztályokban már megjelenik az a kód. Hasonlóan működik a DI is.

Például: Össze akarom gyűjteni az összes Command osztályt egy List -ba.
Csinálok két annotációt és egy statikus függvényt, mondjuk registerCommand(Command c). A függvény feladata mindössze annyi, hogy a fenti listába belerakja a Command osztályt (és csinál vele minden egyebet, amit csak akarunk), az annotációk pedig kb. így néznek ki:

@Target(ElementType.TYPE)
public @interface CollectionMarker {
    String collection();
    String enabled() default true; // erről meg ejtek pár szót
}

@Target(ElementType.METHOD)
public @interface CollectionMethod {
    String collection();
}

Az utóbbit elhelyezem a fenti függvényen, az előbbit pedig minden egyes Command osztályon, amit megírok.

Ezután a processzorok úgy működnek, hogy az egyik összegyűjt egy listát az összes CollectionMarkerrel megjelölt osztályról, a másik megjegyzi, hogy hol a CollectionMethod, majd a feldolgozás végén a megjegyzett osztályba berakunk egy static initializer blokkot, amibe belekerül egy csomó registerCommand(new AkarmilyenCommand());. S lőn, kész is van a chain.

Amit vesztünk ezzel, az
- A dinamikusság: Nem elég átírni egy XML-t és újraindítani az alkalmazást, mint a Spring-nél, ha változtatni akarunk valamit a dependencián. Mindenképpen újra kell fordítanunk az alkalmazást.
- Fordítási sebesség: A fordítás valamivel lassabb lesz
- Rugalmasság: Nyilvánvalóan van egy csomó olyan DI alkalmazás, amit nem lehet megoldani statikus módon.

Amit nyerünk vele:
- Némi performancia
- Átlátható kód extra frameworkok nélkül
- A Spring XML-hez viszonyítva: compile time ellenőrzés a dependenciákra
- Automatizált taszk végrehajtás (Az új command hozzáadása esetén csak az osztály kell létrehoznunk és ellátnunk a megfelelő annotációval, nem kell még különböző ponton beszúrni listákba)

Említettem fent, hogy az enabled attribútumról még ejtek pár szót: ha a gyűjtő egység figyeli ennek is az értékét, és nem gyűjti be azokat, ahol az enabled értéke false, akkor ennek a kapcsolónak az átbillentésével ki-be kapcsolhatjuk a gyűjtését. Akár még azt is megtehetjük, hogy a true-false helyett szinteket definiálunk, és bizonyos osztályokat csak debug módban használunk, productionben nem (ezeket, ha jól sejtem, akár ki is törölhetjük a feldolgozás során..).
Ez így magában még nem is annyira érdekes, de van még egy ennél is fontosabb szerepe: ha a fenti annotation definiciót (a Markert) megfejeljük egy @Inherited annotációval, akkor a leszármazottak is öröklik. Innentől elvileg minden leszármazottat begyűjtünk.. kivéve, ahol specifikusan kitesszük a markert a az enabled attribútum értékét false-ra állítva.

Akik az IDE integráció miatt aggódnak, azoknak sem kell igazából. Az Eclipse-t biztosan tudom, hogy támogatja ezt, a Netbeans és IDEA esetén pedig csak sejtem, hogy támogatják ezt a dolgot (vsz. 5 perc lenne kideríteni a google-lel).

Egyelőre csak a fejemben született meg ez a dolog, nem valósítottam meg, de a horizonton van. Ha lesz konkrétum, vsz kirakom valami elérhető helyre, mondjuk google code.

    static {
        boolean assertsEnabled = false;
        assert assertsEnabled = true; // Intentional side effect!!!
        if (!assertsEnabled)
            throw new RuntimeException("Asserts must be enabled!!!");
    }

Zsenialisan 1xu.

For GAE, Client Login works like this:
- Get the Auth string from “https://www.google.com/accounts/ClientLogin” with a post
- Using the Auth string, get the cookie from “http://myapp.appspot.com/_ah/login” using the abovementioned auth string as a get param, (and add a “continue” param too)
- Use the cookie for subsequent calls

When I tried the second part, I got this:

500 Server Error
Error: Server Error
The server encountered an error and could not complete your request.

If the problem persists, please report your problem and mention this error message and the query that caused it.

The solution:
Someone showed me this: SyncProxy.java

So I inlined the get parameters, like in the code above, or this:
HttpGet get = new HttpGet("http://yourapp.appspot.com/_ah/login?continue=" + URLEncoder.encode("http://yourapp.appspot.com/") + "&auth="+auth.substring(5));
(Where auth is the String from the first request, starting with: “Auth:”)
And it was working.

A few other possible pitfalls I’ve seen:
- If the first url is wrong (should be: https://www.google.com/accounts/ClientLogin) you will get cookies in the reply, a well known one is “PREF=ID=..”
- The information that you send in should be the body of the “post” call for the first request, and “get” parameters in the second.
- The Auth string from the first call comes back as content (in the body), not as header/cookie
- For the second call, it’s better to turn off redirect following
- You don’t need a CookieManager to capture the cookie. You can just look for “Set-Cookie” in the headers
- You can send the cookie as a “Cookie” header field, containing the same content.
- The expiry date of the cookie depends on your GAE app settings. The default is one day, you can change it to 1 or 2 weeks.

Let’s start with an example: You have a moderately large library. You have different sections, each section has some subsections and each subsection has a handful of books in it.
You want to make a catalogue from your books. Today is the day when you will process the books in the nature section, animals subsection. You want to get the name of the author, the title and the number of pages in each books.
You work hard, and in five hours, you are finished with all of the 81 books in that section.

Your manager is not satisfied with this, so he thinks for a while, and tell you to change the process. There is one additional step to make: you need to make copy of the whole book, collect the required data from it and then you trash it. He insists that you must do this, so you start working again. As an experiment, you create the same catalogue from the same 81 books.
How long did it take? Let’s see..

In programming terms, the first solution looks like this:

NodeList nl = (NodeList) bookPath.evaluate(doc, XPathConstants.NODESET);

        for(int i = 0;i<nl.getLength();i++) {

            Node n = nl.item(i);

            String str = (String)title.evaluate(n, XPathConstants.STRING);

            str = (String)author.evaluate(n, XPathConstants.STRING);

            str = (String)numberOfPages.evaluate(n, XPathConstants.STRING);

        }

Running this piece of code took 5406ms alltogether. 5 seconds seems to be a bit long, so let’s put in some copying:

        NodeList nl = (NodeList) bookPath.evaluate(doc, XPathConstants.NODESET);

        for(int i = 0;i<nl.getLength();i++) {

            Node n = nl.item(i);

            Node n2 = n.cloneNode(true);

            String str = (String)title.evaluate(n2, XPathConstants.STRING);

            str = (String)author.evaluate(n2, XPathConstants.STRING);

            str = (String)numberOfPages.evaluate(n2, XPathConstants.STRING);

        }

Running this piece (remember, there is one extra step in it for each and every one of the “book” nodes) took: 688ms. It is 8 times faster.

I’ve run this example on a 200kB XML using the built-in XPath library of Java 5 (which is to my best knowledge Xalan).

Something is definitely wrong here. I’ve tried to find out on Google what others think about this, and it seems that there are a lot of people who were running into this problem. I hope it will be fixed in the close future, because this workaround is not at all obvious.

Kezdek kicsit szétdarabolódni..

Nagyon örülök neki.