JNA love NXT

I started working on something to bring some Ruby love to my Lego Mindstorm NXT. I know, there is already one or two projects that aims to do that, but the first one is dormant and the other one is at a fairly early stage.

The first thing that needs to be done before accomplishing anything useful on the NXT, is to connect to it via USB or (preferably) Bluetooth. This can be done in two ways : use the Lego Fantom API or use direct communication via a serial link.

Communicating via a serial link from Ruby involves using the ruby-serialport gem which seems to be no longer maintained. The other options to work with the serial port (besides using platform specific stuff) would be to use JRuby and the Java Communication API. But Java Comm only support Solaris and Linux. There is also RXTX but it is not that simple to get started with. If serial port communication was the only option, I would probably choose ruby-serialport to get things done (which ruby-nxt is using).

However, there is the option of using the Lego Fantom API. This API comes as a shared library that gets installed whenever you install the Lego Mindstorm software. It is available for Windows and Mac OS. This API is used by the Lego Mindstorm software so I assume it is well tested and will be maintained as long as there is a market for the NXT. It has the added benefit to completely abstract the actual communication link between the PC and the NXT. This mean that you no longer have to choose to work with USB or Bluetooth, the same code will work with both. While I would have likes to see the Fantom API available for Linux, this is not a big showstopper for me as I guess that most Mindstorm users use either Windows or Mac OS.

To use the Fantom API from Ruby, I need a way to map shared library functions to Ruby. This can be done using Ruby DLX, Ruby/DL or ruby-dl2. Again, none of these projects seems to be active at the moment. On the other hand, in Java land, there is JNA. Which is currently gathering some buzz and is being actively maintained. JNA allows to map shared library functions to Java interfaces is a really easy and concise way. Even if I want to ultimately use Ruby to talk to my robot, I don't care if my Ruby code must run under JRuby. So JNA is a good fit for me to get quickly started and not care to much about the low level communication stuff needed to connect to the NXT.

It is now time to start playing around and send some commands to my Lego robot.

So the first thing that need to be done, is to create the Java interface that will be used to map the Fantom API. Here is a minimalist version of it (it just contains the API function needed for this example):

Fantom.java

package treelaws.fantom;

import com.sun.jna.Native;
import com.sun.jna.Pointer;
import com.sun.jna.ptr.ByteByReference;
import com.sun.jna.ptr.IntByReference;
import com.sun.jna.win32.StdCallLibrary;
import java.nio.Buffer;

/**
 * Provides access to the Lego Mindstorm Fantom Library
 * @author Emmanuel Pirsch
 */
public interface Fantom extends StdCallLibrary {
    Fantom INSTANCE = (Fantom) Native.loadLibrary("fantom", Fantom.class);
    
    /**
     * 
     * @param searchBluetooth indicate if we also search bluetooth devices.
     * @param bluetoothSearchTimeout the timeout in second when searching bluetooth devices.
     * @param status OUT
     * @return
     */
    Pointer nFANTOM100_createNXTIterator(boolean searchBluetooth, int bluetoothSearchTimeout, Status status);
    /**
     * 
     * @param iNXTIterator
     * @param resourceName length must be at least 256.
     * @param status
     */
    void nFANTOM100_iNXTIterator_getName(Pointer iNXTIterator, byte[] resourceName, Status status);
    void nFANTOM100_iNXTIterator_advance(Pointer iNXTIterator, Status status);
    Pointer nFANTOM100_iNXTIterator_getNXT(Pointer iNXTIterator, Status status);
    void nFANTOM100_destroyNXTIterator(Pointer iNXTIterator, Status status);
    int nFANTOM100_iNXT_sendDirectCommand(Pointer iNXT, boolean requireResponse, Buffer commandBuffer, int commandBufferLength, byte[] responseBuffer, int responseBufferLength, Status status);
}

 

So now, I can easily connect to the NXT using something like :

String name= "T-NXT";
Status status= new Status();
Pointer iNXTIterator= fantom.nFANTOM100_createNXTIterator(true, 30, status);
try {
    while(!Status.Statuses.NO_MORE_ITEMS_FOUND.equals(status.getStatus())) {
        byte[] resourceName= newResourceName();
        fantom.nFANTOM100_iNXTIterator_getName(iNXTIterator, resourceName, status);
        System.out.println(asString(resourceName));
        if (asString(resourceName).contains(name)) {
            Pointer iNXT= fantom.nFANTOM100_iNXTIterator_getNXT(iNXTIterator, status);
            if (Status.Statuses.SUCCESS.equals(status.getStatus())) {
                return iNXT;
            } else {
                throw new UnableToCreateNXTException();
            }
        }
        fantom.nFANTOM100_iNXTIterator_advance(iNXTIterator, status);
    }
} finally {
    if (iNXTIterator != null) {
        fantom.nFANTOM100_destroyNXTIterator(iNXTIterator, status);
    }
}

 

And then start a robot program (already uploaded to the NXT) with :

String filename= "scorpion.rxe"
Status status= new Status();
byte[] filenameBytes= filename.getBytes();
ByteBuffer command= ByteBuffer.allocate(filenameBytes.length+1+1);
command.put((byte)0x00);
command.put(filenameBytes);
command.put((byte)0x00);
fantom.nFANTOM100_iNXT_sendDirectCommand(nxtPointer, false, command, command.capacity(), null, 0, status);

 

And this is just the beginning... Over the next few weeks (as time permit), I will add support to the rest of the Fantom API and add some nice Java classes to wrap this functionality and completely hide the Fantom API. Once this is done, I will release this code and start working on the JRuby side of things.

For completion, here are the NXT, Status and FantomUtils classes (The NXT class is the starting point of the Fantom API wrapper):

NXT.java

package treelaws.mindstorm;

import com.sun.jna.Pointer;
import java.nio.ByteBuffer;
import treelaws.fantom.Fantom;
import treelaws.fantom.Status;

import static treelaws.fantom.FantomUtils.*;

/**
 * 
 * @author Emmanuel Pirsch
 */
public class NXT {
    private static Fantom fantom= Fantom.INSTANCE;
    
    private String name;
    private Pointer nxtPointer;

    public NXT(String name) {
        this.name= name;
        this.nxtPointer= connect(name);
    }
    
    public void startProgram(String filename) {
        Status status= new Status();
        byte[] filenameBytes= filename.getBytes();
        ByteBuffer command= ByteBuffer.allocate(filenameBytes.length+1+1);
        command.put((byte)0x00);
        command.put(filenameBytes);
        command.put((byte)0x00);
        fantom.nFANTOM100_iNXT_sendDirectCommand(nxtPointer, false, command, command.capacity(), null, 0, status);
        System.out.println(status.getStatus().toString());
    }
    
    public void stopProgram() {
        ByteBuffer command= ByteBuffer.allocate(1);
        command.put((byte)0x01);
        fantom.nFANTOM100_iNXT_sendDirectCommand(nxtPointer, false, command, 1, null, 0, new Status());
    }

    private Pointer connect(String name) {
        Status status= new Status();
        Pointer iNXTIterator= fantom.nFANTOM100_createNXTIterator(true, 30, status);
        try {
            while(!Status.Statuses.NO_MORE_ITEMS_FOUND.equals(status.getStatus())) {
                byte[] resourceName= newResourceName();
                fantom.nFANTOM100_iNXTIterator_getName(iNXTIterator, resourceName, status);
                System.out.println(asString(resourceName));
                if (asString(resourceName).contains(name)) {
                    Pointer iNXT= fantom.nFANTOM100_iNXTIterator_getNXT(iNXTIterator, status);
                    if (Status.Statuses.SUCCESS.equals(status.getStatus())) {
                        return iNXT;
                    } else {
                        throw new UnableToCreateNXTException();
                    }
                }
                fantom.nFANTOM100_iNXTIterator_advance(iNXTIterator, status);
            }
        } finally {
            if (iNXTIterator != null) {
                fantom.nFANTOM100_destroyNXTIterator(iNXTIterator, status);
            }
        }
        throw new NXTNotFoundException();
    }
}

Status.java

package treelaws.fantom;

import com.sun.jna.Structure;
import java.util.EnumSet;
import java.util.HashMap;

/**
 *
 * @author Emmanuel Pirsch
 */
public class Status extends Structure {
    public int code;
    public byte[] filename= new byte[MAX_FILENAME_LENGTH];
    public int lineNumber;
    
    public Statuses getStatus() {
        return Statuses.fromCode(code);
    }
    
    static int MAX_FILENAME_LENGTH= 101;
    
    public enum Statuses {
        SUCCESS,
        FIRST(0),
        PAIRING_FAILED(5),
        BLUETOOTH_SEARCH_FAILED(6),
        SYSTEM_LIBRARY_NOT_FOUND(7),
        UNPAIRING_FAILED(8),
        INVALID_FILENAME(9),
        INVALID_ITERATOR_DEREFERENCE(10),
        LOCK_OPERATION_FAILED(11),
        SIZE_UNKNOWN(12),
        DUPLICATE_OPEN(13),
        EMPTY_FILE(14),
        FIRMWARE_DOWNLOAD_FAILED(15),
        PORT_NOT_FOUND(16),
        NO_MORE_ITEMS_FOUND(17),
        TOO_MANY_UNCONFIGURED_DEVICES(18),
        COMMAND_MISMATCH(19),
        ILLEGAL_OPERATION(20),
        BLUETOOTH_CACHE_UPDATE_FAILED(21),
        NON_NXT_DEVICE_SELECTED(22),
        RETRY_CONNECTION(23),
        POWER_CYCLE_NXT(24),
        FEATURE_NOT_IMPLEMENTED(99),
        FW_ILLEGAL_HANDLE(189),
        FW_ILLEGAL_FILENAME(190),
        FW_OUT_OF_BOUNT(191),
        FW_MODULE_NOT_FOUND(192),
        FW_FILE_EXISTS(193),
        FW_FILE_IS_FULL(194),
        FW_APPEND_NOT_POSSIBLE(195),
        FW_NO_WRITE_BUFFERS(196),
        FW_FILE_IS_BUSY(197),
        FW_UNDEFINED_ERROR(198),
        NO_LINEAR_SPACE(199),
        FW_HANDLE_ALREADY_CLOSED(200),
        FW_FILE_NOT_FOUND(201),
        FW_NOT_LINEAR_FILE(202),
        FW_END_OF_FILE(203),
        FW_END_OF_FILE_EXPECTED(204),
        FW_NO_MORE_FILES(205),
        FW_NO_SPACE(206),
        FW_NO_MORE_HANDLES(207),
        FW_UNKNOWN_ERROR_CODE(208),
        LAST(999);
        
        private int code;
        private static HashMap<Integer, Statuses> code_map= new HashMap<Integer, Statuses>();
        static {
            for(Statuses s : EnumSet.allOf(Statuses.class)) {
               code_map.put(s.code, s);
            }
        }
        Statuses() {
            this.code = 0;
        }
        Statuses(int code_offset) {
            this.code= -142000 - code_offset;
        }
        
        static Statuses fromCode(int code) {
            return code_map.get(code);
        }
    }
}

 

FantomUtils.java

package treelaws.fantom;

import com.sun.jna.Native;

/**
 *
 * @author Emmanuel Pirsch
 */
public class FantomUtils {
    static int DEFAULT_RESOURCE_NAME_ALLOCATION_SIZE = 256;
    
    public static final byte[] newResourceName() {
        return new byte[DEFAULT_RESOURCE_NAME_ALLOCATION_SIZE];
    }
    
    public static final String asString(byte[] s) {
        return Native.toString(s);
    }
}

RE: To Groovy or to JRuby

Shay Banon ask the question : To Groovy or to JRuby?
It seems that these days, whenever we see Groovy mentioned in blog post, it must be compared to Ruby or more specifically to JRuby. Then, in the comments, we have some Groovy lover express that Groovy is better than JRuby for the Java developer. What is funny is that people who are working on JRuby suggests that you try both and see for yourself.

Now that Groovy achieved to deliver a 1.0 version (and now a 1.1), it seems that this trend is increasing... Groovy advocate are becoming more vocal about their stuff. Instead of pushing Groovy/Grails for what it's worth, they are pushing it by saying how easy it is to learn compared to Ruby/Rails for a Java developer perspective. This is the argument that's being repeated over and over! And, as we see in some post comments, this makes some manager feel better.

This makes me feel sad!

Now this argument has some truth in it, but this does not necessarily make Groovy a better alternative than JRuby for scripting the Java platform. Learning a new language is always a good thing... Be it Groovy, Ruby, Python, etc. You never really loose by learning a new language... You never really loose by learning a new API, a new framework. Learning new things makes you a better programmer.

This is why I prefer Ruby/Rails than Groovy/Grails... It's not because Ruby/Rails is superior, it's just that learning a completely new language/framework help me open my mind to new ways of thinking. Learning Groovy will also do that, but to a much smaller extent. This will help me become a better programmer... Even better it will keep my mind awake enough to not become a dinosaur who know only one language/framework.

And to the manager that may read this... Having developers who have an open mind and bring new ideas is way better than having developers that will not stray away from their main comfort zone. It cost less in training and allow you to get advantage of new technologies as they become mainstream instead of missing the boat.

Don't take this post as telling you to choose JRuby, like Charles Nutter said, "don't let anyone make the choice for you."

Grails vs Rails... The number game just started!

A first Grails vs Rails Performance benchmark has just been published The benchmark shows that Grails is faster than Rails. Rails developer Jared Richardsons commented on some flaws with the benchmark (aren't all benchmarks flawed in some way? And as the benchmark author said, the benchmark is not about scalability).

But looking at these numbers, I found that Rails is not far behind Rails. And I find that very encouraging for the Rails community. Yes, I would expect much better performance on the Grails side... Most of Grails is Java code. Java code running inside a modern JVM should run pretty fast. But beside the insert test (which I guess is significantly slower due having just one Mongrel instance running), Grails is not very far ahead Rails. Rails is mostly Ruby code. The Ruby interpreter is just that, an interpreter. There is no just-in-time compilation, no enhanced optimization for often used code in Ruby.

I can't wait till we have Ruby running on some VM (maybe the JRuby team will run the same benchmark and give us their numbers) and I guess we will see some great improvement on the Rails side (I guess that just rerunning the benchmark with multiple Mongrel instances will close much of the current gap).

This may be the very start of the Grails vs Rails number game!

JRuby on Grails?

There is a small ongoing debate about Ruby on Grails... Graeme Rocher, the Grails founder, posted in thoughts on the topics too.

I don't know were Graeme takes it's numbers about Grails apps that are in production, but I guess that Ruby on Rails numbers are much higher. He argue that for a Java shop, Grails is much better framework than JRuby on Rails. His main argument to support this is that it will take less training. He seems to forget that to learn Grails (even if it's closer to Java than Ruby) also require some training. I would even add that learning the convention for framework like Rails or Grails is where most of the training is required... Learning the language is easier than learning the conventions.

In my point of view, JRuby on Rails has much more potential than Grails. First, Grails is playing catchup with Rails (still waiting for Migration in Grails). Second, Grails biggest strength reside on the integration layer between well know frameworks. However, these same frameworks are Java based and they come with the same limitations that the Java language has. Limitation that are not present in Rails because of the dynamic nature of the language it's build on.

Why would it be a good idea to run Rails on JRuby? Yes, integration with Java is a plus. But performance may be part of the answer. Ruby is not very fast (but it should get a lot faster very soon with the use of a Ruby VM), Java is quite fast. So running Rails on Java can make it faster. Even with Ruby VM coming along, JRuby may be faster because Java VM had a lot more work done in the VM performance than Ruby will have when it get it's own VM. If I'm able to run a Rails app in Tomcat or Weblogic, it will also have more appeal for existing Java shops. It could also give you the benefit of having access to existing Java libraries that have no equivalent in the Ruby world.

Instead of working on making JRuby as a language to run on top of Grails, if integration with existing libraries (like Hibernate) is desired, I think that Integrating the existing library in Rails would be much better. One could easily write a simple layer around Hibernate so one could have model objects inheriting something like HibernateRecord instead of ActiveRecord. The dynamic nature of Ruby could make this HibernateRecord behave the same way as an ActiveRecord. As for presentation framework, I don't think that component-based framework are such a good idea anymore. So there is no need to ever want to integrate them in Rails.

Graeme argue that Sun should not longer put resources in Phobos because the project is unpopular... I could argue that same thing about Grails... Compared to Rails, Grails looks like the unpopular guy in the schoolyard. But I won't do that... While I personally don't see why would someone would choose Grails over Rails, I thing that competition between frameworks is good. And when Grails will start to innovate (instead of playing catch-up) then it may become really interesting. The unpopular guy of today can always become the most popular guy of tomorrow (this is the Ruby and Java story).

2007 Predictions

Everybody doing it! Not wanting to be left in the cold, I will follow the herd and make predictions for 2007.

RoR will run flawlessly on a Java Application Server

The good people from JRuby are making incredible progress since they've been hired by Sun. We will soon be able to run RoR applications with no (or very little and trivial) modifications on Glassfish or other Web Container. Will that makes RoR application more scalable? I'm not sure. But it will increase RoR momentum in Java shops. When you talk about RoR to a java shop, the first question they have is "Will it run on Tomcat|WebLogic|WhebSphere|...?"

JRuby will take the top spot for scripting the Java platform

To the demise of Groovy, JRuby will take the spotlight. The main reason is the previous prediction. Once you can run RoR on a Java application server, nobody will need Grails. Grails is the only use of Groovy that create adoption.... Unless some other killer Groovy application, tool or framework is created.

RoR adoption continue

That's a fairly conservative prediction. RoR as a lot of momentum.

Java 6 adoption will start slowly in Q3

While Java 6 is out. It's adoption will grow slowly and will mainly be driven the scripting language support. Java 5 had many compelling reason to get adopted by many shops, but many many many application are still running on Java 1.4. Beside scripting support, there is nothing compelling enough in Java 6 to make it's adoption within the enterprise faster that Java 5.

Something will happen in Java build tool space

A lot of people are tired of Ant . Maven is cool, but lacks flexibility. Scripting language is getting more attention... We have Gant, Raven, JRake and many other existing or yet to be created build tool that use scripting language. By the end of the year, one of them will start getting a lot of traction. My best guess for now is Raven.

Java Specification Requests will become less and less relevant

The JCP is not working very well. It's too lengthy. Design by comity does not work well. Many JSR do not stand the sand of time. Today JSR mostly reflect some existing API with less feature (JPA vs Hibernate for example). They do not offer innovation, they just tend to seal mature innovative technology in the platform. New language features takes too long to traverse the JCP and with the open sourcing of Java, it will become easier for one group to add new feature to the language and to make a Java distribution of it's own. And with JSR like JSR-277 that create more turmoil than solve any problem, more and more people loose interest and faith in the process. Sun is not even interested in really supporting existing JSRs. We have a JSR for Groovy, but Sun hire the JRuby developers without giving any resource to Groovy. That's it! I made my prediction for 2007. This is probably my last post of the year, I will try to post a bit more often next year! In the mean time, have a Great Christmas and a really Happy New Year!