Technology exploration: Vaadin and jMonkeyEngine applet?

February 6th, 2010 by slarson

We’re getting excited over here about the possibilities of deploying the Whole Brain Catalog in a completely different way.  I’ve decided to start exploring these possibilities to see if the technologies we are looking at could bear fruit.  This may be a dead end, but its always fun to play with new tools!

The first thing that has gotten us excited is the discovery of a technology called Vaadin.  Vaadin lets you write user interface code in Java only and then it magically converts that user interface into a web application written in JavaScript.  If you are familiar with the Google Web Toolkit (GWT), this will sound familiar.  However, it appears that Vaadin has some significant advantages over GWT.  The ones we are interested in are the running of the client code on the server.  This would eliminate the need for WBC users to download a ~50 mb webstart package.  Also, regular GWT doesn’t allow the usage of any arbitrary Java libraries, which Vaadin apparently does.

Vaadin only handles 2D interfaces though, so the puzzle is to try to get a workable 3D interface involved as well.  After a lot of investigation into the next generation 3D-in-a-browser solutions that are being put forward, it seems clear that there is still a long way to go before there is a clearly better technology for WBC than jMonkeyEngine.  The advantage of switching to Vaadin, however, would be to deploy the 3D view as a much smaller lighter-weight java applet.  The 2D interface would load up immediately as the user arrived at the web page, and would make waiting for the applet to load much friendlier.

The rest of this post will be devoted to my exploration trying to get Vaadin to work with a jMonkeyEngine applet.

Read the rest of this entry »

Adding the full set of Allen Brain Regions

January 8th, 2010 by jrmartin

We have added over two hundred Allen Atlas brain regions to the latest version of the Whole Brain Catalog.  These changes haven’t been incorporated into the main release yet, but they are ready to be evaluated.  The addition of the Allen Atlas brain regions gives a more a detailed and elegant view of the mouse brain, but visualizing over 200 brain meshes at once has proven to be challenging. The following post intends to elaborate on this new feature and the challenges we have come across during the implementation phase.

A Whole Brain Catalog user has the option of visualizing all brain regions at once or each individually, but for the purpose of simplicity only five brain regions are visible by default at program startup. In order to display all the brain regions in the scene, we first have to download the corresponding data files from our server. Once we have downloaded the data files, we create objects  (Tangibles)  for all the brain regions using a geometry mesh called “TriMesh”, which is available to us through the game engine platform JMonkeyEngine. As of this version we are using “.obj” files to store the brain region’s data, but we also have access to these files in Collada format which  we intend to use in future releases.

Once we have finished downloading the data files and creating the objects (Tangibles) we need to render them on the scene. In order to do this we use the objects we previously created and make spatials (TangibleViews) out of them. A spatial is created by attaching the object (Tangible) to a JmonkeyEngine graph scene node called a Node. When all spatials have been created, we attach them to the 3D scene and this is how we are able to visualize the entire  Allen Atlas brain inside the Whole Brain Catalog. One of the challenges we have come across while implementing this feature  is the high amount of video card resources it takes to render over 200 meshes at the same time. Each brain region is represented by a mesh and each mesh can be represented by hundreds of verticies which can make the 3D scene really crowded. The more crowded the scene, the less performance we get from the application. For future releases, we will explore different rendering techniques to increase the performance of the application when the entire Allen Atlas brain is visible.

Be sure to visit our developer documentation for more information or post to our google groups mailing list if you have questions.

How to get started coding in “Whole Brain Catalog-style”

January 5th, 2010 by slarson

Ever wondered how the technologies used in the production of a tool like the Whole Brain Catalog come together?  Our new team member, Ruggero Carloz, has put together a bare-bones “Hello World” example that marries the basic Whole Brain Catalog open source technologies.  This provides a great template for building other applications and demos in this style.  The basic framework is already in place in our volume rendering example, our MCell COLLADA example, and our neuron cloud example.

The technologies used here include:

You can check out the Hello World developer example documentation to see how we pull these technologies together to make a simple, deployable Swing app that is version controlled and managed by Maven.

Real time monitoring of the Whole Brain Catalog

December 2nd, 2009 by slarson

Today we’re making available access to the statistics of how well the Whole Brain Catalog plant is running, as provided by Larry Lui of our talented systems group.  We have put a detailed page containing links to various stats such as server up-time and data consumption on our wiki.  This is in service of our commitment to building an open community platform.  Please have a look around and let us know your comments on our user forum or on our developer forum.

Status on Two new Features and Developer

November 12th, 2009 by caprea

CAPREA: I’ve been making progress on the MCell rendering collaboration with Tom Bartol, but I am not in direct contact with him. The current status of the MCell visualization is that the molecule diffusion animation is beautiful and faster than the the Millennium Falcon on a Kessel Run, but there is a mesh model with some 2.4 Million faces which needs to be seen in context with the molecules. Visualizing this has proven to be a challenge. Simply loading it into Blender or MeshLab gives both of these professional products problems. I have made significant progress on this in efficiency and refactoring, using a LODMesh in JME, and now we can view both the molecule animation and the background landscape of the cell parts. However, it must be made more efficient in before we show Bartol. The loading time is horrendous, Stephen say’s he’ll fix that with a strategy to load meshes straight from binary.

I’ve been working closesly with Han on volume rendering. Maven has been a pain for setting up such complicated projects. The dependency tree becomes more of a grey hazy bush in fog (It can be circular at times). Progress has been really slow, but yesterday we met a goal. Progress has been steady and painful, nothing has been slowing us down, but we weren’t going very fast to begin with.

In other developments, we have a new hire to our team, Carloz. We set him up with our development environment which is a day’s worth of learning curve to even an experienced programmer. I’m glad I got it piece-wise rather than all-at-once. It’s a lot to take in at once. Eclipse is complicated enough, and Maven’s complexity rivals that of a time machine cyborg with Unicorn magic.

How to render 1500 full neuron trees in real time

November 10th, 2009 by slarson

We spent a lot of effort in the Whole Brain Catalog to create the necessary components to render 1500 neurons in real time in order to visualize the Gage lab’s dentate gyrus network.  We have collected this effort together into a stand-alone example that illustrates the Whole Brain Catalog architecture.  In this post, I want to explain how this works.

The first thing to understand is that we have a set of multi-compartment models of neurons that we have downloaded from NeuroMorpho.org and converted into NeuroML.  These models describe the tree structure of the neurons in 3D.  The challenge to render a single tree is to arrange approximately 500 3d rectangular solids so that it creates the shape of the tree, one compartment (a.k.a “segment”) at a time.

Just arranging these rectangles, however, only gets you so far.  In the 3D world, its important to limit the number of verticies in your shapes in order to get good performance.  The vertex count of a single tree can be quite high at full detail.  So to get more performance out of the system, we use a trick in the game engine we use (JMonkeyEngine) called “Geometry instancing”.  This lets us take a single rectangular solid and copy it 500 times to make a tree, and save on both vertex count and memory consumption.  Its a very nice feature of the game engine.

But we ran into the same problem again when we reached the requirement to render 1500 full neuron trees, because we had one rectangular solid per tree, and 1500 solids is still a large number of verticies.  Our solution was to build a “neuron cloud”.  Any neuron morphology that participates in the cloud will be rendered by a single rectangular solid.  This trick requires us to do a significant amount of book keeping up front, but results in the ability to render the complete complement of trees.

You can check out the source code for this example by checking out our wiki page on the CloudRenderExample.  Questions?  Ask us on our developers forums.

Introduction to Maven2

October 17th, 2009 by slarson

One of the crucial technologies we have taken advantage of in the development of the Whole Brain Catalog has been Maven2.  Since not everyone is familiar with it, I thought it would be good to get a quick overview to it here.

The first thing to know about Maven is it provides the “single command” ability to build an entire code base.  With SVN and Maven installed you can issue the following commands to download and build the entire source of the Whole Brain Catalog:

svn co http://wholebrain.googlecode.com/svn/wbc/trunk wbc

cd wbc

mvn clean install

The one caveat to this is that you need to place a single configuration file called settings.xml in your maven directory.  Your maven directory is located at ~/.m2 on linux/mac, C:\Users\[username]\.m2 on Vista, and C:\Documents and settings\[username]\.m2 in Windows XP.  You can download our copy of the settings.xml file here.  This will point your copy of Maven at our proxy maven repository, which contains many of the third party dependencies necessary for the Whole Brain Catalog.

This last process (mvn clean install) will take a long time because it does a lot of magic things for you.  First of all it downloads all the jar files that the Whole Brain Catalog depends on to your local machine, under your maven home directory.  This will take a while as the WBC depends on a lot of jars.  It will also download Maven specific jar files that are used to build the code, run tests, create java docs, do deployment, and so on.  Once all these jars are downloaded, it will go step-by-step through the wbc modules, compile them, run their test cases, and package them into jars.

We prefer Maven to other build systems, notably Ant, because it hides a lot of functionality away in its plugins, allowing the build configuration to be specified within a single XML structure known as the “pom”, short for Project Object Model.  This file, pom.xml, contains all details about the dependencies needed to build a system, how to configure compilation, how to package up the code, what the current version of the code is, and much more.  More about Maven2 can be found online at its website.

How 3D objects are selected by the mouse

October 17th, 2009 by caprea

My name is Christopher Aprea, commonly reffered to as caprea (my default username has an advantage of being a cool internet handle) and I am a developer of the Whole Brain Catalog.

One bit of code that is extremely useful is the method psudoPick() in the WBCMouseListener class. But it could use some explaining, as it is likely an area to be tweaked in the future.

The way object-picking works: The method for picking objects in a 3D world on a 2D interface requires some heuristics. First, when a user clicks to select an object an invisible ray makes a line that passes through two points; the point that corresponds to the position of the mouse in the world on the first plane in the frustrum, and position that corresponds to the position of the mouse in the world at a nearly infinite distance away. All objects (Tangibles) that have a ‘face’ that intersect this ray are put onto an ordered list that lists the closest of these objects at the 0th index of the list. Secondly, for most cases, a second parameter in the method pseudoPick() omitsentries on this list by use of a ranking system. This allows objects that are actually behind the closest object to be picked instead. For example, if a subcellular component was inside a cell body, the user likely intended to pick the subcellular object, even through the cell was the ‘closest’ object. All TangibleView objects have a pickPriority element, this defines the ranking system. For now, almost all objects are MEDIUM, with BrainRegions being UNPICKABLE. If you intend for an enveloped Tangible to always be picked, make it’s pickPriority higher than the object that contains

Welcome to the Whole Brain Catalog for developers!

October 16th, 2009 by slarson

Welcome to the Developer’s blog.  On the Whole Brain Catalog developer’s site, we’ll be putting up materials that explain the current design and architecture of the WBC, outline the future directions we have for the system, and interact with the user and developer community to make the WBC the best tool available for synthesizing neuroscience information.  While the system may appear fairly large and sprawling, a few key concepts will take you a long way to understanding how it works.

First and foremost, the Whole Brain Catalog is client/server system, which means that we have separate code bases for the code that runs on your personal computer, and the code which runs on the back end systems.  The client is composed of 3 sub-modules: wbc-core, wbc-client and wbc-view.  The wbc-core module is the heart of the Whole Brain Catalog, containing code whose modification has sweeping implications for the whole system.  It is here that we have a master schema (expressed as an XSD) which describes the way that data is fundamentally expressed within the Whole Brain Catalog.  If you don’t have a good viewer for reading XSD files, I recommend you check out Oxygen, which will do a nice job of visualizing its structure.  There you will see the fundamental data types of the WBC, which include “TangibleStates” and “DataWrappers”.  A TangibleState is any item that can be visualized or interacted with in the WBC, and stores information about its position, rotation, scale, and other properties.  A DataWrapper is a generic envelope for any kind of data.  DataWrappers can wrap time series data, mesh models, tree structures, web services, and anything else.  A fundamental relationship in the WBC is between TangibleStates and DataWrappers.  A special sub-type of Tangible State called DataTangibleState allows the connection between these two types to be made.  For example, a SlideState is a DataTangibleState that has a relationship through a DataWrapper to an image. So the SlideState says where the image should be located in the space of the WBC, and the DataWrapper says where to go get the data that will be visualized.

The wbc-client module depends on the wbc-core module and provides the code that manages TangibleStates for the purposes of user interaction, editing, updating, and deleting.  A key class within this module is the TangibleManager class, which holds on to references for all TangibleStates in the system.  In the client, the TangibleState is wrapped inside of a “Tangible” which is the root class for all of the types that can be visualized.  The reason for using a different class for TangibleState and Tangible is that the TangibleState provides the basic setters and getters of the data model, while the Tangible class contains the functions to express how to manipulate that data model with respect to the other parts of the system.

Finally the wbc-view module depends on the wbc-client module, and contains the code that produces the Swing GUI as well as the JMonkeyEngine 3D GUI.  This is the code that processes mouse clicks and button presses and translates it into the manipulation of objects in the world.  The wbc-view contains counterpart classes to the TangibleManager and Tangible classes in wbc-client called TangibleViewManager and TangibleView, respectively.  Our team has had to learn a lot about the JMonkeyEngine to produce this code.  Hopefully you can find some quick solutions to problems you run into using JME here.

On the server side there is a single module called wbc-server.  This depends on the wbc-core module.  Both the client and the server use the Restlet library for its interactions via a REST interface. We have specified our REST API to the WBC server on our wiki.

We hope that you’ll get started looking into the code base and playing around.  You can find a page explaining how to get started on our wiki.  Please don’t hesitate to ask us questions on our forum.  Also, please follow us on Twitter!