NYC Groovy/Grails Meetup Presentation

On Tuesday of this week I gave a short presentation to the New York Groovy/Grails Meetup, summarizing some of the content from the Gr8 conference earlier this year in Copenhagen. Numan Salati then gave a great presentation looking at various metaprogramming techniques in Groovy and Grails. The slides are now available online.

Everyday Git

So, you’ve got git installed, you have created (or cloned someone elses) repository, and you have done an initial commit of a couple of files. What are the day to day commands you’ll need to use on a regular basis? This posting looks through some common tasks and how to accomplish them simply using git. It’s not focused on some of the cooler features in git – just getting you passably productive in as few commands as possible.

Create a Branch
Before you do almost anything to a live project, you’re going to want to start to create a branch. By default you’re on “master” (which is similar to the “trunk” convention in subversion). Unless you’re just doing a quick patch that needs to be tested and deployed immediately (e.g. a quick bug fix for the production server) you’re going to want to create a branch for your work. That way if you *do* need to move to another story, you’ll be able to put aside the code you are working on in a branch and to return it easily. It also means that master will always be deployable. If you ever make changes to master, master then becomes unavailable for deployment until you’ve finished and tested your work, so a 5 minute patch might take a week to deliver if you have a bunch of half finished code in master.

Lets say we want to implement a “user authentication” story for our site. We’d start by creating a branch: “git branch user-authentication”. We’ve now created a branch called “user-authentication”. Type “git branch” and you’ll see there are now two branches – master and user-authentication. You’ll see from the * next to master that while we have created a user-authentication branch, we haven’t switched to it yet (the * denotes which branch you are currently in).

So, the next step is to “git checkout user-authentication” and both a “git branch” and “git status” will show (in their own ways) that you are on the user-authentication branch. Lets add a new file – login.txt that relates to user=authentication, then “git add .” and “git commit -m ‘added login notes’”.

Now we have a branch with an extra file. Imagine you now need to make a quick patch to master and deploy it. All you do is “git checkout master” and you’ll see the login.txt is gone. Now imagine that was a bunch of changes to your core code that would have broken your unit tests/website. Now we can make a tweak to our master branch, deploy from that and then simply “git checkout user-authentication” to get back to the user-authentication story.

Committing Changes
You’ve done a little more work on the user-authentication story and you want to capture all of the changes so you can revert to this point. Simply “git add .” and then “git commit -m ‘meaningful commit message’” and you’re done. With git you can afford to commit changes really often as later you’ll be able to use the rebase command to collapse down the number of commit messages (although we’re not going to cover that in this 101 tutorial). If in doubt, commit. Note that if all you are doing is editing existing files, you can just collapse down the two commands to a single “git commit -a -m ‘commit message’” (note the “-a” flag), but that won’t pick up any untracked files, so if in doubt, just do both steps. (Some UI based tools like the git commands in TextMate will automatically handle the git add as part of the commit.)

Rolling back
Ugh. We’re still on the user authentication story. We have done a lot of work and committed regularly, but since the last commit we tried some different things and they weren’t very good. Worse, we can’t remember how to get the code back to the last commit which was working (although not complete). It’s time to roll back to an earlier commit (it’s git and commits are cheap and easy, so you might be a couple of commits into this path). If you “git log” you’ll see a list of previous commits – each of which has a GUID like df9e8fd2a1d3769e42012a29f0ac4b56f6053a62. Lets say we wanted to revert back to the commit with that GUID. All we do is “git reset df9e8″ (note that we just needed a few characters – enough to uniquely identify the element) and it reverts to the status of the branch at that time, allowing you to get back to a known good state and to try something else.

Of course, if you’re doing something experimental, you might just want to create a branch – “git branch user-authentication-experimental-login” and then “git checkout user-authentication-experimental-login” so if it doesn’t work you can just “git checkout user-authentication” and then delete the experimental branch using “git branch -d user-authentication-experimental-login” (if the branch wasn’t a child of the current branch you’ll need -D, not -d).

Merging a Branch to Master
OK, so we’re finally happy with our new user-authentication branch. Let’s merge it into master. From the user-authentication branch we’re going to “git checkout master” to get back to the master and then “git merge user-authentication” to merge the user authentication branch into the master.

Tagging a Release
Often there will be special points you want to tag – such as production releases. When you want to create a tag, just type “git tag release1″ (where release1 is the name you want to give it), and then to revert to that tagged point at any time, you can just “git checkout release1″.

Pushing Your Changes
If this is a shared project, you may want to upload your files to Unfuddkle or github. Depending on exactly how your project is setup this may well be as simple as just typing “git push”, which will update all branches on the remote location with those from your local copy.

Next Steps
There is a lot more you can do with git. The default documentation is at http://book.git-scm.com/, although there are also many other tutorials available. Any feedback or questions appreciated!

Git and Unfuddle

I happen to be using Unfuddle for a number of my projects. Here are instructions for working with a git repo in Unfuddle. Start by signing up for an account with Unfuddle, then create a test project (make sure the initial repository type is git). Click on the repositories tab and you’ll see that there is an initial repo. Click on the repo and it’ll provide initial instructions for getting started. Here is a summary of what you need to do.

Before you can connect to Unfuddle securely, you’re going to need an SSH key pair.

On a Mac, start by entering “cat ~/.ssh/id_rsa.pub”. If you have a file, you’re already set up with an SSH key pair. If not you’ll need to “ssh-keygen -t rsa”. Just OK all of the default options, and then “cat ~/.ssh/id_rsa.pub” to get the key.

On a windows machine, create the project directory in windows explorer. Right click on the directory and select “Git Bash here”. At the command prompt, enter “ssh-keygen -C “username@email.com” -t rsa” (with your own username and email). Use the default location, and when prompted, enter (and remember) your keyphrase. Keep a note of where it stored the keyfile, open it with a text editor, and then copy the contents of that file as that is the key.

Once you have your key (whether on a Mac or Windows),log into Unfuddle, click on “personal settings” (top right), scroll down to “public keys”, click “new public key” with a meaningful title (e.g. “work computer” or “Dell laptop”) and paste the value of your key into the “value” textarea. Click on “add” and within a couple of minutes you’ll be able to use that SSH key to connect to Unfuddle.

Now that you’ve entered your SSH key, you will want to create a local repo: “mkdir projectname”, “cd projectname”, “git init”. You then need to associate the Unfuddle repo to the local one. The commands will be something like: “git remote add unfuddle git@accountname.unfuddle.com:accountname/projectname.git” (replace the accountname and projectname as appropriate) followed by “git config remote.unfuddle.push refs/heads/master:refs/heads/master”. Then you need to “git add .” and “git commit -m ‘initial commit’” to add and then commit any initial files. Finally, “git push unfuddle master” to push your initial commit to Unfuddle.

Once you’ve done this, all additional users will just have to “git clone git@accountname.unfuddle.com:accountname/projectname.git” (replacing the accountname and projectname with your account and project names) to get a copy of the repository. It’ll ask you to enter a copy of the passphrase and then will download a copy of the repository to your machine. When you later want to push your changes to Unfuddle, just “git push”, enter the passphrase and you’ll upload all your changes (and their history) to the Unfuddle repo.

Getting Started with Git (Windows)

These instructions are for getting started quickly on a Windows PC. I’ve used XP, but you shouldn’t find *too* many differences if you’re using Vista (please comment below with any changes you notice!).

Start by going to the downloads page on the git website. Click on the “msysgit” link (it’s better than the old cygwin implementation). You’ll find install instructions, click on the download link and select the executable installer. Run the exe and when you get to the “Path” screen, select “Use Git Bash only”. On CR/LF behavior, select the “Use Unix Style Line Endings”.

Once the install is completed, created a test directory in Windows explorer. Right click on the directory and select “Git Bash here”. This will create you a terminal window for executing git commands. Go back to windows explorer and create a test file – say test.txt. Then go into the terminal window and type “git init” to create a repo. Then type “git status” and you’ll see test.txt is untracked. “git add .” will add all files (right now, test.txt) to the staging area. “git status” to see that. Then “git commit -m ‘initial commit’” to commit test.txt. Now “git status” and you’ll see that the working directory is clean. “git log” and you’ll see there has been an initial commit. “gitk” and you’ll get a visual tool for viewing/browsing the repository and it’s history. That is it – you’re up and running with git!

Getting Started with Git (Mac)

This post provides simple instructions for getting git installed and set up on a recent (Leopard) Mac.

Start by going to the download page on the git website. Click on the “OSX installer” link, and then on the website, select the latest .dmg file (currently 1.6.3.1). Double click on the dmg file, and then double click on the git installer. You’ll need to enter your root password and when the installer says it was successful, you can eject and delete the dmg.

To test git, open a terminal and type “git”. You should see a list of commands. That’s it, you’re ready to get started with git. Go into a directory, “mkdir testproject”, “cd testproject” and “git init” and now you have an initial repo for that directory. Create a file (e.g. “touch myfile.txt”), “git add .” to add the file to the staging area, “git status” to check out the status, and “git commit -m ‘initial commit’” to commit the file to the repository. Now a “git status” will confirm that the working directory is clean and if you “gitk” you’ll get an (ugly) visual representation of the repo.

Looking for a prettier view of the repo? Download gitx – a nice repo viewer for git on the Mac. Download and install it, make sure to enable command line integration when installing, and then just type “gitx” within a git controlled directory and you’ll get a more attractive visual interface for repo viewing.

If you use TextMate, it has good support for core git commands. Just follow the download instructions here and remember to add the TM_GIT variable so TextMate can find git. Using the default installer, you’ll probably find that the git binaries are installed in /usr/local/git/bin.

You might also want to check out gitnub and changes, If you do, let me know how they work out for you as I haven’t had a chance/need to evaluate either to date.

Getting with Git for Groovy/Grails Developers

The official Grails repository moved to git back in March of this year. For anyone not yet using git for source control, I thought it might be useful to provide a general introduction to the benefits of git and how to get started – on a Mac or a PC. This posting tries to answer the question “Why git?” My next three postings will cover the practicalities of getting a Mac or a PC set up to work with git and then some initial instructions to start working with git.

Why Git?

Git is a distributed version control system. There are a number of key benefits it provides when compared to a centralized vcs like subversion, perforce, visual source safe or cvs (I’ll be comparing it to svn as that’s probably the most popular of the alternatives).

• Offline access – If you ever work offline or from a location that has a slow internet connection, you’re going to love git. Instead of just having a copy of a given tag, branch or the trunk, with git, you have the entire repo with all of the revision history, so if you want to revert to another branch or look at the history of revisions and run diffs between them, you can do that anywhere, anytime. And because you’re working with local files it is *really* fast.
• Fast, easy branching – The power of version control systems comes from the efficient use of branches to keep different streams of work separate, so you can be working on multiple stories at the same time while still keeping a clean trunk in case you need to push a patch live to production quickly. Branching in subversion is easy, but merging can often be painful (and changing branches can be slow as it’s a remote operation). With git you can make very quick changes between branches, and the merge conflicts are handled much more efficiently. The basic process is similar to subversion, but the number of conflicts that require manual intervention is usually much lower. With git, branching is easy enough that you’ll find yourself creating a branch for each story.
• Quick operations – Nothing sucks away developer time worse than slow operations – whether it’s checking out code, reverting to a tag or running a merge. With git, all of the operations are local (except for clone, pull and push) and they’re all extremely quick.
• No more politics – If you’re running an open source project (or any project with ad-hoc contributors) there is usually a whole set of politics and planning around who should and should not be a core committer. With git, anyone can clone the repo, make some changes and send you a patch. You can then just check out the patch into a branch, run regression tests against it, diff against the master to see what they changed, and then decide whether or not to merge their changes into master. It really lowers the barriers to contributions.
• No more orphaned projects – As long as one person has a copy of a git repo for a project, if they continue to maintain it, that can become the definitive resource for that project. There is no problem with a project lead getting busy and the project dying because patches aren’t being committed. As long as *someone* is interested in the project, it’ll continue to live.
• No central repository required – Do you create a repository for every single thing you do? Your notes? Your presentations? Your client meeting summaries? Your little “hack projects”? If not, the reason is probably the overhead of having to go to your subversion server and create a repo. With git, you just cd into a directory and “git init”. That’s it – instant repository. Suddenly there is no reason not to have a repo for everything, and you can always push your repo to a remote server if and when you want to create a central version (you can also just zip up your file system for the project and the user will have a copy of the repo and all of its history).
• A single git directory – Anyone else fed up with having to add code to your build scripts to kill the .svn directories scattered throughout your project? With git there is a *single* .git directory containing all of the revision history.

You might think that having a copy of the entire repo would make initial checkouts unwieldy, but because of an extremely efficient system for representing deltas, git repos are usually much smaller than a corresponding svn repo would be (with the usual disclaimer that lots of large, changing binaries in your repo are as bad an idea with git as with anything else). Often a git repo will only be 20-30% larger than a checkout of the trunk in svn – and it typically downloads extremely quickly.

There are a lot of other benefits that you’ll find as you start to get more familiar with git. You’ll learn to love the “stash” command and to appreciate the difference between the staging area and the repository. You’ll think up all kinds of non-traditional workflows that are possible when you aren’t limited to a single authoritative server, and you’ll find the power of everything from bisect (for narrowing down what part of a commit broke the build) to rebasing (flattening a set of minor commits into a single commit to balance the benefits of small steps with making your history log more meaningful). However, even if you use git as an enhanced replacement for subversion, you’re going to enjoy the improved speed and flexibility that you get so you can focus on coding, not waiting for your vcs. In the next article, I’ll look at getting started with git on a Mac.

Getting Groovy: What’s Next after gr8conf?

I had a great time at gr8conf earlier this week, and feel like I did a reasonable job of blogging a summary of the sessions (some more than others – sorry Griffon fans!) So, what’s next?

Well, I’m definitely going to be doing some more work in Groovy and Grails and I’ll continue to add my experiences to this blog (so remember to add it to your feed reader if you haven’t already!). I’ll also be posting some content on Domain Specific Languages that’s relevant to Groovy devs, and next week I’ll be doing a series on “Getting with Git” for Groovy/Grails developers who’ve noticed the trend and want to know how to kick their subversion habit and join us in the world of distributed vcs’s!

I had a great time meeting everyone at gr8conf this year, and look forward to seeing you all again next year!

The Gr8 Conference: Industrial strength Groovy

Paul King presented the final session of the conference. He has worked on a number of fairly large, Agile projects using Groovy and Grails and wanted to share some thoughts on best practices for building large Groovy and Grails applications. (His slides are available online)

Topics
- Testing/mocking JUnit, TestNG, EasyB, Spock, Instinct, MockFor, Gmock, EasyMock
- DI: Spring, Guice
- Coverage: Cobertura
- Code STyle: CodeNarc, IntelliJ
- Duplication: Simian
- Documentation: GroovyDoc
- Builds: Ant, Gant, GMaven, Gradle, Hudson
- Modularization: Grapes, OSGi

Groovy
- Out of the box it runs junit3, junit4 (if in class path), testNG
- Check out GMock and also expandometaclass mocking in grails

Paul provided an introduction to the common testing and mocking frameworks including JUnit, TestNG, EasyMock and GMock. He also introduced Instinct – a test framework he’s done some work with, and then talked briefly about Spock and EasyB – two interesting frameworks for capturing and running functional/acceptance tests.

Spock vs EasyB
- Spock is a BDD inspired framework that gives devs tools to make their lives easier
- EasyB is a great way to capture requirements and build the acceptance tests
- EasyB scenarios can be stories, and can leave them blank to make them descriptive stories. Then start adding closure code and can put in selenium, etc

DI In Dynamic Languages
In static languages, Dependency Injection is almost essential to create testable code. Without it, hard coded references to “new” objects can’t be replaced (I suppose you could come up with work arounds using test-aware object factories, but really, DI is the way to go). In Groovy (and most other dynamic languages) there are a number of tricks you can use to get the ability to stub/mock out objects without using a DI framework. For example, the object.metaClass.Constructor can be overloaded to point to a specific instance or do something else, so you can get some of testability benefits of DI in Groovy without having to use Spring or Guice. That said, as projects get larger there are still real benefits to using a DI framework. Paul provided a quick overview of two of the most common Java DI frameworks – Spring and Google Guice.

He then went on to look at code coverage using Cobertura, suggested that 100% coverage is a “necessary but not sufficient condition” (I’ll agree that it’s not sufficient – whether 100% coverage of all of your code is necessary, I’ll leave for another time and place!).

He spent some time looking at the ability to enforce code styles with Code Narc (such as setting maximum lines per method or class) and some of the IDE capabilities of IntelliJ for improving the style of your code. His suggestion was that you should allow your code checking tool to break the build, but should only put rules you’re willing to enforce 100% – otherwise you’ll just get more and more warnings and genuine coding issues will get lost in the noise in the reports.

He also introduced a tool called Simian for checking your code base for duplication. It supports Groovy, is free for OSS projects, $99 (Australian) for a single user (with restrictions) or $499 (Australian) per project for larger teams.

He talked briefly about GroovyDoc (which for Java devs familiar with JavaDoc is pretty much what you’d expect), and then went on to look at using Groovy within Ant to script your Ant files, and using Ant within Groovy to fire off Ant scripts from Groovy. He also recommended Gant – a layer on top of the Groovy Ant builder (Ant uses XML config, so a builder is a natural way to script it in Groovy), discussed GMaven (a Groovy Maven wrapper) and Gradle which is focused on bringing the benefits of dependency management (think Maven or Ivy), but in a Groovier style.

Continuous Integration
An important part of any non-trivial project is CI – especially with dynamic languages where you have to rely more heavily on tests because the compiler catches many fewer errors. There are a number of different CI servers such as CruiseControl, Continuum and TeamCity (commercial), but Paul presented on Hudson which is my favorite as it’s free, widely used, and much easier to get started with and to configure than Cruise.

Hudson has:
- Gant plugin – Can invoke Gant build script as the main build setup
- Gradle plugin – Allows hudson to invoke Gradle build scripts as the main build step
- Grails plugin – to invoke Grails tasks as build steps
- Hudsons CLI and the Groovy shell – Have access to runtime in Groovy so can ask what plugins, what is state of info inside it, etc.

Paul also briefly discussed modularization using either Grapes or OSGi.

It was a great overview of the various elements of pulling together a professional set of tools for testing and building Groovy and Grails apps.

The Gr8 Conference: Creating a Griffon Front End

On the final afternoon of gr8conf, Jim Shingler provided an introduction to Griffon – a system for creating Java wing apps in a very Grails like way. Griffon is “a Grails-like Rich Internet Framework” and Jim did a great job of showing how to install it, create a hello world project and get it up and running in about 10 minutes.

He then proceeded to take the Twitter client that Graeme had created earlier during the conference and to provide a rich Swing front end using very little code. The project is still in the early days (it’s currently a 0.1.1 release), but it was extremely impressive to see just how quickly you could build an application using it.

Right now I have my work cut out with getting up to speed with Groovy and Grails, but if you have any interest in building swing apps I’d thoroughly recommend checking out Griffin, and I plan to find some time before next years conference to play with this and post some more thoughts.

The Gr8 Conference: Using IntelliJ IDEA

After Graeme was done, Vaclav Pech did a half hour overview of some of the many features available for working with Groovy and Grails in IntelliJ IDEA, an IDE from JetBrains.

Given that you typically use Groovy and Java together, the theme of the presentation was the importance of making it easy to go between the two of them and to provide a consistent experience between them. There were lots of nice features like the ability to run Groovy and Java tests together, the ability to preview interfaces, generate stubs for extending java interfaces, etc. It even has support for teaching the IDE about specific dynamic methods so if (for instance), you mix or even inject via AST transformations a particular method into an object at runtime, you get tell IDEA that the class will have that method at runtime and it will provide some IDE support for that. There was also support for various refactorings, the ability to do command line Grails operations within the IDE and so on.

There is no question that currently, IntelliJ is the premier IDE for working with Groovy and Grails (although it’s also clear that SpringSource is working hard to make the Eclipse experience more usable than it currently is). Personally I find that for ad-hoc work, using TextMate on a Mac is ideal, but I’ve already picked up a personal license for IntelliJ (in the scheme of things, if you can’t invest $250 in improving your productivity, you’re probably not charging enough for your code!) and am slowly working through learning all of the keyboard shortcuts required to become really proficient in an IDE. I can see that as I start to work on larger projects it’ll really help with productivity.

Anyone used to using an IDE with a statically typed language like Java or C# is going to get a little surprised when they see the limitations imposed by working with a dynamic language. For example, I can’t even reliably “rename method”, because in a dynamic language, you can concatenate a string literal with the name of a method and use that to call the method, so it isn’t possibly to reliably rename all method references. There are some possible approaches that help with some of the issues using dynamic as well as static code analysis (you have a running copy of the application and use that to introspect it) but for a Java developer it is going to seem like a bit of a change of pace (luckily I mainly program in dynamic languages so I have little sense of what I’m missing!).

It was a good overview of some of the capabilities of the premier IDE for Groovy and Grails development, although a number of people I spoke to would have preferred a slightly slower pace, cutting down on the number of features covered to give us a bit of a better sense of how to use the features. Nobody would expect to become proficient in an IDE after a half hour demo, but by (for example) showing some of the features using the menus rather than using the keyboard shortcuts, it would have probably done a better job of presenting the features of the tool even though obviously nobody would actually develop that way.