Build and release Scala/Java Gradle project in GitLab using Jenkins to Artifactory

I am going to show in detail how to regularly build your project and then how to make a release build. It involves cooperation of a number of tools which I found tricky to set up properly, that’s why I wrote this.

The goal

I am about to show you how to achieve two following scenarios. The first one is how to make a regular development non-release build:

Implement something, commit and push it to GitLab.
Trigger Jenkins build by a web hook from GitLab.
Build, test, assemble and then publish binary JAR to Artifactory repository.

The second and more interesting goal is when you want to build a release version:

Run parametric Jenkins build(s) that uses Gradle release plugin to:
1. Verify that the project meets certain criteria to be released.
2. Create Git tag with the release version number.
3. Modify Gradle project version to allow further development.
4. Commit this change and push it to GitLab.
Trigger another generic parametric Jenkins build to publish release artifact(s) to Artifactory.

The situation

I will demonstrate the process describing a real Scala project which I build using Gradle. The build server is Jenkins. Binary artifacts are published to a server running free version of Artifactory. Version control system is a free community edition of GitLab. I am sure that you can follow this guide for any Java application. For clarity of this guide let’s assume that your URLs are following:

GitLab repository (SSH) = git@gitlab.local:com.buransky/release-example.git
Jenkins server = http://jenkins/
Artifactory server = http://artifactory/

Project structure

Nothing special is needed. I use common directory structure:

<project root>
  + build (build output)
  + gradle (Gradle wrapper)
  + src (source code)
  + main
    + scala
  + test
    + scala
  - build.gradle
  - gradle.properties
  - gradlew
  - gradlew.bat
  - settings.gradle

Gradle project

I use Gradle wrapper which is just a convenient tool to download and install Gradle itself if it is not installed on the machine. It is not required. But you need to have these three files:

settings.gradle – common Gradle settings for multi-projects, not really required for us

rootProject.name = name

gradle.properties – contains group name, project name and version

group=com.buransky
name=release-example
version=1.0.0-SNAPSHOT

build.gradle – the main Gradle project definition

buildscript {
  repositories {
    mavenCentral()
    maven { url 'http://repo.spring.io/plugins-release' }
  }
  ...
}

plugins {
  id 'scala'
  id 'maven'
  id 'net.researchgate.release' version '2.1.2'
}

group = group
version = version

...

release {
  preTagCommitMessage = '[Release]: '
  tagCommitMessage = '[Release]: creating tag '
  newVersionCommitMessage = '[Release]: new snapshot version '
  tagTemplate = 'v${version}'
}

Add following to generate JAR file with sources too:

task sourcesJar(type: Jar, dependsOn: classes) {
  classifier = 'sources'
  from sourceSets.main.allSource
}

artifacts {
  archives sourcesJar
  archives jar
}

Let’s test it. Run this from shell:

$ gradle assemble
:compileJava
:compileScala
:processResources
:classes
:jar
:sourcesJar
:assemble

BUILD SUCCESSFUL

Now you should have two JAR files in build/libs directory:

release-example-1.0.0-SNAPSHOT.jar
release-example-1.0.0-SNAPSHOT-sources.jar

Ok, so if this is working, let’s try to release it:

$ gradle release
:release
:release-example:createScmAdapter
:release-example:initScmAdapter
:release-example:checkCommitNeeded
:release-example:checkUpdateNeeded
:release-example:unSnapshotVersion
> Building 0% > :release > :release-example:confirmReleaseVersion
??> This release version: [1.0.0]
:release-example:confirmReleaseVersion
:release-example:checkSnapshotDependencies
:release-example:runBuildTasks
:release-example:beforeReleaseBuild UP-TO-DATE
:release-example:compileJava UP-TO-DATE
:release-example:compileScala
:release-example:processResources UP-TO-DATE
:release-example:classes
:release-example:jar
:release-example:assemble
:release-example:compileTestJava UP-TO-DATE
:release-example:compileTestScala
:release-example:processTestResources
:release-example:testClasses
:release-example:test
:release-example:check
:release-example:build
:release-example:afterReleaseBuild UP-TO-DATE
:release-example:preTagCommit
:release-example:createReleaseTag
> Building 0% > :release > :release-example:updateVersion
??> Enter the next version (current one released as [1.0.0]): [1.0.1-SNAPSHOT]
:release-example:updateVersion
:release-example:commitNewVersion

BUILD SUCCESSFUL

Because I haven’t run the release task with required parameters, the build is interactive and asks me first to enter (or confirm) release version, which is 1.0.0. And then later it asks me again to enter next working version which the plugin automatically proposed to be 1.0.1-SNAPSHOT. I haven’t entered anything, I just confirmed default values by pressing enter.

Take a look at Git history and you should see a tag named v1.0.0 in your local repository and also in GitLab. Also open the gradle.properties file and you should see that version has been changed to version=1.0.1-SNAPSHOT.

The release task requires a lot of things. For example your working directory must not contain uncommitted changes. Or all your project dependencies must be release versions (they cannot be snapshots). Or your current branch must be master. Also you must have permissions to push to master branch in GitLab because the release plugin will do git push.

Setup Artifactory

There is nothing special required to do at Artifactory side. I assume that it is up and running at let’s say http://artifactory/. Of course your URL is probably different. Default installation already has two repositories that we will publish to:

libs-release-local
libs-snapshot-local

Jenkins Artifactory plugin

This plugin integrates Jenkins with Artifactory which enables publishing artifacts from Jenkins builds. Install the plugin, go to Jenkins configuration, in Artifactory section add new Artifactory server and set up following:

URL = http://artifactory/ (yours is different)
Default Deployer Credentials
- provide user name and password for an existing Artifactory user who has permissions to deploy

Click the Test connection button to be sure that this part is working.

Continuous integration Jenkins build

This is the build which is run after every single commit to master branch and push to GitLab. Create it as a new freestyle project and give it a name of your fancy. Here is the list of steps and settings for this build:

Source Code Management – Git
- Repository URL = git@gitlab.local:com.buransky/release-example.git (yours is different)
- Credentials = none (at least I don’t need it)
- Branches to build, branch specifier = */master
Build Triggers
- Poll SCM (this is required so that the webhook from GitLab works)
Build Environment
- Gradle-Artifactory integration (requires Artifactory plugin)
Artifactory Configuration
- Artifactory server = http://artifactory/ (yours is different)
- Publishing repository = libs-snapshot-local (we are going to publish snapshots)
- Capture and publish build info
- Publish artifacts to Artifactory
  - Publish Maven descriptors
- Use Maven compatible patterns
  - Ivy pattern = [organisation]/[module]/ivy-[revision].xml
  - Artifact pattern = [organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]
Build – Invoke Gradle script
- Use Gradle wrapper
- From Root Build Script Dir
- Tasks = clean test

Run the build and then go to Artifactory to check if the snapshot has been successfully published. I use tree browser to navigate to libs-snapshot-local / com / buransky / release-example / 1.0.1-SNAPSHOT. There you should find:

binary JARs
source JARs
POM files

Every time you run this build new three files are added here. You can configure Artifactory to delete old snapshots to save space. I keep only 5 latest snapshots.

Trigger Jenkins build from GitLab

We are too lazy to manually run the continuous integration Jenkins build that we have just created. We can configure GitLab to do it for us automatically after each push. Go to your GitLab project settings, Web Hooks section. Enter following and then click the Add Web Hook button:

URL = http://jenkins/git/notifyCommit?url=git@gitlab.local:com.buransky/release-example.git
- Hey! Think. Your URL is different, but the pattern should be the same.
Trigger = Push events

If you try to test this hook and click the Test Hook button, you may be surprised that no build is triggered. A reason (very often) can be that mechanism is very intelligent and if there are no new commits then the build is not run. So make a change in your source code, commit it, push it and then the Jenkins build should be triggered.

Have a break, make yourself a coffee

This has already been a lot of work. We are able to do a lot of stuff now. Servers work and talk to each other. I expect that you probably may need to set up SSH between individual machines, but that’s out of scope of this rant. Ready to continue? Let’s release this sh*t.

Generic Jenkins build to publish a release to Artifactory

We are about to create a parametric Jenkins build which checks out release revision from git, builds it and deploys artifacts to Artifactory. This build is generic so that it can be reused for individual projects. Let’s start with new freestyle Jenkins project and then set following:

Project name = Publish release to Artifactory
This build is parameterized
- String parameter
  - Name = GIT_REPOSITORY_URL
- Git parameter
  - Name = GIT_RELEASE_TAG
  - Parameter type = Tag
  - Tag filter = *
- String parameter
  - Name = GRADLE_TASKS
  - Default value = clean assemble
Source Code Management – Git
- Repository URL = $GIT_REPOSITORY_URL
- Branches to build, Branch Specifier = */tags/${GIT_RELEASE_TAG}
Build Environment
- Delete workspace before build starts
- Gradle-Artifactory Integration
Artifactory Configuration
- Artifactory server = http://artifactory/ (yours is different)
- Publishing repository = libs-release-local (we are going to publish a release)
- Capture and publish build info
- Publish artifacts to Artifactory
  - Publish Maven descriptors
- Use Maven compatible patterns
  - Ivy pattern = [organisation]/[module]/ivy-[revision].xml
  - Artifact pattern = [organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]
Build – Invoke Gradle script
- Use Gradle wrapper
- From Root Build Script Dir
- Tasks = $GRADLE_TASKS

Generic Jenkins build to release a Gradle project

We also need a reusable parametric Jenkins build which runs the Gradle release plugin with provided parameters and then it triggers the generic publish Jenkins build which we have already created.

Project name = Release Gradle project
This build is parameterized
- String parameter
  - Name = GIT_REPOSITORY_URL
- String parameter
  - Name = RELEASE_VERSION
- String parameter
  - Name = NEW_VERSION
Source Code Management – Git
- Repository URL = $GIT_REPOSITORY_URL
- Branches to build, Branch Specifier = */master
Additional Behaviours
- Check out to specific local branch
  - Branch name = master
Build – Invoke Gradle script
- Use Gradle wrapper
- From Root Build Script Dir
- Switches = -Prelease.useAutomaticVersion=true -PreleaseVersion=$RELEASE_VERSION -PnewVersion=$NEW_VERSION
- Tasks = release
Trigger/call builds on another project (requires Parameterized Trigger plugin)
- Projects to build = Publish release to Artifactory
- Predefined parameters
  - GIT_RELEASE_TAG=v$RELEASE_VERSION
  - GIT_REPOSITORY_URL=$GIT_REPOSITORY_URL

Final release build

Now we are finally ready to create a build for our project which will create a release. It will do nothing but call the previously created generic builds. For the last time, create new freestyle Jenkins project and then:

Project name = Example release
This build is parameterized
- String parameter
  - Name = RELEASE_VERSION
- String parameter
  - Name = NEW_VERSION
Prepare an environment for the run
- Keep Jenkins Environment Variables
- Keep Jenkins Build Variables
- Properties Content
  - GIT_REPOSITORY_URL=git@gitlab.local:com.buransky/release-example.git
Source Code Management – Git
- Use SCM from another project
  - Template Project = Release Gradle project
Build Environment
- Delete workspace before build starts
Build
- Use builders from another project
  - Template Project = Release Gradle project

Let’s try to release our example project. If you followed my steps then the project should be currently in version 1.0.1-SNAPSHOT. Will release version 1.0.1 and advance current project version to the next development version which will be 1.0.2-SNAPSHOT. So simply run the Example release build and set:

RELEASE_VERSION = 1.0.1
NEW_VERSION = 1.0.2-SNAPSHOT

Tools used

Scala 2.11
Gradle 2.4
- Gradle Release plugin 2.1.2
Jenkins 1.617
- Artifactory plugin 2.3.1
- Parameterized trigger plugin 2.29
Artifactory 3.3.0
GitLab 7.7.2

Jenkins plugins (thanks Andreas Mack)

Conclusion

I am sure there must be some mistakes in this guide and maybe I also forgot to mention a critical step. Let me know if you experience any problems and I’ll try to fix it. It works on my machine so there must be a way how to make it working on yours.