DevOps With Jenkins: How to Build and Push Your Image to Docker Hub

There are horrible things in this world. And then there are absolute abominations. And then there's the documentation for Jenkins and its plugins.

Ugh. What a mess.

Because the documentation is so poor, I used search engines to find the info that I needed. I ended up piecemealing tidbits of info gleaned from StackOverflow and Medium into one solution.

Well that's not how it should be. So here we are.

In this guide, I'll show you how to create a Jenkins scripted pipeline that builds your Spring Boot application as a JAR file, creates a Docker image with it, and pushes that image to Docker Hub.

So let's get started.

Gettin' Jiggy With Jenkins

It's outside of the scope of this article to show you how to download and install Jenkins, so instead I'll refer you to this documentation on that subject.

(Warning: that documentation is from Jenkins so be sure to double-check everything you read there on StackOverflow).

Of course, you'll need Docker as well. But you probably knew that because of the title of this article.

Go here to find info on getting Docker installed. Make sure you install it on the same server where you're running Jenkins.

But back to Jenkins.

Once you've got Jenkins installed, it's time to add the necessary plugins. Here are the plugins I used for my DevOps solution:

• Docker Pipeline
• Pipeline

The rest you should have for free.

To install the plugins, just go to Manage Jenkins on the left-hand sidebar of your Jenkins home page.

 

Then, select Manage Plugins.

 

On the next page that appears, just type the name of the plugin you're looking for in the Filter field.

 

Then, check the plugin you want to install and click on the Download now and install after restart button.

Once you've clicked that button, Jenkins will give you a message to the effect of: "This plugin will be installed after you restart."

But it doesn't tell you how to restart.

Because that would be too easy.

So I will tell you how to restart. You have to do it with a URL.

Just add /restart after your base Jenkins URL.

For example, if your base Jenkins URL is http://myjenkins.com then go to http://myjenkins.com/restart to restart Jenkins.

You'll get a confirmation prompt asking if you really want to restart. Before you click Yes, make sure that no jobs are running because the restart will stop all jobs.

Once you've got the plugins, you're ready to get something set up on Docker Hub.

Dancin' With Docker Hub

Docker Hub is where you'll push your Docker images so you (or other DevOps practitioners) can later pull them down and deploy them where needed.

It's well beyond the scope of this article to get into the weeds on Docker and container technology. I'd recommend you start at the Docker website if you're interested in learning more.

Unlike the folks at Jenkins, the people at Docker know how to do documentation.

If you don't have an account on Docker Hub, go through the process of getting one set up. You'll have to go through the email verification process that I'm sure you're familiar with.

But it's free to get started. If you want some additional services, Docker Hub will set you back beteen $5 and $7 per month.

You don't need those premium services to continue with this guide, though.

Once you've got your account, create a repository (hereafter called "repo").

Keep in mind: there's a 1:1 relationship between repo and image in Docker Hub.

You can think of a repo in Docker Hub just like a repo in GitHub. There's typically one repo in GitHub per application.

So for me it's a best-practice to name the repo after the application.

In this guide, I'll push a Docker image for ecosystem-user-service to Docker Hub. So I gave my repo the same name.

 

And yeah, that repo is public. I mean, if the source code is public, why not the Docker image?

So to recap: there's a 1:1 relationship between repo and image. 

But... there's a 1:many relationship between repo and tags.

In other words, you'll push the same image to the repo over and over again, but each image will have a different tag.

In the pic above, you can see I've got a single tag: 9. 

Each image I build gets a new tag. It's the build ID from Jenkins.

Why? That seems to be a best-practice for DevOps these days. No duplicate tags in image builds.

At least when it comes to deployments. That's what the cool kids recommend.

Your mileage may vary, though. Do what makes the most sense for your shop.

Street Cred With Credentials

Before you can create a Jenkins pipeline, you need to create credentials.

Specifically, you'll need two sets of credentials:

• One for your source control repo (probably GitHub)
• One for Docker Hub

Go back to your Jenkins home page and once again click on that magical Manage Jenkins link on the left-hand sidebar.

Then select Manage Credentials from the page that appears.

Jenkins did its usual user-hostile thing and didn't make it easy for you to figure out how to create credentials. So you'll probably have to click on the (global) link below Stores scoped to Jenkins.

That should take you to the Global Credentials page where you can click on Add Credentials.

 

When you click on Add Credentials, you'll see the following:

 

If you're going the username/password route, you can leave the first field alone. 

I leave the second field alone as well. But since this is a security issue I'm reluctant to give any recommendations on scope.

From there, the rest of the fields are self-explanatory. Just keep in mind that you'll reference this set of credentials using the contents of the ID field.

That's why you should put something very descriptive in that field. For my Docker Hub credentials, I use "docker-hub" as the ID.

Once you're done creating both sets of credentials, it's time to get started on a Pipeline.

Piping Hot

Go back to your Jenkins home page and click on New Item on the left-hand sidebar. You should see a screen that looks like this:

Start by entering the name of the pipeline. I like to keep things consistent so the name of the pipeline is ecosystem-user-service. As we've seen, that's also the name of the Docker Hub repo and the GitHub repo.

Click on the Pipeline option from that list above. Then click OK.

Scroll down a bit on the new screen that appears until you get to the Pipeline section. In the Definition dropdown you see at the top of that section, select Pipeline script from SCM.

For SCM, select GIt. Assuming that's what you're using.

Then enter the URL for your source control repository. If you're using GitHub, you can find it by clicking on the green Code button.

 

Just copy and paste that URL into Repository URL field.

In the Credentials dropdown, select the credentials that will work for your source repository.

For Branch Specifier, use the branch you're currently working on. In my case, it's 0.2.7-devops-work. Just make sure to put the */ in front of it.

So the whole thing should look something like this:

 

Click that Save button at the bottom.

Back to the IDE

You're all set with Jenkins. Now it's time to code a pipeline script.

Remember: you chose Pipeline script from SCM in the previous section. That means Jenkins is going to look for a pipeline script in the root of your source tree.

Specifically, it's going to look for a file called Jenkinsfile.

So create that file in the root of your source tree. Then, populate it with code that looks like this:

node {
	def app
	def image = 'registry.hub.docker.com/careydevelopment/ecosystem-user-service'
	def branch = '0.2.7-devops-work'
	
	try {
		stage('Clone repository') {               
	    	git branch: branch,
	        	credentialsId: 'GitHub Credentials',
	        	url: 'https://github.com/careydevelopment/ecosystem-user-service.git'
	    } 
	
		stage('Build JAR') {
	    	docker.image('maven:3.6.3-jdk-11').inside('-v /root/.m2:/root/.m2') {
	        	sh 'mvn -B clean package'
	        	stash includes: '**/target/ecosystem-user-service.jar', name: 'jar'
	    	}
	    }
	     
	    stage('Build Image') {
	    	unstash 'jar'
			app = docker.build image
	    }
	    
	    stage('Push') {
	    	docker.withRegistry('https://registry.hub.docker.com', 'docker-hub') {            
				app.push("${env.BUILD_NUMBER}")
	        }    
	    }
	} catch (e) {
		echo 'Error occurred during build process!'
		echo e.toString()
		currentBuild.result = 'FAILURE'
	} finally {
        junit '**/target/surefire-reports/TEST-*.xml'		
	}
}

There's quite a bit going on there. I'll cover it one step at a time.

First of all, that code you see above is a limited form of Groovy syntax. But you don't need to be a Groovy expert to make something great happen in Jenkins.

You can read more about Groovy here.

Next, note that the whole thing begins with node.  That's because it's using a scripted as opposed to a declarative pipeline.

Why? Because scripted pipelines offer more flexibility. That's for starters.

But also because the code needs a handle on the image when it invokes docker.build. If you want to do that in a declarative pipeline, you have to do so within a script { } block and that seems like it's defeating the purpose of moving to a declarative pipeline.

So scripted pipeline it is.

And if you want a rundown of the differences between scripted and declarative pipelines, check out this article that advises you to go declarative. But I explained my reasoning for rejecting that approach.

Anyhoo, the code defines three variables at the top:

• app - a reference to the application image created via docker.build
• image - the image name
• branch - the branch name

The image name uses the Docker Hub registry domain name, my namespace, and the repo name. That's quite intentional and it's the pattern you should follow.

Next, note that the whole thing operates in a try/catch block. That way if something goes wrong, the finally block will run the junit command to publish unit test result reports.

Inside the catch, the code reports the nature of the error and sets the build status to FAILURE. Without manually setting the build status, Jenkins would report a status of UNSTABLE instead. 

But now take a look at what's happening inside of the try block. There, you'll see a variety of stages.

When the Jenkins script runs, each stage will get its own column in the final report. You'll see that in a moment.

As you can tell by looking at the stage names, they each represent a different unit of work.

If it's not clear, you can see the name of each stage by looking inside the parentheses next to the word stage.

And yes, the stages run in order from top to bottom.

"Clone Repository"

The first stage ("Clone repository") grabs the latest source from source control. In this case, it's gretting the source from good ol' GitHub.

The branch is defined by the variable towards the top of the whole script.

The credentialsId attribute identifies the required credentials to access the GitHub repository. It's the ID of the GitHub credentials you created earlier.

The url attribute is the base URL of the source tree. You saw how to get that in a previous section.

That's it for the first stage. That will pull down the latest code for a specific branch from your GitHub repo and store it in your Jenkins working directory.

For me that's /var/lib/jenkins/workspace

By the way, it's a great idea to use Putty or some other similar tool to login to the server where you're running Jenkins and check out the source code yourself. I'd especially recommend that when you're first starting out.

"Build JAR"

The next stage ("Build JAR") unsurprisingly builds the JAR file for your Spring Boot application.

Note that you can use that code without installing Maven on your local server. That's because it uses a Docker container to handle the whole process.

The Docker image for the container (maven:3.6.3-jdk-11) is suitable for running Maven builds on Java source code compiled with JDK 11. 

The inside() function starts the Docker container and keeps it running for the duration of the body.

Note that the code mounts a volume: -v /root/.m2:/root/.m2

That maps the Maven directory on the host to the Maven directory inside the container. As a result, the Maven instance running in the container will have access to dependencies from the outside world.

The sh command does exactly what you think it does: it executes a command in the container. In this case, it runs Maven (mvn) in batch mode (-B) so that it runs without prompts.

Maven runs the clean and package build phases to create a brand new JAR file every time.

But that JAR file will stay in the container. How can you use it in a later stage when creating the Docker image?

Enter stash. That's going to save the JAR file under the name "jar" for further use.

Oh, one other thing: Maven will also run all the unit tests in the source as part of the build process. If any of those tests fail, the build will end here and execution will transfer to the catch and finally blocks.

"Build Image"

The next stage ("Build Image") uses Docker to create the image.

But you'll need a Dockerfile to make that happen. That file also needs to be in the root of your source.

Here's what it should look like:

FROM adoptopenjdk/openjdk11:jre-11.0.10_9-alpine
COPY ./target/ecosystem-user-service.jar /
EXPOSE 32010
ENTRYPOINT ["java", "-jar", "./ecosystem-user-service.jar"]

It's way outside of the scope of this guide to get too deep into the weeds on what goes into a Dockerfile. I'll just cover the highlights.

The Docker image uses the adoptopenjdk/openjdk11:jre-11.0.10_9-alpine base image. That's suitable for running a Spring Boot application conpiled with JDK 11.

The COPY line takes the stashed JAR file and copies it to the image.

The EXPOSE line tells the container to listen for incoming requests on Port 32010. The Spring Boot application is also configured to listen on that port.

And, finally, ENTRYPOINT configures the container to run as an executable. In this case, it will run the Spring Boot JAR file as an application.

And now back to Jenkinsfile.

That code block unstashes the JAR file created from the previous stage. That's so it's accessible to the Docker build process.

Then the code invokes docker.build. It uses the image name defined towards the top of the whole file.

Once the image is built, it's assigned to the app variable. Once again, that's so it can be referenced and used in later stages.

"Push"

At this point the code has pulled down the source from GitHub, used that source to create a Spring Boot application JAR file, and generated a Docker image that will run that JAR file as an executable.

Now it needs to push the image to the Docker Hub repo.

It does that with the aid of docker.withRegistry. The function specifies the Docker Hub URL ('https://registry.hub.docker.com') in the first parameter.

The second parameter ('docker-hub') identifies the credentials that Jenkins will use to access the Docker Hub repo. That's the ID of the credentials you created earlier.

Inside the docker.withRegistry() block you'll see just a single line of code: app.push("${env.BUILD_NUMBER}")

And that's all you need to see. That will push the image, tagged with the build number, to the repo.

Testing It Out

Okay. Will this thing work?

Sure it will. Think positively.

Head over to Jenkins and find the pipeline you created right there on the home page. If you're building ecosystem-user-service, then that's also the name of the pipeline if you followed the instructions above.

Click that link. Then click Build Now on the left-hand sidebar.

 

The cool thing is you can watch the build process happen in real time. Once the build launches, just click on the blinking blue (hopefully blue) circle below.

 

And let me repeat: click on the circle. Not the date and time. Not the build number.

That will take you to console output that should end with something like this:

 

+ docker rmi registry.hub.docker.com/careydevelopment/ecosystem-user-service:19
Untagged: registry.hub.docker.com/careydevelopment/ecosystem-user-service:19
Untagged: registry.hub.docker.com/careydevelopment/ecosystem-user-service@sha256:d182f69d8619e10005a56e8bf58657155a7d2d925dbd762044f07e4d3ea8e710
Deleted: sha256:d1a4f25a57d8ac3632d7a611ce1e76787156cffd0caff6ee9c1ea4b825cda3cf
Deleted: sha256:e05273a28ef052dd2fee6aefd1e43342497af466bfa90f9bd8c50d5dad91cf4a
Deleted: sha256:11ff9a9660ff5ee851c56d59cafbb816aeeddc43dc521ba99a486b187960c934
Deleted: sha256:e8ecf07acbdba3daaac5888019b66f2ccfd06ca9db2a68c8a65d1e6b97faeae2
[Pipeline] }
[Pipeline] // stage
[Pipeline] junit
Recording test results
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

That last word ("SUCCESS") is the most beautiful word in the English language to a DevOps engineer. It means everything worked as expected.

And you can also see how each of the stages performed by once again clicking on the pipeline from the Jenkins home page and looking at Stage View.

Wrapping It Up

There you have it. 

But keep in mind: this is only a template. You'll undoubtedly need to make some tweaks to suit your own requirements.

And if this guide didn't answer all the questions you have, feel free to lobby the folks at Jenkins to come up with some decent documentation.

You can also grab the code I'm using from GitHub. It includes the Jenkinsfile and the Dockerfile.

In fact, you can try to build that image and push it to your own Docker repo. That's a great way to learn.

For that exercise, you won't need GitHub credentials. That's because you'll only be reading from my GitHub repo.

Have fun!

Photo by Andrea Piacquadio from Pexels