Continuous Build and Deployment for iOS and Android Apps

< Back to Blog
Blog

Kevin Ford

August 8th, 2012

Last fall I undertook a project to set up an automated build server for iOS apps, and after a while we added Androidbuilds as well. At the time we had very little understanding of how Apple’s code-signing process worked, so our developers would revoke the distribution certificate and set up a new certificate and profile every time they wanted to do an ad hoc build. We also had issues with outside contractors sending us code that wouldn’t compile on any machine but their own. Add to that the nightmare of keeping track of all the device IDs for testers, product managers, and various people that just wanted to try out the apps before we released. For awhile it was taking a third of my time to manage the builds for just a handful of apps.

We knew we needed something to automate the build process and I had heard of Hudson and Team City, but after some research I decided to use Jenkins CI. Unfortunately, when I first started setting up the build server there wasn’t much information about Jenkins for Mac, or at least it wasn’t easy to find. More information has come out lately, especially around iOS but there still isn’t much info about doing iOS and Android builds on the same machine.  Hopefully this write-up will be helpful.

Some background on the technical side of our company: Historically NativeX has been an all-Microsoft shop, but with the ramp-up of mobile development we’ve had to expand our horizons quickly.  At first we tried using Team Explorer Everywhere with TFS for our source control system on Mac but it was difficult and buggy to use with Eclipse, and it really didn’t play well with Xcode.  We quickly switched to Git and have been working to improve our development processes as we go.  It’s taken awhile to set up and refine the build process, partly because it can be a hassle to figure out all problems, and partly because of the maze that Apple has created for provisioning profiles, certificates and signing.  Here is the basic flow: Github -> Jenkins -> Testflight (iOS) / Email (Android)

I should clarify the title here and say we are talking about internal deployment of these apps for testing and project management.  (It’s not really possible to do continuous deployment for mobile apps, and may not be a good idea anyway.)

step0-gallery-macmini-image4The Machine

Since one of the primary goals was to build iOS apps we were limited to Apple products for our build server. After some preliminary testing on a MacBook Pro, we decided on using the $999 Mac mini.  Memory is crucial, and the extra processing power of the server version is nice, but in hindsight we might have been able to get by with the $799 version.

The Software

Optional Software

The Xcode installation from the mac app store is fairly simple, although you will need to run Xcode and download the mobile development components (Preferences => Downloads). For Eclipse and ADT, follow the installation guide.

jenkins_logo

Jenkins has a handy pkg installer, which installs to /Applications/Jenkins/jenkins.war.  It also creates a launchctl config and a shell script for starting the Jenkins server process at boot time.  Unfortunately the default settings were completely unworkable for us.  It defaults to running as a daemon user, which sets the home directory to /var/root and has all sorts of permissions issues.  (Here’s a thorough explanation from Laird Nelson)  Instead I created a Jenkins local user (non-admin), and modified the launchctl file (/Library/LaunchDaemons/org.jenkins-ci.plist).

The UserName and GroupName options here were required for me to even get it running.  The SessionCreate flag allows access to the jenkins user’s Keychain.  This is critical, otherwise the Jenkins CI process will not be able to sign iOS builds.  After editing the plist be sure to set the proper ownership of the file or launchctl will refuse to load it:

sudo chown root:wheel /Library/LaunchDaemons/org.jenkins-ci.plist

To start the process without a reboot:

sudo launchctl  load /Library/LaunchDaemons/org.jenkins-ci.plist

The site should be up and running at  http://localhost:8080/. Jenkins includes the Winstone Servlet (http://winstone.sourceforge.net/), so there’s no need to install a web server to run it.  Unfortunately I was unable to figure out a way to get it to run on any port but the default port 8080 (jenkins.war seemed to ignore any command line parameters). It may be possible to set up Apache Tomcat to run Jenkins on port 80 like a standard website, but since I had the Mac Server app available I used it use it to set up a redirect from the default site to port 8080.

Jenkins Plugins:

Jenkins GIT plugin – required for connecting to git repositories (including GitHub)
XCode integration – required for iOS builds
Testflight Plugin – required for uploading iOS Apps to TestFlight for distribution
ant     – required for doing Android builds with Apache Ant scripts
Email-ext plugin  – adds an editable email template as a post-build option that can be used to distribute Android APK files

Optional Plugins:

GitHub Plugin – adds some useful links to the Github repository for the project
Clang Scan-Build Plugin – can used for running the Clang static analysis tool on an iOS project
Android Emulator Plugin– starts up the Android Emulator prior to a build in order to run unit tests on an Android project
Jenkins Emma plugin – plugin to capture output from the Emma code-coverage tool for Java ()
Jenkins Active Directory plugin– allows Jenkins to authenticate a username and the password through Active Directory

GitHub

Originally we had our central Git repository on a shared network drive, but I wasn’t able to find a way to mount that drive at boot time, and any mount scripts that run at login didn’t seem to play nice with Active Directory.  After evaluating some options we opted to switch to a private GitHub account for our central repository, and we definitely don’t regret the move.   The web interface has some neat features, and the ease of access for remote developers is fantastic.

There are a few possible ways allow the Jenkins server access to the private repository, but I decided to create a Github account for the Jenkins user, and grant access to all of our repositories under a read-only team. This seemed like the best way to manage access in a way that wouldn’t lead to the Jenkins server getting accidentally blocked from Github.  The Git plugin for Jenkins doesn’t seem to handle any logins or passwords, so I created an SSH key with no password, and added that public key to the Github account for the jenkins user.  It is possible to set up callback URLs in Jenkins so that Github can trigger builds when any commits are pushed up, but this requires exposing the Jenkins server outside the firewall.  We decided this wasn’t worth the hassle or security risk and opted instead to simply poll for changes (a built-in option for Jenkins that can be set up with simple cron job syntax).

iOS Builds

The Xcode build setup for iOS is fairly simple. For most builds you only need to set the Target, Configuration (typically Release), and then unlock the Keychain with the password of the jenkins user.  You will need to be logged in the first time that Jenkins tries to build a signed IPA file because there will be a GUI prompt to allow access to Keychain.  Click “Always Allow” and you shouldn’t need to do this again.

One additional recommendation is to set the Technical version to: ${BUILD_NUMBER}.  This will override the CFBundleVersion in the info.plist, and that’s useful for a few reasons.  First, Testflight only seems to allow a certain number of builds to be uploaded with the same version (overwriting the previous), and then it starts ignoring new builds.  Second, test users can better keep track of the actual build number they are testing. And third, Jenkins appends the bundle version to the IPA file, and overriding it means you won’t have to update it when the actual bundle version changes (i.e. just set the path to something like this $WORKSPACE/build/Distribution-iphoneos/AppName-Release-${BUILD_NUMBER}.ipa)

The best option we’ve found for distribution of test apps for iOS TestFlight. All you need to set up the TestFlight plugin is the API key for your account and the Token for your Team.  At first I configured the TestFlight plugin using my API key, but after fielding a number of questions about each build I decided to make a separate account for the Jenkins server.  This way it was clear to testers and Product Managers that the build was automated.  You will also need to set up a distribution list in TestFlight for the users you wish to receive the build notifications.

The only remaining challenge is managing test users for ad hoc builds.  Once a new device is registered through TestFlight, we still have to add the UDID for that device to the Provisioning Portal on developer.apple.com, then I download and install the Ad Hoc Distribution Profile onto the Jenkins build sever (adding it to the jenkins user’s Keychain).  While Apple has made some improvements to the provisioning process, it seems unlikely this part be automated anytime soon. (Unless Apple buys Testflight.)

Android Builds

Even without the hassle of a provisioning process, I’ve found maintaining the Android build process to be far more challenging than iOS.  With every update to ADT the build process changes (usually a breaking change), and the documentation rarely stays up-to-date. This example in the Jenkins Wiki was mostly correct at the time this blog post was written (the only problem I had was that the ‘install’ build option didn’t work, I had to use ‘installi‘). On the upside, once I am able to get the Ant builds working on my Mac they do typically work on the Jenkins server as well.

While there are a few services that offer features similar to TestFlight for Android, they seem to be overkill.  It’s easy enough to distribute Android apps for testing as an email attachment.  The editable email notification provided by the Email-ext plugin can be set to trigger off success (default is for failure), and it can attach the APK file after a build (e.g. MyApp/bin/MyAppActivity-debug.apk)

UpcomingUnity_logo

We’ve started doing some work internally with Unity engine and we plan to build out a couple test apps soon. Fortunately there’s already a plugin for Unity builds. Hopefully soon we’ll be able to set up an automated process for building Unity apps through Jenkins as well.

Other Resources

Also, the Jenkins project puts out updates quite frequently, so I made a list of shell commands for quickly updating the Jenkins service:

### Update jenkins.war
cd ~
# stop jenkins
sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist
# back up old version
sudo mv /Applications/Developer/Jenkins/jenkins.war /Applications/Developer/Jenkins/jenkins-prev.war
sudo mv jenkins.war /Applications/Developer/Jenkins
# set permissions and clear the security flags
sudo chown root:admin /Applications/Developer/Jenkins/jenkins.war
sudo chmod 755 /Applications/Developer/Jenkins/jenkins.war
sudo xattr -c /Applications/Developer/Jenkins/jenkins.war
# verify new permissions match old version
ls -@le /Applications/Developer/Jenkins/*
# restart service
sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

Kevin Ford

For all resources have them written by Kevin Ford, Marketing Manager

2012-08-08

Leave a comment

Breakdown of China’s Android Market

Kevin Ford - June 24th, 2019


The android market in China offers the chance at success. Still, it also requires a high level of understanding about […]

Read More >

Combat Advertising Fraud in the APAC (Asia-Pacific) Region

Kevin Ford - June 18th, 2019


Fraud continues to increase in the Asia-Pacific region. Knowing this, advertisers and publishers must take steps to avoid becoming the […]

Read More >

Mobile Gaming is Larger Than Ever

Kevin Ford - June 18th, 2019


There’s no question that the mobile gaming industry is growing. PCs, tablets and smartphones are more attainable than ever before. […]

Read More >