AppVeyor

Sharing configuration between branches in appveyor.yml

2017-08-01T00:00:00+00:00

The problem

There are two options to have branch-specific configuration with appveyor.yml:

Commit appveyor.yml into each branch with branch-specific settings.
Use Conditional build configuration where appveyor.yml has a list of configurations for different branches.

The problem with 1st approach is merging as you are overriding appveyor.yml in the base branch with one from the branch being merged.

2nd approach requires appveyor.yml of the following format:

# configuration for "master" branch
# build in Release mode and deploy to Azure
-
  branches:
    only:
      - master

  configuration: Release

  deploy:
    provider: AzureCS
    ...

# configuration for all branches starting from "dev-"
# build in Debug mode and deploy locally for testing
-
  branches:
    only:
      - /dev-.*/

  configuration: Debug

  deploy:
    provider: Local
    ...

# "fall back" configuration for all other branches
# no "branches" section defined
# do not deploy at all
-
  configuration: Debug

While this approach works great in the most cases there is one incovenience though - with large configuration and many branches appveyor.yml becomes really unmanageable and error-prone as you have to repeat (copy-paste) entire configuration for every branch.

The solution

We just deployed an update to AppVeyor that allows sharing common configuration between branches in a single appveyor.yml!

There is new for node with a list of branch-specific configurations overriding common configuration defined on the top most level, for example:


# common configuration for ALL branches
environment:
  MY_VAR1: value-A

init:
- do_something_on_init.cmd

install:
- do_something_on_install.cmd

configuration: Debug

# here we are going to override common configuration
for:

# override settings for `master` branch
-
  branches:
    only:
      - master

  configuration: Release

  deploy:
    provider: FTP
    ...

# override settings for `dev-*` branches
-
  branches:
    only:
      - /dev-.*/

  environment:
    MY_VAR2: value-B

  deploy:
    provider: Local
    ...

In the example above we define environment, init and install sections for all branches as well as stating that default configuration is Debug. Then, for master branch we override default settings by changing configuration to Release and adding deployment with FTP provider. For dev-* branches we define a second environment variable MY_VAR2 and enable deployment to Local environment.

Configuration merging rules:

Scalar values such as image, version, configuration, platform, etc. defined on branch level override default ones;
Script sections such init, install, before_build, test_script, etc. defined on branch level override default ones;
Environment variables defined in environment sections are merged (new) and overridden (existing);
Build matrix defined on branch level merges with default one;
deploy, artifacts, notifications section can be either overridden or extended.

For example, consider the following configuration:

artifacts:
- path: bin

deploy:
- provider: Local
  ...

notifications:
- provider: Email
  ...

for:
  branches:
    only:
      - master

  artifacts:
  - path: docs

  deploy: off

  notifications:
    provider: Slack
    ...

In the example above we do the following:

For master branch we adding docs folder to artifacts definition, so both bin and docs folders collected. Both default and branch-specific collections were merged.
For master branch we disable any deployment. off or none on branch-level clears default collection.
For master branch we replace all notifications on default level with a single Slack notification.

Best regards,
AppVeyor team

CodeCov Test Coverage Integration

2017-03-17T00:00:00+00:00

This is a guest post by Cedd Burge, Software Developer Lead at RES.

CodeCov visualises code coverage and can enforce standards via GitHub and AppVeyor. More information is available on CodeCov.io.

This post is written from the point of view of someone (me) who is already proficient in C# and unit testing, but was new to AppVeyor and integrating CodeCov with GitHub.

It contains from scratch steps to run CodeCov test coverage on a sample C# project using GitHub, AppVeyor, OpenCover and XUnit. You can look at the repo I created to test this post if you get stuck.

Create a new GitHub Repo

If you are new to GitHub, see this getting started guide, otherwise simply create a new repo (YourRepositoryName from now on) and git clone it somewhere convenient.

Create a new System Under Test (SUT) project and a new Test project

In my version of Visual Studio (Community 2015), you can do this by clicking on “File - New - Project” on the main menu, then “Class Library” from “Templates - Visual C#”. Give it a interesting name, which I will assume to be YourSUTProjectName for the rest of this post.

Repeat this for the test project, which I will assume to be YourTestProjectName.

Add the xunit, xunit.runner.visualstudio and xunit.runner.console nuget packages to your test project.

Add an XUnit test to YourTestProjectName and the corresponding thing under test to YourSUTProjectName, or copy and paste mine.

Run this test to make sure it passes.

Setup AppVeyor Integration

You will need to link an AppVeyor account to your GitHub one, so let’s do that:

Navigate to your repo in GitHub
Click “Settings” on the repo
Click “Integrations and services”
Click “Browse Directory”
Click “AppVeyor”
Click “Configure”
Click “Grant Access”

Now Log in to AppVeyor.com, probably using your GitHub account

Click “Projects”
Click “New Project”
Choose YourRepositoryName and click “Add”

Go to https://codecov.io/ and click “Sign up with GitHub”
Click “Add new repository” and choose YourRepositoryName.
Make a note of the Upload token (YourUploadToken from now on)

Run Coverage Analysis Locally

When working with AppVeyor, it always makes sense to test on your own computer first. The feedback is immediate and you iterate very quickly. It takes a lot longer to modify the appveyor.yml file, push it and wait for a build to go through. Also, if it works locally but doesn’t work on AppVeyor, you know the problem is a configuration difference between your computer and the AppVeyor environment (eg a different version of msbuild).

However, this step is not required, so skip to “Run Coverage Analysis on AppVeyor” if you wish.

CodeCov.io is a tool for visualising and integrating coverage data, which we need to create. We will use OpenCover to do this.

Analyse

Add the OpenCover nuget package to the solution (which will install OpenCover.Console.exe, probably in packages\OpenCover.4.6.519\tools) and then run the following in a command window.

YourTestDLL is the relative path to the dll for your test project, eg. Tests\bin\Debug\tests.dll
-register[:user] asks OpenCover to register the code coverage profiler under HKEY_CURRENT_USER, which doesn’t require the user to have administrative permissions.
-target: asks OpenCover to run XUnit
-targetargs: are arguments that OpenCover passes to XUnit
-output: is the file to contain the coverage results
YourSUTNamepace, YourTestNamespace are the namespaces for your Test and SystemUnderTest projects. If these filters aren’t specified OpenCover will try and analyse everything, including the XUnit dlls.

packages\OpenCover.4.6.519\tools\OpenCover.Console.exe -register:user -target:"packages\xunit.runner.console.2.2.0\tools\xunit.console.x86.exe" -targetargs:"YourTestDLL -noshadow" -output:".\coverage.xml" -filter:"+[YourSUTNamepace*]* -[YourTestNamespace*]*"

Upload

Download and install Python 3.x, make sure to tick the box to add Python to your path (or do so manually).

Run the following on the command line. You may need to close and reopen the console to pick up changes to the path. YourUploadToken is the CodeCov token that you took a note of earlier, or available on the project settings page on CodeCov (eg https://codecov.io/gh/YourGitHubUserName/YourRepositoryName/settings)

pip install codecov
codecov -f "coverage.xml" -t YourUploadToken

The output will show a url with the results (eg https://codecov.io/gh/YourGitHubUserName/YourRepositoryName)

Run Coverage Analysis on AppVeyor

Now that the coverage upload is working locally, we can run it on AppVeyor.

Add and commit an appveyor.yml file to the root of the repository as below.

YourSolution is the relative path of the .sln file to build (eg codecov-on-appveyor.sln)
YourUploadToken is not required if you are uploading from a public repo with AppVeyor, but is included in case you are using a private repo.
%xunit20%. XUnit is installed on the AppVeyor build environment by default, and AppVeyor provide the %xunit20% environment variable for the installation path.

before_build:
 - nuget restore
build_script:
 - msbuild /verbosity:quiet "YourSolution"
test_script:
 - .\packages\OpenCover.4.6.519\tools\OpenCover.Console.exe -register:user -target:"%xunit20%\xunit.console.x86.exe" -targetargs:"YourTestDll -noshadow" -output:"coverage.xml" -filter:"+[YourSUTNamepace*]* -[YourTestNamespace*]*"
after_test:
 - "SET PATH=C:\\Python34;C:\\Python34\\Scripts;%PATH%"
 - pip install codecov
 - codecov -f "coverage.xml" -t YourUploadToken

Add Coverage Graphics to the Repository

There are various badges and graphs available. Click on your project in CodeCov, then “Settings” and “Badge” (eg https://codecov.io/gh/YourGitHubUserName/YourRepositoryName/settings/badge) to see what’s available.

Copy and paste a code snippet from this in to your README.md, such as this one .

You can look at my readme for example badges

Integrate CodeCov with Pull Requests

CodeCov automatically integrates with GitHub pull requests (as long as you signed up to CodeCov via your GitHub account) which you can see on this pull request.

It will show some statistics, such as the increase or decrease in coverage, and by default will fail the build if coverage decreases. You can configure this with a ‘codecov.yml’ file in the root of your repository.

Install the browser plugin

There is a CodeCov browser plugin, for all reputable browsers, that adds coverage when browsing GitHub.com. Code is highlighted in red / green, and a coverage percentage is shown.

Conclusions

It can be difficult keeping control of test coverage for a project, especially with distributed and transient team structures. The combination of GitHub, AppVeyor OpenCover and CodeCov make it very easy to visualise the coverage, and allow you to enforce standards to ensure that it improves over time.

Best regards,
Cedd Burge

Follow Cedd on Twitter: @cuddlyburger
Follow AppVeyor on Twitter: @appveyor

SonarQube Analysis

2016-12-23T00:00:00+00:00

This is a guest post by Cedd Burge, Software Developer Lead at RES.

SonarQube / SonarSource analyzes code, highlights quality issues and calculates metrics such as technical debt. More information is available on SonarSource.com.

This post is written from the point of view of someone (me) who is already proficient in C#, and had even used SonarQube, but was new to AppVeyor and integrating SonarQube with GitHub.

It contains from scratch steps to run the SonarQube analysis on a sample project and to publish the results to the publicly available Nemo instance of SonarQube. You can look at the repo I created to test this post if you get stuck.

Create a new GitHub repository

If you are new to GitHub, see this getting started guide, otherwise simply create a new repo and git clone it somewhere convenient.

Create a new project

Add some code that has some quality issues (e.g. a variable that is declared but never used). You can use the the full list of SonarQube C# issues for inspiration. Alternatively you can copy and paste some of mine.

Install the SonarLint Visual Studio Plugin. This highlights quality issues in your code as you type and gives you a chance to fix them before committing.

Integrate with AppVeyor

You will need to link an AppVeyor account to your GitHub one, so let’s do that:

Navigate to your repo in GitHub
Click “Settings” on the repo
Click “Integrations and services”
Click “Browse Directory”
Click “AppVeyor”
Click “Configure”
Click “Grant Access”

Now Log in to AppVeyor.com, probably using your GitHub account

Click “Projects”
Click “New Project”
Choose your GitHub repository and click “Add”

Create an account at sonarqube.com/sessions/new
Click on sonarqube.com/account/security/
Enter a token name and click “Generate”
Make a note of the generated token (YourSonarQubeToken from now on)

Run SonarQube Analysis Locally

When working with AppVeyor, it always makes sense to test on your own computer first. The feedback is immediate and you iterate very quickly. It takes a lot longer to modify the appveyor.yml file, push it and wait for a build to go through. Also, if it works locally but doesn’t work on AppVeyor, you know the problem is a configuration difference between your computer and the AppVeyor environment (e.g. a different version of msbuild).

Instead of committing SonarQube executables to the repo, we will download them during the build using Chocolatey.

Install chocolatey

Install Chocolatey from an administrator command prompt / powershell.
Close the command prompt

Install SonarQube MSBuild Runner

Open a new administrator command prompt / powershell.
choco install "msbuild-sonarqube-runner" -y

Analyze and upload to SonarQube

MSBuild.SonarQube.Runner.exe begin /k:"**YourUniqueProjectName**" /d:"sonar.host.url=https://sonarqube.com" /d:"sonar.login=**YourSonarQubeToken**"
"**YourPathToMSBuild**\MSBuild.exe" "**YourProjectName**.sln"
MSBuild.SonarQube.Runner.exe end /d:"sonar.login=**YourSonarQubeToken**"

When finished, you will be able to see the results at sonarqube.com/. If it isn’t working, make sure you are using MSBuild 14 and Java 1.8 or later. The SonarQube Getting Started page is excellent if these instructions become out of date.

Run SonarQube Analysis on AppVeyor

Now that this is working locally, we can run it on AppVeyor.

Add and commit an appveyor.yml file to the root of the repository as follows

before_build:
  - nuget restore
build_script:
  - choco install "msbuild-sonarqube-runner" -y
  - MSBuild.SonarQube.Runner.exe begin /k:"YourUniqueProjectName" /d:"sonar.host.url=https://sonarqube.com" /d:"sonar.login=YourSonarQubeToken"
  - msbuild "YourProjectName.sln"
  - MSBuild.SonarQube.Runner.exe end /d:"sonar.login=YourSonarQubeToken"

Again, you can check the results at sonarqube.com.

Add a SonarQube badge to the repo

There are are variety of Quality Gate and Metrics badges available.

To add a standard badge, add the following to readme.md.

[![Quality Gate](https://sonarqube.com/api/badges/gate?key=YourUniqueProjectName)](https://sonarqube.com/dashboard/index/YourUniqueProjectName)

Integrate SonarQube with Pull Requests

SonarQube can analyze Pull Requests for quality issues, which you can see on this pull request.

This requires a GitHub authentication token, which must be secured, secure variables to be enabled in pull requests and a differential build for Pull Requests.

Get a GitHub Authentication token

Go to your profile and click “Edit Profile”. Click on “Personal access tokens” in the “Developer settings” section. Give the token any name and tick on the “public_repo” scope. Make a note of the created token (GitHubAuthToken from now on)

Secure the GitHub Authentication token

Anyone with access to this token can alter your data, contact information and billing data, so we don’t want that.

On AppVeyor, click your user name in the top right hand corner and then click “Encrypt data” from the drop down menu. Enter GitHubAuthToken in to “Value to encrypt” and click “Encrypt”. AppVeyor will then display a token which you can use in place of the real value (EncryptedGitHubAuthToken from now on).

Allowing Secure Variables in Pull Requests

Normally AppVeyor will not decrypt secure variables in Pull Requests, as in this case a Hacker could send you a PR and then read all of your secure data. However, for SonarQube to analyze Pull Requests, it is necessary. You need to decide whether you can live with this.

If you can, go to AppVeyor, click on your project, click “Settings”, tick “Enable secure variables in Pull Requests from the same repository only” and click “Save”.

Create a Pull Request Build

Modify AppVeyor.yml to ask SonarQube to publish results on standard builds, and to integrate with pull request builds. To achieve this, extra parameters are given to the SonarQube runner when if ($env:APPVEYOR_PULL_REQUEST_NUMBER) detects a Pull Request build.

environment:
  github_auth_token:
    secure: EncryptedGitHubAuthToken
before_build:
  - nuget restore
build_script:
  - choco install "msbuild-sonarqube-runner" -y
  - ps: if ($env:APPVEYOR_PULL_REQUEST_NUMBER) { MSBuild.SonarQube.Runner.exe begin /k:"YourUniqueProjectName" /d:"sonar.host.url=https://sonarqube.com" /d:"sonar.login=YourSonarQubeToken" /d:"sonar.analysis.mode=preview" /d:"sonar.github.pullRequest=$env:APPVEYOR_PULL_REQUEST_NUMBER" /d:"sonar.github.repository=YourRepositoryUrl" /d:"sonar.github.oauth=$env:EncryptedGitHubAuthToken" }
  - ps: if (-Not $env:APPVEYOR_PULL_REQUEST_NUMBER) { MSBuild.SonarQube.Runner.exe begin /k:"YourUniqueProjectName" /d:"sonar.host.url=https://sonarqube.com" /d:"sonar.login=YourSonarQubeToken" }
  - msbuild "YourProjectName.sln"
  - MSBuild.SonarQube.Runner.exe end /d:"sonar.login=YourSonarQubeToken"

Wrapping Up

SonarQube is maturing fast and is becoming industry standard, and happily it is easy to integrate Open Source projects with the publicly available SonarQube server and AppVeyor. The SonarLint Visual Studio Plugin is fantastic at spotting problems before you commit them, and the GitHub integration allows you to control the quality of contributions.

Best regards,
Cedd Burge

Follow Cedd on Twitter: @cuddlyburger
Follow AppVeyor on Twitter: @appveyor

Creating a Signed and ZipAligned APK (for Google Play) from Xamarin

2016-11-22T00:00:00+00:00

Introduction

This is a guest post by Cedd Burge, Software Developer Lead at RES.

This post is written from the point of view of someone (me) who is already proficient in C#, but was new to Xamarin, Mobile phone development, and AppVeyor.

It contains from scratch steps to create a Xamarin Android application (in Visual Studio), to build it on AppVeyor and to publish it to the Play Store. You can look at the repo I created to test this post if you get stuck.

First, install Xamarin from https://www.xamarin.com/download.

Create a new repository on GitHub

If you are new to GitHub, see this getting started guide, otherwise simply create a new repo and git clone it somewhere convenient.

Create a new Xamarin Portable Class Library (PCL) project

In my version of Visual Studio (Community 2015), this is done by clicking “File - New - Project” on the main menu and then selecting “Blank App (Xamarin.Forms Portable)” from “Templates - Visual C# - Cross-Platform”. Give it a interesting name, which I will assume to be YourAppName for the rest of this post.

Run the app!

Select the YourAppName.Droid project and run it. This should show the bare bones app in an emulator.

If you have Hyper-V enabled (maybe you use Docker), then you might get an Deployment Error when doing this. Disable Hyper-V and restart your machine to fix this.

You might also run in to compile errors due to ridiculous dependency weirdness.

Create APK file manually

To tell Google about your app, you have to make some changes to the Properties\AndroidManifest.xml file of the YourAppName.Droid project.

Add a package="com.yourappname" (or similar) attribute to the root manifest node. The package name must be unique on Google Play and must follow normal java package name conventions. Most people use their url in reverse (eg com.yourappname instead of yourappname.com) and stick to lower case.
Add an android:versionCode="1" attribute to the root manifest node. This is an integer and it must be incremented every time you upload an apk on Google Play.
Add an android:versionName="0.1 attribute to the root manifest node. This value can be anything you like and is displayed in Google Play.
Change the label attribute on the application node to YourAppName.

Visual studio has some tools to create an APK, and they seem to be in constant churn, but at the time of writing, the process is as follows.

Change the “Build Configuration” to “App Store”.
Select the YourAppName.droid project and click “Build - Archive” from the main menu.
The archive manager window will appear and build your app.
Click “Distribute”, which will pop up the Distribute window.
Initially you won’t have a signing identity, so click on the green plus button and fill in the details to create one.
Once created, double click on it and note where it is on disk (YourKeyStoreFilename from now on)
Click “Save As” to create an APK.

Upload the APK to Google Play

Create a Developer Account on Google Play Developer Console (this costs $25)
Click “Add New Application” and add YourAppName
Upload the APK that you saved in the previous step

Publish the App on Google Play (probably just to Alpha or Beta)

There are some requirements when publishing an application to Google Play, and these are likely to change, but happily google tells you what they all are. If you click on “Why can’t I publish?”, near the top right corner of the page, you will get a list of things to do.

It’s all simple stuff that can be done within the Developer Console. Some screenshots and pictures are required. If you just want to get a test version up quickly, then feel free to use mine temporarily.

There are a lot of optional things you can do as well, which can be worthwhile if you want to publish a killer app. The Google Launch Checklist, is comprehensive, but takes a long time to read.

Automate APK creation locally

When working with appveyor, it always makes sense to test on your own computer first. The feedback is immediate and you iterate very quickly. It takes a lot longer to modify the appveyor.yml file, push it and wait for a build to go through. Also, if it works locally but doesn’t work on AppVeyor, you know the problem is a configuration difference between your computer and the AppVeyor environment (eg a different version of msbuild).

Being as we are making a new version of the apk, we need to increment android:versionCode in Properties/AndroidManifest.xml.

There are some Xamarin MSBuild targets, which we can use to create a Signed and ZipAligned apk as below.

There are 2 passwords required in the command because java KeyStores can contain multiple Alias’. So the first password is to access the KeyStore, and the second one is to access the specific alias. Visual studio hides this complexity from you and assigns the same password to both places.

I do a lot of work in GIT Bash, but this statement only works in Batch (the windows command line), I think because of parameter escaping.

MSBuild "/t:SignAndroidPackage" "/p:Configuration=Release" "/p:AndroidKeyStore=true" "/p:AndroidSigningKeyAlias=YourKeyAlias" "/p:AndroidSigningKeyPass=YourKeyStorePassword" "/p:AndroidSigningKeyStore=YourKeyStoreFilename" "/p:AndroidSigningStorePass=YourKeyStorePassword" "YourAppName.csproj"

This will create com.yourappname-Signed.apk in the bin\release folder. Upload this to Google Play to make sure that everything is working properly.

Automate APK creation on AppVeyor

You will need to link an AppVeyor account to your GitHub one, so let’s do that:

Navigate to your repo in GitHub
Click “Settings” on the repo
Click “Integrations and services”
Click “Browse Directory”
Click “AppVeyor”
Click “Configure”
Click “Grant Access”

Now Log in to AppVeyor.com, probably using your GitHub account

Click “Projects”
Click “New Project”
Choose your GitHub repository and click “Add”

MSBuild needs to access your KeyStore file in order to sign the apk, so copy YourKeyStoreFilename in to the folder of YourAppName.Droid project (called YourKeyStoreLocalFilename from now on).

When we created the apk from the command line, we entered in some passwords, and we obviously can’t save these passwords to a public Git repository. Happily AppVeyor have thought of this, and you can convert passwords in to tokens that can be exposed publically.

To do this, click your user name in the top right hand corner and then click “Encrypt data” from the drop down menu. Enter YourKeyStorePassword in to “Value to encrypt” and click “Encrypt”. AppVeyor will then display a token which you can use in place of the real value.

Now that we have everything we need, add an appveyor.yml file to the root of your repository as below. Note that YourLocalKeyStoreFilename is relative to the csproj file being built (the YourAppName.Droid folder below).

environment:
  keystore-password:
    secure: DSwAr4fYt3Q35Sjob5qAN5uj # YourPassword for keystore
before_build:
  - nuget restore
build_script:
  - msbuild "/t:SignAndroidPackage" "/p:Configuration=Release" "/p:AndroidKeyStore=true" "/p:AndroidSigningKeyAlias=YourKeyAlias" "/p:AndroidSigningKeyPass=%keystore-password%" "/p:AndroidSigningKeyStore=YourLocalKeyStoreFilename" "/p:AndroidSigningStorePass=%keystore-password%"  "YourAppName.Droid\YourAppName.csproj"
artifacts:
  - path: YourAppName.Droid\bin\Release\com.yourappname-Signed.apk

Remember to update android:versionCode and then push to GitHub. This will trigger a build on AppVeyor. The build_script section will call msbuild to create the signed and zipaligned apk file, and the artifacts section will archive the apk file so we can download it later.

Go to AppVeyor.com, click on your project, click on “Artifacts”, download the apk file, and then upload it to google to check that everything has worked properly.

Conclusion

There is a lot to learn to get everything working, and I couldn’t find a single source for all of these things, but having done it once, the process is actually quite simple, and the tools and services involved are generally a pleasure to work with.

Best regards,
Cedd Burge

Follow Cedd on Twitter: @cuddlyburger
Follow AppVeyor on Twitter: @appveyor

AWS Elastic Beanstalk

2016-11-07T00:00:00+00:00

Appveyor does not support AWS Elastic Beanstalk deployment out of the box right now. However it can be automated in Appveyor with help of some scripting. Here is small guide based on this support forum discussion.

In the root folder of your web application create text file named awsdeploy.txt.

Add the following to awsdeploy.txt:

Template = ElasticBeanstalk
Container.ApplicationHealthcheckPath = /healthcheck

Add AWSAccessKeyId as environment variable and AWSSecretKey as secure environment variable.
Set Package Web Applications for XCopy deployment in build stage.

Set the following as a deployment script:

$packageweb = $artifacts.values | Where-Object { $_.path -like '*WebApplication1.zip' }
$exe = "C:\Program Files (x86)\AWS Tools\Deployment Tool\awsdeploy.exe"
&$exe -r "-DDeploymentPackage=$($packageweb.path)" "-DEnvironment.Name=MyAppWeb-test123" "-DApplication.Name=MyAppWeb123" "-DRegion=eu-west-1" "-DAWSAccessKey=$env:AWSAccessKeyId" "-DAWSSecretKey=$env:AWSSecretKey" "C:\projects\WebApplication1\awsdeploy.txt"

Note that this script assumes that application was already deployed at least once to Beanstalk, otherwise you need to replace -r switch with -w for single first deployment.

Here is an example YAML (only relevant parts):

environment:
  AWSAccessKeyId: AKIAIODIUCY3ETD6TEST
  AWSSecretKey:
    secure: LEvzbXpiLkWVvswonFHnAYV9ZS6fEFL3wswjTcIQ6ZXC5j1nynd6N0Bs/VFtest
build:
  publish_wap_xcopy: true
deploy_script:
- ps: |
    $packageweb = $artifacts.values | Where-Object { $_.path -like '*WebApplication1.zip' }
    $exe = "C:\Program Files (x86)\AWS Tools\Deployment Tool\awsdeploy.exe"
    &$exe -r "-DDeploymentPackage=$($packageweb.path)" "-DEnvironment.Name=MyAppWeb-test123" "-DApplication.Name=MyAppWeb123" "-DRegion=eu-west-1" "-DAWSAccessKey=$env:AWSAccessKeyId" "-DAWSSecretKey=$env:AWSSecretKey" "C:\projects\WebApplication1\awsdeploy.txt"

Here is an example web application folder structure:

Directory of C:\Projects\WebApplication1

09/15/2016  02:48 AM    <DIR>          .
09/15/2016  02:48 AM    <DIR>          ..
07/17/2016  12:34 PM               505 .gitattributes
07/17/2016  12:34 PM             2,858 .gitignore
09/15/2016  02:40 AM                80 awsdeploy.txt
08/30/2016  03:22 PM    <DIR>          WebApplication1
07/21/2016  06:51 PM             1,012 WebApplication1.sln

Enjoy!

The new build cache

2016-09-28T00:00:00+00:00

AppVeyor runs every build on a clean virtual machine. Virtual machine state is not preserved between builds which means every build downloads sources, installs NuGet packages, Node.js modules, Ruby gems or pulls dependencies. Build cache allows you to preserve contents of selected directories and files between project builds.

AppVeyor was the first hosted CI to introduce a build cache and over the time it became a very popular build tool and important infrastructure component. There were some limitations though, such as maximum cache entry size of 500 MB and intermittent save/restore lags due to ever changing networking conditions.

Increasing requirements from larger customers running builds with “heavy” dependencies made us to re-visit cache architecture and thus the new and updated build cache was born! The new build cache lives close to build worker VMs, it’s fast and offers virtually unlimited possibilities for scale and has lower update/restore times.

Cache size

With the introduction of the new cache we are also changing the way it’s metered.

The total size of build cache is limited per account and depends on the plan:

Free	Basic	Pro	Premium
1 GB	1 GB	5 GB	20 GB

It’s a hard quota which means the build will fail while trying to upload cache item exceeding the quota. The maximum size of a single cache entry cannot be larger than the size of cache.

Cache speed vs size

The new cache uses 7z to compress/uncompress files before transferring them to the cache storage. We chose 7z over built-in .NET compression library because it’s generally faster, produces smaller archives and works with hidden files out-of-the-box.

While compressing cache item, by default AppVeyor uses 7z with zip algorithm and compression level 1 (“Fastest”) thus producing archive faster, but with larger size (-tzip -mx=1 args). However, you can change compression behavior of 7z by providing your own command line args in APPVEYOR_CACHE_ENTRY_ZIP_ARGS environment variable. For example, to enable LZMA compression method with the highest possible compression ratio set this variable to -t7z -m0=lzma -mx=9.

Availability

The new build cache is currently in beta. It’s automatically enabled for all new accounts.

We are rolling out new cache to existing accounts in batches while observing performance. If you want participate in beta sooner or noticed any issues with the build cache please let us know.

Best regards,
AppVeyor team

Migrating build environment to RackSpace

2016-07-16T00:00:00+00:00

Dear customers,

We have great news! Today we moved all open-source and most private accounts to a new build environment hosted at RackSpace.

For open-source projects that means their builds will start and run faster and private projects may notice better performance as well.

Virtual machines in the new environment have 2 CPU cores and up to 4 GB of RAM. The new public IP address for that environment is 74.205.54.20 - you may need to update your firewalls.

We are still going to maintain Google Compute Engine (GCE) environment for selected accounts and as a backup build cloud. If your private builds were previously running on GCE they will remain there.

Some open-source projects may experience issues while building on a new environment. Due to implementation differences between GCE and Hyper-V platforms we have separate build worker images for GCE and Hyper-V environments, so there might be minor discrepancies in installed software. Please report any issues you notice - your help with fixing those issues is much appreciated.

We have a good reason for this move as some customers already reported 10x performance increase!

We would love to hear your feedback!

Best regards,
AppVeyor team

Deployment projects

2015-11-04T00:00:00+00:00

Sometimes your deployment requirements cannot fit into AppVeyor built-in deployment providers such as FTP, Web Deploy, S3 and others, for example, you are deploying to Elastic Beanstalk, or you have to recycle application pool during the deployment.

You can write a script doing custom deployment job, however this script can be run only on build worker VM during the build. “Environments” do not support custom scripts and there is no “Script” provider. This is because “Environment” deployments run on a shared background worker severs where potentially insecure custom scripting is not allowed.

This article demonstrates how you can simulate “Script” environment with regular builds and deploy project artifacts to any environment with your own custom script.

Solution overview

The basic idea is having two AppVeyor projects:

Main project - this is your main project running tests and producing artifacts.
Deployment project - helper project downloading artifacts from specific build of “Main project” and deploying them with your custom script. Here, to “deploy” the build of “Main project” you could either manually run new build of deployment project from UI or use AppVeyor API.

Deployment project

The “core” of deployment project is PowerShell script downloading artifacts. The script uses AppVeyor API to find specific build and download its artifacts. Artifacts are downloaded to the root of build directory (%APPVEYOR_BUILD_FOLDER%).

The following environment variables must be set for script to work:

api_token - AppVeyor REST API authentication token. Can be found/generated on this page.
deploy_project - project slug to download artifacts from. Project slug can be seen in the project URL.
deploy_version - build version to deploy. If not specified artifacts from the most recent version will be downloaded.
deploy_artifact - file name or deployment name of artifact to download. If not specified all artifacts will be downloaded.

These environment variables can be set on Environment tab of deployment project settings or in appveyor.yml:

environment:
  api_token:
    secure: ABC123==
  deploy_project: my-web
  deploy_version: ''            # download artifacts from latest build if no version specified
  deploy_artifact: ''           # download all artifacts if empty

To download artifacts add the following PS script into “Before deploy” section of your project settings or before_deploy scripts of appveyor.yml:

before_deploy:
  - ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/deploy.ps1'))

Your own deployment logic can be put under deploy_script section, for example:

deploy_script:
  - command stopping app pool
  - '"C:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:package="%webdeploy_package%" -dest:auto,ComputerName="%webdeploy_server%",UserName="%webdeploy_username%",Password="%webdeploy_password%",AuthType="Basic" -setParam:"IIS Web Application Name"="%webdeploy_site%" -allowUntrusted'
  - ...
  - command starting app pool

Add these to disable automatic build and test phases:

build: off
test: off

Setup notifications if required:

notifications:
  - provider: <provider_1>
    settings: ...

Complete appveyor.yml example:

environment:
  # AppVeyor API token for your account, project, version and artifact(s) to download
  api_token:
    secure: ABC123==
  deploy_project: my-web
  deploy_version: ''            # download artifacts from latest build if no version specified
  deploy_artifact: ''           # download all artifacts if empty

  # deployment-specific settings
  # we are going to deploy using Web Deploy, so...
  webdeploy_package: '%appveyor_build_folder%\MyWebApp.zip'
  webdeploy_server: https://test.scm.azurewebsites.net:443/msdeploy.axd?site=test
  webdeploy_site: test
  webdeploy_username: $test
  webdeploy_password:
    secure: AAABBBCCC123==

# download project artifacts
before_deploy:
  - ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/deploy.ps1'))

# here is your own custom deployment code
deploy_script:
  - '"C:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:package="%webdeploy_package%" -dest:auto,ComputerName="%webdeploy_server%",UserName="%webdeploy_username%",Password="%webdeploy_password%",AuthType="Basic" -setParam:"IIS Web Application Name"="%webdeploy_site%" -allowUntrusted'

# notifications
notifications:
  - provider: <provider_1>
    settings: ...

# disable build and test pahses
build: off
test: off

deploy.yml

If main and deployment projects share the same repository (most probably they do) you can put YAML deployment settings to a separate deploy.yml file, next to appveyor.yml and then set “Custom configuration .yml file name” on General tab to deploy.yml. To disable automatic triggering of deployment project on webhook you can either remove its webhook from repository or set branch filter to some non-existent branch, for example:

branches:
  only:
    - deployment-project

Main project

Main project is the project creating application packages and uploading them to build artifacts. With artifacts in place you can call deployment project (either manually or via API) to download and deploy artifacts.

To start a new build of deployment project during the build use Start-AppveyorBuild cmdlet:

environment:
  api_token:
    secure: ABC123==

deploy_script:
  - ps: Start-AppveyorBuild -ApiKey $env:api_token -ProjectSlug deploy-project -EnvironmentVariables @{ "deploy_version" = $env:appveyor_build_version }

Enjoy!

Visual Studio 2015 RTM on Pro environment

2015-08-02T00:00:00+00:00

Great news for AppVeyor customers! We’ve finally managed to run Visual Studio 2015 image on Pro environment!

If you are on a paid plan use Visual Studio 2015 image (“OS” setting on Environment tab of project settings or os: Visual Studio 2015 in appveyor.yml).

It still takes around 40-50 seconds before build starts - this is the time required to boot up build worker VM with Visual Studio 2015 image. But everything is going on Pro environment and build start time is quite predictable there. We hope it’s acceptable trade-off between performance and convenience.

Those customer accounts that were moved to a “new” faster OSS environment where Visual Studio 2015 is installed on build workers by default can be moved back to Pro environment, so make sure you switch your builds to use Visual Studio 2015 image.

How AppVeyor Helps Snipcart Automate Product Development & Deployment

2015-07-30T00:00:00+00:00

This is a guest blog post from Charles Ouellet of Snipcart. Charles is a co-founder and lead engineer at Snipcart, a solution empowering developers to turn any website into a customizable e-commerce platform. He likes code, scotch, and colourful socks. You can follow him on Twitter.

When we launched our developer-centric e-commerce platform, finding a premium cloud-based continuous integration solution was a top priority. We used Jenkins at first, and, while we liked it, we were also looking forward to handling those operations directly in the cloud. For months, we kept our eyes and ears opened, searching for such a solution as we kept developing our product and growing our business.

A few months ago, I stumbled upon AppVeyor while exploring the Tweetosphere one night. Upon skimming through their home page and documentation, I quickly realized this was exactly what our team and product needed. Since Snipcart’s API is built on top of ASP.NET Web API, we needed to find a web-based solution that was supporting .NET. And that’s exactly what I found that night with AppVeyor.

The day after that, we spent maybe an hour setting it up, and it worked like a real charm.

How we use AppVeyor exactly for our Snipcart application

One of AppVeyor’s killer feature is that it gives you the ability to configure your whole build with a YAML file. This allows us to have the build configuration in our source control, making it very easy to maintain.

To give you a little context: Snipcart is basically a web application, a worker process that processes queued events, and two JavaScript applications that consume our API. Let’s get more specific with each of these components.

Web application

Like I mentioned earlier, our web application is an ASP.NET Web API. Once the build is completed and all our tests have passed (yep, AppVeyor supports running unit tests too!), we deploy our application to our Azure web apps via WebDeploy. The build process and the deployment process are all configured in the YAML file. We just push to our production git branch, and it triggers a build and a deployment.

Once it’s deployed, Azure takes care of spawning multiple instances when needed.

Worker

Our worker is a Cloud Service hosted on Azure. Once the build is completed, it is automatically deployed by AppVeyor as well, which supports a wide range of deployment processes, including Cloud Services. Before making the switch to AppVeyor, this was a pain for us because our developers needed to deploy the worker through Visual Studio directly; it wasn’t automated, which could have led to errors.

JavaScript applications

All of our client applications are single page apps built on top of Backbone.js. We use webpack to bundle our application. We have a gulp task that does all the job, bundling the numerous JavaScript files into a single file and uglifying the output. AppVeyor also allows us to run this gulp task inside our build process, which is pretty amazing. It doesn’t only allow to build .NET applications: you can use npm, gem, and any NodeJS modules.

We also have unit tests for these two applications; we run those in AppVeyor as well, using a gulp task.

CDN

Once our JavaScript applications are built, we automatically push all of our assets on an Azure Blob Storage container. This is another deployment option provided by AppVeyor. We upload all of the static files that are included on our customers websites, such as our default stylesheet, our snipcart.js file and our custom web fonts.

On top of the storage, we are also using KeyCDN. We have a pull zone that fetches the content of our Blob storage. KeyCDN takes care of the caching and everything needed to make sure our static files are served as fast as possible. We discuss it further in this blog post.

When we make changes to snipcart.js or to any other static file that is cached by the CDN, we need to invalidate the cache. If we don’t, customers would need to hit refresh multiple times, which would not make any sense. With AppVeyor, we make an HTTP request to the KeyCDN API to purge our cache zone after the Blob storage has been updated. By doing so, we are sure our customers always have the latest version of our static files.

Conclusion? We friggin’ love AppVeyor

Since we automated everything with AppVeyor, we are much more confident with our deployments. Sometimes, we even start a build and go play a few ping pong games at the office while it deploys. We know it’ll just work. With our solid test suite in place, we trust AppVeyor won’t deploy something that is broken.

As I finish writing this post, I realize that I don’t see how we could work without this amazing tool today. It’s a crucial part of our product development process.

AppVeyor

Sharing configuration between branches in appveyor.yml

The problem

The solution

CodeCov Test Coverage Integration

Create a new GitHub Repo

Create a new System Under Test (SUT) project and a new Test project

Setup AppVeyor Integration

Sign up with CodeCov

Run Coverage Analysis Locally

Analyse

Upload

Run Coverage Analysis on AppVeyor

Add Coverage Graphics to the Repository

Integrate CodeCov with Pull Requests

Install the browser plugin

Conclusions

SonarQube Analysis

Create a new GitHub repository

Create a new project

Integrate with AppVeyor

Sign up with SonarQube and generate an Authentication Token

Run SonarQube Analysis Locally

Install chocolatey

Install SonarQube MSBuild Runner

Analyze and upload to SonarQube

Run SonarQube Analysis on AppVeyor

Add a SonarQube badge to the repo

Integrate SonarQube with Pull Requests

Get a GitHub Authentication token

Secure the GitHub Authentication token

Allowing Secure Variables in Pull Requests

Create a Pull Request Build

Wrapping Up

Creating a Signed and ZipAligned APK (for Google Play) from Xamarin

Introduction

Create a new repository on GitHub

Create a new Xamarin Portable Class Library (PCL) project

Run the app!

Create APK file manually

Upload the APK to Google Play

Publish the App on Google Play (probably just to Alpha or Beta)

Automate APK creation locally

Automate APK creation on AppVeyor

Conclusion

AWS Elastic Beanstalk

The new build cache

Cache size

Cache speed vs size

Availability

Migrating build environment to RackSpace

Deployment projects

Solution overview

Deployment project

deploy.yml

Main project

Visual Studio 2015 RTM on Pro environment

How AppVeyor Helps Snipcart Automate Product Development & Deployment

How we use AppVeyor exactly for our Snipcart application

Web application

Worker

JavaScript applications

CDN

Conclusion? We friggin’ love AppVeyor