AppVeyor runs every build on a clean virtual machine. Virtual machine state is not preserved between builds which means every build downloads sources, installs NuGet packages, Node.js modules, Ruby gems or pulls dependencies. Build cache allows you to preserve contents of selected directories and files between project builds.

AppVeyor was the first hosted CI to introduce a build cache and over the time it became a very popular build tool and important infrastructure component. There were some limitations though, such as maximum cache entry size of 500 MB and intermittent save/restore lags due to ever changing networking conditions.

Increasing requirements from larger customers running builds with “heavy” dependencies made us to re-visit cache architecture and thus the new and updated build cache was born! The new build cache lives close to build worker VMs, it’s fast and offers virtually unlimited possibilities for scale and has lower update/restore times.

Cache size

With the introduction of the new cache we are also changing the way it’s metered.

The total size of build cache is limited per account and depends on the plan:

It’s a hard quota which means the build will fail while trying to upload cache item exceeding the quota. The maximum size of a single cache entry cannot be larger than the size of cache.

Cache speed vs size

The new cache uses 7z to compress/uncompress files before transferring them to the cache storage. We chose 7z over built-in .NET compression library because it’s generally faster, produces smaller archives and works with hidden files out-of-the-box.

While compressing cache item, by default AppVeyor uses 7z with zip algorithm and compression level 1 (“Fastest”) thus producing archive faster, but with larger size (-tzip -mx=1 args). However, you can change compression behavior of 7z by providing your own command line args in APPVEYOR_CACHE_ENTRY_ZIP_ARGS environment variable. For example, to enable LZMA compression method with the highest possible compression ratio set this variable to -t7z -m0=lzma -mx=9.

Availability

The new build cache is currently in beta. It’s automatically enabled for all new accounts.

We are rolling out new cache to existing accounts in batches while observing performance. If you want participate in beta sooner or noticed any issues with the build cache please let us know.

Best regards,
AppVeyor team

Follow us on Twitter: @appveyor

Dear customers,

We have great news! Today we moved all open-source and most private accounts to a new build environment hosted at RackSpace.

For open-source projects that means their builds will start and run faster and private projects may notice better performance as well.

Virtual machines in the new environment have 2 CPU cores and up to 4 GB of RAM. The new public IP address for that environment is 74.205.54.20 - you may need to update your firewalls.

We are still going to maintain Google Compute Engine (GCE) environment for selected accounts and as a backup build cloud. If your private builds were previously running on GCE they will remain there.

Some open-source projects may experience issues while building on a new environment. Due to implementation differences between GCE and Hyper-V platforms we have separate build worker images for GCE and Hyper-V environments, so there might be minor discrepancies in installed software. Please report any issues you notice - your help with fixing those issues is much appreciated.

We have a good reason for this move as some customers already reported 10x performance increase!

We would love to hear your feedback!

Best regards,
AppVeyor team

Follow us on Twitter: @appveyor

Sometimes your deployment requirements cannot fit into AppVeyor built-in deployment providers such as FTP, Web Deploy, S3 and others, for example, you are deploying to Elastic Beanstalk, or you have to recycle application pool during the deployment.

You can write a script doing custom deployment job, however this script can be run only on build worker VM during the build. “Environments” do not support custom scripts and there is no “Script” provider. This is because “Environment” deployments run on a shared background worker severs where potentially insecure custom scripting is not allowed.

This article demonstrates how you can simulate “Script” environment with regular builds and deploy project artifacts to any environment with your own custom script.

Solution overview

The basic idea is having two AppVeyor projects:

• Main project - this is your main project running tests and producing artifacts.
• Deployment project - helper project downloading artifacts from specific build of “Main project” and deploying them with your custom script. Here, to “deploy” the build of “Main project” you could either manually run new build of deployment project from UI or use AppVeyor API.

Deployment project

The “core” of deployment project is PowerShell script downloading artifacts. The script uses AppVeyor API to find specific build and download its artifacts. Artifacts are downloaded to the root of build directory (%APPVEYOR_BUILD_FOLDER%).

The following environment variables must be set for script to work:

• api_token - AppVeyor REST API authentication token. Can be found/generated on this page.
• deploy_project - project slug to download artifacts from. Project slug can be seen in the project URL.
• deploy_version - build version to deploy. If not specified artifacts from the most recent version will be downloaded.
• deploy_artifact - file name or deployment name of artifact to download. If not specified all artifacts will be downloaded.

These environment variables can be set on Environment tab of deployment project settings or in appveyor.yml:

environment:
  api_token:
    secure: ABC123==
  deploy_project: my-web
  deploy_version: ''            # download artifacts from latest build if no version specified
  deploy_artifact: ''           # download all artifacts if empty

To download artifacts add the following PS script into “Before deploy” section of your project settings or before_deploy scripts of appveyor.yml:

before_deploy:
  - ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/deploy.ps1'))

Your own deployment logic can be put under deploy_script section, for example:

deploy_script:
  - command stopping app pool
  - '"C:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:package="%webdeploy_package%" -dest:auto,ComputerName="%webdeploy_server%",UserName="%webdeploy_username%",Password="%webdeploy_password%",AuthType="Basic" -setParam:"IIS Web Application Name"="%webdeploy_site%" -allowUntrusted'
  - ...
  - command starting app pool

Add these to disable automatic build and test phases:

build: off
test: off

Setup notifications if required:

notifications:
  - provider: <provider_1>
    settings: ...

Complete appveyor.yml example:

environment:
  # AppVeyor API token for your account, project, version and artifact(s) to download
  api_token:
    secure: ABC123==
  deploy_project: my-web
  deploy_version: ''            # download artifacts from latest build if no version specified
  deploy_artifact: ''           # download all artifacts if empty

  # deployment-specific settings
  # we are going to deploy using Web Deploy, so...
  webdeploy_package: '%appveyor_build_folder%\MyWebApp.zip'
  webdeploy_server: https://test.scm.azurewebsites.net:443/msdeploy.axd?site=test
  webdeploy_site: test
  webdeploy_username: $test
  webdeploy_password:
    secure: AAABBBCCC123==

# download project artifacts
before_deploy:
  - ps: iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/deploy.ps1'))

# here is your own custom deployment code
deploy_script:
  - '"C:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:package="%webdeploy_package%" -dest:auto,ComputerName="%webdeploy_server%",UserName="%webdeploy_username%",Password="%webdeploy_password%",AuthType="Basic" -setParam:"IIS Web Application Name"="%webdeploy_site%" -allowUntrusted'

# notifications
notifications:
  - provider: <provider_1>
    settings: ...

# disable build and test pahses
build: off
test: off

deploy.yml

If main and deployment projects share the same repository (most probably they do) you can put YAML deployment settings to a separate deploy.yml file, next to appveyor.yml and then set “Custom configuration .yml file name” on General tab to deploy.yml. To disable automatic triggering of deployment project on webhook you can either remove its webhook from repository or set branch filter to some non-existent branch, for example:

branches:
  only:
    - deployment-project

Main project

Main project is the project creating application packages and uploading them to build artifacts. With artifacts in place you can call deployment project (either manually or via API) to download and deploy artifacts.

To start a new build of deployment project during the build use Start-AppveyorBuild cmdlet:

environment:
  api_token:
    secure: ABC123==

deploy_script:
  - ps: Start-AppveyorBuild -ApiKey $env:api_token -ProjectSlug deploy-project -EnvironmentVariables @{ "deploy_version" = $env:appveyor_build_version }

Enjoy!

Follow us on Twitter: @appveyor

Great news for AppVeyor customers! We’ve finally managed to run Visual Studio 2015 image on Pro environment!

If you are on a paid plan use Visual Studio 2015 image (“OS” setting on Environment tab of project settings or os: Visual Studio 2015 in appveyor.yml).

It still takes around 40-50 seconds before build starts - this is the time required to boot up build worker VM with Visual Studio 2015 image. But everything is going on Pro environment and build start time is quite predictable there. We hope it’s acceptable trade-off between performance and convenience.

Those customer accounts that were moved to a “new” faster OSS environment where Visual Studio 2015 is installed on build workers by default can be moved back to Pro environment, so make sure you switch your builds to use Visual Studio 2015 image.

This is a guest blog post from Charles Ouellet of Snipcart. Charles is a co-founder and lead engineer at Snipcart, a solution empowering developers to turn any website into a customizable e-commerce platform. He likes code, scotch, and colourful socks. You can follow him on Twitter.

When we launched our developer-centric e-commerce platform, finding a premium cloud-based continuous integration solution was a top priority. We used Jenkins at first, and, while we liked it, we were also looking forward to handling those operations directly in the cloud. For months, we kept our eyes and ears opened, searching for such a solution as we kept developing our product and growing our business.

A few months ago, I stumbled upon AppVeyor while exploring the Tweetosphere one night. Upon skimming through their home page and documentation, I quickly realized this was exactly what our team and product needed. Since Snipcart’s API is built on top of ASP.NET Web API, we needed to find a web-based solution that was supporting .NET. And that’s exactly what I found that night with AppVeyor.

The day after that, we spent maybe an hour setting it up, and it worked like a real charm.

How we use AppVeyor exactly for our Snipcart application

One of AppVeyor’s killer feature is that it gives you the ability to configure your whole build with a YAML file. This allows us to have the build configuration in our source control, making it very easy to maintain.

To give you a little context: Snipcart is basically a web application, a worker process that processes queued events, and two JavaScript applications that consume our API. Let’s get more specific with each of these components.

Web application

Like I mentioned earlier, our web application is an ASP.NET Web API. Once the build is completed and all our tests have passed (yep, AppVeyor supports running unit tests too!), we deploy our application to our Azure web apps via WebDeploy. The build process and the deployment process are all configured in the YAML file. We just push to our production git branch, and it triggers a build and a deployment.

Once it’s deployed, Azure takes care of spawning multiple instances when needed.

Worker

Our worker is a Cloud Service hosted on Azure. Once the build is completed, it is automatically deployed by AppVeyor as well, which supports a wide range of deployment processes, including Cloud Services. Before making the switch to AppVeyor, this was a pain for us because our developers needed to deploy the worker through Visual Studio directly; it wasn’t automated, which could have led to errors.

JavaScript applications

All of our client applications are single page apps built on top of Backbone.js. We use webpack to bundle our application. We have a gulp task that does all the job, bundling the numerous JavaScript files into a single file and uglifying the output. AppVeyor also allows us to run this gulp task inside our build process, which is pretty amazing. It doesn’t only allow to build .NET applications: you can use npm, gem, and any NodeJS modules.

We also have unit tests for these two applications; we run those in AppVeyor as well, using a gulp task.

CDN

Once our JavaScript applications are built, we automatically push all of our assets on an Azure Blob Storage container. This is another deployment option provided by AppVeyor. We upload all of the static files that are included on our customers websites, such as our default stylesheet, our snipcart.js file and our custom web fonts.

On top of the storage, we are also using KeyCDN. We have a pull zone that fetches the content of our Blob storage. KeyCDN takes care of the caching and everything needed to make sure our static files are served as fast as possible. We discuss it further in this blog post.

When we make changes to snipcart.js or to any other static file that is cached by the CDN, we need to invalidate the cache. If we don’t, customers would need to hit refresh multiple times, which would not make any sense. With AppVeyor, we make an HTTP request to the KeyCDN API to purge our cache zone after the Blob storage has been updated. By doing so, we are sure our customers always have the latest version of our static files.

Conclusion? We friggin’ love AppVeyor

Since we automated everything with AppVeyor, we are much more confident with our deployments. Sometimes, we even start a build and go play a few ping pong games at the office while it deploys. We know it’ll just work. With our solid test suite in place, we trust AppVeyor won’t deploy something that is broken.

As I finish writing this post, I realize that I don’t see how we could work without this amazing tool today. It’s a crucial part of our product development process.