Managing your git repository via REST API in VSTS

Let's say you'd like to store some results of a build or a release inside a repository. It could be any reason - easy access, versioning, the only tool you have access to. By default VSTS doesn't provide any kind of git-related steps, which could be helpful in such case. Fortunately, once more its REST API comes to the rescue, giving us an opportunity to fully manage repositories with the possibility to push multiple commits.

Before we start, take a look at the overview to understand what are the general capabilities of this API.

Making an initial commit

Once you have your git repository created in VSTS, you can either initialize it with a commit from the GUI or push an initial commit using API. If you take a look at the example from the documentation:

/
POST /_apis/git/repositories/{repository}/pushes?api-version={version}

and its body:

/
{
  "refUpdates": [
    {
      "name": "refs/heads/master",
      "oldObjectId": "0000000000000000000000000000000000000000"
    }
  ],
  "commits": [
    {
      "comment": "Initial commit.",
      "changes": [
        {
          "changeType": "add",
          "item": {
            "path": "/readme.md"
          },
          "newContent": {
            "content": "My first file!",
            "contentType": "rawtext"
          }
        }
      ]
    }
  ]
}

you'll see mostly self-descriptive properties, which build this JSON. The only thing - oldObjectId - is not so obvious. Basically it's SHA1 of the commit this commit is based on - since there're no commits yet, it's basicaly a string full of zeros.

Pushing data

Making an initial commit is a piecie of cake. What if we'd like to update an existing file? The main issue here is to find an oldObjectId, which is required to actually make a request successful. Once more the API becomes handy here - what we can do is to fetch a list list of all pushes and take the last one. Take a look at the signature from the documentation:

/
GET https://{instance}/DefaultCollection/_apis/git/repositories/{repository}/pushes?api-version={version}[&fromDate={dateTime}&toDate={dateTime}&pusherId={guid}&$skip={integer}&$top={integer}]

What is great about this request is the possibility to filter the data - we don't have to download all pushes, only those from the date interval, made by a specific pusher or maybe only the top N. The response gives us a list of pushes ordered from the newest to the oldest. That is important here is to pass includeRefUpdates=true parameter in the query string. This way we'll get following additional property in the response:

/
{
          "repositoryId": "04baf35b-faec-4619-9e42-ce2d0ccafa4c",
          "name": "refs/heads/master",
          "oldObjectId": "0000000000000000000000000000000000000000",
          "newObjectId": "5e108508e2151f5513fffaf47f3377eb6e571b20"
}

and we're able to refer to the newObjectId property to make an update. Once we have it, we can use once more the endpoint used to create an initial commit with a slightly modified body:

/
{
  "refUpdates": [
    {
      "name": "refs/heads/master",
      "oldObjectId": "5e108508e2151f5513fffaf47f3377eb6e571b20"
    }
  ],
  "commits": [
    {
      "comment": "Added a few more items to the task list.",
      "changes": [
        {
          "changeType": "edit",
          "item": {
            "path": "/readme.md"
          },
          "newContent": {
            "content": "Modified readme file!",
            "contentType": "rawtext"
          }
        }
      ]
    }
  ]
}

Once we post this request, a new commit should be pushed and visible when you access a repository.

Building feature branches on VSTS automatically

Currently VSTS lacks(see edit below) really handy feature, which lets you trigger a build based on a branch wildcard. For instance, I'd like to run my build definition only for refs/heads/feature/* branches. While it's super-easy when using e.g. TeamCity, with VSTS it requires us to prepare a smart workaround.

Service hooks

VSTS provides us a wide collection of different service hooks. They can be used to command builds, analyze logs or trigger any kind of an event. In fact the list of available services is already impressing and gets bigger - I strongly recommend you to take a look when you have a minute.

Besides integrating with third-party services like Slack, Bamboo or Trello, it allows us to use a simple Web Hook, which can send a specific event to a specific endpoint. Because we can send any event to any endpoint, the sky is the only limit - you can create a "lambda" using Azure Function(like me), a simple WebAPI or old but still reliable MVC application using technology stack of your choice. Whatever solution you'll create, it will work as long as it can process HTTP requests.

Kick it back!

Retrieving an event from VSTS is cool but gives information only. What we really need is to orchestrate VSTS to schedule a build using a specific build definition on a specific branch. Once again REST API provided by VSTS comes to the rescue:

/
POST https://{instance}/DefaultCollection/{project}/_apis/build/builds?api-version={version}

The important part of this request is its body:

/
{
  "definition": {
    "id": 25
  },
  "sourceBranch": "refs/heads/master",
  "parameters": "{\"system.debug\":\"true\",\"BuildConfiguration\":\"debug\",\"BuildPlatform\":\"x64\"}"
}

There're two things which are required to make it work: build definition ID and source branch. You can obtain the former from the URL when you go directly to the definition you choose, once the latter is being passed along with the whole request in data.resource.refUpdates[0].name field.

Code example

An example of a Azure Function which can handle the functionality:

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
    log.Info("C# HTTP trigger function processed a request.");

    // Get request body
    dynamic data = await req.Content.ReadAsAsync<object>();

    if(data.eventType == "git.push" && data.resource.refUpdates[0].name.Value.Contains("refs/heads/feature")) {
        using(var client = new HttpClient()) {
            var branch = data.resource.refUpdates[0].name.Value;
            log.Info($"Build will be scheduled for {branch} branch.");

            client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", "yourToken");
            await client.PostAsync("https://{yourInstance}.visualstudio.com/DefaultCollection/{projectId}/_apis/build/builds?api-version=2.0", 
                new StringContent($"{{\"definition\": {{\"id\": {definitionId}},\"sourceBranch\": \"{branch}\"", Encoding.UTF8, "application/json"));

            log.Info($"Build scheduled successfully!.");
            return req.CreateResponse(HttpStatusCode.OK, "Build scheduled");
        }
    }

    return req.CreateResponse(HttpStatusCode.BadRequest, "Error occured");
}

Combining it all

To summarize steps needed here:

  1. Go to Service Hooks tab in VSTS and create new Web Hook pointing to your service endpoint using Code pushed as event
  2. Create a service, which will handle requests sent by VSTS and call its API to schedule a build
  3. Enjoy builds including only filtered branches

Edit:

After this was pointed I checked recently updated docs and it seems wildcard are now officially supported! You can treat this post as an inspiration for building new features around VSTS. You can find new documentation here -> https://www.visualstudio.com/en-us/docs/build/define/triggers