SHIFT

--- Sjoerd Hooft's InFormation Technology ---

User Tools

Site Tools


azuredevops
Differences

This shows you the differences between two versions of the page.

Link to this comparison view

azuredevops [2019/05/06 14:32] (current)
sjoerd created
Line 1: Line 1:
 += Getting Started with Azure DevOps for AWS = 
  
 +After [[aws]], [[o365]] and [[federatedloginforaws]] and [[transferdomaintoaws]] I now also want use Azure DevOps in combination with AWS. The goal is to use Azure DevOps as the code repository and use a build task to upload my code to an AWS S3 bucket which is hosting the domain. This article describes the steps to do so. Used technologies:​
 +
 +* Azure DevOps
 +* Azure AD
 +* AWS IAM
 +* S3 Static Hosting
 +* Azure DevOps Build Pipeline
 +* Azure AD Enterprise App
 +
 += Azure DevOps =
 +
 +First you need to sign up for Azure DevOps:
 +* Go to https://​go.microsoft.com/​fwlink/?​LinkId=307137
 +* Select your Azure AD Work Account with administrative privileges we created [[o365|earlier]]
 +* Create new organization
 +** Name: cbwdh
 +** Hosted in: West Europe
 +* Create project to get started
 +** cbwdhcom
 +
 +
 +> Note that when you create th Azure DevOps Organization like this Azure DevOps is automatically connected to your Azure AD. 
 +
 +
 +== Invite Team Member from Azure AD ==
 +
 +Because we do not want to develop the code using the administrative account you need to select a user from your AD to assign permissions in the Azure DevOps organization:​
 +* Organization Settings
 +* Users -> Add new users
 +** Select users from the list
 +** Access level: Basic for users who will contribute to the code base
 +** Select all projects
 +** Azure DevOps Groups: Project Contributors
 +** Enable package management
 +** Send invite
 +* Click Add
 +
 += Marketplace =
 +
 +Azure DevOps allows for extenstions to be installed so you can modify your build and release pipelines as you desire. AWS created the AWS Tools for Azure Devops (still named after VSTS, the previous name for Azure DevOps)
 +
 +== Install the AWS Tools for VSTS Extension == 
 +
 +* Log into Azure DevOps and click on the marketplace icon in the right top corner -> Browse marketplace
 +* Search for and select "AWS Tools for Microsoft Visual Studio Team Services"​ and click "Get it free"
 +* Click the install button after you selected your correct Azure DevOps organization
 +* Return to your organization when you're done
 +
 += AWS =
 +
 +Now we need to setup a few things in AWS. First we need to create an AWS IAM user so Azure DevOps can access the correct AWS Account and have the correct permissions to place files in the correct S3 bucket. ​
 +
 +== Create an IAM user ==
 +To use the AWS Tools for VSTS, you must have a set of valid AWS credentials,​ which consist of an access key and a secret key. To create one, log on with the root user (or [[federatedloginforaws|federated equivalent]] ) of the AWS account you want to use and follow these steps:
 +
 +* Open the IAM console
 +* Click Users -> Add user
 +** Provide username: AWSVSTSS3Upload
 +** Access Type: Programmatic access
 +* Click on "Next: Permissions"​
 +** Click on "​Attach existing policies directly"​
 +** Search and select the "​AmazonS3FullAccess"​ policy
 +* Click on "Next: Tags"
 +* Click on "Next: Review"​
 +* Review your settings and click Create User 
 +
 +Note the user it's access key ID and secret access key as you'll need it later to configure access in Azure DevOps. Also, add the username and the credentials to the lastpass root user account info for future reference or any other password management tool you keep. 
 +
 +== Create the S3 Bucket ==
 +
 +We will now create the S3 bucket. As you are creating a S3 bucket for a real domain here we have some specifics to consider:
 +
 +> We only need one bucket to upload the files to, and it needs to have the same name as the domain it will service (warmetal.nl in this case). We do need a second one however to also redirect www.warmetal.nl to this site. See [[https://​docs.aws.amazon.com/​AmazonS3/​latest/​dev/​website-hosting-custom-domain-walkthrough.html#​root-domain-walkthrough-s3-tasks|here]] for more information.
 +
 +* Open the S3 concole
 +* Click "​Create bucket"​
 +** bucket name: warmetal.nl
 +** Region: EU (ireland)
 +* Click next
 +** Enable encryption using AES-256 ( Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3))
 +** Leave all other options disabled as they are by default
 +* Click next
 +** As this bucket will host an internet website disable this two settings:
 +***  Block new public bucket policies (Recommended)
 +***  Block public and cross-account access if bucket has public policies (Recommended)
 +* Click next, review and click create to finish creating the bucket
 +
 +== Enable Static Website Hosting and Enable Public Access ==
 +
 +* Select the bucket from the S3 console
 +* Go to permissions -> Bucket Policy
 +* Enter the code below and click Save
 +<​code>​
 +{
 +    "​Version":​ "​2012-10-17",​
 +    "​Statement":​ [
 +        {
 +            "​Effect":​ "​Allow",​
 +            "​Principal":​ "​*",​
 +            "​Action":​ "​s3:​GetObject",​
 +            "​Resource":​ "​arn:​aws:​s3:::​warmetal.nl/​*"​
 +        }
 +    ]
 +}
 +</​code>​
 +
 +
 +> Note that the version is a version supplied by AWS and should not be changed
 +> Note that you now see a warning indicating that the bucket is publicly available
 +
 +
 +* Go to properties
 +* Click the static webhosting tile
 +** Click the "Use this bucket to host a website"​
 +** Set the index file to: index.html
 +** Note the endpoint: ​ http://​warmetal.nl.s3-website-eu-west-1.amazonaws.com ​
 +* Click Save
 +
 +=== Create the WWW S3 Bucket ===
 +
 +Create a second bucket as described above only then with the name www.warmetal.nl ​
 +
 +When done, follow these steps to redirect the bucket static website towards the first bucket:
 +* Go to properties
 +* Click the static webhosting tile
 +** Click the "Use this bucket to host a website"​
 +** Click Redirect requests. In the Target bucket or domain box, type the name of the first bucket: warmetal.nl. ​
 +* Click Save.
 +
 +
 +> Note that you do not need to change policy defaults here
 +
 += Azure DevOps =
 +
 +It is now time to return to Azure DevOps to glue everything together. We first create the service connection so an Azure DevOps project can connect to AWS and then we create the actual build so we have something to upload the files with.
 +
 +== Create Service Connection ==
 +
 +To make the AWS Connection available over all pipelines follow these steps:
 +
 +* Go to Project Settings - > Pipelines -> Service connections
 +* Click on new service connection -> AWS 
 +* Provide a connection name and fill in the Access Key ID and the Secret Access Key
 +
 +== Build Pipeline ==
 +
 +Pipelines -> New -> New build pipeline
 +* Source
 +** Team project: Shift
 +** Repository: Shift
 +** Default Branch: master
 +Click Continue
 +Click "empty job" to create an empty pipeline
 +* Name: AWS Warmetal
 +Under Agent job1 click + to add a task to the pipeline
 +Search and add the "AWS S3 Upload"​ task
 +* Display name: S3 Upload: Getshifting - Staging
 +* Select the correct AWS credentials from the dropdown
 +* Set the region code to eu-west-1
 +* Set the bucket name to warmetal.nl
 +* Set the sourcefolder accordingly to your needs by browsing to it
 +* Leave the target folder empty as we wil upload to the root directory
 +* Leave Access Control to private (if changed you might run into access denied errors)
 +* Leave all advanced and other options empty
 +* Click save to save the pipeline
 +
 +Now you can run the pipeline to upload the files. Afterwards you can check the full configuration by going to the s3 bucket endpoint. ​
 +
 +
 +> Note that under some circumstances you might not be able to select an Agent Pool. I'm not sure yet if this by design or a bug, but I had to login as the administrative account and configure the build pipeline to use the "​Hosted VS2017"​ agent pool. 
 +
 +
 += Azure AD =
 +
 +Now we need to ass an enterprise app just so we can use the integrated login. Note that this is nothing fancy here. It just adds the enterprise app en this should work for all websites that use Azure AD integrated login.
 +
 += Enterprise App =
 +
 +Create new Enterprise Application:​
 +* App Registration -> New Application Registration
 +* Name: Azure DevOps
 +* Application Type: Web app / API
 +* Sign-on URL: https://​dev.azure.com/​cbwdh
 +
 +After creation, click on the Application Name under Managed Application in local directory: Azure DevOps
 +* Users: Assign the users you want to assign
 +* Properties: User assignment required: Yes
 +
 +Now the app is available in the [[https://​myapps.microsoft.com|Microsoft MyApps Portal]]. ​
 +
 += Slow =
 +
 +During testing I found a few times that the installation for the AWS Tools took quite some time. Even though the installation says it's done, it took hours (!) before the actual tasks became available in the buils tasks. ​
 +
 +
 += Resources =
 +* [[https://​docs.microsoft.com/​en-us/​azure/​devops/​user-guide/​sign-up-invite-teammates?​view=azure-devops|Sign up for Azure DevOps]]
 +* [[https://​docs.aws.amazon.com/​vsts/​latest/​userguide/​welcome.html|AWS Tools for Azure DevOps UserGuide]]
 +* [[https://​docs.aws.amazon.com/​AmazonS3/​latest/​dev/​example-bucket-policies.html|S3 bucket example policies]]
 +
 +
 +
 +{{tag>​aws azure cloud devops}}
azuredevops.txt ยท Last modified: 2019/05/06 14:32 by sjoerd