Using VRO to query tags for a VRA-Cloud Assembly dropdown

We've just started building servers in our vSphere with NSX-T environment using VRA to deploy the servers. The security group placement is controlled via NSX policies that are determined by the machine's tagging. I wanted to provide a dynamic dropdown in VRA so that I did not have to recode the cloud template and subsequent service broker custom form each time a new tag value was added to vSphere. I noticed in VRA that the NSX tags were being discovered there. 

This routine runs from VRO to read the tags in VRA (/iaas/api/tags) with a specific KEY, and returns the VALUE in a sorted array.

Prerequisites:

1. VRO server integrated with VRA-Cloud Assembly. 
2. VRA Plug-in to VRO (reference: here) - need to download and install on your VRO server.
3. Cloud template that needs a dropdown of tag values. 

Warning:  Without a filter, we have over 7,000 tags coming from NSX. When I tried to query all our tags, I crashed my VRO server and needed to reboot it to kill the query. 

   

Configuration:

1. Set default VRA host 
1. VRO - select Configurations - vRA plug-in
2. Variables tab - click on defaultHost
3. Click on the value field and select the VRA host you want to query.
4. Save
2. Create new Action
1. VRO - select Library - Actions menu item 
2. New Action
3. type in a name and module name 
1. typing in a 'new' name for the module will create a new module. 
4. Click on the Script tab and make sure Javascript is selected for Runtime.
5. On the right, click Add New Input button and create a new input
1. name: tagFilter
2. type: string
6. Change the Return type to string and check the checkbox for Array.
7. Paste the code below.
8. Save.

Last step, TEST! Click the RUN option up top before closing the Action screen and enter a tagFilter value that matches a tag showing in your VRA - Configure - Tags. 

Replication:

1. Log into your VRA instance
2. Scroll down to Integrations and open the VRO integration
3. Click the Start Data Collection button and wait...

Use:

1. Log into your VRA instance
2. Open a cloud template
3. Browse to the Inputs menu. 
4. Locate or create a new input. Mine:
1. Name: ApplicationTag
2. Display name: Application
3. Type: string
4. Default Value: leave blank.
5. Scroll to the bottom of the screen and expand More Options
1. Pairs: External Source
2. Action: click Select and search for the name you input into 2.3 in VRO above. 
3. Unselect BIND and type in the tag KEY value to filter on. 
4. SAVE to save select.
5. SAVE to close Input properties
6. Test cloud template - The dropdown should load for a few seconds then complete. 

VRA API Programming Lessons Learned

I've been working for so long inside the closed environment of Exchange powershell, that when a co-worker suggested programming against a REST API, I sort of cringed in fear. Here was a brand new programming method that I had no concept of where to start. Luckily, VMWare vRealize Automation (reference) has a full-featured API that you can easily write PowerShell commands against to both pull information from or push configurations up to. The documentation is well rounded with examples (written in CURL format) that can be easily translated to Powershell. 

Our friend on this journey will be invoke-restMethod. (msdn link) (yes, I have started writing my commands in camel case). This command has a few basic components:

1. headers - a variable that contains the credentials that you'll need to access this URI
2. uri - the URL to the aspect that you are looking to read from/post to. 
3. method - GET information, POST information PATCH update information etc. 
4. body - the information you'd like to post up to this interface. Usually empty when doing a GET.

Headers = Authorization - This differs per the API. VMWare requests that a 'refresh' token is generated in the UI, then exchanged for a bearer token. The bearer token can then be used for up to 30 minutes on API calls. AWS needs an Access key and Secret key encoded in a basic credential (i.e. 'get-credential') object. 

URI - This is the basic URL to reach this object type. From my experience, this URL follows a basic format. 

    API URL / object

For example, to query all of the VRA Cloud Accounts (reference) in my environment, the URI would be 

$myURL = "https://api.mgmt.cloud.vmware.com" + "/iaas/api/cloud-accounts"

Method - Are actions telling invoke-restMethod what is to be performed at the URL. 90% of the time I used only GET to query and POST to create. PATCH can be used to update an existing entry. (see the msdn link for more info)

Note: Powershell on a GET method will return an object that you can extract information from. Unfortunately, I was never able to read modify and post the same object back to a site. I ended up reading it, then creating a new body and posting that. 

$Results = Invoke-RestMethod -Uri $myURL -Method get -ContentType "application/json" -UseBasicParsing -headers $headers 

 

$results | convertto-json

(if the query is successful, the results should look like this)

whereas, you can manipulate the object:

$results.content.name 

and get "my-name"

Body - This was the largest part of the learning curve. The body variable is looking for a JSON formatted variable. For example, peek at the cloud-accounts POST method. The completed body JSON should look like something like this:

Sample Cloud-Account Body from code.vmware.com

In Powershell, you can attempt to create this two different ways. Simply, it can be a giant text string that you format to contain all of the properties and values. Seems easy enough, but will be rather hard to support later on.  The second option you can create a PowerShell hash table and convert it to JSON using the (obviously named) convertTo-JSON command. Once we identify come clues, I feel you'll agree, the hash table to JSON method is much easier.

Observation one: Items are enclosed in curly brackets. These enclose a concept or object. This will be replaced with a hash. "@{ }"

Observation two: There are a few items enclosed in square brackets as well. These are actually arrays of results. Often when there is a one-to-many relationship. @( ) 

Observation three: Backslashes "\" surround almost all variable values. Ignore them. 

Observation four: concepts are separated by commas. Replace these with a semi-colon. 

Observation five: the colon is used to separate a key with its value. "name": "string". replace colons with an equals sign.

Observation six: variable names are in 'camelCase' sensitive (only uppercase the first letter of the following words). This is EXTREMELY important when doing REST calls in JSON. You can build the prettiest JSON in the world, but if your variable names are not perfectly matching the case shown in the documentation, the rest call will fail. regionIds -ne RegionIDs. Going from PowerShell to JSON was a shock because of this! 

This next example includes a multi-value array for the tags:

The hardest part of the process is making sure that the hash tables and arrays are closed correctly. 

Here's a snippet of what I use to create new AWS Cloud Accounts in VRA. I pass a number of variables to this JSON, but I think you can guess how they correlate to your AWS account. 

$CloudRegionArray = @("us-west-2") # we only put stuff here.

$NewCloudAccount = @{"accessKeyId"=$strCloudAccountAccessKey;"secretAccessKey"=$strCloudAccountSecretKey;"createDefaultZones"="true";"name"=$strCloudAccountName;"regionIds"=$CloudRegionArray;"tag"=@(@{"key"="ca";"value"=$($strCloudAccountName+"-aws")})}  | convertto-json

Invoke-RestMethod -uri "https://api.mgmt.cloud.vmware.com/iaas/api/cloud-accounts-aws" -Method POST -ContentType "application/json" -headers $headers -Body $NewCloudAccount

Using Powershell against vRealize Automation Cloud Assembly

Over the last few weeks, I have been working to automate the steps to onboard a new customer into vRealize Automation (VRA) Cloud Assembly. The plan is as a new customer signs up with our solution, we will create a new AWS account that all of their resources will be deployed into. VRA will then be configured with a new 'cloud account'. Since we could potentially have different support areas needing access, we will create a new project to grant the permissions. With the new cloud account, we need to touch each of the flavor mappings and image mappings, create network profiles for each AWS region and storage profiles.

Prerequisite # 1: Get a refresh token
A refresh token is used to initiate the authenticated communications with your environment. This refresh token is exchanged by VRA for a bearer token. You then pass this bearer token in the header of each REST call to VRA.

1. Log into your VMware Cloud Services console.
2. Click down caret by your name in the upper right corner. 
3. Select My Account from the dropdown menu.
4. Select API Tokens from the top menu. 
5. Populate the form so that it includes the permissions you want to include. 
6. Click the Generate button and it will pop up a screen with a 64 character token. 
7. Copy down this code, you'll need it to populate the variable in the AuthenticateWithVRA function.  (line 53)

Prerequisite 2: AWS Programmatic Access Token

The script configures the AWS cloud account in the very beginning. Therefore script requires an AWS Access key and secret key credentials for a programmatic access account. The account will need to have power user role access to the AWS environment if it is to actually provision objects there. 

Ref: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html

Prerequisite 3: Master Account configured

To limit the scope of the script, I clone most of the configuration of an existing 'master account'. This account has the flavor and image mappings configured similar to how all new projects would be configured. To simplify the script I've embedded this value in the script.  (line 33)

# VRA Dev Account ID -> Script copies this account for new customers to be.

$DevAccountID = 'abc12345667890'

This value can be pulled from the UI fairly easily. Log into VRA Cloud console, open the Cloud Account that you want to replicate then look at the URL for the project. The end of the URL, after the last %2F contains the cloud account ID to use. Mine is 29 hex digits long.

sorry this is either really BIG or really small.

Copy down that code and then locate (see reference above) and replace the ID value in the code. 

The full script can be pulled from GitHub.