Blizzard API Project

The Blizzard team has realized that WoW players love to reminisce about their game. We love to play old dungeons, level more characters, and collect appearances from this legacy content. Last summer, Blizzard introduced a gameplay mode called "Remix". This limited-time playstyle encouraged leveling new characters in order to collect all of the available items. Each of these collectable items provided a cosmetic appearance that all of your characters could use. The interesting dilemma here is tracking appearances your account still needed to collect.

For many, the UI was the easiest method to track. You could walk up to a vendor and compare what your account 'knew' versus what was available. "Easy" except the UI required that you change your class to see what you collected. Does my druid have this set? Open appearances, change class to druid, scroll until finding Mists of Pandaria collections, check the various raid-tier levels. Second option was a spreadsheet variant. Some diligent people went through all the vendor offerings, got the names and cost of all the items. When you went to the vendor, you'd mark off all the items you purchased. Neither of these options really appealed to me. 

In an effort to find a better way, I investigated coding against the API. Could a Google Sheets make a call and populate the completion statistics for me? Would it be easier to code against WoWhead's api and query data? I even investigated learning LUA and writing my own addon that would help. This lead to my own analysis paralysis and soon after the end of the Remix event. 

Fast forward 6 months and Blizzard introduces two new achievements based on past expansions. "A Farewell to Arms" and "A World Awoken" are meta-achievements that require completing a number of events, reputation grinds, and collecting treasures in the game. Completing each of these achievements, rewards the grand prize of an iconic mount from that expansion. I mean, who wouldn't want to transform into "Jani Lord of Thieves, God of Garbage, Master of Minions, and Ruler of Rubbish". This, along with my current work with python had peaked my interest. 

Project plan: Using local copy of Python, connect to Blizzard's API and query the character achievement tree. Look for any achievements that are marked as incomplete and provide them as a simple HTML page. 

Step 1: Create an account on Blizzard's developer portal. After agreeing to all the legal things, you're provided with a client id and secret. These credentials are passed in the headers to the authentication portal and a refresh token is provided. This refresh token is put into a JSON structure to create a bearer token. The bearer token is passed each time calling the Blizzard API. All this is identical to how to authenticate with Aria. So far so good.

Step 2: Read character's achievement stack. This is an easy call to the API. The returned JSON provides (almost) every achievement that is available and their status. Issues I found:

• Achievements are presented in a flat format. The main meta-achievement has 8 child criteria. The child criteria, while listed there are not achievement ids, they are criteria ids. Took me awhile to figure that out.  
• There are holes in the results. None of my characters have completed the "Storm Chaser" achievement. This meta-achievement is to complete all 4 different 'elemental storms' events in all 4 zones. This achievement doesn't show in the API results for the characters. (I bet if I complete one of the 4 zone achievements, Storm Chasers will appear in my query.)
• Last, some of the achievements are character-based. So, while my main character may have petted all the dogs in DragonFlight, my new alt still shows the achievement incomplete. 

New Step 2: Read the generic achievements stack from the API. Using a recursive loop, go through the achievement id by id and pull all children. Return all of the child achievement ids in an array. Loop through the character achievement results for completed achievements and report back on what is incomplete. This solution avoided the holes in the API. I was able to query the entire tree and present a high-level result. Unfortunately, it looks like I still have close to 70 achievements to complete. I decided to implement a few optimizations:

1. While querying the achievements tree, I now build a dictionary (ie PowerShell hash table). The dictionary equates parent achievements to its children. This allowed me to optimize the code to say, if you completed this parent, all the children can be assumed completed.
2. Everything is written to the local drive. I was having issues where the time between runs was several minutes, especially when recursing the achievement tree. So on initial run, each achievement JSON, the achievement tree and the achievement dictionary is written locally.  At the moment, I am also reusing old character achievement JSON dumps, but that may be updated. The final report now completes in seconds instead of minutes. 
3. The final report only includes items with no children. Using the dictionary, I look for achievement entries that don't have children listed and focus on those. My report that used to show I needed "Storm Chasers" and "Chasing Storms in Waking Shores" and "Firestorms in Waking Shores", now it only shows the "Firestorms" entry. 

This leaves one issue unresolved. Some of the incomplete achievements are character based. As of writing this, my solution is to run the same code against my alternates that played through that expansion and manually compare notes. The druid leveled via hunts, where the paladin favored the pvp type activities and so on. My plan is to pull all my max level toons from my account, query their completion status and combine into a master list. If any of these 8 characters says this achievement is complete, then consider it complete. I am running into a permissions issue as my developer credentials do not allow querying private account details. 

With the Legion Remix on the books for "soon"(tm), I am that much closer to achieving my goal of a dynamic shopping list. Once I have code, need to consider how I can share it. I personally like the idea of an external website that I can review from outside the game (while here in office). I'll try to keep you posted. 

VRO Action to find tagged network

 In Aria Automation, one of the most frustrating aspects of using constraint tags is the unknown. On my vCenter builds, I have created a simple cloud template that utilizes 4 tags to constrain new virtual machine build to a specific network. To make sure the dropdown options are legitimate, I have automation that will query the tags based on the selections. Once the engineer selects a project, it populates the tenant dropdown. When they select a tenant, it proceeds to populate the application, environment and tier tags. 

Unfortunately, some tenants (like IT) have hundreds of options for tag combinations and not all of them are legitimate. This causes builds to fail only after the engineer had populated the template and hit submit. "no matching fabric network with these tags..."

This JavaScript action will take the provided constraint tags and return any networks that match that combination. I added a radio-button set to read-only to my custom form and set the source to this VRO action. As I didn't want it to list ALL networks when the form is loaded, I have it only update after the tenant tag is populated. Functionally, it is currently only returning results when all 4 tags are populated, but it should return each time an additional tag is selected. 

VRA ABX Action to set Deployment Lease

 Running vRealize Automation (VRA), I wanted to create a new catalog item in Service Broker that would basically self-destruct after a certain time. My intent is to allow our Windows engineers to quickly spin up a new machine in AWS with a lease applied to it. 

I could find a catalog template setting that allowed me to configure this, so I found that I could do this via a REST call to the VRA. So I decided to work on creating an extensibility action using Powershell. 

Note, once a lease is set, I don't think it's possible to remove the lease. 

Pre-requisites:

• Extensibility Actions on Prem integration server configured.
• VRA refresh token defined. This will be used in step 0.
• BlueprintID of a catalog template that will apply the lease to. We'll need this for step 2. The blueprint id can be captured easily from the URL when viewing it in the design menu. Copy everything after the %2F, for example (the bolded portion): 
  
  https://www.mgmt.cloud.vmware.com/automation-ui/#/blueprint-ui;ash=%2Fblueprint%2Fedit%2F123456-7890123456-123456789

Step 0: setup Action Constant

1. Inside VRA - Extensibility menu, select Actions on the sidebar. 
2. Select Action Constants and add a + New Action Constant
3. Enter the name of "refreshToken" and paste the refresh token string into the value field and select toggle to enable the Encrypt the action contact value. 
   
   
4. Click SAVE.

Step 1: create the action 

1. Extensibility menu, Actions on sidebar and click + New Action
2. Define the new action Name "SetDeploymentLease" and select the VRA Project that this will run against then click Next.
3. On the top, change the dropdown from PYTHON to POWERSHELL. 
4. Copy the pastebin code below and paste it into the body of the action.
5. Under Default Inputs, add two more input fields (for a total of 3). These fields will be used during the run-time of the action. 
   
   
   
   
1. Default - LeasePeriod - Positive integer for the number of days to define the lease.
2. Default - deploymentId - can be anything, this will be sent to the action from the subscription event defined in step 2. I copied the deployment id off an existing 'test' deployment directly from the URL. (like prerequisites, it's everything after the %2F). 
3. Action Constant - refreshToken - this will pull the value from Step 0.
6. SAVE the action

Step 2: create a subscription - The subscription will trigger after the blueprint is run.

1. Extensibility menu, Subscriptions option on sidebar and click + NEW SUBSCRIPTION
2. Give subscription name. 
3. For Event Topic select "Deployment Resource Completed"
4. Enable the Condition and enter a (case matters) single value of: 
   event.data.blueprintId == "value from pre-requisites"
5. Action/Workflow: select the new action you created in Step 1.
6. Projects: Select the project you specified in step 1, item 2.
7. SAVE
   
   
   
   
   
   

Testing: 

1. Option: If valid values are entered in step 1 part 5, the test button on the Action will run. 
   
   
   
   
   
2. Option: Deploy a new machine using the catalog item defined in the prerequisites. Within a minute after the deployment completes the build, the lease should be applied via the subscription. 
   
   
   

---

Using VRO to query tags for a VRA-Cloud Assembly dropdown

We've just started building servers in our vSphere with NSX-T environment using VRA to deploy the servers. The security group placement is controlled via NSX policies that are determined by the machine's tagging. I wanted to provide a dynamic dropdown in VRA so that I did not have to recode the cloud template and subsequent service broker custom form each time a new tag value was added to vSphere. I noticed in VRA that the NSX tags were being discovered there. 

This routine runs from VRO to read the tags in VRA (/iaas/api/tags) with a specific KEY, and returns the VALUE in a sorted array.

Prerequisites:

1. VRO server integrated with VRA-Cloud Assembly. 
2. VRA Plug-in to VRO (reference: here) - need to download and install on your VRO server.
3. Cloud template that needs a dropdown of tag values. 

Warning:  Without a filter, we have over 7,000 tags coming from NSX. When I tried to query all our tags, I crashed my VRO server and needed to reboot it to kill the query. 

   

Configuration:

1. Set default VRA host 
1. VRO - select Configurations - vRA plug-in
2. Variables tab - click on defaultHost
3. Click on the value field and select the VRA host you want to query.
4. Save
2. Create new Action
1. VRO - select Library - Actions menu item 
2. New Action
3. type in a name and module name 
1. typing in a 'new' name for the module will create a new module. 
4. Click on the Script tab and make sure Javascript is selected for Runtime.
5. On the right, click Add New Input button and create a new input
1. name: tagFilter
2. type: string
6. Change the Return type to string and check the checkbox for Array.
7. Paste the code below.
8. Save.

Last step, TEST! Click the RUN option up top before closing the Action screen and enter a tagFilter value that matches a tag showing in your VRA - Configure - Tags. 

Replication:

1. Log into your VRA instance
2. Scroll down to Integrations and open the VRO integration
3. Click the Start Data Collection button and wait...

Use:

1. Log into your VRA instance
2. Open a cloud template
3. Browse to the Inputs menu. 
4. Locate or create a new input. Mine:
1. Name: ApplicationTag
2. Display name: Application
3. Type: string
4. Default Value: leave blank.
5. Scroll to the bottom of the screen and expand More Options
1. Pairs: External Source
2. Action: click Select and search for the name you input into 2.3 in VRO above. 
3. Unselect BIND and type in the tag KEY value to filter on. 
4. SAVE to save select.
5. SAVE to close Input properties
6. Test cloud template - The dropdown should load for a few seconds then complete. 

VRA API Programming Lessons Learned

I've been working for so long inside the closed environment of Exchange powershell, that when a co-worker suggested programming against a REST API, I sort of cringed in fear. Here was a brand new programming method that I had no concept of where to start. Luckily, VMWare vRealize Automation (reference) has a full-featured API that you can easily write PowerShell commands against to both pull information from or push configurations up to. The documentation is well rounded with examples (written in CURL format) that can be easily translated to Powershell. 

Our friend on this journey will be invoke-restMethod. (msdn link) (yes, I have started writing my commands in camel case). This command has a few basic components:

1. headers - a variable that contains the credentials that you'll need to access this URI
2. uri - the URL to the aspect that you are looking to read from/post to. 
3. method - GET information, POST information PATCH update information etc. 
4. body - the information you'd like to post up to this interface. Usually empty when doing a GET.

Headers = Authorization - This differs per the API. VMWare requests that a 'refresh' token is generated in the UI, then exchanged for a bearer token. The bearer token can then be used for up to 30 minutes on API calls. AWS needs an Access key and Secret key encoded in a basic credential (i.e. 'get-credential') object. 

URI - This is the basic URL to reach this object type. From my experience, this URL follows a basic format. 

    API URL / object

For example, to query all of the VRA Cloud Accounts (reference) in my environment, the URI would be 

$myURL = "https://api.mgmt.cloud.vmware.com" + "/iaas/api/cloud-accounts"

Method - Are actions telling invoke-restMethod what is to be performed at the URL. 90% of the time I used only GET to query and POST to create. PATCH can be used to update an existing entry. (see the msdn link for more info)

Note: Powershell on a GET method will return an object that you can extract information from. Unfortunately, I was never able to read modify and post the same object back to a site. I ended up reading it, then creating a new body and posting that. 

$Results = Invoke-RestMethod -Uri $myURL -Method get -ContentType "application/json" -UseBasicParsing -headers $headers 

 

$results | convertto-json

(if the query is successful, the results should look like this)

whereas, you can manipulate the object:

$results.content.name 

and get "my-name"

Body - This was the largest part of the learning curve. The body variable is looking for a JSON formatted variable. For example, peek at the cloud-accounts POST method. The completed body JSON should look like something like this:

Sample Cloud-Account Body from code.vmware.com

In Powershell, you can attempt to create this two different ways. Simply, it can be a giant text string that you format to contain all of the properties and values. Seems easy enough, but will be rather hard to support later on.  The second option you can create a PowerShell hash table and convert it to JSON using the (obviously named) convertTo-JSON command. Once we identify come clues, I feel you'll agree, the hash table to JSON method is much easier.

Observation one: Items are enclosed in curly brackets. These enclose a concept or object. This will be replaced with a hash. "@{ }"

Observation two: There are a few items enclosed in square brackets as well. These are actually arrays of results. Often when there is a one-to-many relationship. @( ) 

Observation three: Backslashes "\" surround almost all variable values. Ignore them. 

Observation four: concepts are separated by commas. Replace these with a semi-colon. 

Observation five: the colon is used to separate a key with its value. "name": "string". replace colons with an equals sign.

Observation six: variable names are in 'camelCase' sensitive (only uppercase the first letter of the following words). This is EXTREMELY important when doing REST calls in JSON. You can build the prettiest JSON in the world, but if your variable names are not perfectly matching the case shown in the documentation, the rest call will fail. regionIds -ne RegionIDs. Going from PowerShell to JSON was a shock because of this! 

This next example includes a multi-value array for the tags:

The hardest part of the process is making sure that the hash tables and arrays are closed correctly. 

Here's a snippet of what I use to create new AWS Cloud Accounts in VRA. I pass a number of variables to this JSON, but I think you can guess how they correlate to your AWS account. 

$CloudRegionArray = @("us-west-2") # we only put stuff here.

$NewCloudAccount = @{"accessKeyId"=$strCloudAccountAccessKey;"secretAccessKey"=$strCloudAccountSecretKey;"createDefaultZones"="true";"name"=$strCloudAccountName;"regionIds"=$CloudRegionArray;"tag"=@(@{"key"="ca";"value"=$($strCloudAccountName+"-aws")})}  | convertto-json

Invoke-RestMethod -uri "https://api.mgmt.cloud.vmware.com/iaas/api/cloud-accounts-aws" -Method POST -ContentType "application/json" -headers $headers -Body $NewCloudAccount