Wednesday, May 18, 2016

Rapid Math Tricks - Mathematical Curiosity #5

Finally digging out an old book out of the closet. I found this little math curiosity and decided to explore it further. 

12² = 144 while 21² = 441
13² = 169 while 31² = 961
112² = 12,544 while 211² = 44,521
...

function sqr($a) { return $a*$a}
function Rev($b) { return -join([System.Linq.Enumerable]::Reverse($b))}

for($i=0; $i -le 1000;$i++){$ir = [int]$(rev([string]$i));$s = sqr($i);$sir = sqr($ir);$rs = rev([string]$sir);if($rs -eq $s -and $i -ne $ir){write-host $i,$s,$ir,$sir }}

12 144 21 441
13 169 31 961
21 441 12 144
31 961 13 169
102 10404 201 40401
103 10609 301 90601
112 12544 211 44521
113 12769 311 96721
122 14884 221 48841
201 40401 102 10404
211 44521 112 12544
221 48841 122 14884
301 90601 103 10609
311 96721 113 12769

Wednesday, May 11, 2016

Simple Progress Bar

I use the Powershell progress bar (write-progress) quite often. I put it into scripts that have long execution time (like cycling through all 100,000 objects in our GAL. I put it into place when manipulating data from multiple sources. I try to show some progress everywhere things happen that take more than a few seconds.

Problem being, the progress bar requires some setup. You need to know how many objects you are going to touch. You need to maintain an index of what object you are currently on. That I why I started working on these two 'skins' for the progress bar.

My first function uses a global variable that it maintains. This first copy takes the maximum count of items you want to review, then using a the global variable, tracks which item it's currently displaying. [0 .. Item Count]

Function WP {
 [CmdletBinding()] param(
  [Parameter(Mandatory=$true,ValueFromPipeline=$true)][Int]$ArrayItemCount,
  [Parameter()][String]$JobName="Counter"
 )
 $OCount = $ArrayItemCount
 $envVar = get-Variable -Name $JobName -Scope Global -ErrorAction SilentlyContinue -ValueOnly
 if ($EnvVar -eq $null) { 
  #Global Variable doesn't exist, create one called based on $JobName
  $Env_WPIndex = 0
  New-Variable -Name $JobName -Scope Global -Value 0 #-Visibility Private
 } else {
  #Use current global variable value.
  $env_WPIndex = [double]$EnvVar
 }
 #display basic progress bar
 Write-Progress -Activity $JobName -Status $([string]$Env_WPIndex + ":"+[string]$ocount) -PercentComplete (($Env_WPIndex / $oCOunt)*100) 
 $env_WPIndex = $env_wpIndex + 1
 if ($env_wpIndex -lt $OCount) { 
  #if less then max object count, increment the global variable by one
  Set-Variable -Name $JobName -Scope Global -ErrorAction SilentlyContinue -Value $env_WPIndex
 } else {
  #if already greater than max, remove the global variable from machine. 
  Set-Variable -Name $JobName -Scope Global -Value $null # -ErrorAction SilentlyContinue
 }
}



For example:

$Services = get-service
$Services | %{wp $Services.count; write-host $_.name}

The problem I've found with Function WP is sometimes the global variable doesn't get reset at the end of the previous run. This causes your next progress bar to 'wrap around' (started at 11 goes to 100, then back to 11 again). I added the JobName field so that it could spin up a new global for each different iteration if you wish.

The second script is much simpler, but requires you send a copy of the entire array. You pass the function a copy of your array, plus what item you're currently on, and it displays a progress bar based of it's location in the array.

Function WP2 {
 [CmdletBinding()] param(
  [Parameter(Mandatory=$true,ValueFromPipeline=$true)][array]$Array,
  [Parameter(Mandatory=$true,ValueFromPipeline=$true)] $Item
 )
 #Find Index of current item in Array
 $Index = [array]::IndexOf($array, $item)
 #Count items in array
 $ocount = $array.count
 
 Write-Progress -activity "Counter" -Status $([string]$Index + ":"+[string]$ocount) -PercentComplete (($Index/$OCount)*100)
}

For example:

$Services = get-service
$Services | %{wp2 $Services $_ ; write-host $_.name}



Tuesday, January 12, 2016

GAL Cleanup Part 1 - Expire old contacts

Over the 8 years that I've worked here, we've managed to virtually triple the number of contacts we host in our Global Address List. At this point, we have 90,000 mailboxes and 60,000 mail enabled contacts. I suspect that a vast majority of these contacts have not been sent to in several years! Heck, considering the volatile work environment, it's more than likely a good portion of those are no longer good. Customer at another company, leaves and we never delete their contact.

Phase 1:

This got me thinking.. Read in the primary smtp address from 100 or so contacts. Search the message tracking logs (on server hosting Internet bound email) for something going to this contact. Each contact I touch, I would mark Custom Attribute 9, with date/time. If I found an entry in the tracking logs, mark Custom Attribute 10. Relatively easy script to write.

#This is too easy!!
get-contacts -resultsize 100 | ForEach {get-messagetrackinglog -start (7 days ago) -recipient $_.externalemailaddress.addressstring.tostring()}

Unfortunately, that TOOK FOREVER!
  1. We have 12 hub transports that send out email to the Internet. This means to effectively scan for a single recipient, I'd need to scan all 12 servers. 
  2. We process probably a million messages each day going out to the Internet.
  3. As I said, we have 60,000 mail enabled contacts.
  4. We keep tracking logs back 30 days.
If I let it run in this state, we'd be scanning 1/30th contacts every day each month..

Phase 2:

I noticed that the Recipient field on get-messagetrackinglog uses OR logic. I could technically buffer up a big handful of recipients into that field and search for them all at once. 20, I'll start with twenty recipients per search. 

These are not the results I was hoping for...
Evidently, I've stumbled onto a 'known bug' with the cmdlet. Your search has to be under so many characters (256 iirc). Once you exceed that, it fails. Only workaround is to reduce the # of entries in your query. At one point, I reduced my # of recipients to only 5 addresses and the script was failing. What next? Only 2 people at a time? Not much of a time savings. 

Phase 3:

While walking out to my car that night I was discussing this project with a co-worker. During explaining the concept to him, I came up with an interesting idea. Scan message tracking logs for non-mailbox users. OK, it sounds worse, but it pays off. 
  1. Create giant string of every accepted domain. This will be used to filter out every mailbox recipient.
  2. Find and fine-tune 'directory searcher' function to validate email address is in GAL. 
So here's my basic process. On each hub transport:
  •  read message tracking logs and spit out all recipients
  • filter where internal domains -notmatch external email address domain
  • check GAL to see if contact exists for recipient.
  • get mailcontact - put today's date in CA10.
Now some contacts appear to get messages hourly as part of scheduled tasks. So I created a second filter on already touched contacts. 

Exchange 2010 RPC Client Access Logs + Powershell + LogParser

I've been working on a few projects revolving around analyzing Outlook clients connecting to our email environment. The latest request was for a report that would detail how many clients from one office are connecting to email via OWA vs Outlook. While I've seen the function that details CAS connections, it won't work for me. My customers range from over 45 different offices and they include vastly different #s of concurrent users. Many of the offices share the same CAS servers/pools.

This got me looking at the RPC Client Access logs on each box. From here I can filter my connections based on individual mailboxes, members of the same OU, or even distribution group members. Sadly with 40,000 active users, connecting to possible 60+ CAS boxes, that's easily a 9GB of data per person of log data to go through. While I have developed fairly efficient powershell to process this data, (it requires effectively reading in the log file for each server as a CSV and processing it individually) I believe I can do better.

LogParser 2.2

ExchangeServerPro published an article about using LogParser to query RPC Client Access logs. Reviewing this article, I was able to pick up the basics about building a LP query for RPC client. This got me to this: 

#Set up some basics for the script.
#Default path to the RPC Client Access logs. Going to build paths to.
$LogPath = "\C$\Program Files\Exchange\Logging\RPC Client Access\"
#Default path for LogParser executable
$LogparserExec = "C:\Program Files (x86)\Log Parser 2.2\logparser.exe"


#Final Report Path
$TodayString = (Get-Date -Format "yyyyMMdd").tostring()
$ReportPath = "c:\reports\OutlookClientReport_"+$TodayString + ".CSV"

#Get names for all CAS boxes
$CASPool = get-clientaccessServer | %{$_.name}

#Build source statement for all log files. (Reading only today's logs).
[array]$logPaths = $null
$CASPool | %{$logPaths += "'\\"+$_+$LogPath+"RCA_"+$todayString + "*.log'"}
$allServers = $logPaths -join(";")

#Build one REALLY BIG query.
$Query = "SELECT EXTRACT_SUFFIX(client-name,0,'=') as Name,client-software as Software,client-software-version as Version INTO '"+$ReportPath+"' FROM "+$allServers + " where software in ('outlook.exe';'OUTLOOK.EXE') Group BY Name,Software,Version ORDER BY Name"

#Execute LogParser search
& $LogparserExec $Query -i:CSV -nSkipLines:4  # -Stats:Off


When executed, this generates a CSV in the report path specified. It contains each customer who's connected today, and all the Outlook client version they connected with.
Statistics:
-----------
Elements processed: 9140619
Elements output:    53206
Execution time:     135.88 seconds (00:02:15.88)
As the 'Client-Name' field is based off the end-user's mailbox alias and/or legacyExchangeDN, you can search for a specific user based off it.


#Get specific mailbox path
$cn = (Get-Mailbox "End-User, Joe").LegacyExchangeDN

#Query that specifies client we are looking for.
$Query = "SELECT EXTRACT_SUFFIX(client-name,0,'=') as Name,client-software as Software,client-software-version as Version INTO '"+$ReportPath+"' FROM "+$allServers + " where software in ('outlook.exe';'OUTLOOK.EXE') and client-name='"+$cn+"' Group BY Name,Software,Version ORDER BY Name"

Now you are searching all of your client access servers for Outlook connections from this specific user. My average search time for a day's log files is about the same 2 minutes per person. 2,000 people will take 4,000 minutes, or about 2 days to run.

Grabbing certain GB of Mailboxes From a DB

We have a few databases that are reaching capacity. These DBs are around 600-700gb in the 1 TB drives we used.

$TotalMoveSize=0;$move = @();get-mailbox -Database DB08 | %{$size = (get-mailboxstatistics -identity $_.identity).totalitemsize.value.tobytes();if ($totalMoveSize + $size -le 300gb){$totalMoveSize += $size;$Move+= $_.identity}}

This one-liner, will read DB08, and keep adding mailboxes to an array until at or near 300gb in size.

Wednesday, October 14, 2015

Checking OWA.. Is Mine Hacked?

We've recently had a number of our staff ask about the recent OWA hack. To appease their fears, I went through and checked my OWA boxes to make sure that the OWAAuth.DLL hadn't been replaced or re-registered using a hacked version.

$servers = @("OWAServer1","OWAServer2") # get-ExchangeServer
$sbFileVersion = {
$FilePath = "C:\Program Files\Exchange\ClientAccess\Owa\auth\OWAAuth.dll"
Get-ChildItem $FilePath |  Select-Object Name,length,@{Name="Version";Expression={$_.versionInfo.FileVersion}},LastWriteTime
}
Invoke-Command -ScriptBlock $sbFileVersion -ComputerName $servers # | group fileversion
$SBRegistry = {
 #Return installed folder path for OWAAuth.DLL
 $RegKey = "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Components\66A06D0DD155D354CB4C311E0ED2EE9D"
 $RegValue="AE1D439464EB1B8488741FFA028E291C"
 (Get-ItemProperty $regkey).$regvalue
}
Invoke-Command -ScriptBlock $SBRegistry -ComputerName $servers 
This does two things. First off, it checks the install path of the OWAAuth.DLL and returns the version and size of the file. I skimmed these to look for differences in the installed files.

Second step, it checks the registry on these same servers and looks to see what's registered. I visually checked to see if the path in the registry matches the install path. (Those registry values I found by searching one of my OWA box registries for the filename.)

My server running Exchange 2010 SP3 RU10 returned:

Name               : OWAAuth.dll
Length             : 104632
Version            : 14.03.0248.002
LastWriteTime      : 5/27/2015 1:47:42 PM
PSComputerName     : OWASERVER1

C:\Program Files\Exchange\ClientAccess\Owa\auth\OWAAuth.dll

Monday, May 11, 2015

Powershell IsNumeric

For the longest time, I've been using a visual basic trick to determine if a variable is a numeric value.

[reflection.assembly]::LoadWithPartialName("'Microsoft.VisualBasic")
function isNumeric([string] $a) {
    $b = ([Microsoft.VisualBasic.Information]::isnumeric($a))
    return $b
}

IsNumeric "12"
$True

IsNumeric "Bob"
$False



Looking at the code, all I am doing is type-casting the variable as a integer and seeing if I get an error. So I could simply.

Try {
      [Int]$Variable -is [Int]
} Catch {
     $false
}

So, now in my code instead of evaluating the variable if it's a numeric value, then run through a "if numeric then ____ else _____". Now I simply encapsulate my THEN _ ELSE _  portions into my Try _ Catch _.

try {
 $DaysInt = [int]$daysBack
 $EndDate = Get-Date
 $StartDate = $EndDate.AddDays(-1 * $DaysBack).ToShortDateString()
} catch {
 $StartDate = Get-Date $daysback -Format g -ErrorAction silentlycontinue
  if ($StartDate -ne $null) {
  Write-Host "you entered a date" $StartDate 
  $EndDatestr = (Read-Host "Specify an End Date (enter for today)").trim()
  if ($enddatestr -eq "") {
   $EndDate = Get-Date
  } else {
   $endDate = Get-Date $EndDateStr  
  }
 }
}