I often present PowerShell at SQL Saturdays. One question I’m often asked is “Why learn PowerShell? I can do everything in T-SQL.” Well here’s a great example.

I was developing some SSIS packages that took a set of text files and loaded them into multiple dimension and fact tables. In the process I had one particular value that was creating duplicate values in one of the target tables. Obviously, I needed to find that value in the various files and see what’s different about it versus the other data.

The problem was there are close to 200 files I’m loading. Each file has hundreds of lines, and each line is over 200 characters long. Yikes! That’s a lot of data to try to look through. It would have been very time consuming to look in all of those files manually.

PowerShell to the rescue! In just a few minutes I created a PowerShell script to loop over the files, find the ones that had the value I was looking for, and list those file names. Made it easy then to open just the files I needed and look at the rows. I could have gone further, actually printed out the data, or perhaps load it into some array where it would count it, or more.

I decided to share the script, which you will see below. It actually took me longer to write up all the comments than it did to write the initial code. With the comments removed there’s only 9 lines of real code in the script.

Using PowerShell I was able to quickly find the rows I needed and find the issue. This is just one simple example of how PowerShell can be used to quickly solve a problem that might take a long time to accomplish in other languages.

 

#-------------------------------------------------------------------------
# Script: Search for a string in a group of files
# Author: Robert C. Cain
#
# Revision History
#   V01 - 05/28/2012 - Initial version
#-------------------------------------------------------------------------
# This PowerShell script will loop over a collection of text files 
# which are delimited somehow (comma, tab, etc) and look for a 
# specific value. It will then print out the name of the file. 
#-------------------------------------------------------------------------

# Here I've hardcoded a value to look for, you could also
# pass this in as a paramter if you convert this to a function
$ValueToLookFor = '123456789'

# Our data files have a header row, this should match the 
# names of those columns. This will allow us to address 
# the columns by name later in the routine
$header = "Column1", "Column2", "ColumnToSearch", "Column3", "Column4"

# Move to the location with all the files
Set-Location 'C:\MyData'

# Retrive a list of files that we need to look thru
# Note the use of the filter to only select some files
# If you are doing all files you can omit the filter
$files = Get-ChildItem -Filter "Data*.txt" | Select-Object Name

foreach($file in $files)
{
  # Load the data in the file into $data
  #
  # In this example the files are tabbed delimted, hence the need
  # to specify the tab character as a delimter. If your file is
  # comma delimted you can omit the -Delimter, otherwise specify
  # whatever yours is. 
  #
  # We also need to pass in the $header variable so PowerShell 
  # will be able to associate each column of data with a name
  $data = Import-Csv $file.Name -Delimiter "`t" -Header $header

    foreach($line in $data)
    {
      # Here's where the header comes in handy, I can now use the 
      # column name as the .Property to look in
      if ($line.ColumnToSearch -eq $ValueToLookFor)
      { 
        # Print out the name of the file. You could do other things
        # like print out the actual line, or load it into a variable,
        # just to name a few. 
        $file.Name
        
        # Use a break if you only want the file name listed once
        # I actually let it print out multiple times to indicate how
        # many times it's in the file

        # break  
      }
    } # foreach($line in $data)
} # foreach($file in $files)

Last week when I blogged about iPowerShell from Sapien, I mentioned they had a few free community tools. One of those is PowerShell Help Community Edition.

As you can see, it provides a simple, easy to you way to navigate help in PowerShell. I find this especially nice as I often have my editor open in one window and want help files or other reference materials open in a second. Down the left is a tree which allows you to navigate to the type of object you are seeking help for. It also supports search, you can see the search results pane in the lower right.

 

Like iPowerShell, it uses the standard PowerShell help format, so you can add in any PowerShell help file, just as you can with iPowerShell. Best part is the price, it’s free! Just go to Sapien’s website, to the download center, and navigate to the Community Tools section. Select your flavor (32 or 64 bit).

One little quirk I was having that someone in the support forums helped me with, and this was a Windows issue not a PowerShell Help issue. I wanted to create a docked shortcut for this right in the Windows 7 task bar, but it wasn’t letting me. Turns out if you have the word Help in your file name Windows “protects” you by not letting you add it to the task bar. I created a shortcut, renamed the shortcut to remove the e in Help from the file name, and all was well with the universe.

OK, I admit it. I love PowerShell! It’s the way to go when you want to do automation. Not only that, it has the ability to hook into all aspects of the Microsoft tool stack. Not just SQL Server, but SharePoint, Exchange, Windows Server, and more. So here are some of my favorite resources for learning PowerShell.

A quick disclaimer, some of the links below are by co-workers or other people I have an affiliation with, financial or otherwise. That’s because I’m lucky enough to work with some of the best people in the field. Also, in the case of the books I’ve linked to the Kindle version where possible, mostly because I’m a Kindle junkie. There are paper versions of the books, and you are free to buy from your favorite retailer.

Books

Windows PowerShell in Action, Second Edition – If you are only going to buy one PowerShell book, this is “The” book as folks say. It’s one of the two that gets referenced quite often. Note the prior link is to Amazon, where you can only get the paper (aka “dead tree”) version. You can get electronic versions (Kindle, PDF, and ePub) directly from the publishers website.

PowerShell In Practice -  This is the other book in my collection that gets a good workout. Lots of great examples and easy to understand. Like the previous book, the link is to the Amazon dead tree version, you can also get the electronic version from the publishers website.

Windows PowerShell 2.0 Best Practices – So you want to know if you are doing PowerShell right? As it’s title implies, this book lays out best practices to help your PowerShell solutions succeed.

Note there are a lot of other really good PowerShell books on the market that focus on using PowerShell with specific technologies such as SharePoint, SQL Server, Exchange, and Windows Server. The books I’ve listed above cover PowerShell in general.

Blogs

Richard Siddaway – This guy blogs more about PowerShell than anyone I know. It’s an invaluable resource for PowerShell. In addition he is also the author of PowerShell in Practice, listed above.

PowerShell.com – This website is a treasure trove of resources. Indeed, it could have been put into all of the sections in this post as it holds not just blogs but free e-books, forums, and webinars.

Hey Scripting Guy – Ed Wilson, author of the Best Practices book above, is “The Scripting Guy”. Ed works on the PowerShell team at Microsoft and is very active in the PowerShell community.

Podcasts

PowerScripting Podcast – A great show out of Atlanta, has a lot of information about PowerShell especially for beginners. The website also has a lot of useful links. I always learn something new listening to their show.

Get-Scripting – If you are out of the UK you’ll appreciate this PowerShell Podcast. Has a focus on PowerShell with VMWare’s PowerCLI.

Videos

Pluralsight – I know Pluralsight has a great series on PowerShell, as I’m the one who authored them! In addition Pluralsight has an extensive catalog of other courses you can pick from. It’s subscription bases so there is a modest fee (starts at $29 US per month last I checked) but well worth it for the training you can get. There’s also a free trial.

For a quick link direct to this post, you can use http://bit.ly/arcaneps