Category: Books

Fun With PowerShell – Showing Book Data at the Library of Congress with Start-Process

In my previous post, Fun With PowerShell – Opening Websites with Start-Process, I showed how to use the Start-Process cmdlet to open a website. This is part of my ongong ArcaneBooks Project, in which I created a new function to display the webpage for a book at the OpenLibrary website by using the ISBN.

I wanted to create a similar function to work with the Library of Congress website, and so let me present the Show-LCCNBookData function.

Show-LCCNBookData

The function I created, Show-LCCNBookData is almost identical to the Show-ISBNBookData function I covered in the previous post, so I won’t go into a lot of depth in this post.

As with the ISBN version, I made this an advanced function so users could pipe data into it.

function Show-LCCNBookData
{
  [CmdletBinding(HelpURI="https://github.com/arcanecode/ArcaneBooks/blob/1ebe781951f1a7fdf19bb6731487a74fa12ad08b/ArcaneBooks/Help/Get-ISBNBookData.md")]
  [alias("slccn")]
  param (
         [Parameter( Mandatory = $true,
                     ValueFromPipeline = $true,
                     HelpMessage = 'Please enter the LCCN (Library of Congress Control Number).'
                     )]
         [string] $LCCN
        )

Note I still need to update the help URL to the correct one, but the rest of the function opening is complete, with the sole parameter being the $LCCN.

Now we fall into the process block.

  process
  {
    foreach($number in $LCCN)
    {
      Write-Verbose "Beginning Show-LCCNBookData for $ISBN at $(Get-Date).ToString('yyyy-MM-dd hh:mm:ss tt')"

      $lccnCleaned = $LCCN.Replace('-', '').Replace(' ', '')
      $lccnPrefix = $lccnCleaned.Substring(0,2)
      $lccnPadded = $lccnCleaned.Substring(2).PadLeft(6, '0')

      # Now combine the reformatted LCCN and save it as a property
      $lccnFormatted ="$($lccnPrefix)$($lccnPadded)"

      $baseURL = "https://lccn.loc.gov/"

      $url = "$($baseURL)$($lccnFormatted)"

      Write-Verbose 'Opening the Book on Library of Congress Number'

      Start-Process $url

      Write-Verbose "Finished Getting Data for $($LCCN)"
    }

    Write-Verbose "Done opening the web pages at Library of Congress"

  }

When we fall into the process loop we first need to clean up the LCCN that was passed in. As was documented in my LCCN overview post the LCCN is the two digit year at front, then six digits. If the number of digits after the first two isn’t six in length we have to zero pad it to become six, which will make the entire LCCN string eight digits.

We then append the formatted LCCN to the base URL for the LOC website. Then we use the Start-Process cmdlet to open the webpage.

Calling Show-LCCNBookData

Calling the function is pretty easy, you can either pass in a Library of Congress Control Number as a parameter or via the pipeline. All these examples should open the Library of Congress website, in your default browser, with the book associated with the LCCN you passed in.

# Pass in a single LCCN as a parameter
$LCCN = '54009698'
Show-LCCNBookData -LCCN $LCCN -Verbose

# Alias
$LCCN = '54009698'
slccn -LCCN $LCCN -Verbose

# Pipe in a single ISBN
$LCCN = '54-9698'
$LCCN | Show-LCCNBookData

.EXAMPLE
# Pipe in an array of LCCNs
$LCCNs = @( '54-9698'
          , '40-33904'
          , '41-3345'
          , '64-20875'
          , '74-75450'
          , '76-190590'
          , '71-120473'
          )
$LCCNs | Show-LCCNBookData -Verbose

In the final example we can actually pipe in an array of LCCNs, it should open up a page for each one.

Note the Library of Congress isn’t perfect, sometimes it will bring up a page with multiple items for the number passed in as it may have multiple entries. It’s still faster though than having to do manual searches on the LoC website.

See Also

You may find more helpful information at the links below.

ArcaneBooks Project at GitHub

ArcaneBooks Project Introduction

ArcaneBooks – Library of Congress Control Number (LCCN) – An Overview

Fun With PowerShell – Advanced Functions

Fun With PowerShell – Opening Websites with Start-Process

Fun With PowerShell – Strings

Fun With PowerShell – Write-Verbose

Conclusion

This post and the previous one demonstrates how easy it can be to create helper functions for your modules. My two show functions are designed to let users quickly bring up the webpage for the books they are working with.

If you like PowerShell, you might enjoy some of my Pluralsight courses. PowerShell 7 Quick Start for Developers on Linux, macOS and Windows is one of many PowerShell courses I have on Pluralsight. All of my courses are linked on my About Me page.

If you don’t have a Pluralsight subscription, just go to my list of courses on Pluralsight . At the top is a Try For Free button you can use to get a free 10 day subscription to Pluralsight, with which you can watch my courses, or any other course on the site.

ArcaneBooks – Parsing Library of Congress Control Number (LCCN) Data With PowerShell

Introduction

In my previous post in this series, ArcaneBooks – Library of Congress Control Number (LCCN) – An Overview, I provided an overview of the LCCN and the basics of calling its public web API to retrieve data based on the LCCN.

In this post I will demonstrate how to call the API and dissect the data using PowerShell. This will be a code intensive post.

You can find the full ArcaneBooks project on my GitHub site. Please note as of the writing of this post the project is still in development.

The code examples for this post can be located at https://github.com/arcanecode/ArcaneBooks/tree/main/Blog_Posts/005.00_LCCN_API. It contains the script that we’ll be dissecting here.

XML from Library of Congress

For this demo, we’ll be using an LCCN of 54-9698, Elements of radio servicing by William Marcus. When we call the web API URL in our web browser, we get the following data.

<zs:searchRetrieveResponse xmlns:zs="http://docs.oasis-open.org/ns/search-ws/sruResponse">
  <zs:numberOfRecords>2</zs:numberOfRecords>
  <zs:records>
    <zs:record>
      <zs:recordSchema>mods</zs:recordSchema>
      <zs:recordXMLEscaping>xml</zs:recordXMLEscaping>
      <zs:recordData>
        <mods xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns="http://www.loc.gov/mods/v3" version="3.8" xsi:schemaLocation="http://www.loc.gov/mods/v3 http://www.loc.gov/standards/mods/v3/mods-3-8.xsd">
          <titleInfo>
            <title>Elements of radio servicing</title>
          </titleInfo>
          <name type="personal" usage="primary">
            <namePart>Marcus, William. [from old catalog]</namePart>
          </name>
          <name type="personal">
            <namePart>Levy, Alex,</namePart>
            <role>
              <roleTerm type="text">joint author</roleTerm>
            </role>
          </name>
          <typeOfResource>text</typeOfResource>
          <originInfo>
            <place>
              <placeTerm type="code" authority="marccountry">nyu</placeTerm>
            </place>
            <dateIssued encoding="marc">1955</dateIssued>
            <issuance>monographic</issuance>
            <place>
              <placeTerm type="text">New York</placeTerm>
            </place>
            <agent>
              <namePart>McGraw Hill</namePart>
            </agent>
            <dateIssued>[1955]</dateIssued>
            <edition>2d ed.</edition>
          </originInfo>
          <language>
            <languageTerm authority="iso639-2b" type="code">eng</languageTerm>
          </language>
          <physicalDescription>
            <form authority="marcform">print</form>
            <extent>566 p. illus. 24 cm.</extent>
          </physicalDescription>
          <subject authority="lcsh">
            <topic>Radio</topic>
            <topic>Repairing. [from old catalog]</topic>
          </subject>
          <classification authority="lcc">TK6553 .M298 1955</classification>
          <identifier type="lccn">54009698</identifier>
          <recordInfo>
            <recordContentSource authority="marcorg">DLC</recordContentSource>
            <recordCreationDate encoding="marc">820525</recordCreationDate>
            <recordChangeDate encoding="iso8601">20040824072855.0</recordChangeDate>
            <recordIdentifier>6046000</recordIdentifier>
            <recordOrigin>Converted from MARCXML to MODS version 3.8 using MARC21slim2MODS3-8_XSLT1-0.xsl (Revision 1.172 20230208)</recordOrigin>
          </recordInfo>
        </mods>
      </zs:recordData>
      <zs:recordPosition>1</zs:recordPosition>
    </zs:record>
  </zs:records>
  <zs:nextRecordPosition>2</zs:nextRecordPosition>
  <zs:echoedSearchRetrieveRequest>
    <zs:version>2.0</zs:version>
    <zs:query>bath.lccn=54009698</zs:query>
    <zs:maximumRecords>1</zs:maximumRecords>
    <zs:recordXMLEscaping>xml</zs:recordXMLEscaping>
    <zs:recordSchema>mods</zs:recordSchema>
  </zs:echoedSearchRetrieveRequest>
  <zs:diagnostics xmlns:diag="http://docs.oasis-open.org/ns/search-ws/diagnostic">
    <diag:diagnostic>
      <diag:uri>info:srw/diagnostic/1/5</diag:uri>
      <diag:details>2.0</diag:details>
      <diag:message>Unsupported version</diag:message>
    </diag:diagnostic>
  </zs:diagnostics>
</zs:searchRetrieveResponse>

Let’s see how to retrieve this data then parse it using PowerShell.

Parsing LCCN Data

First, we’ll start by setting the LCCN in a variable. This is the LCCN for "Elements of radio servicing" by William Marcus

$LCCN = '54-9698'

To pass in the LCCN to the web API, we need to remove any dashes or spaces.

$lccnCleaned = $LCCN.Replace('-', '').Replace(' ', '')

After 2001 the LCCN started using a four digit year. By that time however, books were already printing the ISBN instead of the LCCN. For those books we’ll be using the ISBN, so for this module we can safely assume the LCCNs we are receiving only have a two digit year.

With that said, we’ll use the following code to extract the two digit year.

$lccnPrefix = $lccnCleaned.Substring(0,2)

Since digits 0 and 1 are the year, we’ll start getting the rest of the LCCN at the third digit, which is in position 2 and go to the end of the string, getting the characters.

Next, the API requires the remaining part of the LCCN must be six digits. So we’ll use the PadLeft method to put 0’s in front to make it six digits.

$lccnPadded = $lccnCleaned.Substring(2).PadLeft(6, '0')

Now combine the reformatted LCCN and save it to a variable.

$lccnFormatted ="$($lccnPrefix)$($lccnPadded)"

Now we’ll combine all the parts to create the URL needed to call the web API.

$baseURL = "http://lx2.loc.gov:210/lcdb?version=3&operation=searchRetrieve&query=bath.lccn="
$urlParams = "&maximumRecords=1&recordSchema=mods"
$url = "$($baseURL)$($lccnFormatted)$($urlParams)"

It’s time now to get the LCCN data from the Library of Congress site. We’ll wrap it in a try/catch so in case the call fails, for example from the internet going down, it will provide a message and exit.

Note at the end of the Write-Host line we use the PowerShell line continuation character of ` (a single backtick) so we can put the foreground color on the next line, making the code a bit more readable.

try {
  $bookData = Invoke-RestMethod $url
}
catch {
  Write-Host "Failed to retrieve LCCN $LCCN. Possible internet connection issue. Script exiting." `
    -ForegroundColor Red
  # If there's an error, quit running the script
  exit
}

Now we need to see if the book was found in the archive. If not the title will be null. We’ll use an if to check to see if the LCCN was found in their database. If not, the title property will be null. If so we display a message to that effect.

If it was found, we fall through into the else clause to process the data. The remaining code resides within the else.

# We let the user know, and skip the rest of the script
if ($null -eq $bookData.searchRetrieveResponse.records.record.recordData.mods.titleInfo.title)
{
  Write-Host = "Retrieving LCCN $LCCN returned no data. The book was not found."
}
else # Great, the book was found, assign the data to variables
{

To get the data, we start at the root object, $bookData. The main node in the returned XML is searchRetrieveResponse. From here we can use standard dot notation to work our way down the XML tree to get the properties we want.

Our first entry gets the Library of Congress Number. The syntax is a little odd. If we walk XML tree, we find this stored in:

<identifier type="lccn">54009698</identifier>

If we display the identifier property using this code:

$bookData.searchRetrieveResponse.records.record.recordData.mods.identifier

We get this result.

type #text
---- -----
lccn 54009698

The LCCN we want is stored in the property named #text. But #text isn’t a valid property name in PowerShell. We can still use it though if we wrap the name in quotes.

  $LibraryOfCongressNumber = $bookData.searchRetrieveResponse.records.record.recordData.mods.identifier.'#text'

From here we can process other properties that are easy to access.

  $Title = $bookData.searchRetrieveResponse.records.record.recordData.mods.titleInfo.title
  $PublishDate = $bookData.searchRetrieveResponse.records.record.recordData.mods.originInfo.dateIssued.'#text'
  $LibraryOfCongressClassification = $bookData.searchRetrieveResponse.records.record.recordData.mods.classification.'#text'
  $Description = $bookData.searchRetrieveResponse.records.record.recordData.mods.physicalDescription.extent
  $Edition = $bookData.searchRetrieveResponse.records.record.recordData.mods.originInfo.edition

Now we get to the section where an XML property can contain one or more values.

Books can have multiple authors, each is returned in its own item in an array. One example is the book subjects. Here is a sample of the XML:

<subject authority="lcsh">
  <topic>Radio</topic>
  <topic>Repairing. [from old catalog]</topic>
</subject>

As you can see, this has two topics. What we need to do is retrieve the root, in this case subject, then loop over each item.

For our purposes we don’t need them individually, a single string will do. So in the PowerShell we’ll create a new object of type StringBuilder. For more information on how to use StringBuilder, see my post Fun With PowerShell – StringBuilder.

In the loop if the variable used to hold the string builder is empty, we’ll just add the first item. If it’s not empty, we’ll append a comma, then append the next value.

  $authors = [System.Text.StringBuilder]::new()
  foreach ($a in $bookData.searchRetrieveResponse.records.record.recordData.mods.name)
  {
    if ($a.Length -gt 1)
      { [void]$authors.Append(", $($a.namePart)") }
    else
      { [void]$authors.Append($a.namePart) }
  }
  $Author = $authors.ToString()

As a final step we used the ToString method to convert the data in the string builder back to a normal string and store it in the $Author variable.

From here, we’ll repeat this logic for several other items that can hold multiple values. The books subjects is one example.

  $subjects = [System.Text.StringBuilder]::new()
  $topics = $bookData.searchRetrieveResponse.records.record.recordData.mods.subject | Select topic
  foreach ($s in $topics.topic)
  {
    if ($subjects.Length -gt 1)
      { [void]$subjects.Append(", $($s)") }
    else
      { [void]$subjects.Append($s) }
  }
  $Subject = $subjects.ToString()

A book could have multiple publishers over time. The author could shift to a new publisher, or more likely a publishing house could be purchased and the new owners name used. The data is returned as an array, so combine them as we did with authors and subjects.

Note that in the returned data, the publisher is stored as an "agent". We’ll use the name Publisher to keep it consistent with the ISBN data.

  $thePublishers = [System.Text.StringBuilder]::new()
  foreach ($p in $bookData.searchRetrieveResponse.records.record.recordData.mods.originInfo.agent)
  {
    if ($thePublishers.Length -gt 1)
      { [void]$thePublishers.Append(", $($p.namePart)") }
    else
      { [void]$thePublishers.Append($p.namePart) }
  }
  $Publishers = $thePublishers.ToString()

Since there could be multiple publishers, logically there could be multiple publishing locations. This section will combine them to a single location.

  $locations = [System.Text.StringBuilder]::new()
  foreach ($l in $bookData.searchRetrieveResponse.records.record.recordData.mods.originInfo.place.placeTerm)
  {
    if ($locations.Length -gt 1)
      { [void]$locations.Append(", $($l.'#text')") }
    else
      { [void]$locations.Append($l.'#text') }
  }
  $PublisherLocation = $locations.ToString()

All done! We’ll give a success message to let the user know.

  Write-Host "Successfully retrieved data for LCCN $LCCN" -ForegroundColor Green

Finally, we’ll display the results. Note some fields may not have data, that’s fairly normal. The Library of Congress only has the data provided by the publisher. In addition some of the LCCN data dates back many decades, so the data supplied in the 1940’s may be different than what is supplied today.

  "LCCN: $LCCN"
  "Formatted LCCN: $lccnFormatted"
  "Library Of Congress Number: $LibraryOfCongressNumber"
  "Title: $Title"
  "Publish Date: $PublishDate"
  "Library Of Congress Classification: $LibraryOfCongressClassification"
  "Description: $Description"
  "Edition: $Edition"
  "Author: $Author"
  "Subject: $Subject"
  "Publishers: $Publishers"
  "Publisher Location: $PublisherLocation"
}

The Result

Here is the result of the above code.

LCCN: 54-9698
Formatted LCCN: 54009698
Library Of Congress Number: 54009698
Title: Elements of radio servicing
Publish Date: 1955
Library Of Congress Classification: TK6553 .M298 1955
Description: 566 p. illus. 24 cm.
Edition: 2d ed.
Author: Marcus, William. [from old catalog], Levy, Alex,
Subject: Radio, Repairing. [from old catalog]
Publishers: McGraw Hill
Publisher Location: nyu, New York

As you can see it returned a full dataset. Not all books my have data for all the fields, but this one had the full details on record with the Library of Congress.

See Also

This section has links to other blog posts or websites that you may find helpful.

The ArcaneBooks Project – An Introduction

ArcaneBooks – ISBN Overview, PowerShell, and the Simple OpenLibrary ISBN API

ArcaneBooks – PowerShell and the Advanced OpenLibrary ISBN API

ArcaneBooks – Library of Congress Control Number (LCCN) – An Overview

Fun With PowerShell – StringBuilder

The GitHub Site for ArcaneBooks

Conclusion

In this document we covered the basics of the LCCN as well as the web API provided by the Library of Congress. Understanding this information is important when we integrate the call into our PowerShell code.

ArcaneBooks – Library of Congress Control Number (LCCN) – An Overview

Introduction

This is part of my ongoing series on my ArcaneBooks project. The goal is to provide a module to retrieve book data via provided web APIs. In the SEE ALSO section later in this post I’ll provide links to previous posts which cover the background of the project, as well as how to use the OpenLibrary APIs to get data based on the ISBN.

In this post I will provide an overview of using the Library of Congress API to get data based on the LCCN, short for Library of Congress Control Number.

The next post in this series will provide code examples and an explanation of how to use PowerShell to get data using the Library of Congress API.

LCCN Overview

The abbreviation LCCN, according to the Library of Congress’s own website, stands for Library of Congress Control Number. When the system was first created in 1898, however, LCCN stood for Library of Congress Card Number, and I’ve seen it both ways in publications.

I’ve also seen a few places define it as Library of Congress Catalog Number, although this was never an official designation.

The LCCN was created in 1898 to provide a unique value to every item in the Library of Congress. This not only includes books, but works of art, manuscripts (not in book form), maps, and more.

LCCN Format

The LCCN has two parts, a prefix followed by a serial number. From 1898 to 2000 the prefix was two digits, representing the year. Beginning in 2001 the prefix became four digits, representing the year.

The serial number is simple a sequential number. 45-1 was the first number assigned in 1945. 45-1234 was the 1,234th item assigned in that year.

Be aware from 1969 to 1972 there was an experiment where the single digit of 7 was used for the prefix. They decided this scheme wasn’t going to work out, and reverted to the standard format of year followed by serial number.

Here are a few examples of real LCCNs from books in my personal collection. You can use these in your own testing.

LCCN Title
54-9698 Elements of Radio Servicing
40-33904 Radio Handbook Twenty-Second Edition
41-3345 The Radio Amateur’s Handbook 42nd Edition 1965
64-20875 Early Electrical Communication
74-75450 VHF Handbook for Radio Amateurs
76-190590 Wire Antennas for Radio Amateurs
71-120473 73 Vertical, Beam, and Triangle Antennas

Accessing Book Data from the Library of Congress

The Library of Congress actually provides two web APIs for getting book data. The first API is for accessing assets, such as digital assets. It doesn’t return much data for books.

The second is the LC Z39.50 system, accessible through lx2.loc.gov. Here is an example of calling it to retrieve a record for the book Elements of Radio Servicing, which has the LCCN of 54-9698. (It should, of course, all be used as a single line just in case your web browser wraps it.)

http://lx2.loc.gov:210/lcdb?version=3&amp;operation=searchRetrieve&amp;query=bath.lccn=54009698&amp;maximumRecords=1&amp;recordSchema=mods

Breaking it down, the root call is to http://lx2.loc.gov:210/lcdb. After this is a question mark ?, followed by the parameters.

The first parameter is version=3. This indicates which format to use for the return data. It supports two versions, 1.1 and 3. For our purposes we’ll use the most current version, 3.

Following the ampersand &amp; is operation=searchRetrieve. This instructs the Library of Congress’s API that we want to do a search to retrieve data.

Next is the core piece, we need to tell it what LCCN number to look up, query=bath.lccn=54009698. The root object is bath, then it uses the property lccn.

The LCCN has to be formatted in a specific way. We start with the two or four digit year. In the above example, 54-9698, this would be the two digit year of 54.

Next is the serial number. If the number is less than six digits, it must be left zero padded to become six. Thus 9698 becomes 009698. The year and serial number are combined, removing any dashes, spaces, or other characters and becomes 54009698.

Following is maximumRecords=1, indicating we only expect one record back. That’s all we’ll get back with a single LCCN anyway, so this will work fine for our needs.

The final parameter is recordSchema=mods. The API supports several formats.

Record Schema Description Notes
dc Dublin Core (bibliographic records) Brings back just the basics (Name, author, etc)
mads MADS (authority records) Brief, not a lot of info
mods MODS (bibliographic records) Very readable XML schema, most info
marcxml MARCXML – the default schema Abbreviated schema, not readable
opacxml MARCXML (wth holdings attached) As above with a bit more info

You are welcome to experiment with different formats, but for this module we’ll be using mods. It provides the most information, and is in XML. XML is very easy to read, and it works great with PowerShell.

ISBN and Library of Congress

It is possible to use the Library of Congress to look up the ISBN. In my testing though, the interface provided by OpenLibrary provided more data. Thus we’ll be using it for looking up ISBNs in this module.

We’ll use the LCCN API for books where we only have the LCCN.

See Also

The ArcaneBooks Project – An Introduction

ArcaneBooks – ISBN Overview, PowerShell, and the Simple OpenLibrary ISBN API

ArcaneBooks – PowerShell and the Advanced OpenLibrary ISBN API

Conclusion

In this document we covered the basics of the LCCN as well as the web API provided by the Library of Congress. Understanding this information is important when we integrate the call into our PowerShell code.

Eric Ligman’s FREE Microsoft eBook Giveaway–Revising the download script

Every year, Eric Ligman, director of Sales Excellence for Microsoft, creates a blogpost in which he gives away tons of FREE Microsoft eBooks. This year has 361 in the list.

You name it, it’s in the list. SQL Server, Azure, PowerShell, .NET, BizTalk, SharePoint, Windows Server, and more. You can find Eric’s post at:

https://blogs.msdn.microsoft.com/mssmallbiz/2017/07/11/largest-free-microsoft-ebook-giveaway-im-giving-away-millions-of-free-microsoft-ebooks-again-including-windows-10-office-365-office-2016-power-bi-azure-windows-8-1-office-2013-sharepo/#comments

While there are individual links to each file, what if you want every one of them? He explains on the post why he doesn’t provide a big zip file. He does, however, provide a PowerShell script (attributed to David Crosby) that will do the job.

However, I found some issues with the script. Not that it didn’t work, it did, but there were several things I felt could be done to improve it.

First, there was no progress message issued during the download. As a user, I had no idea which file I was on, so had no concept of how much longer it would take. Thus, I’ve added a little progress message.

I then thought “Hmm, what if my downloads were interrupted, I don’t want to have to start all over”. So, I added some code that sees if the file we’re downloading already exists. This way it won’t re-download a file it already has.

But then another problem arose. What if it had partially downloaded a file? Just checking the file names wouldn’t catch that. So I added further code to compare the file size at the source with the file size on disk. If different, then it will re-download.

So far so good, now it will skip the file only if the file name is already on the local disk, and the file sizes match.

I now encountered my next concern. Crappy internet. I live out in the country, and while I love my privacy and rural living, my internet sucks. It is prone to go down or drop packets. If it had issues during a  download I didn’t want it to crash, but instead go onto the next file.

Thus I added a try/catch error handler, which displays an error message and continues on.

At this point I thought I was done. Just I was about to call it finished though, a typical afternoon Alabama thunderstorm came up. Kaboom! House rattled and power blinked.

This presented my final concern, what if the power went out? I’d want to know where it got to with the downloads. So I added some further code such that when the downloading starts it creates a new log file and appends each message to it.

I realize some of you have superfast gigabit internet and will be able to download these almost instantly. (I hate you by the way. #jealous). Therefore I made logging optional, so it wouldn’t create an extra file if you didn’t want it. Just set the $log variable to $false, and it will skip logging.

So there you go, a revised download script that will handle stopping and restarting the script gracefully, will look for errors, and adds logging so you can track progress.

You’ll find the revised script on my GitHub site, in the PowerShell folder:

https://github.com/arcanecode/PowerShell

Just look for the file “Eric Ligmans Microsoft eBook Giveaway Revised Download Script.ps1”

There’s also a readme style file by the same name, which echoes this blog post.

The Phoenix Project

Just wanted to take a second during my lunch break to let you know about a book called “The Phoenix Project”. Back in ancient times, the 1980’s, a man named  named Eli Goldratt wrote a book entitled “The Goal”. It was about applying a concept called “Theory of Constraints” to the manufacturing process. But this wasn’t a boring textbook, but instead written as a novel. At the time a new concept, in the time since it was published the Theory of Constraints has become integrated into almost every manufacturing operation around the world.

“The Phoenix Project” takes the concepts of The Goal and updates them with new management techniques like Agile and Kanban, then applies them to the crazy world of IT. And just like in The Goal, The Phoenix Project uses the form of a novel to tell the story.

Bill is a newly minted VP of IT for Parts Unlimited. Bill inherits a mess, a project pivotal to the success of the company dubbed The Phoenix Project is over two years late and out of control. Not only that, operations in general is in a shambles. Bill is given just a few months to fix it, or the entire IT division will be outsourced.

Paralleling the goal, a mysterious figure steps in to mentor Bill. Using the Socratic method, he guides Bill in the application of Theory of Constraints, Agile, Kanban, and other techniques to the world of IT. Along the way Bill shifts the organization from a silo model of Dev, Operations, and Projects to the unified model known as DevOps.

So why am I delaying my lunch break (my lovely wife makes a killer cube steak) to blog about this? Well until Midnight, Wednesday April 3rd you can get the book for FREE in Kindle format from Amazon:

http://www.amazon.com/The-Phoenix-Project-Business-ebook/dp/B00AZRBLHO/ref=sr_1_1?s=digital-text&ie=UTF8&qid=1364912906&sr=1-1&keywords=the+phoenix+project

I had actually paid good money and loved the book so much I’d already started this review post, now that it’s free I’m happy to share with all of you. It’s not a long book, I read most of it on a Friday evening and finished it up on a Saturday. Since finishing it I’ve begun applying some of the concepts to my personal and professional life with good results. If you happen to run across this post after April 3rd I still recommend getting it, even if you have to pay for it like I did. It is well worth your time and money.

Don’t have a Kindle? No problem, there is free Kindle software for all the tablets; iPad, Android and yes even Surface. No device? There’s also a Kindle app for your PC and Mac. If all else fails, there’s even a cloud reader so you can read your books on any web browser.

If you want to know more about DevOps, the authors (Gene Kim, Kevin Behr, and George Spafford) run a website called IT Revolution at http://itrevolution.com/ where you can find out more.

This is a great book that should be required reading for everyone in IT. I believe it will have as big an impact to IT as The Goal did to manufacturing. And if you haven’t read The Goal, get it as well, perhaps even read it first. The Phoenix Project makes several references to The Goal, which will make more sense if you read The Goal first. (The Goal is also available as an audio book from Audible, I’m hoping the Phoenix Project will be converted to audio soon.)

Enjoy, now if you’ll excuse me I have a cube steak with my name on it.

T-SQL Tuesdays–Resolutions

There’s a new meme going around the SQL Server blogosphere called T-SQL Tuesdays. A bunch of SQL Bloggers all post about the same SQL related topic on the same day. Back at the beginning of January fellow MVP Jen McGown (blog | twitter) wrote a blog post about resolutions for the new year. Granted I’m way too late to be included in that round of SQL Server related Tuesdays, but thought it was a great idea and wanted to put up my resolutions. Since it’s February it’s a little late for new years, maybe we can call them Ground Hogs’ day resolutions?

Produce more variety – I’ve been doing a lot of content for Pluralsight, I want to continue putting out good content for them, but want to put out other content as well. I want to try and blog more, and hope to self publish a few books on the Kindle platform.

Consume more – As much as I love to produce content, I recognize that I need to do a better job of consuming more. My Kindle has been a big help with that, having a huge library available all the time has made it easy to switch between the subjects I want or need to learn about.

Balance my consumption – While I love technical books, I realize to be a true professional I need a good balance. I’ve started reading a lot of what I call “professional” books. Books that give me insights into things like business, community, time management, and teamwork.

Better balance of family life – Let’s face it, us really aggressive type A folks work a lot. When we’re not doing our 9 to 5 job we’re producing content like blog posts, books, videos, or are off at some user group meeting or weekend event presenting. This has a definite impact on the family. While my family is incredibly understanding, we still miss each other. So I’m working on some creative solutions to this issue.

Get my next certification – Last year I earned my MCTS for SQL Server 2008 Business Intelligence. This year my goal is to advance that and earn my MCITP also in SQL Server BI.

More Beta work – The CTP for the next version of SQL Server is now out. I want to spend more time working with it so when the final version is ready I’ll be well up to speed, plus better able to share the new features with the community.

I suppose this list is identical to the list of many people. but if I put it in writing I’ll be able to both measure it and hold myself accountable.

Step 2 – Learn iteratively

There are many ways to learn. Attending a presentation or live webcast is great because you get what I call “condensed knowledge”. You get the results of someone else spending thirty to forty hours of learning and working to condense it into a one to two hour presentation. You also get immediate feedback, you can interact with the presenter and ask questions. On the downside, if you missed something, you missed it (unless the presentation is recorded). These are great for getting an overview, however you won’t really get deep technical understanding out of a presentation like this.

The next step beyond a live presentation is a recorded webcast or video, such as those produced by DNRTV. These are great because you can pause them, rewind, and listen multiple times. However, you lose the ability to get immediate feedback from the presenter, and like live presentations it’s condensed knowledge.

Online reading is the next area available to us for learning. Blogs, MSDN, TechNet, etc. These have a much deeper level of information than previously mentioned formats. Plus they tend to be updated as changes are made. However, there do tend to be some limitations. Content is not quite as polished as a book as often it does not go through professional editors. Also, while the content will be more in depth than a presentation, it will still be limited in scope. It will be rare to find the equivalent of a book given out for free in a blog. Most of the time it will be equivalent to a really long magazine article.

This then, brings us to good old fashioned books. For true topical mastery, there’s nothing like a good thick book filled with code examples for learning. Some pundits are already predicting the death of the print book. However, in my opinion print is simply a medium. Books could be read online, or downloaded to something like a Kindle. Whatever the form, it’s still an in depth presentation of content.

So does this mean you should favor books over other forms of learning? Absolutely not. If you recall yesterday’s post, I mentioned being able to keep up your base. Podcasts, webcasts, user groups, and blogs can be great ways of doing just that, while using books for honing your expert skills in some topic.

Deciding on a delivery mechanism for your learning is the first step. Now you need to decide the process. Learning is a gradual process. It takes learning a little something every day for it to take hold. Even as little as 20 minutes a day can rapidly bring your understanding up to new levels. Avoid cramming, cramming works for short term but studies show long term retention is not good. For ultimate learning, start with a goal. Decide what you want to know. Then gather the materials you’ll need: books, articles, manuals, blogs, etc. Next, schedule the time. As I mentioned you want at least 20 minutes a day devoted to learning. Turn off your e-mail, close your twitter, turn off the radio, close the home office door, and focus on the material.

Practice what you read as well. Type in the code samples, run them, debug them, step through the code line by line. Make changes and see how it affects the flow of the code. Find the patterns and practices for your environment and try them out.

Using an iterative process you will soon be on your way to expert level knowledge.

Arcane Review: Expert SQL Server 2005 Integration Services

If you recall my “Good Reads” post from June 25th, you will remember I am a big believer in books as a learning medium. I like to employ a lot of different ways to learn: user groups, blogs, podcasts, videocasts, and magazines to name a few. But for really in depth coverage, it’s hard to beat a nice book in your hands. I got some good feedback from my mention last week of Andy Leonard’s new e-book on Data Dude, so I thought that I would continue by adding book reviews to the blog every so often.

Expert SQL Server 2005 Integration Services For this review I thought I’d cover a book that seems to constantly be on my desk lately: Expert SQL Server 2005 Integration Services, by Brian Knight and Eric Veerman. This book does a really good job and is specifically targeted toward the data warehousing professional. One entire chapter is devoted to ETL for dimension tables; another chapter focuses on the fact tables. It was great to have coverage so focused on these topics.

Another favorite part of the book is the two chapters on deploying and managing SSIS packages. So often these topics are glossed over, especially the managing piece. The book does a great job in covering all the tools and practices around this subject. I’ll mention one more chapter, one that focuses on package reliability. They cover logging, auditing, event handling, checkpoint files, and even suggestions on testing error handling logic.

There are many more chapters in the book, such as migration from DTS (SQL Server 2000) and Scalability, for you to discover. The other thing I love about this book is the brevity. The authors cover an amazing amount of information in just 382 pages. As a busy, busy person I very much appreciate the conciseness they achieved without sacrificing any clarity.

I’ve met both authors, and have heard them speak. They are both very nice, knowledgeable individuals, and I highly encourage you to attend one of their presentations if you get the chance, or if not at least buy their book from your favorite retailer; you will find it a great investment.