Tag Archives: PowerShell

Exclude A File From Git Source Code Control in VSCode

Like many developers, I use VSCode for my projects. Primarily PowerShell, but also other things like the PHP code used for my ham radio club website or markdown projects.

It can be useful to have extra files, that I don’t need or want to have saved in Git. One example, I often have a “scratchpad.ps1” file. It’s a place I can prototype code, test out ideas, before I copy them into my main project. If this file gets lost, or damaged, I don’t really care as it’s just temporary.

Another example stems from my need to demonstrate code on video, for my Pluralsight courses, or live at SQL Saturdays and code camps. I often need to login to a website or database, and don’t want to have my credentials hard coded in my script for everyone to see.

To solve this, I simply place my needed information in a text file, then use Get-Content (with the -Raw switch) to read it into a variable. I don’t want this text file though to be placed in my public github repositories.

Excluding a file is simple. In the root folder of your source controlled project is a folder named .git. (Note the period on the front, also note that on some operating systems it may be hidden by default.) Under it is another folder called info.

In .git/info is a file called exclude (with no extension). To it, simply add the names of the files you want to exclude from source control. Here’s an example:

# git ls-files --others --exclude-from=.git/info/exclude
# Lines that start with '#' are comments.
# For a project mostly in C, the following would be a good set of
# exclude patterns (uncomment them if you want to use them):
# *.[oa]
# *~
uid.txt

The lines that begin with a pound sign (#) are comments, and are included by git.

Under it is the file I added, uid.txt. This file is will now be excluded from any git commits, and will not be uploaded to github or whatever tool you use for git. You can see this in the VSCode file tree, the file will appear in an off color font. In a dark mode theme, this will be a gray.

The arrow points to the uid.txt file that was excluded from the project.

That’s all there is to it. Now you can include extra files, such as scratch pads, notes, or even passwords that you don’t need (or want) to have as part of your git repository. Just keep in mind it’s your responsibility to back these files up.

Suppress Write-Verbose When Calling A function in PowerShell

In a project for one of my recent Pluralsight courses, “Everyday PowerShell for Developers on Linux, macOS, and Windows“, I had created some functions in my DataFabricator module. Some of these called many child functions.

I wanted to be able to call the parent function using Verbose in order to track progress. As you may be aware though, when you call a parent function using Verbose, it carries down to all of the functions it calls. This caused a large volume of verbose messages making it difficult to find the information I wanted. Hence I needed to suppress verbose in the called functions.

Let’s see how. First let’s create two simple functions that don’t suppress verbose messages.

function Parent1()
{
  [CmdletBinding()]
  param()

  Write-Verbose "Verbose: Parent1"
  Write-Host "Parent1"
  Child1
}

function Child1()
{
  [CmdletBinding()]
  param()

  Write-Verbose "Verbose: Child1"
  Write-Host "Child1"
}

When we call the parent function with the verbose switch, we get the expected output.

Parent1 -Verbose

VERBOSE: Verbose: Parent1
Parent1
VERBOSE: Verbose: Child1
Child1

Now let’s create an alternate version of the parent function. To suppress verbose on a called child function, all that is needed is to add the verbose switch, but followed by a colon and a $false.

function Parent2()
{
  [CmdletBinding()]
  param()

  Write-Verbose "Verbose: Parent2"
  Write-Host "Parent2"
  Child1 -Verbose:$false
}

Here is the output when we call the new Parent2.

Parent2 -Verbose

VERBOSE: Verbose: Parent2
Parent2
Child1

As you can see, the verbose statements in the Child1 function do not trigger. All that was needed was to add “-Verbose:$false” as a switch when calling Child2.

And that’s all there is to it, using this simple technique you can suppress verbose messages when calling a child function, whether verbose was used to call the parent or not.

Fixing the Failed To Acquire Token Error When Logging Into Azure from PowerShell

When I haven’t used Azure from PowerShell in some time, I’ll get an error “failed to acquire token” when using cmdlets such as Get-AzSubscription or Get-AzContext.

To add to the confusion, sometimes Connect-AzAccount appears to work and log me in, but I’ll still get the error.

Under the hood, the Azure cmdlets are caching some connection tokens. Fixing this issue is pretty simple, although not intuitive or easy to find the solution. All you have to do is issue the following cmdlet:

Clear-AzContext

This will remove old tokens from your system.

After issuing the clear command, just end your current PowerShell session, then login again using Connect-AzAccount and you should be good to go.

Iterate Over A Hashtable in PowerShell

Iterating over an array in PowerShell using a foreach loop is pretty simple. You might think you can do the same thing with a hash table using the same syntax, but if you do you’ll get nothing back.

It is possible to loop over a hash table though, using one of two methods.

First, let’s create a simple hashtable.

$hash = @{
'About Arcane Code' = 'http://arcanecode.me'
'ArcaneCode Blog' = 'https://arcanecode.com'
'ArcaneCode RedGate Articles' = 'http://arcanecode.red'
'ArcaneCode Github Repository' = 'http://arcanerepo.com'
}

In the first method, the one that I prefer, you can use the GetEnumerator method of the hash table object.

foreach ($h in $hash.GetEnumerator() )
{
  Write-Host "$($h.Name) : $($h.Value)"
}

Within the loop, you can use the Name property to get the key part of the hash, and the Value property to retrieve the value. Here is the output:

ArcaneCode Blog : https://arcanecode.com 
ArcaneCode Github Repository : http://arcanerepo.com 
About Arcane Code : http://arcanecode.me 
ArcaneCode RedGate Articles : http://arcanecode.red

In the second method, instead of iterating over the hash table itself, we loop over the Keys of the hash table.

foreach ($h in $hash.Keys) 
{
  Write-Host "$h: $($hash.Item($h))"
}

For each key, we retrieve it’s value from the hash table using the key to indicate the item we want. We could have shortened this slightly, but skipping the Item and just referencing the value by the key, using this syntax:

foreach ($h in $hash.Keys) 
{
  Write-Host "$h: $($hash.$h)"
}

Both of these methods produce the same output as our original version.

ArcaneCode Blog : https://arcanecode.com 
ArcaneCode Github Repository : http://arcanerepo.com 
About Arcane Code : http://arcanecode.me 
ArcaneCode RedGate Articles : http://arcanecode.red

There you go, two simple ways in which you can iterate over a hash table in PowerShell. As I indicated, I prefer GetEnumerator because I have access to both the key and the value in a single variable within my loop. But feel free to use the method that works best for your situation.

Two New PowerShell Courses for Developers on Pluralsight

I’m proud to announce I have not one but two, yes TWO new PowerShell courses available on Pluralsight, targeted for developers. These two courses are designed to work in harmony, to show developers how they can leverage PowerShell to automate and assist in their daily work.

The first is “PowerShell 7 Quick Start for Developers on Linux, macOS, and Windows”. It leverages your knowledge as a developer to bring you up to speed on the PowerShell language. It doesn’t waste time explaining concepts you already know, like variables and loops. It simply shows you how to do things in PowerShell.

The second course is “Everyday PowerShell for Developers on Linux, macOS, and Windows”. It begins showing how to combine PowerShell with Docker to create a PHP container and test a simple website. It then proceeds to create an Azure SQL database and load data in it.

In the second you are taught how to code your own classes and modules by creating a simple module that leverages a USPS website API to lookup a zip code and return the city and state it belongs to.

In the final part of the course you are shown how to use the new DataFabricator module to generate realistic looking, but fake data for use in testing your applications.

While originally developed with PowerShell 7.0.3, all code in the course was tested and videoed using PowerShell 7.1.0.

Additionally, the code was tested on a variety of platforms, including Ubuntu 20.04, 20.10, Windows 10, H1 and H2, macOS Catalina and even macOS Big Sur.

If you don’t have a Pluralsight subscription, just go to the Pluralsight page and click the Try for Free link to get a free 10 day trial.

VSCode User Snippets for PowerShell and MarkDown

I’ve been working a lot on a new course for Pluralsight, Everyday PowerShell 7 for Developers. I’ve been cranking out a lot of code as a result, PowerShell, obviously, as well as Markdown for documentation.

I’m finding the use of VSCode’s User Snippets to be extremely useful for developing this course. Snippets have allowed me to store my most often used code as templates, quickly manifesting new sections of code.

In PowerShell I’m finding these especially useful for writing Pester tests. Most tests fall into a few simple patterns, using a snippet I can quickly create the bulk of my test, leaving me to fill in the few remaining pieces.

If you know Markdown, you know some of the syntax can be a bit arcane. Using snippets makes it easy to insert commonly used ones, such as being able to type in a snippet name of mdlink and get the correct syntax to appear for a hyperlink.

It also helps with consistency. A small example, for doing italics in Markdown, you can use a singe underscore or a single asterisk. For bold, two underscores or asterisks.

I decided (for no particular reason) to use underscores for italics and asterisks for bold. If it’s been a while since I wrote Markdown though, I may not always remember. So I setup mditalic and mdbold to insert the correct formatting for me.

I’ve placed my snippets in a repository on my github site:

https://github.com/arcanecode/VSCode_User_Snippets

If you aren’t familiar with how to use User Snippets, I have a quick overview in a Markdown file in the repository.

Speaking of Markdown, there is one issue with VSCode. By default, intellisense is not enabled in VSCode for Markdown. While snippets can be used without intellisense, it’s much easier to use them with intellisense turned on.

In the repository I have a Markdown  file with information on how to turn intellisense on in VSCode for Markdown files.

Be aware there are also user snippets available through the Extensions Marketplace in VSCode. Just go into the Extensions, and enter Snippet into the search bar.  I just didn’t happen to find any that quite suited my needs, hence I created my own.

You’re welcome to copy and use the user snippets, just some or all of them, as you need. Do be aware some of them have my name and websites, such as the author info snippet I use for the bottom of all my Markdown files, or the header snippet for the top of my PowerShell files.

I hope you find user snippets as helpful as I do.

Getting Started with PowerShell Core on Linux and macOS

My newest course, Getting Started with PowerShell Core on Linux and macOS, is now live on Pluralsight! This course is my eighteenth in a long line of Pluralsight courses.

I begin the course explaining the difference between PowerShell for Windows (version 5.1) and the all-new PowerShell Core (version 6.2 was used for this course), which works not only on Windows but on Linux and macOS as well. I then show how to install PowerShell Core, along with a few other key components such as Visual Studio Code, on both Linux and macOS.

Not familiar with PowerShell? No problem! I quickly cover the basics of PowerShell including cmdlets, the use of the pipeline, how to write functions, and how to put those functions in reusable scripts.

As if that weren’t enough, I show how to do some “cool things” with PowerShell Core, including working with Docker containers, SQL Server, and Azure.

For the course, I primarily used Ubuntu 19.04 and macOS Mojave. The code was also tested on Ubuntu 18.04 LTS and 18.10, as well as macOS High Sierra. In addition, I tested the Linux installs on a variety of distributions including CentOS, Manjaro, and more. The samples include markdown files with information on how to install on these other distributions.

All of the samples are included in the downloadable components of the course on Pluralsight. New with this course I have the samples also available on my GitHub site. As I move into the future the GitHub codebase will be updated with new samples and information.

Also included in the samples are several markdown files that have additional information not included in the course, such as setting VSCode on Windows to use PowerShell Core instead of Windows PowerShell 5.1 as the default terminal.

While you are up on my GitHub site be sure to check out the full list of repositories, I have a lot of examples on it, including some from previous courses such as my recent Reporting Services course. (For a full list of my courses just check out the About ArcaneCode page on this site.)

Note the sample file on Pluralsight will remain static, so if someone watches the course their samples will reflect what is in the course. For the latest updated samples see the GitHub site referenced above.

What? You don’t have a Pluralsight subscription yet? Well, no worries dear reader, just email me, free @ arcanetc.com and I can send you a code good for 30 days with which you can watch all 18 of my courses, plus anyone else’s course at Pluralsight.

Downloading Files with PowerShell and Invoke-WebRequest

Last weekend I was at the Atlanta Code Camp, giving a presentation on PowerShell for Developers. One of the attendees emailed me, asking for a good example of being able to download multiple files from a website.

To do so, we’ll use the Invoke-WebRequest cmdlet. But first, we’ll setup a directory to hold our output. For all my demos I have a root folder, C:\PowerShell. For this article I’ve created a subfolder called Invoke-WebRequest-Demo. The first thing my code does is sets a variable to point to this folder, then changes the current location to it.

$dir = 'C:\PowerShell\Invoke-WebRequest-Demo'
Set-Location $dir

Next, we’ll need a variable to point to the root URL of the website we wish to download from. This will be everything except for the actual file name.  For this demo I will use files from the No Agenda Show podcast. Not only is it a cool podcast, but they have an unrestricted license for their material meaning I can freely reuse it for demo purposes. Each episode of the podcast has it’s own image, so we’ll download images from a few recent episodes.

After I place the base URL into a variable, I create an array, each item in the array having the name of a file to download.

$baseUrl = 'http://adam.curry.com/enc/'
$files = '1537127586.656_na-1069-art-feed.png',
          '1536868338.385_na-1068-art-feed.png',
          '1536264075.484_na-1066-art-feed.png'

Now that everything is setup, we use a simple foreach loop to iterate over the array and download each file via Invoke-WebRequest.

foreach ($file in $files)
{
   Write-Host "Downloading $file"
   $dlUrl = "$($baseUrl)$file"
   $dlPath = "$($dir)$file"
   Invoke-WebRequest $dlUrl -OutFile $dlPath
}

And that’s all there is to it. If you want to learn more about downloading files via the web, this code was extracted from my Testing PowerShell with Pester course on Pluralsight. In it I test a module which gets the RSS feed, then downloads images and audio files for the No Agenda show. Just go to the About Me link at the top and you’ll find a complete list of all my Pluralsight courses.

Speaking of Pester, you might also appreciate the introduction to Pester series of articles I’m currently authoring for Red Gate’s Simple Talk website. You can find a link via the same About Me page, or just jump directly there by going to http://arcanecode.red to see my articles.

Get Notified When Your Home Router IP Address Changes–With PowerShell!

Readers of my blog over may recall I’m an amateur (ham) radio operator (N4IXT that’s me!). One of my ham radio buddies has a cool radio that can be remotely controlled over the internet. With their software he just enters the IP address of his home (his home router that is, after we setup some port forwarding), and he can connect to and operate his radio remotely, make contacts around the world and more.

The tricky part of all this came in the sentence enters the IP address of his home…. His home router tends to change its address on the whim of his service provider. If the router reboots, it will definitely have a new address when it comes up. You may be thinking “well just get a static IP address”. Unfortunately, his internet service provider can’t give him this, something about him using a bonded pair of 10 mb lines to get 20 mb prevents it.

So how could he get notified when his IP address changes? Well I was fairly certain I could solve his problem with some simple PowerShell scripting!

I’ve now published the solution on my GitHub site:

https://github.com/arcanecode/PowerShell/tree/master/IPAddressMailer

Rather than going more in-depth here, the ReadMe has the full details describing the scripts that were created. In addition the code is heavily commented.

In addition to solving this particular problem, the code could be used as a template for:

  • Getting the IP Address of your home router
  • Sending emails via GMail
  • Setting up tasks in the Windows Task Scheduler

Hopefully other people with similar issues will find this little project as useful as my ham radio buddy has!

Azure PowerShell PlaybooK: Azure SQL–Now on Pluralsight!

My latest course is now available on Pluralsight! It’s the Azure PowerShell Playbook: Azure SQL. If you aren’t familiar with Pluralsight’s Playbook format, they are fast past courses that are almost 100% demo driven. They are meant to be consumed quickly, my course is just one hour and four minutes long. Great lunchtime viewing!

This course shows you how to use PowerShell to manage and migrate your on premises database up to Azure SQL. In brief, you’ll learn how to:

  • Create resource groups
  • Create and work with Storage Accounts
  • Create a SQL Server in Azure SQL
  • Package up your local database into a bacpac file
  • Import your bacpac file into a new Azure SQL database
  • Execute commands against your new Azure SQL database
  • Cleanup! I even show how to remove everything you’ve created, individually or as a whole

And all of this with PowerShell!

Additionally, I’ve included functions for just about everything listed, so (assuming your subscription gives you access to the samples) you’ll have a great starting point for your own library of cmdlets. (All the code for the functions appears on screen, so if you have to you could always pause and type it in.)

You can find my new course at:

https://www.pluralsight.com/courses/azure-powershell-sql-playbook

I also wrote an article for RedGate’s SimpleTalk website that aligns well with this course. I dive deeper into the restartability aspect of the way the functions were coded, something I couldn’t get deep into with the video course due to time constraints.

https://www.red-gate.com/simple-talk/sysadmin/powershell/powershell-functions-reusability-restartability-azure/

What’s that? Yes you in the back row, you say you don’t have a Pluralsight subscription? Well no worries, just email me, free@arcanetc.com and I’ll be glad to send you a code that will be good for 30 days at Pluralsight. During that time you can watch my courses, indeed you can watch any course at Pluralsight.

Creating Azure Resource Groups Safely With PowerShell

A few posts back I mentioned that I had become a SimpleTalk author, and that my first post for them was PowerShell Functions For Reusability and Restartability in Azure (read it at http://bit.ly/acred01)

I’ve created a companion video for it, which focuses on creating Resource Groups. You can see it below, or on YouTube at http://bit.ly/acredyt01.

As I mention in the video, you can find the code samples on my github site, at http://bit.ly/acredgit01

I’m Now a SimpleTalk Author!

RedGateSimpleTalkLogoVerticalAdding to my other activities, I’m now writing for SimpleTalk, RedGate’s community hub. My first article just went live.

PowerShell Functions for Reusability and Restartability in Azure is the title, in it I describe how to implement the concepts of reusability and restartability in PowerShell. The functions were written against the Azure platform, however the concepts are valid for any PowerShell implementation. You can read the full article at:

https://www.red-gate.com/simple-talk/sysadmin/powershell/powershell-functions-reusability-restartability-azure/

If you want to keep up with all my articles, I’ve setup a special URL which will jump you to my author page at SimpleTalk:

http://arcanecode.red

It’s small now, but expect it to grow quickly. Happy reading!

Eric Ligman’s FREE Microsoft eBook Giveaway–Revising the download script

Every year, Eric Ligman, director of Sales Excellence for Microsoft, creates a blogpost in which he gives away tons of FREE Microsoft eBooks. This year has 361 in the list.

You name it, it’s in the list. SQL Server, Azure, PowerShell, .NET, BizTalk, SharePoint, Windows Server, and more. You can find Eric’s post at:

https://blogs.msdn.microsoft.com/mssmallbiz/2017/07/11/largest-free-microsoft-ebook-giveaway-im-giving-away-millions-of-free-microsoft-ebooks-again-including-windows-10-office-365-office-2016-power-bi-azure-windows-8-1-office-2013-sharepo/#comments

While there are individual links to each file, what if you want every one of them? He explains on the post why he doesn’t provide a big zip file. He does, however, provide a PowerShell script (attributed to David Crosby) that will do the job.

However, I found some issues with the script. Not that it didn’t work, it did, but there were several things I felt could be done to improve it.

First, there was no progress message issued during the download. As a user, I had no idea which file I was on, so had no concept of how much longer it would take. Thus, I’ve added a little progress message.

I then thought “Hmm, what if my downloads were interrupted, I don’t want to have to start all over”. So, I added some code that sees if the file we’re downloading already exists. This way it won’t re-download a file it already has.

But then another problem arose. What if it had partially downloaded a file? Just checking the file names wouldn’t catch that. So I added further code to compare the file size at the source with the file size on disk. If different, then it will re-download.

So far so good, now it will skip the file only if the file name is already on the local disk, and the file sizes match.

I now encountered my next concern. Crappy internet. I live out in the country, and while I love my privacy and rural living, my internet sucks. It is prone to go down or drop packets. If it had issues during a  download I didn’t want it to crash, but instead go onto the next file.

Thus I added a try/catch error handler, which displays an error message and continues on.

At this point I thought I was done. Just I was about to call it finished though, a typical afternoon Alabama thunderstorm came up. Kaboom! House rattled and power blinked.

This presented my final concern, what if the power went out? I’d want to know where it got to with the downloads. So I added some further code such that when the downloading starts it creates a new log file and appends each message to it.

I realize some of you have superfast gigabit internet and will be able to download these almost instantly. (I hate you by the way. #jealous). Therefore I made logging optional, so it wouldn’t create an extra file if you didn’t want it. Just set the $log variable to $false, and it will skip logging.

So there you go, a revised download script that will handle stopping and restarting the script gracefully, will look for errors, and adds logging so you can track progress.

You’ll find the revised script on my GitHub site, in the PowerShell folder:

https://github.com/arcanecode/PowerShell

Just look for the file “Eric Ligmans Microsoft eBook Giveaway Revised Download Script.ps1

There’s also a readme style file by the same name, which echoes this blog post.

What happened to Save-AzureRmProfile?

I’ve been working a lot in the Azure PowerShell area of late. One thing I wanted to be able to do is have my scripts login automatically to Azure. In many examples the cmdlet Save-AzureRmProfile was used to save your Azure credentials, then later you could use Import-AzureRmProfile to import them.

But, when I attempted to run Save-AzureRmProfile I got the error ‘Save-AzureRmProfile is not recognized as the name of a cmdlet, function, script file, or operable program’.  Huh? I checked the docs, and it does include a listing for Save-AzureRmProfile.

https://docs.microsoft.com/en-us/powershell/module/azurerm.profile/save-azurermprofile?view=azurermps-3.8.0

This is a case of the PowerShell AzureRM module getting ahead of the docs. After beating my head against the wall, I found the cmdlets had been replaced with the new noun of AzureRmContext.

To use them, first login to Azure manually. Then, use the new Save-AzureRmContext to save your information to a file.


# Setup – First login manually per previous section
Add-AzureRmAccount

# Now save your context locally (Force will overwrite if there)
$path = "C:\Azure\PS\ProfileContext.ctx’
Save-AzureRmContext -Path $path -Force

Once that’s done, from then on you can use the Import-AzureRmContext to automate the login.


# Once the above two steps are done, you can simply import
$path = C:\Azure\PS\ProfileContext.ctx’
Import-AzureRmContext -Path $path

Be warned, this does present a security issue. If someone were to steal your context file, they could then login as you. You need to be sure your context file is stored in a safe location no one can get to.

IT ops and news talk–Episode 3 Secure that Jump Server

I owe everyone an apology, I missed blogging about this at the time it occurred. Last December I was a guest on Don Box’s podcast, “IT ops and news talk”. I appeared on Episode 3, Secure that Jump Server.

In the podcast we discuss the testing of PowerShell code with Pester. After that we got into an interesting discussion on the current state of DevOps. Give it a listen I think you’ll enjoy, it’s about half an hour in length. Don is a great interviewer it was a lot of fun.

As a follow on to the discussion, you might want to learn more about Pester. I have a complete course on the subject in my Pluralsight course Testing PowerShell with Pester. If you don’t know anything about PowerShell, but want to learn, then I’d suggest my Beginning PowerShell Scripting for Developers. Note that even though it says “…for Developers” we don’t mean programmers (although it could), instead it refers to people who wish to develop scripts in PowerShell.

What? What was that? You in the back row waving your hand? You say you don’t have a Pluralsight subscription? Hey, no problem. Just email me, free <at> arcanetc.com and I can send you a code good for 30 days of free access to Pluralsight, with which you can watch not just my courses, but any course from Pluralsight’s library of over 5,000 courses.