Fun With PowerShell Hash Tables

Introduction

Hash tables are powerful, flexible items. In some languages, hash tables are also called dictionaries. In this post we’ll start with a few basics, then get into some fun things you can do with hash tables.

A quick note, some documentation calls them a hash table, others read hashtable, one word. For this post I’ll use the two word hash table, but it’s the same thing no matter what documentation or blog you read.

For all of the examples, we’ll display the code, then under it the result of our code. In this article I’ll be using PowerShell Core, 7.1.3, and VSCode. The examples should work in PowerShell 5.1 in the PowerShell IDE, although they’ve not been tested there.

To run a snippet of code highlight the lines you want to execute, then in VSCode press F8 or in the IDE F5. You can display the contents of an array variable (or any variable) by highlighting it and using F8/F5.

Hash Table Basics

To create a hash table you will need to use the @{} syntax to create a set of key / value pairs.

$hash = @{'Key'                 = 'Value'
          'PowerShell'          = 'PowerShell.com'
          'Arcane Code'         = 'arcanecode.com'
          'Pluralsight'         = 'pluralsight.com'
          'GitHub - ArcaneCode' = 'github.com/arcanecode'
         }
$hash                  # Display all values

Result:
Name                           Value
----                           -----
Arcane Code                    arcanecode.com
Key                            Value
Pluralsight                    pluralsight.com
PowerShell                     PowerShell.com
GitHub - ArcaneCode            github.com/arcanecode

In this example I lined up the equal signs because, in my humble opinion, it’s easier to read. But it’s not required.

Similar to an array, you can use positional notation to retrieve a value, only instead of a numeric value you pass in the key.

$hash['PowerShell']

Result:
PowerShell.com

The hash table manifests each key as a property. This means instead of positional notation you can instead use . (dot) notation.

$hash.PowerShell

Result:
PowerShell.com

But Arcane, you may ask, some of those keys have spaces in them. How does that work?

Well all you have to do is put the key in quotes.

$hash.'Arcane Code'

Result:
arcanecode.com

Well that was fun, but what about updating a value for a key? All you have to do is assign the new value.

$hash.'Arcane Code' = 'ArcaneCode.me'
$hash

Result:
Name                           Value
----                           -----
GitHub - ArcaneCode            github.com/arcanecode
Key                            Value
PowerShell                     PowerShell.com
Pluralsight                    pluralsight.com
Arcane Code                    arcanecode.me

As you can see the value for the 'Arcane Code' key has changed from arcanecode.com to arcanecode.me.

In addition to updating, you will have times you want to add a new item to an array. Doing so isn’t intuitive, but it’s easy. Just use the hash table, provide the new key and assign the value to it.

$hash.Ubuntu = 'Ubuntu.com'
$hash

Result:
Name                           Value
----                           -----
GitHub - ArcaneCode            github.com/arcanecode
Key                            Value
Ubuntu                         Ubuntu.com
PowerShell                     PowerShell.com
Pluralsight                    pluralsight.com
Arcane Code                    arcanecode.me

Fun Properties and Methods of Hash Tables

Hash tables have a variety of useful properties and methods you can call. The first, Keys, will return a list of all the keys.

$hash.Keys

Result:
GitHub - ArcaneCode
Key
PowerShell
Pluralsight
Arcane Code
Ubuntu

As you might guess, there’s a corresponding property to return all the values.

$hash.Values

Result:
github.com/arcanecode
Value
PowerShell.com
pluralsight.com
arcanecode.me
Ubuntu.com

Just like the array data type, hash tables also have a Count property to let you know how many key/value pairs are in the table.

$hash.Count

Result:
6

Earlier in this post I showed how to add a new key/value pair. Before adding, it would probably be a good idea to check and see if it is already there. There is a ContainsKey method that will do that.

$hash.ContainsKey('Arcane Code')

Result:
True

This method will return either True or False depending on the presence of the key.

Likewise, there is a method to check to see if a value is present.

$hash.ContainsValue('pluralsight.com')

Result:
True

What if we’d wanted to use ContainsKey to see if a key was present, and if so we wanted to remove it? There is a Remove method that will delete a key/value pair from our hash table.

$hash.Remove('Ubuntu')
$hash

Result:
Name                           Value
----                           -----
GitHub - ArcaneCode            github.com/arcanecode
Key                            Value
PowerShell                     PowerShell.com
Pluralsight                    pluralsight.com
Arcane Code                    arcanecode.me

As you can see in the results, the key / value pair for the Ubuntu entry is now gone.

The final method we’ll look at in this section will allow us to remove all key / value pairs from the hash table.

$hash.Clear()
$hash

Result:
(Nothing is displayed because the hash table is empty)

Creating an Empty Hash Table

Much as with arrays, there are times you will want to create an empty hash table. You could then enter a loop that added new key / value pairs to the hash table.

To declare an empty hash table, you simply assign @{} to a variable.

$empty = @{}

Now you can assign values to it. Here we’ll do it in individual lines, but as I indicated this technique is most useful in a loop.

$empty['Pluralsight'] = 'pluralsight.com'
$empty['DataFabricator'] = 'datafabricator.com'
$empty

Result:
Name                           Value
----                           -----
Pluralsight                    pluralsight.com
DataFabricator                 datafabricator.com

Ordered Hash Tables

Let’s reset our hash table, then look at the output.

$hash = @{'Key'                 = 'Value'
          'PowerShell'          = 'PowerShell.com'
          'Arcane Code'         = 'arcanecode.com'
          'Pluralsight'         = 'pluralsight.com'
          'GitHub - ArcaneCode' = 'github.com/arcanecode'
         }
$hash

Result:
Name                           Value
----                           -----
Arcane Code                    arcanecode.com
Key                            Value
Pluralsight                    pluralsight.com
PowerShell                     PowerShell.com
GitHub - ArcaneCode            github.com/arcanecode

Look at the output carefully. The returned list is not in the same order we added them to the hash table. Most of the time you are referencing the hash table via a specific key, so the ordering isn’t a concern.

What if you needed them to be returned in the same order they were added in? Fortunately PowerShell has an [ordered] instruction you can place before declaring the hash table.

$hash = [ordered]@{
  'Key'                 = 'Value'
  'PowerShell'          = 'PowerShell.com'
  'Arcane Code'         = 'arcanecode.com'
  'Pluralsight'         = 'pluralsight.com'
  'GitHub - ArcaneCode' = 'github.com/arcanecode'
}
$hash

Result:
Name                           Value
----                           -----
Key                            Value
PowerShell                     PowerShell.com
Arcane Code                    arcanecode.com
Pluralsight                    pluralsight.com
GitHub - ArcaneCode            github.com/arcanecode

As you can see, the results are returned in the exact same order they were added to the hash table. This example also demonstrates you can begin the list of key / value pairs on the next line after the single squiggly bracket {.

Iterating (Looping) Over a Hash Table

It is possible to loop over the contents of a hash table. But it isn’t quite as intuitive as it is with arrays. If you were to use foreach($item in $hash) you’d get an error, because PowerShell isn’t sure if you want to loop over the keys, values, or something else.

Instead, the hash table has a special method called GetEnumerator which will allow you to iterate over the table, returning one key / value pair each time through the loop.

foreach($item in $hash.GetEnumerator())
{
  "$($item.Name) has a value of $($item.Value)"
}

Result:
Key has a value of Value
PowerShell has a value of PowerShell.com
Arcane Code has a value of arcanecode.com
Pluralsight has a value of pluralsight.com
GitHub - ArcaneCode has a value of github.com/arcanecode

This is the way I prefer to iterate over a hash table. Just my personal opinion, but I think the code is cleaner and easier to read. That said, there is a second method you may like better.

In it you loop over the Key values, then reference the value from the hash table, using the key as the position. Do notice that instead of using the key[] notation, you have to reference the Item method then pass the key into it.

foreach($key in $hash.Keys)
{
  "$key value is $($hash.Item($key))"
}

Result:
Key value is Value
PowerShell value is PowerShell.com
Arcane Code value is arcanecode.com
Pluralsight value is pluralsight.com
GitHub - ArcaneCode value is github.com/arcanecode

As you can see, the result is the same either way. Use which ever method makes the most sense for you.

Splatting

Right now you are probably asking yourself “Splatting? Hey Arcane, are you trying to confuse us with silly names?”

Fear not, splatting is a real thing. First, let’s start with an example. Here, we are going to take the output of GetChildItem and pipe it to a cmdlet called Join-String. If you aren’t familiar with it, Join-String will take a list of objects and join them to a single string.

Get-ChildItem -Directory D:\ |
  Join-String -Property Name -Separator ', '

Result:
Backups_Temp, CamtasiaWorkspace, Hyper-V, ISOs, Mobile Report Publisher

With splatting, we create a hash table, Each key is the name of a parameter that needs to be passed in, and the value is what we want to use for that parameter.

$splat = @{Property = 'Name'
           Separator = ', '
          }

Here you can see the Property and Separator parameter names were passed in as keys, and the values are what we want to pass in.

Now we can call our cmdlet, passing in the name of our hash table. Only, and this is important, instead of a $ we use an @ at the front of the name of our hash table.

Get-ChildItem -Directory D:\ | Join-String @splat

Result:
Backups_Temp, CamtasiaWorkspace, Hyper-V, ISOs, Mobile Report Publisher

Whenever PowerShell sees a @ in front of a name that corresponds to a hash table, it understand you are using a splat and expands each key / value pair into parameters. As you can see, it shortens the call to Join-String significantly.

What if we want to change the value of one of our parameters? That’s easy, we just assign a new value. The rest of the hash table remains the same.

$splat.Separator = ' - '
Get-ChildItem -Directory D:\ | Join-String @splat

Result:
"Backups_Temp" - "CamtasiaWorkspace" - "Hyper-V" - "ISOs" - "Mobile Report Publisher"

All of the parameters don’t have to be included in the splat. For example, Join-String has a switch that, when included, will wrap each item in single quotes.

$splat = @{Property = 'Name'
           Separator = ', '
          }

Get-ChildItem -Directory D:\ |
  Join-String @splat -SingleQuote

Result:
'Backups_Temp', 'CamtasiaWorkspace', 'Hyper-V', 'ISOs', 'Mobile Report Publisher'

Here I used a switch, but it could be another parameter with a value added after the splat.

What if you wanted to include the switch in the hash table though? Switches don’t have a value that is passed in.

That turns out to be a simple answer, just include the switch as a key, and for the value use either $True or $False.

$splat = @{Property = 'Name'
           Separator = ', '
           SingleQuote = $True
          }
Get-ChildItem -Directory D:\ | Join-String @splat

Result:
'Backups_Temp', 'CamtasiaWorkspace', 'Hyper-V', 'ISOs', 'Mobile Report Publisher'

You could also include the switch in the splat, but set the value to $False if you didn’t want to use single quoted every time.

While this example only had a few parameters, I’ve worked with some cmdlets that had a dozen or more parameters that were required to pass in. SQL Server’s cmdlets come to mind.

I often had scripts that had a dozen parameters, but the only one that changed with each call was the actual SQL command I was sending to the server. Using a splat let me reduce the amount of code significantly.

Conclusion

In this post, we saw a lot of fun things you can do with hash tables. A few were creating, updating, adding and removing values. We also covered iterating over a hash table, and even the very useful concept of splatting.

Now you can use these features in your own scripts where a hash table comes into play.

The demos in this course came from my Pluralsight course PowerShell 7 Quick Start for Developers on Linux, macOS and Windows, one of many PowerShell courses I have on Pluralsight. All of my courses are linked on my About Me page.

If you don’t have a Pluralsight subscription, just go to my list of courses on Pluralsight. At the top is a Try For Free button you can use to get a free 10 day subscription to Pluralsight, with which you can watch my courses, or any other course on the site.

Fun With PowerShell Arrays

Introduction

In this article, we’ll look at the different ways to use Arrays in PowerShell. We’ll cover the basics, then move onto more advanced topics.

For all of the examples, we’ll display the code, then under it the result of our code. In this article I’ll be using PowerShell Core, 7.1.3, and VSCode. The examples should work in PowerShell 5.1 in the PowerShell IDE, although they’ve not been tested there.

To run a snippet of code highlight the lines you want to execute, then in VSCode press F8 or in the IDE F5. You can display the contents of an array variable (or any variable) by highlighting it and using F8/F5.

A quick note for my non-native English speaking friends, I use the term aka a few times. AKA is shortcut for “also known as“, it’s just a quick way to say “a is another word for b”.

Array Basics

The formal array to declare an array is to use @() around the contents, assigning it to a variable.

$array = @('Robert', 'Cain')
$array

Result:
Robert
Cain

Note that in the output, each item (aka element) in the array is displayed on a single line.

There is a little easier way to create an array, you can omit the @() . If you are passing in more than one item, separated by a comma, PowerShell converts it to an array.

$array = 'Robert', 'Cain'

If you have a long list of items, you can break the list over multiple lines. As long as a comma is the last thing on the line PowerShell will assume the rest of the array contents continue on the next line.

$array = 'Robert',
         'Cain'

You can access individual items in the array by using what is called positional notation. After the name of the array, use square brackets [] and inside put a number that indicates the position in the array.

Numbering begins at zero and goes to the maximum number of elements. (Well, technically the max number of elements minus one, but more on that in a moment.) If you try to use a number greater than the maximum number of elements, you will get an error.

$array = 'Robert', 'Cain'
$array[0]

Result:
Robert

$array[1]

Result:
Cain

Of course you can use variables in place of the numeric values for positional notation.

$x = 1
$array[$x]

Result:
Cain

So how do you know how many items are in an array? The array variable type has a Count property that can tell you that.

$array.Count

Result:
2

There is also a property called Length, which returns the same thing. They can be used interchangeably, most people however use the Count property.

One thing to be careful of with Count, is that it returns the number of items in the array. But arrays start with position 0 (zero), so the last item in an array will be Count - 1, not Count.

$array[$array.Count - 1]

Result:
Cain

PowerShell does provide a shortcut to get the last element. You can pass in a -1 (negative one) into the position, and it will get the last item.

$array[ -1 ]

Result:
Cain

Fun Tricks with Positional Notation

There are a few fun tricks you can do with positional notation. So far, we’ve only used a single value in the square brackets to get a single value from the array.

Did you know you can pass in multiple numbers and get back multiple elements from the array in one command?

$array = 'Zero', 'One', 'Two', 'Three',
         'Four', 'Five', 'Six', 'Seven'
$array[2, 4, 6]

Result:
Two
Four
Six

If you pass in the same number twice, you’ll get that array item back twice in the results.



$array = 'Zero', 'One', 'Two', 'Three',
         'Four', 'Five', 'Six', 'Seven'
$array[2, 4, 2]

Result:
Two
Four
Two

Finally, you can use PowerShell’s range operator to return a series of values.



$array = 'Zero', 'One', 'Two', 'Three',
         'Four', 'Five', 'Six', 'Seven'
$array[1..4]

Result:
'One'
'Two'
'Three'
'Four'

Adding Items to an Array

Adding items is pretty easy, you can use the += (plus equal) to add a new item into the array. In this example, we’ll add two new items, then display the new array, and finally the count of items.

$array = 'Robert', 'Cain'

$array += 'Arcane'
$array += 'Code'
$array

$array.Count

Result:
Robert
Cain
Arcane
Code
4

Here you can see are my original two items, followed by the new ones we just added. Finally the count, 4, is displayed.

Updating an Array Element

What if we want to update a value in the array? Turns out that is as easy as adding a new one. Simply address the item to replace with positional notation, then assign a new value.

$array[0] = 'Mr.'
$array

Result:
Mr.
Cain
Arcane
Code

Empty Arrays

When you need to create an empty array, you use the formal array declaration syntax we saw in the first example, only you put nothing in it.

$empty = @()
$empty

Result:
(Nothing is displayed because the array is empty)

So why would you need an empty array?

Let’s say you were looping over a collection of some type, and were building a new array to output. It might be a collection (aka array) of files, or records return from a SQL Server query.

You would create the empty array, enter the loop, extract and format the information you want then add it to the empty array using the += syntax.

Iterating Over Arrays

There are a few methods we can use to iterate, or loop over the contents of our arrays. The most common, and one I use the most, is a foreach loop.

foreach($item in $array)
{
  "The current item is $item"
}

Result:
The current item is Mr.
The current item is Cain
The current item is Arcane
The current item is Code

The next option is to pipe our array into the ForEach-Object cmdlet, passing a script block to the cmdlet with our instructions.

$array | ForEach-Object { "Current Value: $PSItem"}

Result:
Current Value: Mr.
Current Value: Cain
Current Value: Arcane
Current Value: Code

The variable $PSItem is a built in PowerShell variable, meant to represent the current item in a loop.

Finally, an Array datatype has a ForEach method built right in. You call the method, then as you did with the ForEach-Object cmdlet pass in a script block with the instructions.

$array.ForEach({ "Current Value: $_"})

Result:
Current Value: Mr.
Current Value: Cain
Current Value: Arcane
Current Value: Code

In this example we used the variable $_ to represent the current item. The $_ was the original way to create a placeholder, later PowerShell added the $PSItem variable. $PSItem was introduced way back in version 3, so unless you are on a really really REALLY old version of PowerShell, you can use $_ and $PSItem interchangeably.

Erasing the Contents of an Array

Clearing out the contents of an array is easy, just call the array’s Clear method.

$array.Clear()
$array

Result:
(Nothing is displayed because the array is empty)

Mixed Data Types

Unlike a lot of other languages, the values going into an array can be a mix of data types. In the example below I’m going to load an array with some strings, integers, and a floating point type.

$array = 'Arcane', 'Code', 33, 42,
         'Alabama', 3.14, 'In the Morning'
$array

Result:
Arcane
Code
33
42
Alabama
3.14
In the Morning

Removing Items From Arrays

We now move to a topic that doesn’t have a happy answer. You cannot remove an item from an array. That said, you could create a brand new array, filtering out the unwanted elements.

In this example, we pipe our array into the Where-Object cmdlet. We pass in a script block, in which we say only pass out items where the value of the current item is not equal ( -ne ) to the state of Alabama. After that we display the results.

Note too we are using another built in line continuation feature of PowerShell. If the pipe symbol, a | , is the last thing on the line PowerShell assumes what is on the next line is a continuation of the line above. (I like to indent the second line to the right of my equal sign, but it’s not required.)



$array = 'Arcane', 'Code', 33, 42,
         'Alabama', 3.14, 'In the Morning'
$newArray = $array |
            Where-Object { $PSItem -ne 'Alabama' }
$newArray

Result:
Arcane
Code
33
42
3.14
In the Morning

As you can see, the state of Alabama is no longer in our list.

Fun Operators – In and Compare

There are other operators in PowerShell that can let us do fun things with arrays.

First, what if we need to know if an item is already in the array? There’s actually two methods to handle that. In the first, we’ll see if the item is in the array.

$array = 'Arcane', 'Code', 33, 42,
         'Alabama', 3.14, 'In the Morning'
'Arcane' -in $array

Result:
True

In the second method we’ll use the contains operator.

$array -contains 'Code'

Result:
True

More Fun Operators – Join

What if you wanted to convert all of the elements in the array into a single string? To handle that we have the join operator.

After the join, you pass in a character to use to separate each element. You also have the option to pass in an empty string if you don’t want anything separating your results.

In the last example, you’ll see that you can pass in more than one character to separate the results.

$array = 1, 3, 5, 7, 9
$result = $array -join ' '
$result

Result: 1 3 5 7 9

$array = 1, 3, 5, 7, 9
$result = $array -join '-'
$result

Result: 1-3-5-7-9

$array = 1, 3, 5, 7, 9
$result = $array -join ','
$result

Result: 1,3,5,7,9

$array = 1, 3, 5, 7, 9
$result = $array -join ''
$result

Result: 13579

$result = $array -join ', '
$result

Result: 1, 3, 5, 7, 9

Even More Fun Operators – Replace, Match, Split

There are three more operators we’ll cover here. The first is Replace, which will search through the array and replace one string with another.

$servers = 'HOLD-001', 'HOLD-002', 'SQL-001', 'SQL-002'
$newServers = $servers -replace 'HOLD', 'TEST'
$newServers

Result:
TEST-001
TEST-002
SQL-001
SQL-002

You might use this to develop a script to run against your servers. When designing the script you might use a string like HOLD-001, HOLD-002, etc where HOLD is just a place holder.

At runtime you pass a parameter into your script such as DVLP (development), TEST, or PROD (production) so you can use the same script in multiple environments.

The next operator we’ll look at is match. Match can be used as a filter. Let’s take a look at this example:

$servers = 'SRV-001', 'SRV-002', 'SQL-001', 'SQL-002'
$newServers = $servers -match 'SQL'
$newServers

Result:
SQL-001
SQL-002

As you can see, only servers with SQL in the name were returned. This might be useful if you had a full list of all servers you wanted to iterate over, but in certain sections of your script you only wanted to do things to your SQL Servers.

The final operator to look at is split. You can use split to further divide the elements in your array into smaller parts.

In this example we’ll reuse our list of servers, but split the items based on a dash (or hyphen).

$servers = 'SRV-001', 'SRV-002', 'SQL-001', 'SQL-002'
$newServers = $servers -split '-'
$newServers

Result:
SRV
001
SRV
002
SQL
001
SQL
002

Here you can see each server name was split into two parts, each part becoming a new element in our output array. Also note that the character we split on, the dash, is discarded from the result set.

Iterating Over an Array of Objects

Just like arrays of simple items like strings and integers, you can also loop over an array of objects.

Here, we’ll get a list of files in my temp directory using the Get-ChildItem cmdlet. We’ll also use the empty array technique you saw earlier. A foreach loop will be used to go over each file object in the array and format a string with some of its properties, then add that string to our empty $newFiles array.

Set-Location "C:\Temp"
$files = Get-ChildItem
$newFiles = @()
foreach($file in $files)
{
  $newFiles += "$($file.Name) is $($file.Length) in size"
}
$newFiles

Result:
2019-ARRL-FieldDay-Rules-RevA.pdf is 111434 in size
ARRL-FIELD-DAY.adi is 76110 in size
azuredatastudio-windows-user-setup-1.13.1.exe is 91789368 in size
curl-7.74.0.zip is 6068670 in size
standard-notes-3.6.14-win.exe is 105761936 in size
SysinternalsSuite.zip is 41889407 in size

Within my loop $file represents the current item in my $files array. Inside I access the properties of my file object. I could, for example, have used if ($file.Length -gt 1) and done something useful.

In this case I’m using string interpolation to display the value of a property. Note that because I want it to access the property, within the string I have to wrap it in $(), such as $($file.Name).

This forces PowerShell to first evaluate what is inside the $(), then return that into the string.

Conclusion

This post turned out longer than I’d intended, but there are so many fun things you can do with a PowerShell Array it was hard to stop at just a few.

With luck you’ll have found some new techniques you can employ with arrays in your own PowerShell scripts.

The demos in this course came from my Pluralsight course PowerShell 7 Quick Start for Developers on Linux, macOS and Windows, one of many PowerShell courses I have on Pluralsight. All of my courses are linked on my About Me page.

If you don’t have a Pluralsight subscription, just go to my list of courses on Pluralsight. At the top is a Try For Free button you can use to get a free 10 day subscription to Pluralsight, with which you can watch my courses, or any other course on the site.

Fun With PowerShell String Formatting

Introduction

In last Monday’s blog post, Fun With PowerShell Strings, I covered a lot of the basics around PowerShell Strings. One thing we didn’t get into though was some of the special formatting commands you can use with PowerShell.

Specifically, this can control the output when we embed a numeric value inside a string. Passing in special formatting instructions will make it easy to display values with commas, as currency, or even as hexidecimal.

For all of the examples, we’ll display the code, then under it the result of our code. In this article I’ll be using PowerShell Core, 7.1.3, and VSCode. The examples should work in PowerShell 5.1 in the PowerShell IDE, although they’ve not been tested there.

To run a snippet of code highlight the lines you want to execute, then in VSCode press F8 or in the IDE F5. You can display the contents of an array variable (or any variable) by highlighting it and using F8/F5.

How it Works

PowerShell supports the C# / C / C++ style of string formatting. Let’s take a look at this example.

[string]::Format("There are {0} items.", $items)

Inside our string we have {0}. This is a placeholder, when a C style language sees this it will look for the first variable after the string and put that into placeholder 0.

PowerShell of course always wants to make life easier on us, so we can use this much simpler format.

"There are {0} items." -f $items

In PowerShell we have our string, including the placeholder. After it we have a -f to let PowerShell know the following items will be copied into the placeholders. In this case, the value in $items will be inserted into the {0} placeholder.

It’s not necessary to use a variable, we could have just hardcoded a value.

"There are {0} items." -f 33

What if we had multiple items? Well we’d just use multiple placeholders (all on one line).

"There are {0} items in the location {1}." -f $items, $loc

In this example, $items will go into the {0} and the value in $loc will get inserted into placeholder {1}.

You can even repeat a placeholder.

"There's {0} items in {1}. {0} is a lot of items!" -f $items, $loc

Now, if you’ve read my previous post you’re probably thinking “wait, wouldn’t string interpolation make all these unnecessary?”

And in these examples you are correct, with string interpolation you could have used the much easier to read syntax.

"There are $items items."

So why use the placeholder style? Well it turns out the placeholder syntax includes some extra syntax which will allow you to format values passed in. Let’s take a look at some examples.

Number

In the placeholder, after the placeholder position ( 0 in these examples ) you can put a colon : , then a letter that indicates the formatting style. Then another number which indicates the decimal places to retain.

"N0 {0:N0} formatted" -f 12345678.119    

Result:
N0 12,345,678 formatted

In this example, after the colon we have the letter N. This lets PowerShell know we want this to be formatted as a numeric value, and to include commas to separate the numbers.

In this example after the letter N we have another number, a 0. This indicates how many decimal places to retain in the output.

Here’s another example where we’ll include two decimal places.

"N2 {0:N2} formatted" -f 12345678.119

Result:
N2 12,345,678.12 formatted

As you can see, it follows the normal rules for rounding up the output. The next to the last last number, 1, was rounded up to 2 since the number after it was a 9.

Often we want to display numbers as right aligned, so the last digit will line up in our output. This means we want to display a number of spaces in front of it. PowerShell includes a feature for this too.

"N0 {0,8:N0} formatted" -f 123.119

Result:
N0      123 formatted

After the place holder we have a comma then a number ( here 0,8 ). This tells PowerShell we want the value for this placeholder to be eight characters in width.

In the output, PowerShell saw the final value, 123, was three characters wide. So it subtracted three from eight and put five spaces in front of the 123 so the entire output became eight characters in width.

A common question is “what if the output is longer than the width passed in?”. For example, you used 0,3 but your input to the placeholder was a value of 123456? In that case PowerShell will still format it as 123,456 but will not add any spaces before the number. This could result in some misaligned output in your display, but at least the accurate value will still be shown.

Currency

Using the C format instruction, PowerShell can display the results in currency format. It will use the default currency symbol for your computer’s language / location. Since I’m in the United States it will use a dollar sign.

"C0 {0:C0} formatted" -f 12345678.1234
"C2 {0:C2} formatted" -f 12345678.1234
"C2 {0,20:C2} formatted" -f 12345678.1234

Result:
C0 $12,345,678 formatted
C2 $12,345,678.12 formatted
C2       $12,345,678.12 formatted

Currency acts very similar to the Number format, the only real difference is the addition of your location’s currency symbol in the formatted output.

Percentage

PowerShell also has the ability to display values in a percentage format.

"P0 {0:P0} formatted" -f 0.1234
"P2 {0:P2} formatted" -f 0.1234
"P2 {0,8:P2} formatted" -f 0.1234

Result:
P0 12% formatted
P2 12.34% formatted
P2   12.34% formatted

Make sure to notice that PowerShell will take the decimal amount and automatically convert the percentage for you. Thus, if you want to display 12%, you have to pass in 0.12.

Hex

Next up on the list is Hexadecimal notation. PowerShell supports two methods for doing Hex values. In the first, it will take a standard number (base 10) and convert it to hexadecimal then display it.

"X0 0x{0:X0} formatted" -f 1234

Result:
X0 0x4D2 formatted

In this case we passed in the number 1234. PowerShell converted it to hex then displayed it.

What if the value were already in hex format though, and we just wanted to display it? In PowerShell, we indicate a number is in hex by placing a 0x in front of it. So we just pass in our number with 0x in front to the string.

"X0 0x{0:X0} formatted" -f 0x4D2

Result:
X0 0x4D2 formatted

When PowerShell saw the 0x on the front of 0x4D2, it knew this value was already in hex format and didn’t try to convert it, it simply displayed our value.

Decimal

The name of the next formatter we’ll look at is Decimal. It’s a bit misnamed though, because it’s really just for formatting integer based numbers. As a matter of fact, if you try use with a non integer value, such as 33.42, it will error out.

Let’s look at the most basic form of Decimal.

"D0 {0:D0} formatted"   -f 123

Result:
D0 123 formatted

Pretty simple, it just displays the value. So you can guess D is the formatter for decimal, but what is D0? The number after the D indicates how many spaces we want to zero pad to.

In other words, we want the string to be a certain number of characters long, so the number indicates that length. It will then put zeros to the left to pad it out. Let’s look at this example to make it clear.

"D8 {0:D8} formatted"   -f 123

Result:
D8 00000123 formatted

In this case, the D8 indicated we wanted eight characters in decimal formatting. PowerShell converted 123 to a string that was three characters long, so it put five zero’s in front to make the total length eight.

You can also add the placeholder comma value formatting to make a value a certain width wide. In these two examples, the first will act like a Number format. The second though will let you use leading zeros, but still set the total space.

"D0 {0,9:D0} formatted" -f 123
"D0 {0,9:D6} formatted" -f 123

Result:
D0       123 formatted
D0   0000123 formatted

In the first example, it made the output nine characters wide by placing six spaces in front of the 123. In the second example, it first made the 123 six characters by placing 000 on front, resulting in 000123.

It then took the resulting 000123 and made that nine wide by placing three spaces in front. As you can see, PowerShell is very flexible when formatting this way.

A reminder, as stated earlier Decimal is only for integers. If I were to try using -f 123.1 as an input it would error out.

Custom Date Formatting

In addition to the other formats, PowerShell provides the ability to create custom date outputs. You do so using these characters, note they are case sensitive.

M / MMOne or Two character month number
d / ddOne or two character day of the month number.
yy / yyyyTwo or four digit year number.
h / hhOne or two digit hour of the day in 12 hour format
H / HHOne or two digit hour of the day in 24 hour format
m / mmOne or two digit minute of the hour
s / ssOne or two digit second of the minute

Let’s see these used in a series of examples.

"Today is {0:M/d/yy}."                 -f $(Get-Date)
"Today is {0,10:MM/dd/yyyy}."          -f $(Get-Date)
"Today is {0,10:dd/MM/yyyy}."          -f $(Get-Date)
"Today is {0,10:yyyyMMdd}."            -f $(Get-Date)
"Today is {0,10:MM/dd/yyyy hh:mm:ss}." -f $(Get-Date)
"Today is {0,10:MM/dd/yyyy HH:mm:ss}." -f $(Get-Date)

Result:
Today is 7/18/21.
Today is 07/18/2021.
Today is 18/07/2021.
Today is   20210718.
Today is 07/18/2021 04:59:56.
Today is 07/18/2021 16:59:56.

In all of these, I passed in the current date (as I’m writing this) using Get-Date, but any date based variable would have worked.

In the output I’ve arranged the date parts in a variety of common manners. You can use these or create your own custom formats. For example you may wish to use a dash instead of a slash to separate the date parts.

Custom Number Formatting

Let’s say you lived in London, where the pound was your default currency. But you are part of a multinational company and need to produce output to send to your coworkers in the United States. You don’t want to have to change your computer to a US language just to get the $ symbol.

You can fix this by using custom formatting. In these examples, when it sees a character such as the $ it will just keep it in the output. When it sees #, (a pound sign followed by a comma) it will know to use commas in the output.

When it sees just a # sign it will put a number there if it has one, otherwise a space, and when it sees 0 it will put a number from the passed in data, or a zero if there is no number there. These examples will clarify it.

Note each one is on it’s own line, WordPress is wrapping the -f part to the next line for space reasons. Additionally, in this and other examples I’ve chosen to add spaces to line up the demos for readability, this isn’t a requirement and you can format however you need.

"Custom 0, 25 $ #,##0.0000  = {0,25:$ #,##0.0000} " -f 123456789.012000005
"Custom 0, 25 $ #,##0.0000  = {0,25:$ #,##0.00} "   -f 123456789.012000005
"Custom 0, 25 $ ##0.0000    = {0,25:$ ##0.00} "     -f 123456789.012000005

Result:
Custom 0, 25 $ #,##0.0000  =        $ 123,456,789.0120
Custom 0, 25 $ #,##0.0000  =          $ 123,456,789.01
Custom 0, 25 $ ##0.0000    =            $ 123456789.01

Using Formatting with Variables

It’s possible to create a formatted value, then assign that to a variable. Let’s take a look at this slightly more complex example.

$vTday = "{0,8:N0}" -f 134567
$vYest = "{0,8:N0}" -f 23546

$tday = "{0:MM/dd/yyyy}" -f $(Get-Date)
$yest = "{0:MM/dd/yyyy}" -f $((Get-Date).AddDays(-1))

$output = "Visitors to ArcaneCode.com`r`n"
$output += "$tday had $vTday visitors`r`n"
$output += "$yest had $vYest visitors"
$output

Result:
Visitors to ArcaneCode.com
07/18/2021 had  134,567 visitors
07/17/2021 had   23,546 visitors

The first two lines you’ve seen a few times by now. I’m simply taking a number, formatting it with commas and making it eight spaces wide.

The next line you’ve seen as well, I take the current date, format to MM/dd/yyyy format and assign to $tday.

The $yest line is slightly more complex, but only because I’m doing some date math. The (Get-Date) will get the current date, but create a date object from it. Essentially I have a date variable type but without a variable name.

Because I have a date datatype I can employ a method of the date datatype, AddDays. This will let me add a number of days to the date, or if you pass in a negative value subtract days. Thus AddDays(-1) will give me yesterday’s date. I then have to wrap it all in $() so PowerShell will evaluate the entire expression before passing it back to the placeholder.

Finally I’m using the += operand to concatenate several strings together for a nice output. Note the formatted dates, and how the visitors number right aligns nicely.

Conclusion

While string interpolation makes creating output strings easy, knowing how to use placeholders can make creating formatted output even easier, allowing you to create output with numeric values that align neatly, or creating custom date formatting.

The demos in this course came from my Pluralsight course PowerShell 7 Quick Start for Developers on Linux, macOS and Windows, one of many PowerShell courses I have on Pluralsight. All of my courses are linked on my About Me page.

If you don’t have a Pluralsight subscription, just go to https://pluralsight.com. At the top is a Try For Free button you can use to get a free 10 day subscription to Pluralsight, with which you can watch my courses, or any other course on the site. Once there just search for my name, Robert Cain, to see all the courses I have.

Introduction to the SQL Server Mobile Report Publisher – Now Live on Pluralsight

My latest Pluralsight course, Introduction to the SQL Server Mobile Report Publisher, just went live!

The Mobile Report Publisher is a powerful, under-appreciated tool that lets you quickly and easily create reports for not just the web, but for a variety of platforms such as Apple’s iOS and Google Android.

In this course, Introduction to the SQL Server Mobile Report Publisher, you’ll learn to quickly and easily create dashboards for the web, as well as mobile devices such as phones and tablets.

First, you’ll explore how to use the Report Publisher designer to create the layout of the dashboard.

Next, you’ll see how to create datasets and bind them to the dashboard.

Finally, you’ll learn advanced features such as filters to limit the data on the dashboard, as well as drillthroughs to launch a detailed report or web page.

When you’re finished with this course, you’ll have the skills and knowledge of the Mobile Report Publisher needed to create dashboards on multiple platforms.

What’s that you say? You don’t have a subscription but want to watch my course? Hey, no problem. Pluralsight has a 10 day free trial. Just go to https://www.pluralsight.com/pricing/free-trial and sign up!

Fun With PowerShell Strings

Introduction

PowerShell has some of the best, most flexible string handling of any language I’ve used. In this post we’ll look at some string basics, then some of the features that make it special.

Note in these demos I’m using PowerShell Core 7.1.3 and VSCode. Everything here should also work in the PowerShell 5.1 and the PowerShell IDE.

Basic Strings in PowerShell

Strings in PowerShell can be denoted with either single quote marks or double quote marks.

"This is a string"
'This is a string too!'

If you want to see the result, simply highlight the strings and press either F8 in VSCode, or F5 in the PowerShell IDE.

This is a string
This is a string too!

Multiline Strings, aka Here Strings

PowerShell has the ability to create a multiline string, also known as a “here” string. To create a here string, use an @ sign, followed by a quotation mark (either single or double) then hit Enter.

On the next lines, enter the value for your string. Finally hit Enter, then on the next line put the closing quotation mark then another @ sign.

This is important, the closing quotation mark and @ sign must be the first two characters on the line. If you attempt to indent them in any way the technique will not work!

$heretext = @"
Some text here
Some more here
     a bit more

a blank line above
"@

$heretext

Some text here
Some more here
     a bit more

a blank line above

Again, this works with single or double quotes. So what’s a common use?

One common thing I use it for is SQL queries. Here’s an example in a traditional language.

$sql = 'SELECT col1' `
     + '     , col2' `
     + '     , col3' `
     + '  FROM someTable ' `
     + ' WHERE col1 = ''a value'' '

Note the single ` (back tick) mark at the end of each line. This is the line continuation character in PowerShell.

As you can see, each line has to be connected with + signs. Now let’s contrast it with a here string version.

$sql = @'
SELECT col1
     , col2
     , col3
  FROM someTable
 WHERE col1 = 'a value'
'@

This is not only much more readable, but easier to work with. You can develop your SQL query in SQL Server Management Studio, or Azure Data Studio, then simply cut and paste it into PowerShell.

Another thing to notice, you can embed single or double quotes within a here string, and PowerShell will ignore them as denoting another string.

Embedded Quote Marks

If you need to embed a quotation mark in a string, you can simply double them up.

"ArcaneCode said ""PowerShell is awesome!"" and the crowd applauded."

ArcaneCode said "PowerShell is awesome!" and the crowd applauded.

This technique works with single quote marks as well. For example one’s used in common words such as shouldn’t.

'You shouldn''t miss the blog posts at arcanecode.com'

You shouldn't miss the blog posts at arcanecode.com

String Interpolation

You may be wondering what the difference between single and double quotes is. Double quotes allow you to use something called string interpolation. String interpolation will take any variables embedded in the string and expand them before returning the string.

Here’s a simple example. First we get the number of items in the current folder. Then we copy the current folder location into another variable. Finally we create a string with these variables in it, surrounding the string in double quote marks.

$items = (Get-ChildItem).Count 

$loc = Get-Location

"There are $items items are in the folder $loc."

There are 3 items are in the folder C:\PSCore-QuickStart.

As you can see in the result (the bottom line) it has taken the value in $items, a 3, and expanded it in the return result.

Likewise, it has taken the value in $loc and expanded it to the name of the current folder.

Now see the result if single quote marks had been used.

$items = (Get-ChildItem).Count 

$loc = Get-Location

'There are $items items are in the folder $loc.'

There are $items items are in the folder $loc.

With single quotes, the string interpolation engine does not kick in.

As a rule of thumb then, only use double quotes if you have to have interpolation. Otherwise use single quotes so you don’t incur the (admittedly minimal) amount of time for the string interpolation engine to needlessly process the string.

Escape Characters

PowerShell also has the concept of escape characters. These are special characters you can embed in the string that provide additional formatting instructions.

Note that escape characters only work inside double quote marked strings. If you use single quotes for your strings, it will just display the escape character.

There are four that are the most commonly used. Tab, Carriage Return, Line Feed, and escaping the $ (dollar sign) so it may be used.

Escape characters are preceded by a back tick ` then the character. Let’s look at the tab, which is a `t.

"`tThis is tabbed in"

        This is tabbed in

The next is the carriage return line feed sequence, which is `r`n.

"Here is some text`r`nAnd some more text`r`n`r`nA blank line before this"

Here is some text
And some more text

A blank line before this

You would most commonly use this when looping over data and building a new output string.

The final escape character we’ll look at allows you to embed a $ character in an double quoted string. Simply use a back tick followed by the dollar sign.

"The `$items variable is $items"

The $items variable is 3

We’ve only covered four here, but note there are many more escape sequences you can use in PowerShell.

Conclusion

In this post you saw all kinds of cool things you can do with PowerShell strings. With this knowledge you can take your PowerShell scripts to the next level, and be able to generate some creative output with your own PowerShell strings.

The demos in this course came from my Pluralsight course PowerShell 7 Quick Start for Developers on Linux, macOS and Windows, one of many PowerShell courses I have on Pluralsight. All of my courses are linked on my About Me page.

If you don’t have a Pluralsight subscription, just go to https://pluralsight.com. At the top is a Try For Free button you can use to get a free 10 day subscription to Pluralsight, with which you can watch my courses, or any other course on the site. Once there just search for my name, Robert Cain, to see all the courses I have.

Fun with PowerShell Get-Random

Introduction

Many people know about PowerShell’s Get-Random cmdlet. Using it you can generate a random value. For example:

Get-Random

Will output a value such as:

64031951

But did you know there’s other features of Get-Random?

Numeric Ranges

With Get-Random you can lock down the return result to a specific range. You can specify a minimum value as well as a maximum.

Get-Random -Minimum 100 -Maximum 200

122

You can run it numerous times and the return value will always be between 100 and 200.

You don’t have to use both the minimum and maximum values, you can just use one or the other.

While be default it returns an unsigned 32 bit integer, you can also use it to return a random floating point value.

Get-Random -Minimum 10.5 -Maximum 20.9

11.9004625960255

Get-Random and Arrays

You can also use Get-Random with Arrays. Let’s say you have an array of integer values. You can pipe it through Get-Random and it will pick one value from the array for you.

$array = 1, 3, 5, 7, 9
$array | Get-Random

7

The cool thing about this is it works with any array. Let’s say you have an array of states, and want to pick a state at random.

$array = (
  'Alabama', 'Alaska', 'Arizona',
  'Arkansas', 'California', 'Colorado',
  'Connecticut', 'Delaware', 'Florida',
  'Georgia', 'Hawaii', 'Idaho',
  'Illinois', 'Indiana', 'Iowa'
)
$array | Get-Random

Florida

This also works with arrays that hold a collection of objects. Let’s get a collection of file objects from our C:\Temp folder and let it pick one for us.

$files = Get-ChildItem -Path "C:\Temp"
$files | Get-Random

    Directory: C:\Temp

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----           1/14/2021  2:31 PM                curl-7.74.0

Getting More Than One Random Item

Get-Random also has a -Count parameter which you can use to get multiple values back. Let’s use our state example, and let it return five states.

$array = (
  'Alabama', 'Alaska', 'Arizona',
  'Arkansas', 'California', 'Colorado',
  'Connecticut', 'Delaware', 'Florida',
  'Georgia', 'Hawaii', 'Idaho',
  'Illinois', 'Indiana', 'Iowa'
)
$array | Get-Random -Count 5

Idaho
Alaska
California
Delaware
Colorado

This works with any type of array, or just random integers.

Get-Random -Count 3

1981714362
1664558041
1474367023

SetSeed

Get-Random has one more parameter, -SetSeed. You can pass in a value, and when you do Get-Random will always return the same result for that seed.

Get-Random -SetSeed 100

1881691673

No matter how many times you run Get-Random, as long as you use the same seed value, it will always return a result of 1881691673. This also works with other types of arrays such as the array of states we used earlier.

$array = (
  'Alabama', 'Alaska', 'Arizona',
  'Arkansas', 'California', 'Colorado',
  'Connecticut', 'Delaware', 'Florida',
  'Georgia', 'Hawaii', 'Idaho',
  'Illinois', 'Indiana', 'Iowa'
)
$array | Get-Random -SetSeed 33

Illinois

Again, run this as much as you want and with the same SetSeed value it will always return the same state, in this case Illinois.

So why use SetSeed? One main use it testing. You want to create a set of reusable tests, for example using Pester, and want to validate you always get the same value back from your array.

Conclusion

In this post we saw how to use Get-Random for a variety of purposes. Hopefully you’ll find it as useful as I do!

I have many PowerShell related courses on Pluralsight. All of my courses are linked on my About Me page.

If you don’t have a Pluralsight subscription, just go to https://pluralsight.com. At the top is a Try For Free button you can use to get a free 10 day subscription to Pluralsight, with which you can watch my courses, or any other course on the site. Once there just search for my name, Robert Cain, to see all the courses I have.

Setting Up an Ubuntu VM for Developers in Hyper-V

Introduction

On occasion I will need to setup a new Ubuntu virtual machine in Hyper-V, for doing development or creating new courses for Pluralsight.

It’s not something I do every day though, so I have instructions to remind me of all the steps involved, what buttons to push, and so on.

The problem is my instructions have been scattered across multiple files in a few repositories. Today I finally decided enough was enough.

The Instructions

This is kind of an unusual blog post, in that it doesn’t contain a lot of content. The instructions were rather long, and if I’d tried to do it as a blog post I’d have had to break it into multiple posts, making it difficult to follow, or have one blog post that was 3 miles long.

Instead, I’ve created a repository on my GitHub site, https://github.com/arcanecode/Setting-Up-an-Ubuntu-VM-for-Developers-in-Hyper-V. There you will find the instructions broken down into small, easily digestible parts.

Buyer Beware

I want to make it understood, this is not meant to be an in-depth tutorial. There aren’t a lot of screen shots, nor do I take time to explain “why” (at least not often).

Instead think of it like a check list. If you are familiar with both Hyper-V and Ubuntu, you should not have any problems following the instructions.

Decisions, Decisions

Many of the configuration choices I made were the result of this being in a VM. I turn off things like screen blanking, auto locking, and the like. I figure the host machine should be the one to manage these types of things.

In addition, the software I chose to install was a result of my needs. VSCode, PowerShell, Azure Data Studio, and gcc/g++ were the core tools installed.

There are too many other options out there. Python, PHP, Ruby, Java, Rust, and on and on and on. I feel these instructions will get you to the basic platform, then you can add on your specific language and other tools from there.

Into The Future

I’ll keep this updated when new versions of Ubuntu or Hyper-V are released that may invalidate the instructions.

Also, it just screams for automation. I’d like to write some PowerShell Core to run on Windows to create and configure the VM, then a shell script to run inside Ubuntu to handle as much installation and configuration as possible.

I actually have some PowerShell Core samples for generating a Hyper-V VM, I just need to assemble them into a useable script. So that may be something I tackle in the near future.

Conslusion

Well this was a rather unusual post, as most of the info is over in my GitHub repository. Please check it out, let me know what you think.

Even if you don’t have a big need for an Ubuntu VM, I’d be curious how you find the experience of most of the info being in GitHub. Was it easy to follow, well organized?

If this works out, I may do some future work where the bulk of the information is on my GitHub site, and my blog post provides a brief overview.

VeraCrypt On the Command Line for Ubuntu Linux

Introduction

In this, the third post on using VeraCrypt from the command line, we’ll cover how to use VeraCrypt from Linux, specifically Ubuntu 20.04.

We’ll cover the basics. Creating a container, mounting a container (aka volume), getting a list of mounted volumes, and finally dismounting your volumes.

As I stated in previous posts, the command line syntax is a bit different for all the major operating systems. In this post we’ll focus on Linux.

Prerequisites

Before you install VeraCrypt, you should be aware it has a dependency on the libwxgtk3.0-gtk3-0v5 library, so we might as well install that first. From the terminal execute these commands.

sudo apt-get update -y
sudo apt-get install -y libwxgtk3.0-gtk3-0v5

Next, in order to format a volume as exFat so it can be used across platforms, you’ll need to install the exFat tools.

sudo apt-get install -y exfat-fuse exfat-utils

Now you’ll need to install VeraCrypt. I went to https://veracrypt.fr/en/Downloads.html and scrolled down to the Linux area. In the Ubuntu 20.04 area, I downloaded veracrypt-1.24-Update7-Ubuntu-20.04-amd64.deb file into my Downloads folder.

In the terminal, I moved to my Downloads folder then issued this command to install it:

sudo dpkg -i veracrypt-1.24-Update7-Ubuntu-20.04-amd64.deb

From here I went into Ubuntu’s menu and launched VeraCrypt to validate it installed correctly.

Code Samples


While I will be providing samples here, you should also check out the project I have on GitHub that goes with this post, https://github.com/arcanecode/VeraCrypt-CommandLine-Examples

I’ll update it over time as needed, and it may be easier for you to download, or cut and paste from it.

OK, let’s get started!

Creating a Container

Before we begin I’d like to make two notes. First, I’ll be storing the container in my Documents folder. So in the terminal be sure to cd into your Documents. This will make it easier as we won’t have to specify full paths to our files.

Second, included in this demo is a randomdata.txt file. We’ll explain its use shortly, but you’ll need to copy this file into Documents as well, or when the time comes include the full path to it.

Let’s start by looking at the long line of code needed to create a container. Please note that while your blog reader may wrap the command below, it should be one line in your shell script or from the command line.

sudo veracrypt --text --create vctest.vc --size 200M --password MySuperSecurePassword1! --volume-type normal --encryption AES --hash sha-512 --filesystem exfat --pim 0 --keyfiles "" --random-source randomdata.txt

OK, that’s a bit hard to read, so let me break it out into each part below.

The first item is the veracrypt command, assuming you installed it in the default location. By default VeraCrypt installs in /usr/bin, but because /usr/bin is in the path you don’t have to specify it when calling VeraCrypt. You do though, need to use all lowercase as Linux is case sensitive. In addition you’ll need to call it using sudo.

sudo veracrypt

The --text parameter says we want to use VeraCrypt in text mode, not GUI. Note, the --text parameter must be the FIRST parameter you pass in, or it will not work.

--text

We next tell VeraCrypt we want to create a new file container, and where it is to be stored at. For this demo, I’m just going to use the Documents directory. Be sure to cd into this folder from the terminal command line before issuing the full command. Otherwise, you can specify the full path to the .vc file you want to create.

--create vctest.vc

Next we indicate how big we want our container to be. If you just list a number, VeraCrypt assumes you mean bytes, but you can also affix M for megabytes, G gigabytes, K kilobytes, and so on.

Here I’m going to keep it small for demo purposes and use 200 megabytes.

--size 200M

Now we provide our password. Normally you would not want to hard code it, but rather pass it into your script as a parameter.

I wanted to keep this demo simple though, and focus on VeraCrypt so I’ve just hard coded it. I’m using the same “super secure” password I’ve used in my last few posts.

--password MySuperSecurePassword1!

Next is the volume type, normal or hidden. In a previous blog post I cover hidden types, and if you are going to do a hidden volume I would suggest using the GUI in order to assure it is done right.

For this demo we’ll go with a normal volume.

--volume-type normal

Now we pick the encryption type. There are quite a few, so refer to the VeraCrypt documentation for a full list. Here we’re using AES.

--encryption AES

Next up, for the encryption method we picked it needs to know the hashing algorithm. For AES I suggest SHA-512.

--hash sha-512

In order to keep this container portable across other OS’s (Windows and macOS) we’ll format using exfat. In order to format as exFAT, you’ll need to have installed the exFAT utilities on your system (see the Prerequisites section above).

--filesystem exfat

The PIM is a special number that allows you to specify the number of times hashing algorithm executes. It’s a bit more complex than that, if you want full details see the VeraCrypt documentation.

For now, we can pass it the value of 0 (zero), which tells it to use the default value for the PIM.

--pim 0

You can mount a volume in VeraCrypt using a keyfile, as opposed to a password. We’ve not done that here, so we’ll just pass in an empty string to indicate we won’t use a keyfile.

--keyfiles ""

As a final parameter, you need to provide random data for VeraCrypt to use when generating its hashes. It needs at least 320 characters, but you can give more.

I’ve provided a sample one as part of this demo (see the GitHub code samples), I just picked up my keyboard and randomly smacked myself in the head until I got around 350 characters. I’d suggest creating one of your own for production, but for just testing and learning (I’m assuming you’ll throw away the vault when you are done) then the one here will be OK.

As stated in the beginning of this section, either copy the randomdata.txt file into the folder where you are executing the commands from, in my case the Documents folder, or specify the full path to it in the command line.

--random-source randomdata.txt

OK, that’s everything you need to create a volume. Just run the command, or execute the script. Now that it’s created, let’s mount it.

Mounting a VeraCrypt Volume

Here is the full command line to mount. (As before, it should be on a single line, ignore any wrapping done by your browser).

sudo veracrypt --text --mount vctest.vc /mnt --password MySuperSecurePassword1! --pim 0 --keyfiles "" --protect-hidden no --slot 1 --verbose

Let’s start breaking it down. First, as before, is the call to the VeraCrypt app.

sudo veracrypt

As with all of these commands, the --text parameter must come first to let VeraCrypt know we want to use text mode and not the GUI.

--text

The mount parameter actually has two values that need to be passed in.

First we pass in the name of the file to mount.

Second we need to provide a mount point. This will let Linux treat it like any other drive you might plug in. In this example we will use /mnt.

--mount vctest.vc /mnt

Using /mnt works fine if you only have one volume to mount. If not, you’ll have to create new mount points. This is pretty easy.

sudo mkdir /media/vc2

Placing these in the /media area is a common practice. From there you can name it anything you want. You could use vc followed by the slot number, or use the name of the file.

Then in your script, you can use:

--mount vctest.vc /media/vc2

Note that once you create the directory under /media it is persistent, you don’t have to create it again. This does mean your /media will accumulate these mount points over time. Once you are sure you’ll no longer need them consider using rmdir to remove them.

You could of course use /media for all your volumes, and avoid using /mnt completely. Just be aware you’ll have to create a name under /media first.

Next is our “super secure” password. If your password has spaces, you’ll need to wrap this in double quote marks.

--password MySuperSecurePassword1!

If you overrode the default PIM when creating your volume, you’ll need to provide it. Otherwise, we can pass it the value of 0 (zero), which tells it to use the default value.

--pim 0

If you created your volume using a keyfile or files, provide them here. Otherwise, you can just pass in an empty string to indicate no keyfile is needed.

--keyfiles ""

If this volume contained a hidden volume, you would need to let VeraCrypt know by using a value of yes, plus some other parameters.

In this case there is no hidden partition in our volume, so we can just give a value of no.

--protect-hidden no

OPTIONAL – Slot

Slot is an optional parameter. If you look at the VeraCrypt, down the left side are a series of slot numbers. If you omit this parameter, VeraCrypt will mount in the first empty slot.

However you can specify a slot, which can be useful if you want to make sure certain volumes always mount in a specific slot. You can then use the slot number when you want to dismount your volumes.

--slot 1

OPTIONAL – Verbose

Verbose is also an optional parameter, but I often include it just to see what is going on under the covers. You can use it with any of the commands in this post, I just included it on this one for example purposes.

If you intend to make this into a script then I would suggest omitting it once your script is debugged and working.

--verbose

OK, hopefully all is going well, and you’ve created and mounted your volume. Let’s next see how to get a list, from the command line, of all your mounted volumes.

Listing Mounted Volumes

Here’s the command line to see what is mounted on your Ubuntu computer.

sudo veracrypt --text --list

As with other commands you have seen, we start by calling the VeraCrypt application. We then use --text to let VeraCrypt know not to use the GUI.

We finish with --list, which tells VeraCrypt to display a list of all mounted containers. This will include the slot number, volume name, and mount directory.

If you’ve been following along, then running the list should produce an output like:

1: /home/arcanecode/Documents/vctest.vc /dev/mapper/veracrypt1 /mnt

Of course your folder will have your username, not arcanecode.

Mom always taught me to put away my toys when I was done playing with them, so in the next section we’ll see how to dismount your volumes once you are done with them.

Dismounting VeraCrypt Volumes

There are four ways to dismount a volume. Three of them will dismount a specific volume, the final will dismount all volumes.

All methods follow the same pattern. Run veracrypt using sudo, followed by the --text parameter to tell VeraCrypt not to launch the GUI.

Finally we give the --dismount to let VeraCrypt know we want to unload our volume. The value passed into the --dismount parameter varies, and will be explained below.

Method 1: Slot Number

sudo veracrypt --text --dismount --slot 1

With the first method, you provide the slot number. If you mounted a volume and used the slot number parameter, for example your personal file vault is always in slot 5, then this can be an easy way to dismount.

On the other hand, if you let VeraCrypt load in the first available slot, you’ll either have to look at the GUI, or run the list command in the previous section, to learn the slot number.

Be aware there is no requirement to load slots in sequential order. Your first mount could go into slot 5, the second into slot 7, and third into slot 10, leaving the other slots empty.

Method 2: Mount Directory

sudo veracrypt --text --dismount /mnt

Or if you created it under /media, it might look like:

sudo veracrypt --text --dismount /media/vc2

Using the volume list command or looking at the “Mount Directory” column in the GUI, you can pass in that value to dismount. Because you may not always use the same mount point on all systems to mount, it can be of an issue trying to be reliable in dismounting the right volume.

Method 3: Volume File Name

sudo veracrypt --text --dismount vctest.vc

Alternatively you can use the full path to the volume.

sudo veracrypt --text --dismount /home/arcanecode/Documents/vctest.vc

This method is the most reliable. Since you know the name of the file you mounted, you can just provide the same file name to dismount.

VeraCrypt doesn’t care what slot it is loaded into, it uses the file name to find it.

The winner – Method 3!

For the reasons above, I highly suggest Method 3 be your go to method for dismounting volumes in your scripts. It is the most reliable, and easiest to understand when looking at the scripts.

But wait, there’s more!

Dismounting ALL Volumes

There is one final method, you can dismount all of the VeraCrypt volumes you have mounted.

sudo veracrypt --text --dismount

If you use just the --dismount parameter, and pass in no values, then VeraCrypt will attempt to dismount ALL volumes you have loaded.

This can be a useful command to run when you’re shutting down your computer, to ensure all volumes are properly shutdown.

If you don’t have any volumes mounted, then VeraCrypt basically shrugs it’s shoulders, does nothing, and ends.

Conclusion

In this post, we learned how to create, mount, and dismount VeraCrypt volumes from the command line in Linux, specifically Ubuntu. In addition, we also saw how to get a listing of volumes currently mounted.

VeraCrypt On The Command Line for Windows

Introduction

This is part of my ongoing series on the VeraCrypt encryption utility. If you’ve not kept up, in my first post of the series, “VeraCrypt – A Powerful Encryption Utility“, I covered the basics of VeraCrypt including how to get it, use it through the GUI, and how the series was inspired by the Grumpy Old Bens podcast.

In the second post, “Creating and Using Hidden Containers in VeraCrypt“, I covered how to create a hidden container, adding an extra level of security to your containers.

My previous post, “VeraCrypt on the Command Line for macOS“, showed how to call VeraCrypt from within a script on the macOS platform.

The commands to call VeraCrypt from the command line are very different for each platform, As such, I’ve broken them out into individual blog posts.

In this entry you’ll see how to call VeraCrypt on Windows 10.

Prerequisites

Obviously, you’ll need to have VeraCrypt installed. My first post in the series, “VeraCrypt – A Powerful Encryption Utility“, covers where to get it from.

For this post, we’ll also be using the CMD mode to execute the commands. Do note that on most installations of Windows these days, PowerShell has replaced CMD as the default terminal shell. If you open up a command window and see PowerShell, all you have to do is type in CMD and it enter, and you’ll be switched to CMD mode.

Code Samples

While I will be providing samples here, you should also check out the project I have on GitHub that goes with this post, VeraCrypt-CommandLine-Examples.

I’ll update it over time as needed, and it may be easier for you to download, or cut and paste from it.

One item I want to mention, unlike the macOS version, the Windows version of VeraCrypt lacks the ability to list containers. So for this post we’ll only be able to include creation, mounting and dismounting of containers.

OK, let’s get started!

Creating a Container

Let’s begin by looking at the full command to create a container, then we will break it down to it’s individual components. While your blog reader or webpage may wrap the line, in your script (or command line) it should all be entered as a single line of text.

"C:\Program Files\VeraCrypt\VeraCrypt Format.exe" /create "C:\temp\vctest.vc" /size "200M" /password MySuperSecurePassword1! /encryption AES /hash sha-512 /filesystem exfat /pim 0 /silent

First up is the command to call. If you installed VeraCrypt to the default folder, you’ll find it in C:\Program Files\VeraCrypt\

The command to create a new volume is actually a separate executable than the rest of VeraCrypt. It is “VeraCrypt Format.exe

Note there is indeed a space in the file name! Thus you have to enclose the entire thing in double quotes.

"C:\Program Files\VeraCrypt\VeraCrypt Format.exe"

Next is the command to create a volume, /create. You follow it with the path and file name to create. If you omit the path it will create the volume in the current directory you are running the script from.

As with all file names, if it has a space you must include double quotes. Otherwise they are optional, but it doesn’t hurt to have them.

/create "C:\temp\vctest.vc"

We now need to tell VeraCrypt how big to make the volume. VeraCrypt includes shortcuts for M (Megabytes), G (Gigabytes), T (Terabytes) and
K (Kilobytes). If you omit a letter, it assumes bytes.

For this demo we are making it small, so will use 200M to indicate 200 Megabytes.

/size "200M"

Next up is the password to use to encrypt the volume. In a “real world” situation, you should probably pass it into the script or get it using an alternate method.

To keep this demo simple, I’m just going to embed the password using the “super secure” password I’ve used throughout this series of blog posts.

As with file names, if your password has spaces you’ll need to enclose it in double quotes.

/password MySuperSecurePassword1!

Now we need to provide the encryption algorithm to use. VeraCrypt supports a vast array of algorithms, see their documentation for the supported list.

For this demo, we’ll use the popular AES.

/encryption AES

Many algorithms require you to provide an encryption hashing method. For AES, we’ll use the strong SHA-512.

/hash sha-512

In order to keep this container portable across OS’s we’ll format using exfat. Be aware though that to use exfat on a Mac, you’ll have to install macFUSE (see my previous post on macOS for more info).

/filesystem exfat

The PIM is a special number that allows you to specify the number of times the hashing algorithm executes. It’s a bit more complex than that, if you want full details see the VeraCrypt documentation at:

https://documentation.help/VeraCrypt/Personal%20Iterations%20Multiplier%20(PIM).html

For now, we can pass it the value of 0, which tells it to use the default value.

/pim 0

The final parameter is /silent. By default VeraCrypt will display dialogs notifying you of its progress, as well as when it is complete.

In a scripting situation you would normally not want this, so we add the silent switch to suppress the messages.

Note this does have one side affect, if there are any errors those too are also suppressed, so you won’t be aware of them. The most common of these would be the attempt to create a volume name that already exists.

/silent

You now have everything you need to create a VeraCrypt volume. Note that there is one more parameter that we didn’t use in the example, but you may want to know about.

/force

Normally, if you are trying to create a new volume and that file already exists, VeraCrypt will pop up a dialog (assuming you’ve not used /silent) warning you the volume you are trying to create already exists. It will then give you the choice of canceling or overwriting the existing file.

The /force parameter suppresses the message and always overwrites the file.

So hopefully you’ve now created your own volume using the commands in this section. Let’s now see how to mount it.

Mounting a VeraCrypt Volume

Mounting is very simple, here is the full command, then we’ll take a look at each part. As before, it should be all a single line.

"C:\Program Files\VeraCrypt\VeraCrypt.exe" /volume "C:\temp\vctest.vc" /letter x /password MySuperSecurePassword1! /quit /silent

We start with the command to VeraCrypt. This assumes you have installed to the default folder.

"C:\Program Files\VeraCrypt\VeraCrypt.exe"

Next we provide the /volume parameter, with the path to and the file name of the file to mount.

/volume "C:\temp\vctest.vc"

Volumes in VeraCrypt appear as a drive letter to Windows. As such we need to provide a letter to use. Note if you use a drive letter
already in use, you’ll get an error.

The letter can be provided in either or upper or lower case.

If you don’t know a drive letter, or don’t care what letter is used, then you can omit this parameter completely. When you do, VeraCrypt will use the first available drive letter it finds.

/letter x

Next up is the password to use to encrypt the volume. In a “real world” situation, you should probably pass it into the script, or get it using an alternate method.

To keep this demo simple, I’m just going to embed the password using the “super secure” password I’ve used throughout this series of blog posts.

As with file names, if your password has spaces you’ll need to enclose it in double quotes.

/password MySuperSecurePassword1!

Next we provide the quit parameter. By default, if you omit it then the VeraCrypt dialog will remain on the screen. Using quit will close the
VeraCrypt dialog, something usually desired if you are running a script.

/quit

Finally we’ll add the /silent parameter. This has the same affect as it did in the create function, suppressing any dialogs. Be aware, that for /silent to work, you must also have used the /quit parameter.

/silent

At this point hopefully all went well, and you have created a volume as well as mounted it. Once you are done with a volume, you’ll need to dismount it, the subject of the next section.

Dismounting VeraCrypt Volumes

The command to dismount a volume is the simplest of all.

"C:\Program Files\VeraCrypt\VeraCrypt.exe" /dismount H /quit /silent /force

Let’s look at the individual components of the command.

We start with the command to VeraCrypt. This assumes you have installed to the default folder.

"C:\Program Files\VeraCrypt\VeraCrypt.exe"

Next is the dismount parameter. You pass in the drive letter associated with the volume to dismount. As with mounting, the case of the drive letter does not matter.

If you omit the drive letter, VeraCrypt will dismount ALL currently mounted volumes.

/dismount X

We now provide the quit parameter. By default, if you omit it then the VeraCrypt dialog will remain on the screen. Using quit will close the
VeraCrypt dialog, something usually desired if you are running a script.

/quit

Now we append the /silent parameter, to suppress any dialogs as we did in the previous sections. As with mounting, for /silent to work we must also include /quit.

/silent

Finally we provide the force parameter. If some app is accessing the volume, for example Windows File Explorer, it will prevent VeraCrypt from dismounting.

The force parameter tells VeraCrypt to shut down, no matter what. Your inclusion of force is up to you, depending on your situation.

For the demo, we’ll include it.

/force

And that’s all there is to it. It’s a best practice to dismount all of your volumes when you are done with them, and especially before shutting down your computer.

This will ensure any operations that are copying data to your encrypted volume have completed, ensuring the data does not get corrupted.

Conclusion

This post covered how to create, mount and dismount VeraCrypt volumes in Windows 10. The technique should also be applicable to Windows 7 and 8, if you are still on those platforms.

VeraCrypt on the Command Line for macOS

Introduction

Automation is the key to success, especially in the DevOps world. In addition, security is an ever increasing need in today’s world of hacks and ransomware.

In my last two posts I showed how to use the encryption utility, VeraCrypt, to create encrypted containers and drives to securely store data. In this, and the next two posts, we’ll see how we can automate VeraCrypt from the command line.

We’ll cover the basics. Creating a container, mounting a container (aka volume), getting a list of mounted volumes, and finally dismounting your volumes.

As it turns out, the command line syntax is a bit different for all the major operating systems. In this post we’ll focus on macOS, in the next two posts we’ll cover Windows and Linux.

Prerequisites

Naturally, you’ll have to have VeraCrypt installed. You can find out more in my first post of the series, VeraCrypt – A Powerful Encryption Utility.

The next requirement applies to macOS. In order for macOS to be able to read the variety of filesystems VeraCrypt supports, you need to install a free utility, macFUSE.

It’s free, fast to install and takes very little in the way of system resources. It’s compatible with macOS BigSur and Catalina, and probably older versions too, check their site if you have that need.

Code Samples

While I will be providing samples here, you should also check out the project I have on GitHub that goes with this post, VeraCrypt-CommandLine-Examples.

I’ll update it over time as needed, and it may be easier for you to download, or cut and paste from it.

OK, let’s get started!

Creating a Container

First off, let’s look at the long line of code needed to create a container. Please note that while your blog reader may wrap the command below, it should be one line in your shell script.

/Applications/VeraCrypt.app/contents/MacOS/VeraCrypt --text --create "/Users/arcanecode/Documents/temp/vctest.vc" --size "200M" --password MySuperSecurePassword1! --volume-type normal --encryption AES --hash sha-512 --filesystem exfat --pim 0 --keyfiles "" --random-source /Users/arcanecode/Documents/temp/randomdata.txt

OK, that’s a bit hard to read, so let me break it out into each part below.

The first item is the path to the VeraCrypt application, assuming you installed it in the default location.

/Applications/VeraCrypt.app/contents/MacOS/VeraCrypt

The --text parameter says we want to use VeraCrypt in text mode, not GUI. Note, the –text parameter must be the FIRST parameter you pass in, or it will not work.

--text

We next tell VeraCrypt we want to create a new file container, and where it is to be stored at. For this demo, I’ve created a temp folder in my Documents directory.

You’ll want to change this to the location where you store your volumes. In addition, you’ll also want to change the user folder from arcanecode to your username.

--create "/Users/arcanecode/Documents/temp/vctest.vc"

Next we indicate how big we want our container to be. If you just list a number, VeraCrypt assumes you mean bytes, but you can also affix M for megabytes, G gigabytes, K kilobytes, and so on.

Here I’m going to keep it small for demo purposes and use 200 megabytes.

--size "200M"

Now we provide our password. Normally you would not want to hard code it, but rather pass it into your script as a parameter.

I wanted to keep this demo simple though, and focus on VeraCrypt so I’ve just hard coded it I’m using the same “super secure” password I’ve used in my last few posts.

--password MySuperSecurePassword1!

Next is the volume type, normal or hidden. My previous blog post talks about hidden types, and if you are going to do a hidden volume I would suggest using the GUI in order to assure it is done right.

For this demo we’ll go with a normal volume

--volume-type normal

Now we pick the encryption type. There are quite a few, so refer to the VeraCrypt documentation for a full list. Here we’re using AES.

--encryption AES

Next up, for the encryption method we picked it needs to know the hashing algorithm. For AES I suggest SHA-512.

--hash sha-512

In order to keep this container portable across other OS’s (Windows and Linux) we’ll format using exfat. Be aware though that to use exfat on a Mac, you’ll have to install macFUSE (see the prerequisites section above.)

--filesystem exfat

The PIM is a special number that allows you to specify the number of times hashing algorithm executes. It’s a bit more complex than that, if you want full details see the VeraCrypt documentation.

For now, we can pass it the value of 0 (zero), which tells it to use the default value for the PIM.

--pim 0

You can mount a volume in VeraCrypt using a keyfile, as opposed to a password. We’ve not done that here, so we’ll just pass in an empty string to indicate we won’t use a keyfile.

--keyfiles ""

As a final parameter, you need to provide random data for VeraCrypt to use when generating its hashes. It needs at least 320 characters, but you can give more.

I’ve provided a sample one as part of this demo (see the GitHub code samples), I just randomly smacked keys on my keyboard with my eyes closed. I’d suggest creating one of your own for production, but for just testing and learning (I’m assuming you’ll throw away the vault when you are done) then the one here will be OK.

--random-source /Users/arcanecode/Documents/temp/randomdata.txt

OK, that’s everything you need to create a volume. Now that it’s created, let’s mount it.

Mounting a VeraCrypt Volume

Here is the full command line to mount. (As before, it should be on a single line, ignore any wrapping done by your browser).

/Applications/VeraCrypt.app/contents/MacOS/VeraCrypt --text --mount "/Users/arcanecode/Documents/temp/vctest.vc" /Volumes/vctest --password MySuperSecurePassword1! --pim 0 --keyfiles "" --protect-hidden no --slot 1 --verbose

Let’s start breaking it down. First, as before, is the full path to the VeraCrypt app.

/Applications/VeraCrypt.app/contents/MacOS/VeraCrypt

As with all of these commands, the --text parameter must come first to let VeraCrypt know we want to use text mode and not the GUI

--text

The mount parameter actually has two values that need to be passed in.

First we pass in the name of the file to mount.

Second we need to provide a mount point. This will let macOS treat it like any other drive you might plug in.

In macOS, use /Volumes then add on a name. For simplicity I usually use the name of the file (without any extension), but it doesn’t have to be.

In reality though, it doesn’t make a real difference as macOS overrides the name you provide, and instead uses “Untitled”, “Untitled 2”, and so on.

--mount "/Users/arcanecode/Documents/temp/vctest.vc" /Volumes/vctest

Next is our “super secure” password. If your password has spaces, you’ll need to wrap this in double quote marks.

--password MySuperSecurePassword1!

If you overrode the default PIM when creating your volume, you’ll need to provide it. Otherwise, we can pass it the value of 0 (zero), which tells it to use the default value

--pim 0

If you created your volume using a keyfile or files, provide them here. Otherwise, you can just pass in an empty string to indicate no keyfile is needed.

--keyfiles ""

If this volume contained a hidden volume, you would need to let VeraCrypt know by using a value of yes, plus some other parameters.

In this case there is no hidden partition in our volume, so we can just give a value of no.

--protect-hidden no

OPTIONAL!
Slot is an optional parameter. If you look at the VeraCrypt GUI in macOS, down the left side are a series of slot numbers. If you omit this parameter, VeraCrypt will mount in the first empty slot.

However you can specify a slot, which can be useful if you want to make sure certain volumes always mount in a specific slot. You can then use the slot number when you want to dismount your volumes.

--slot 1

OPTIONAL!
Verbose is also an optional parameter, but I often include it just to see what is going on under the covers. You can use it with any of the commands in this post, I just included it on this one for example purposes.

If you intend to make this into a script then I would suggest omitting it once your script is debugged and working.

--verbose

OK, hopefully all is going well, and you’ve created and mounted your volume. Let’s next see how to get a list, from the command line, of all your mounted volumes.

Listing Mounted Volumes

Here’s the command line to see what is mounted on your Mac.

/Applications/VeraCrypt.app/contents/MacOS/VeraCrypt --text --list

As with other commands you have seen, we start with the full path to the VeraCrypt application. We then use --text to let VeraCrypt know not to use the GUI.

We finish with --list, which tells VeraCrypt to display a list of all mounted containers. This will include the slot number, volume name, and mount directory.

Mom always taught me to put away my toys when I was done playing with them, so in the next section we’ll see how to unmount your volumes once you are done with them.

Dismounting VeraCrypt Volumes

There are four ways to dismount a volume. Three of them will dismount a specific volume, the final will dismount all volumes.

All ways follow the same pattern. Give the path to the VeraCrypt app, followed by the --text parameter to tell VeraCrypt not to launch the GUI.

Finally we give the --dismount to let VeraCrypt know we want to unload our volume. The value passed into the –dismount parameter varies, and will be explained below.

Method 1: Slot Number

/Applications/VeraCrypt.app/contents/MacOS/VeraCrypt --text --dismount --slot 1

With the first method, you provide the slot number. If you mounted a volume and used the slot number parameter, for example your personal file vault is always in slot 5, then this can be an easy way to dismount.

On the other hand, if you let VeraCrypt load in the first available slot, you’ll either have to look at the GUI, or run the list command in the previous section, to learn the slot number.

Method 2: Volume Name

/Applications/VeraCrypt.app/contents/MacOS/VeraCrypt --text --dismount "/Volumes/Untitled"

Using the volume list command or looking at the “Mount Directory” column in the GUI, you can pass in that value to unmount. Because this is typically “Untitled”, “Untitled 1” and so on, it can be of an issue trying to be reliable in unmounting the right volume.

Method 3: Volume File Name

/Applications/VeraCrypt.app/contents/MacOS/VeraCrypt --text --dismount "/Users/arcanecode/Documents/temp/vctest.vc"

This method is the most reliable. Since you know the name of the file you mounted, you can just provide the same file name to unmount.

VeraCrypt doesn’t care what slot it is loaded into, it uses the file name to find it.

The winner – Method 3!

For the reasons above, I highly suggest Method 3 be your go to method for dismounting volumes in your scripts. It is the most reliable, and easiest to understand when looking at the scripts.

But wait, there’s more!

Dismounting ALL Volumes

There is one final method, you can dismount all of the VeraCrypt volumes you have mounted.

/Applications/VeraCrypt.app/contents/MacOS/VeraCrypt --text --dismount

If you use just the --dismount parameter, and pass in no values, then VeraCrypt will attempt to dismount ALL volumes you have loaded.

This can be a useful command to run when you’re shutting down your Mac, to ensure all volumes are properly shutdown.

If you don’t have any volumes mounted, then VeraCrypt basically shrugs it’s shoulders, does nothing, and ends.

Conclusion

In this post, we learned how to create, mount, and dismount VeraCrypt volumes from the command line in macOS. In addition, we also saw how to get a listing of volumes currently mounted.

In the next two posts we’ll see how to do the same things, but in Windows, then Linux.

Creating and Using Hidden Containers in VeraCrypt

Introduction

In my previous post I explained the fundamentals of the powerful encryption tool, VeraCrypt. If you are not familiar with VeraCrypt, I’d suggest going back and reading it first so that this post will make some sense.

In this post I’ll cover how to use VeraCrypt’s hidden containers feature, explaining what it is for then seeing step by step screen shots on how to set it up and use it.

What Are Hidden Containers For?

Let’s say you have a Bitcoin wallet with a sizable amount of money. You travel a lot, and are concerned with someone tampering with your data.

Often times a countries border agents will want to search your computer. I’m not casting aspersions on anyone’s honesty, but you never know.

Now you could setup a standard VeraCrypt container, but what if a tech savvy person noticed it? They could demand you give up your password to the vault. At some point you would wind up giving in, giving access to data you didn’t want shared (your Bitcoin wallet). This is where hidden containers come into play.

VeraCrypt allows you to create a container inside a container. Into what VeraCrypt refers to as the “outer” container, you can place information that looks important, but in reality isn’t.

You can then create the inner or “hidden” container, giving it a different password. To anyone who inspects the outer container, there is no way to tell it contains a hidden container.

In the above scenario, you simply give up, after some fake complaining of course, the password to the outer container. You interrogator will have no way to tell there’s more in there.

When you want to get to the hidden area, you mount it using the password to your hidden container. Let’s see how to setup a hidden container, then how to use it.

Setting up a Hidden Container

Let’s start by opening VeraCrypt.

Begin the process by clicking the Create Volume button.

For this demo I’ll use an encrypted file container, but these steps will also work when you encrypt a non-system partition/drive, as we did in my previous post using a USB thumb drive.

I’m going to take the default of the encrypted file container and click the Next button.

Now we begin creating a hidden container by changing the option to Hidden VeraCrypt Volume and clicking Next.

Now we begin specifying how we’ll be creating our container. In Normal Mode it assumes we have no container. This is the most common method, and what we’ll use here.

In Direct Mode, VeraCrypt will let us add a hidden container to an already existing container. The steps are similar, although since a container already exists it will skip over the next few screens picking up where we start configuring the hidden container.

For this demo we’ll keep the default of Normal Mode and click Next.

Now we need to specify where to store the new file container. Here I just typed in C:\Temp\DemoHidden.hc and clicked Next.

Now we begin the process of configuring the Outer volume. Just click Next to proceed.

Here we’ll select the encryption method. These were discussed in the previous post, so for this demo I’ll just take the default of AES and click Next.

Now we need to specify how big to make the entire container. This will need to be big enough to hold the data we want to put in our hidden container, as well as the space to put our “fake” data we are willing to give up.

For this demo I’m making it a small 250 megabytes, just to go quickly, but you can make this any size you need.

Now we’ll enter the password for the outer container. Follow the same rules for password generation you normally would. Also keep in mind the outer and (when you create it in just a moment) hidden passwords will need to be different.

I’ve checked on the Display Password box so you can see it, and am using the same MySuperSecurePassword1! password I used in the previous post. Once the password is entered, click Next.

Now you jiggle the mouse around to generate some random data VeraCrypt can use to create the real encryption key. Keep going until the bar at the bottom goes green all the way across. Once it does, you can click Format.

When formatting is complete, VeraCrypt is ready for you to copy your fake data into it. Click on the Open Outer Volume button and a file manager for your operating system will open.

Here in Windows it’s the Windows File Explorer, on macOS it will be Finder, and on Linux it will be the file manager for your particular distro.

Once it has opened, copy some fake files you’ve prepared to put in here. Remember this should be data that looks realistic, but isn’t.

One idea might be an Excel Spreadsheet with a family budget, only with fake numbers. Another might be a text file with some realistic looking but fake credit card numbers. Again, data that looks real but isn’t so if someone takes it you won’t be hurt financially or otherwise.

For this demo I’m simply using the same pic I currently (at the time of this writing) use on my Twitter account so you can easily see what is in the outer container.

Be aware this isn’t your only chance, at the end of this post I’ll show how to open the outer container again so you can add, remove or update files in it.

Once done, close the file explorer, return to VeraCrypt and click Next.

Now VeraCrypt lets you know it’s time for configure the hidden volume. Simply click Next to proceed.

You are now asked what encryption method to use on the hidden volume. It is indeed possible to use a different encryption method for the hidden area than you did on the outer container. Doing so isn’t a bad idea, as it can make it more difficult for an advanced hacker to break in.

For this demo I’ll stick to AES, but feel free to pick something different if you wish, then click Next.

Next you need to let VeraCrypt know how much space you want to reserve for the hidden container.

VeraCrypt examines what you have in the outer area, then lets you know how much of the free space you can use for the hidden area. Typically you don’t want to max it out, so you can go update the outer area from time to time.

Here I’m going to use half of my space for the hidden area, in this case 125 megabytes. I enter that, then click Next.

It’s time to enter a password for your hidden volume. Please note, the password for the hidden volume must be different than the outer volume!

Here I will use MySuperSecurePassword2! for illustration purposes, changing the number 1 used in the outer volume to a 2. In real life this would be very easy for someone to guess, so be sure to pick a password that is wildly different from the outer one.

Next we need to format the hidden area. As usual, jiggle the mouse around until the green bar is all the way across the bottom and click Format.

When formatting is complete you are presented with the above informational message. In short, it says if you open the outer volume without taking precautions (which I’ll show in a moment) you can accidentally overwrite the hidden partition.

Simply click OK to dismiss the message.

OK, you are all done with the creation. Simply click Exit to leave the wizard.

Now let’s see how to use the hidden container.

Accessing A Hidden Container

Accessing a hidden container is no different than accessing a regular container that doesn’t have a hidden one. Simply enter the path to the file (or select the device, such as a thumb drive), pick an unused drive letter, and click Mount.

Enter the hidden volumes password, as I did here using the MySuperSecurePassword2!, and click OK.

You can now open the drive letter (in this example W:) in your file explorer and copy files into your hidden container. Here I’ve copied in a photo I took of the historic Boll Weevil Monument from my old home town of Enterprise AL.

Note that this is the only picture here. The photo I use for Twitter doesn’t appear, as it is part of the outer volume.

When you are done, you can close your file explorer, return to VeraCrypt and Dismount the hidden container.

Accessing the Outer Container

What if you need to access the files in the outer container? For example, you may wish to copy updated fake data into it in order to keep it looking realistic.

It is possible to get to the outer container, but you need to take a few extra steps to prevent over writing the data in your hidden container.

As normal, enter the file name for your container (or select the device), pick a drive letter, then click Mount.

When the mount dialog appears, enter your password to the outer container. But WAIT!

Before you click OK, you need to click on the Mount Options button.

Go down to the bottom and check the box that says “Protect hidden volume against damage caused by writing to the outer volume“.

Now enter the password to the hidden volume, then click OK.

Now it displays another message warning against updating the hidden volume. It is possible, but not recommended, to have both the outer and hidden volume at the same time. Writing data to the hidden area could corrupt both the outer and inner areas.

As such I have a personal rule never to have both volumes open at the same time, and I highly suggest you stick to that rule.

Now you can click OK to mount the outer volume. With the outer volume now mounted, you can now access it in your file explorer.

Here you can see my Twitter photo I copied in originally. I can now update it, or copy in a few more files, up to the amount I have space for.

In my case, I have a 250 MB container, but I’ve reserved 125 MB for the hidden space, leaving me roughly 125 MB to put data in the outer area (VeraCrypt does use a little space in the container for its data).

Backup To Prevent Unintentional Damage

Remember how I said you could give the password to the outer container to an agent, or perhaps a bad guy?

Obviously you aren’t going to tell them about the hidden container, as such they won’t use the Mount Options to prevent overwriting the hidden area. Thus it is possible they could wind up destroying your hidden info.

To prevent this, be sure to make a backup of your container. Store it in a safe place away from home, such as a relatives house or your safety deposit box. This way a bad guy could go so far as to destroy your device and your data will still be safe.

Containers in Containers

One last thing, be aware VeraCrypt has no problems storing encrypted containers inside other containers.

For example, you could use VeraCrypt to encrypt a thumb drive. Then you could create a second file container, perhaps one with a hidden volume, and store it on the encrypted thumb drive.

You could go so far as to give it a different extension, perhaps using .dat instead of the default .hc, so a casual observer would not know it is a VeraCrypt container. When you select a file to mount, VeraCrypt doesn’t care what the extension is.

While this may seem a little paranoid, it is possible you may have a need for this level of protection so I just wanted you to be aware this option exists.

Conclusion

In this post I covered how to use the hidden container feature of VeraCrypt, one of it’s advanced options. Using it you can protect your most sensitive data.

In the next and final post we’ll see how to write scripts so you can automate the process of mounting and dismounting containers.

Veracrypt – A Powerful Encryption Utility

Introduction

I’m a huge podcast junkie. I’m subscribed to almost 200 podcasts on a variety of subjects, the majority of which are tech related.

One podcast I listen to is Grumpy Old Bens. The hosts discuss technology in relation to current news events. In an episode earlier this year they were discussing how to protect the data on your computer should, for example, you need to bring your laptop to a computer repair shop, forget it is there, and the owner decide to look through your drive.

One of the tools they mentioned is VeraCrypt. I’ve used VeraCrypt for years, and before that its predecessor, TrueCrypt.

Multi Platform

As with the other tools I’ve described in this series, VeraCrypt is multi platform with versions for Windows, macOS, and a wide variety of Linux distros.

Sadly there are no versions for portable devices such as iOS or Android.

Be aware what you encrypt is portable between platforms. If I encrypt something using VeraCrypt on macOS, I can later open it on Linux for example, or Windows.

Open Source

VeraCrypt is an open source application. Their website allows you to download all the source code. This allows you to inspect the source code and even build your own version of VeraCrypt from it.

How VeraCrypt Works

VeraCrypt has two modes of operation. First, it can encrypt an entire drive. This can be your main hard drive, a secondary drive, or one you plug in such as an external USB hard drive, USB thumb drive, or even an SD or MicroSD card you’ve placed in your computers card reader.

It uses industry standard encryption methods, and there are a wide variety you can select from, to scramble the contents of the drive to make it unreadable.

As a matter of fact, if you plug in an encrypted USB drive Windows will pop up an error message that you must format the drive for it to be used. Naturally you will want to cancel out of it, but this makes the drive more secure. An unknowing person will pop it in, and just assume the drive is bad. They’ll either throw it away or just reformat it, either way your data is kept out of their hands.

To use the encrypted drive, you’ll need to open the VeraCrypt software and select the drive to mount. Provide your password, and it will make it available. We’ll cover the steps more closely in a moment.

The second mode will let you create an encrypted file container. This is a single, encrypted file on your hard drive. Once you mount it, VeraCrypt will add a new drive letter to your system (on Windows) or mount point on other platforms.

You can move files in and out of this “drive”, create folders, or use it like any other drive. When you Unmount it, the files are no longer accessible.

How to Encrypt a Drive – Step 1

I’ll assume you’ve gone to the VeraCrypt Downloads page, and have downloaded VeraCrypt for your operating system. For this demo, I’ll be using Windows.

Begin by opening VeraCrypt. (As with all my posts, you can open the image to see the full sized view).

Begin the process by clicking on Create Volume. This will launch the VeraCrypt Volume Creation Wizard.

Step 2 – Pick the item to encrypt

For this demo, I want to encrypt an entire drive, specifically an 8 GB thumb drive I’ve plugged into a USB port. As such, I need to pick the second option, Encrypt a non-system partition/drive, then click Next.

Step 3 – Volume Type

We’re now asked if we want a standard volume or a hidden one. I’ll take about hidden volumes later in this post, so for now we’ll go with a Standard VeraCrypt volume and click Next.

Step 4 – Volume Location

Now we need to select the drive to encrypt, in this case our thumb drive. Click on the Select Device button so we can get its exact ID.

The laptop I’m writing this post on is configured to dual boot between Windows 10 and Kubuntu 21.04. It has two physical drives. The first one listed is a second 1 TB drive I use for storing VMs.

The second drive, listed as Hard Disk 1, is the 2 TB drive with its various partitions. I have one partition for Windows (C:), one for Kubuntu (listed as Partition 5). The remaining 1.2 TB partition is shared between Kubuntu and Windows. There’s a few other partitions for the GRUB booter and other recovery items.

Now here’s where we need to be careful. There is a listing for Harddisk 2, of 8 GB, and as your recall my thumb drive that I want to encrypt is 8 GB. This partition though is actually a recovery partition.

Any external drives I want to encrypt will be listed as a Removeable Disks. In the image above (surrounded by the red square) is my 8 GB thumb drive. I’d previously formatted for NTFS it and it is empty.

MAKE SURE YOU PICK THE RIGHT DRIVE!

You take full responsibility for picking the right drive, if you pick the wrong one I take no responsibility.

So with the thumb drive now picked, I’ll click on OK. Then back on the Volume Location screen I’ll click Next.

Step 5 – Volume Creation Mode

Next you are given a choice. In the first option, it will delete the contents of the drive and create an encrypted drive. If your drive is empty this is by far the fastest method.

If there is data on the drive you want to keep, then you should pick the second option. It will retain the files and put them into the drive once it is encrypted.

Because this option is slow, I would suggest moving any files to another drive, encrypting it, then moving them back when done. Even if you choose this option, I would highly suggest you backup everything just in case something catastrophic occurs such as a power outage or blue screen during the process.

Because my drive is empty, I’ll take the default and click Next.

Step 6 – Encryption Option

In this step you are asked what encryption algorithm you want to use. I’ve clicked the drop down so you can see the long list. For most instances, the default of AES using the SHA-512 is sufficient, but I have encountered clients who require a specific encryption algorithm be used.

The good part is that all of these are industry standards that have been vetted by security experts. For this demo I will use the default of AES and click Next.

Step 7 – Volume Size

If we were creating a container we would have the opportunity to select a size for it. In this case we are doing an entire drive, so VeraCrypt just informs us of the device name and the size of it as a confirmation we’ve picked the drive we wanted.

This is correct for this demo, so we’ll just click Next.

Step 8 – Volume Password

In the next screen you’ll enter the password that will be used to decrypt the drive. You can also check the Display password box so you can see what you are typing, which I highly recommend.

Because this is a demo, I’ve used a trivial password of MySuperSecurePassword1! but obviously you will want to use something much stronger.

I’d also suggest storing the password in your password manager. I use LastPass (which I’ll blog about in the near future). It has a Secure Note feature which I store my drive passwords in. You could also put them in Standard Notes, the subject of my previous blog post.

Once you have entered your super secure password, click Next.

Step 9 – Large Files

If the drive or container you are creating is larger that 4 GB then you will be asked if you want to store files bigger than 4 GB on it. (If the container or drive is 4 GB or smaller this step is skipped.)

This will determine how VeraCrypt formats the drive. The default is No, but I almost always change it to Yes, just in case.

After changing it to Yes, I clicked Next.

Step 10 – Format it!

Without getting too technical, one of the key factors for good encryption is having a random pool of data that can be used in generating the encryption key.

VeraCrypt gets this from the movement of your mouse around the window. As you jiggle your mouse around randomly, the bar at the bottom will change color from red, to yellow, then green. For best results, keep moving your mouse around until the bar is solid green all the way across the bottom.

Once it is solid green you can click on Format to begin the process.

After clicking format, you are given the warning that any data on the drive will be lost, are we sure we want to proceed?

If you are sure, take a deep breath then click on Yes.

The Volume Format screen now updates to show the progress. You’ll find it in the center, highlighted by the red rectangle in the above image.

The green is a progress bar, and you can see the exact percentage and speed below. To the very right is the time left, 21 minutes when I took this screen shot.

Be aware even on a fast computer this type of encryption can be a slow process. This small 8 GB drive takes about 20 minutes. A multi-terabyte drive can spend several hours encrypting. For that reason, on large drives, I’d suggest kicking off the process then going to bed.

When formatting is done, you will be provided this informational message:

My thumb drive was assigned drive F when it was plugged in, this message just says be sure to pick a drive letter other than F (or another used one) when you go to mount it. Just click OK to dismiss the message.

When you do, VeraCrypt will let you know the volume was created successfully.

You’re then returned to the wizard, where you can create another drive or exit. In our case we’ll click Exit.

All done, let’s use it!

Now that formatting is complete, let’s see how to mount our new drive.

As I mentioned before, when you plug in the VeraCrypt encrypted drive, Windows and other operating systems will complain the device needs to be formatted before it can be used.

Be sure to cancel out of it!

Once you’ve told your operating system to go away and that you know what you are doing, bring up VeraCrypt.

Because we’ll be mounting a thumb drive, we’ll need to click on the Select Device button.

We will need to scroll down to our list of removable disks, and pick our thumb drive then click OK.

With the device filled out, next go to the list of drive letters above and pick out a drive letter that is unused. for this demo I’ve picked the Z: drive. (Note this will differ for other operating systems.)

Now just click on Mount.

Note I check on the Display password box so we can see what we’re typing, and I’ve entered our super secure password. Before we click OK, I just want to call your attention to the Mount Options button.

One option that may be useful is the mount as read only. This is also where you can setup hidden volume protection, which we’ll talk about later. For now we can cancel, then back on the password entry window click OK to mount our drive.

As you can see VeraCrypt now updates to show our Z drive is now mapped to our thumb drive! It shows up as the Z drive in explorer and we can begin to use it like any other drive.

We can now copy files into our encrypted drive using your operating systems file explorer.

Creating an Encrypted Container

As previously described, an encrypted container is a single file which can be used to store data securely. As the steps to create one are almost identical to encrypting a drive, we’ll only highlight the differences here.

We’ll start on the main screen by clicking Create Volume.

On the VeraCrypt Volume Creation Wizard this time take the Create an encrypted file container option and click Next.

In the Volume Type screen, pick Standard VeraCrypt volume and click Next.

On the Volume Location screen, I’ve entered C:\Temp\demo.hc for the file to store my encrypted data in. VeraCrypt uses hc for its default extension, although if you wanted added protection you could give it a different extension. Once that was entered I clicked Next.

The next step is the Encryption Options, I’ll take the default of AES and click Next.

Next up is the volume size, how big do we want our container. You can size it in terms of kilobytes, megabytes, gigabytes, or terabytes.

For this demo, I am making it 100 MB in size. I’ve created containers of several hundred gigabytes for storing client data. I’ve also created some as small as 10 MB because I wanted to securely email information to someone (who I’d given the password to over the phone).

For now, we’ll go with the 100 MB and click Next.

Next up is the Volume Password screen, I’ll be using the same MySuperSecurePassword1! that I used in the previous demo and clicking next.

The final step is the Volume Format, where I’ll jiggle the mouse around a while to create a random data pool to be used in encryption. Once it is green I’ll click Format and let it format my encrypted file.

Because this is so small formatting goes very fast. When done I’ll just click Exit to end the wizard.

Mounting a File

Start with the main VeraCrypt dialog. Then click Select File to locate the C:\Temp\demo.hc file (or whatever your file name is).

I’ll then pick an unused drive letter, in this case the Y drive, and click Mount.

I’ll enter my super secure password and click OK and my encrypted file container will now be mounted as a drive!

Your Y drive will now appear in your file explorer, ready to use.

Dismount Your Drives

When you are done using your encrypted drive or container, be sure to dismount it when you are done. This will ensure the container is safely shut down and you won’t lose any data.

Hidden Containers

Hidden VeraCrypt containers is an advanced subject I’ll save for a future blog post as this has already gotten very long.

In short though, VeraCrypt lets you embed one container in another. Let’s take our 8 gig thumb drive as an example.

In the wizard, when you select the hidden container option, it will first walk you through the creation of a regular container. In this case our 8 gig thumb drive.

You could then copy some files into it that look like something sensitive. Perhaps some spread sheets with your family budget, or images of receipts. Stuff you might want to plausibly remain hidden, but if it got out wouldn’t be overly damaging.

It would then walk you through creating a hidden container. You could specify you wanted 5 of the 8 gig devoted to the hidden partition. Into it you could place your important data such as copies of birth certificates, marriage licenses, that kind of thing.

The idea is if a bad guy forces you to give up your password, you give them the password to the “outer” or regular drive. All they see is the spreadsheets and receipts you placed there.

The bad guys think they have the info, and have no way of telling you have a hidden partition setup.

When you want to use the hidden partition, you simply supply the password to the hidden area instead of the regular one.

As I said this is a bit of an advanced topic, so we’ll cover it more in the near future.

VeraCrypt Version

Please note the information in this post is valid as of the date I wrote the post. For it I used VeraCrypt Version 1.24-Update 7.

The interface and functionality of VeraCrypt has remained steady over the years so I don’t expect any significant changes, but things on the internet stay forever. If you are reading this five years into the future expect some minor differences between the post and what you see in your version of VeraCrypt.

Conclusion

With the proliferation of hackers and other bad guys, keeping data secure is more important than ever. I create containers for each of my clients to keep their data secure, and separate from other clients.

Additionally I have containers for my family documents such as copies of birth certificates, social security cards, and the like. Made small enough these can be easily stored in places like OneDrive or DropBox for easy retrieval.

As stated, in upcoming posts I’ll cover hidden containers, as well as how to script the mounting and unmounting of your containers.

Standard Notes – A Secure, Multi Platform Note Taking App

Introduction

With this post I’ll continue my series on useful tools and utilities. In this post I’ll talk about an outstanding application called Standard Notes. You can find and download Standard Notes from its website, https://standardnotes.org/.

Now, I know what you are thinking, “Yet another note taking app? What makes this so special?”

Well as it turns out a lot. Let me start by covering some of its special characteristics, then we’ll get a short overview on how to use it.

Multi Platform

As I stated in my previous post on Microsoft To Do, I have a strong preference for apps that work across multiple platforms. Standard Notes has apps for Windows, macOS, and almost every Linux Distro. In addition, it also has mobile apps for Android as well as iOS, both phone and tablet.

In addition, you can also login to the Standard Notes with your web browser and access your notes that way. Handy for when you are using a computer or device that you can’t or don’t want to install the Standard Notes application on.

The notes you enter are synced across all of your devices, much like Microsoft To Do. Enter a note on your Mac, and it will be there on your Android device, or any combination of the platforms I just mentioned.

Open Source

Standard Notes is an open source project. All of the source code is available for you to inspect, and even contribute to.

The syncing ability is handled by Standard Notes using their sync server software. The Standard Notes sync server is also an open source project that you can download. As such you can stand up your own private sync server and use it to sync all of your notes. Very useful if you are super concerned about security, or want to stand up your own Standard Notes server for your companies private use.

This also has some very positive implications to the longevity of Standard Notes. Let’s say the owners of Standard Notes win a bazillion dollars in the lottery. They hang up a “going out of business” sign on their website and head off to a remote island in the Bahamas for an early retirement.

You (or your company) could download the server project and spin up your own sync server. You can restore your backup to the new server, make an adjustment in the app to point to your private sync server and you are back in business. This capability should make any business more confident in adopting Standard Notes.

I mentioned backups, by default Standard Notes will perform an automated backup on a regular basis. You can (and should!) also do a periodic manual backup via the Account menu.

Encrypted Notes

Speaking of security, the biggest benefit to Standard Notes is encryption. It uses industry standard encryption, not something they made up themselves.

All of your note data is encrypted on your device. It is then sent to the sync server over an encrypted connection. As Standard Notes doesn’t have your password they have no way to decrypt the data. Even if someone got access to their servers, all the data will be gibberish.

They do make it clear, if you forget your password, they have no way of helping you. You need to make sure to store your password in a safe place like your password vault.

Freemium Model

Standard Notes uses what is known as a “freemium” model. The base application is free. You can take an unlimited number of notes, and these will be synced for you.

In the free model, you are restricted to text only notes. In addition, you are stuck with the basic color scheme of a white or gray background with black text.

With the paid model, you gain access to different color themes. It also enables you to use the rich set of extensions Standard Notes offers. Most of these extensions are around various editors. One is a code editor, that gives syntax highlighting for a huge array of languages.

There is another editor which gives you a “Word” like environment to do bold, italic, and the like. There is also a MarkDown editor if you prefer to edit and save MarkDown data.

Another editor I use a lot is the checklist. It lets you create check lists, and once you mark an item complete it moves to the bottom under completed tasks. This can be useful when the contents of your checklist are sensitive and need to be kept extra secure.

The premium model is done with subscriptions of 1, 3, and 5 years. The 5 year model works out to less than $3 (US) a month (at the time of this writing), which is very reasonable and funds their sync servers and development.

Setting Up Standard Notes

The Standard Notes website has instructions for downloading and installing it on the variety of platforms it supports, so I will refer you to it for your computer or device.

When you open Standard Notes the first time, it will open up with the Account pane. Note that for all the images in this post you can click on it to open up a bigger version.

You’ll need to start by clicking the Register button to create an account. When you do, it will prompt you for an email address and password. It will then send you an email just to confirm you are a real person.

Note the big No Password Reset box, warning you that if you lose your password, there is nothing the Standard Notes group can do to help you. I highly advise the use of a password manager, such as LastPass, which will be the subject of a future blog post.

Also note that the combination of your email and password is used to generate your security encryption keys. This means your email address will be case sensitive.

YourName@email.com, yourname@email.com, and YOURNAME@EMAIL.COM are all different. When you login to your other devices, you must key in both your email address and password in exactly the same case you use to create your account.

Using Standard Notes

Here is the Standard Notes app with some sample notes I’ve created.

The left side panel is the Views panel. You can organize your notes into groups called Tags in Standard Notes. You can enter any text you want for a tag name.

Additionally a note can have multiple tags associated with it. This will let the same note appear in multiple tag views.

The center column is the list of notes. If you click on a tag, the list of notes is filtered to show just the ones with that tag. In addition to the name of the note, it also has the first few words of the note and the last date the note was modified on (although you can change this in the settings).

In the example above, I have a note called “Harbor Freight Shopping List”. Under the title of the note I have two tags, Ham (referring to my Ham Radio hobby) and Lists. I could click in this area to type in the name of another tag if I wish.

In this next example, I’ve clicked on the Household tag in the Views pane.

As you can see, my list of notes is now reduced to just two, the ones who have the tag Household assigned to them.

Uses for Standard Notes

The uses for Standard Notes are only limited by your imagination. Because your notes are encrypted, you could use it to store information like phone numbers, vehicle VIN numbers, even passwords (although I’d suggest a more robust password manager of some type, such as LastPass).

It could also be used to store all kinds of tips and tricks. Items that don’t necessarily need to be encrypted, but you’d like to store and have handy in the future.

Anytime I have to do a search to figure out how to do something, I make a note so I can refer back to it in the future.

Premium Benefits

As a big fan of Standard Notes, I have purchased the premium subscription. I first did a one year subscription to see if I liked it, but got hooked so when it was time to renew I went with the five year subscription and haven’t regretted it.

Here you can see my Standard Notes paid subscription with one of the color themes applied.

As you can see from my list of tags on the left, I have a LOT of notes in Standard Notes. Here you can see a PowerShell script to remind me how to use the StringBuilder in PowerShell. I’m using the code editor, and have selected the PowerShell language.

This is just one of the many languages available, pretty much every programming language is included.

I also use Standard Notes for check lists, regular notes, and more.

In addition to the editors, the extensions also provide for various color themes, plus a selection of enhancements to Standard Notes. One developers may find useful is the GitHub Push extension, allowing you to push a note up to GitHub.

Want an extension but don’t see it? The Standard Notes site on GitHub has instructions for authoring your own extensions. This is also where you can access the source code for Standard Notes, the Standard Notes server, and more. You can also log issues, make suggestions for improvements and new features, and check for solutions to past issues.

Also note that your premium subscription unlocks the premium features on all of your devices. There is no limit to the number of devices or computers you can have Standard Notes (even with your premium subscription) running on.

Passcode Lock

Another useful feature is the Passcode Lock. This is similar to the pin login feature of Windows 10. You can turn on Passcode Lock in the Account menu area.

When you launch Standard Notes it will prompt you for the Passcode Lock. The lock can be any combination of letters, numbers, and special characters you want. I use a long complex gibberish password for my Standard Notes account, but use a little easier to remember and type Passcode Lock.

I consider this a “nosy person” feature. It keeps a nosy family member or coworker from walking up to your computer while you are getting coffee and snooping at what’s in your Standard Notes.

In addition, you can set an Autolock timer. After a certain amount of time Standard Notes will lock itself. By default Autolock is set to Off.

It also has an Immediate setting, which locks Standard Notes the minute you click away from it. There are also timers for 1 minute, 5 minutes, and 1 hour.

You can also remove the Passcode Lock when you decide you no longer need it.

On iOS, you can also lock Standard Notes using either Face ID or the Fingerprint reader. I would imagine Android tablets/phones that support biometric security also have this feature, but my Android tablet lacks biometric security so I’m unable to test this.

Other Notable Features

I just wanted to mention a few other useful features.

You have the ability to sort the list of notes in a variety of ways. By default it uses most recent at the top, but you can also sort alphabetically. You can also reverse the sort so oldest is first, or titles are sorted Z to A.

You can also Pin notes so they will always appear at the top of the note list, regardless of what sort order you select. In addition to pinning, a note can be Locked, which puts it into a read only state.

You can also get rid of a note by selecting Move to Trash, then go to the trash can and choose to delete all notes in the trash can or just individual ones.

The final feature I’ll mention is the ability to Archive a note. Let’s say you have a note on how to do something in Ubuntu 16.04. It’s highly unlikely you’ll need to use this on a normal basis, and don’t want to see it cluttering up your list.

However, the information is valuable, and on the off chance you may need it you really don’t want to trash it. This is where the Archive feature comes in. In any note, simply pick Archive in its Options menu. The note will no longer appear in the list view or searches.

You can easily bring it back though. First, in the list view’s options menu you can turn on the appearance of Archived notes. Later you can use the same option to again suppress the appearance of Archived notes.

If you suddenly find that previously archived note is useful, you can pick the note, then in the note editors options menu just Unarchive it.

Standard Notes Version

The instructions in this post are valid at the time I wrote it, using Standard Notes version 3.6.9.

Standard Notes is under continual development, so depending on when you read this there may be some subtle differences between your version and the one I’m using for this post. The core concepts should remain constant though.

Standard Disclaimer

I just want to be clear this is in no way a paid for post. Nor do I receive any discount for talking about it. I paid my own money for it just like everyone else. I just love the tool and wanted to share it with others.

Conclusion

Give Standard Notes a try. You can do so for free, and I think once you do you will quickly find it an indispensable tool in your kit. It’s a tool I use literally every single day.

Microsoft To Do

Introduction

This post continues my series on useful tools and utilities. Here we’ll be covering Microsoft’s To Do application.

There’s a famous quote “If you don’t write it down, it doesn’t exist.” A “To Do” list is a great way to capture tasks or information you need to remember. Some people use paper, or a variety of other devices or applications. For me, Microsoft To Do is the place to capture this information.

Some of you may remember Wunderlist. Microsoft bought Wunderlist a few years back, and have transformed it into Microsoft To Do.

Multi Platform

I use a variety of devices and platforms on a regular basis. One of the things I find useful about Microsoft To Do is the availability of apps on almost all platforms.

There are apps for Windows, macOS, Android, and iOS. On Linux (or other platforms), you can access Microsoft To Do in any web browser.

I also found a blog post that describes an application called AO that runs on most Linux distros. It’s basically a wrapper around the To Do web page. I use it on my Kubuntu laptop for working with Microsoft To Do.

Using Microsoft To Do

I’ll quickly illustrate the various uses of To Do with some screen shots. (Note that you can see a bigger version of any of these images by clicking on them.) The first one shows the basic layout of To Do.

The default view of To Do is “My Day”. To Do allows you to designate tasks for immediate action called “My Day”. More on that in a moment, but here you can see one task for a new Pluralsight course I’m currently working on, the SQL Server Mobile Report Publisher.

The menu of actions appears down the left side of the application. To add a new list, simply click the “+ New List button”. This is very simple, it just brings up a page and you can start typing in your tasks.

You can rearrange the list by clicking and dragging the tasks into any order. Note that by default new tasks are added to the top of the list, but you can go into the settings menu to change this behavior so new tasks are added to the bottom (which is what I do).

Simple Lists

Microsoft To Do can be used for simple lists. Here you can see a grocery list.

To Do makes generating a grocery list like this easy. I can enter my list on my computer, where I have a full keyboard. To Do then syncs my list to all of my devices.

When I pull out my phone in the grocer store, I can simply mark each item off as I put it in my grocery cart. Completed items are moved to the bottom of the list, making it easy to see the items I still need to get.

When I get home, I can either delete each completed task individually, or right click on the “Completed” header and delete them all at once.

Projects

Another use for To Do is project tracking.

Here is the list for my current project, the Mobile Report Publisher course I’m working on. This list is a high level view of the tasks I need to complete for the course.

For each task, I can create a list of sub tasks that need to be completed.

When I click on the task, a pane pops out on the right. I can enter a series of steps for setting up my virtual machine. Installing Win 10, Installing basic tools, and more.

Here is the detailed information for another task in the list, Create Data Source in the Report Portal.

This task only has one step. However, I’ve clicked on the “Add to My Day” which will add the task to the “My Day” screen To Do opens to.

With To Do I can also set a Due Date. To Do makes it easy, I can set it today, tomorrow, next week, or I can pick a specific date. I can also set a reminder, so To Do will remind me when a task is coming up.

I can get an overview of all my tasks with due dates by clicking on the Planned link on the left side of To Do.

Here you can see the one task I planned with a due date. The nice thing about the planned tab is that it shows tasks coming due across all your lists.

For example, I could have assigned a due date for an item in my grocery list so I’d be sure to have an ingredient for a planned meal. Or perhaps I have another list for planned blog posts.

All tasks with due dates that haven’t been marked as complete will show up here on planned, making it easy for me to get an overview of upcoming tasks no matter what list they are on.

To Do and Multiple Accounts

To Do allows you to manage multiple accounts. By clicking on the account name it will show you a list of all To Do accounts you’ve logged into.

Here you can see I have two accounts logged in, one I use for work, the other for personal items such as grocery lists. This makes it nice as I don’t have to mix work and personal lists.

It’s also useful when I work with multiple clients, when the client provides me an account to use in their organization. I can easily keep each client’s task list separate from each other a well as my personal lists.

List Management

The final item I’d like to show is list management, accessible by right clicking on a list.

Some items are pretty obvious, such as deleting or renaming a list. You can also duplicate a list, or print it out.

The biggest feature I like is the “Share list” option. You can share a list with another Microsoft To Do user. A good example is the grocery list. I share mine with my entire family. When another family member needs a grocery item, they can simply add it to the list.

There’s no need to tell me, or send a text, or anything else. When I get to the store, there’s the item on the list. If it’s something odd or unusual, they can click on the item and in the pop out pane on the right add a note to the item to explain why it is needed.

Another use is for small projects with your coworkers. You can assign a task to another person. They will see the task as assigned to them, and as they mark the task, or each step in a task, complete the others that the list has been shared with will be updated automatically.

This makes it nice for a project manager, as they can easily see the teams progress for each task.

Conclusion

As you can see, Microsoft To Do is a great tool for managing lists, or for tracking progress of your smaller projects. The multi-platform capability makes your data easily accessible across any device you are using.

There are many more uses you can put To Do to. I’ve used it for managing home repair projects. I even have some lists for favorite recipes, that list includes the ingredients as well as cooking instructions.

If you can think of more uses for Microsoft To Do, then by all means leave a comment so we can all learn.

Cut and Copy Fast and Easy with Pantherbar for Windows

Introduction

In a previous post, I showed a tool to make Cut and Copy easy on macOS. In this post we’ll look at a tool, Pantherbar, to provide similar capability on Windows.

Pantherbar

Pantherbar is available in the Microsoft Store. It has a free version, as well as a paid one at the reasonable price of $4.99 (US).

Similar to PopClip when you highlight text, it pops up a toolbar.

As you can see from the image above, Pantherbar appears with several icons. The left most is copy, next is cut. If you have anything in the clipboard the paste icon appears next.

The magnifying glass will launch a search in your default browser. By default it is Google, but through the settings you can change the search engine. Finally is the share with windows icon.

Extensions

Pantherbar has a rich set of extensions you can add to it. You can view the full list at http://pantherbar-app.com/extensions

In the free version you can have two extensions active. In the paid version, you can have as many as you want.

In this next example, I’ve loaded three extensions. In the image below I’ve highlighted some text in notepad.

The three new ones on the right are character count (handy for composing a tweet), reverse the text, and remove spaces. In the image below I’ve clicked the reverse text option.

As you can see, the text is reversed immediately, replacing what had been selected.

Note this is a little different than the behavior of PopClip on macOS. PopClip copies the modified text to the clipboard. Pantherbar immediately pastes the new text in.

To be honest, I prefer this behavior. The majority of the time I’m going to put the corrected text where I had selected, so this saves a few steps.

Conclusion

Pantherbar is a useful addition to your Windows toolkit. With its rich set of extensions, you can save a lot of time performing common tasks. You can try it for free, and if you decide to buy the price is very reasonable.

Do note that this, nor the PopClip blog post, are paid posts in anyway. I just like the tools and was happy to pay for them.

To my knowledge there is no similar extension on the Linux platform, at least for the Ubuntu based distros I tend to use. As I understand it, the current graphics engine makes this kind of extension difficult. It’s hoped though that once the Wayland engine gets into wide spread use tools similar to Pantherbar or PopClip will become available on Linux.

If you know of one that exists, by all means leave a comment. I’d love to check it out!