Programming Possum

I’d like you to meet my Programming Possum. His name is Floyd the Ferocious, and he is one of my biggest helpers in coding.

[Programming Possum]

Floyd was a Christmas gift from my wife, to commemorate the dead possum I found in my hot tub shortly before Thanksgiving. Seems that possum was assigned to write an application in VB4 that generated Java code to be an interop layer between a Perl app and Ruby on Rails program.

Poor guy grabbed his chest and just keeled over in my hot tub, turning himself into a hot possum stew. I found him about four days later, when I went to top off the tub. The smell was, well it was something Mike Row would have wrinkled his nose at. And the stew was now a nice murky black. Yum, soups on!. But I digress.

A few weeks ago I was catching up on my Dot Net Rocks episodes. In episode 205 (http://www.dotnetrocks.com/default.aspx?showNum=205#download) guests Venkat Subramaniam and Andrew Hunt were talking about Agile Programming, and one of them mentioned he keeps a rubber duck on his computer. He keeps his duck around for the same reason Floyd hangs out on top of my monitors, to squash bugs.

Have you ever gone to one of your coworkers and started to explain an issue, when all of a sudden you smacked yourself in the head and gone “OK I know how to fix it now thanks!” Your coworker wittily replies “Uh, OK”.

For me, my Programming Possum is the first person I talk to about my issues. He’s like a hairy therapist with a tail, very attentive listener and never interrupts. (I used to talk to pictures of my wife, but found I kept getting interrupted.) About forty percent of the time, I find that I can work through all the alternatives and come up with a resolution. And if I can’t, when I do have to visit my coworkers at least my thoughts are more organized and coherent.

It doesn’t have to be a possum, it could be a rubber duck or even a rubber chicken. Scott keeps a yellow chicken by his cube.

[White Chicken]

Ben, another coworker, has a yellow chicken which keeps an eye on his code. (Which frankly we all think is a bit weird. I mean, who ever heard of a yellow chicken? White, sure, even brown or black, but yellow? I guess that’s where brown eggs come from.)

Whatever you pick, we all recommend you get your own Programming Possum. Something to focus on so you can talk your issues through while you look for solutions. You’d be surprised at how often it’ll work for you.

And don’t worry about your coworkers thinking you are nuts. I’ve talked to them, and they already think that.

Floyd the Ferocious, Programming Possum

Loading a SQL Server Compact Edition Table From a DataTable in C#

Of all the situations you can be in using SSCE, I think one of the most common will be to pull data from a larger database provider and populate a table in your local SSCE database. While SSCE does have some nifty replication support, it only works with SQL Server. Like many of you I have to support data coming from many different places, such as Oracle, Access, and XML to name but a few.

The easiest way to pull the data is to get it into an ADO.Net DataTable, but what then? I sure didn’t want to have to create the insert statements by hand each time, plus I wanted to be able to reuse my routines. Being a lazy individual, I wanted a routine I could pass any DataTable into and push the data into my SSCE table.

I started with a blank class, and created a few class level variables. One was out of a SqlCeConnection which I named _connection. I also had two strings to hold the database name and password. Next, I created a “ConnectionString” method, identical to what I described a few days ago.

    private string ConnectionString()

    {

      return string.Format(

        “DataSource=\”{0}\”; Password='{1}'”

        , this._CacheDatabase

        , this._Password);

    }

 

I then created an Open method, which is pretty simple:

 

    public void Open()

    {

      if (_connection == null)

        _connection = new SqlCeConnection(this.ConnectionString());

 

      if (_connection.State == ConnectionState.Closed)

        _connection.Open();

    }

 

OK, now for the grand finale.

 

 

    public void Populate(DataTable myDataTable

      , string tableName)

    {

      // If the datatable has no rows, we’re wasting

      // our time, get outta dodge.

      if (myDataTable.Rows.Count == 0)

      {

        return;

      }

 

      // Make sure database is open for business

      this.Open();

 

      // Use a string builder to hold our insert clause

      StringBuilder sql = new StringBuilder();

      sql.Append(“insert into “ + tableName + ” (“);

 

      // Two more, one for the list of field names,

      // the other for the list of parameters

      StringBuilder fields = new StringBuilder();

      StringBuilder parameters = new StringBuilder();

 

      // This cycles thru each column in the datatable,

      // and gets it’s name. It then uses the column name

      // for the list of fields, and the column name in

      // all lower case for the parameters

      foreach (DataColumn col in myDataTable.Columns)

      {

        fields.Append(col.ColumnName);

        parameters.Append(“@” + col.ColumnName.ToLower());

 

        if (col.ColumnName !=

          myDataTable.Columns[myDataTable.Columns.Count

                 - 1].ColumnName)

        {

          fields.Append(“, “);

          parameters.Append(“, “);

        }

      }

      sql.Append(fields.ToString() + “) “);

      sql.Append(“values (“);

      sql.Append(parameters.ToString() + “) “);

 

      // We now have our Insert statement generated.

      // At this point we are ready to go through

      // each row and add it to our SSCE table.

      int rowCnt = 0;

      string totalRows = myDataTable.Rows.Count.ToString();

 

      foreach (DataRow row in myDataTable.Rows)

      {

        SqlCeCommand cmd

          = new SqlCeCommand(sql.ToString(), _connection);

 

        foreach (DataColumn col in myDataTable.Columns)

        {

          cmd.Parameters.AddWithValue(“@”

            + col.ColumnName.ToLower()

            , row[col.ColumnName]);

        }

        // Optional: I created a delegate called message delegate,

        // and assign it to the class level variable Cachemessage

        // It’s a simple method that takes one string and displays

        // the results in a status bar. If you want to simplify

        // things, just remove this entire section (down to

        // the try).

        rowCnt++;

        if (_CacheMessage != null)

        {

          if ((rowCnt % 100) == 0)

            _CacheMessage(“Loading “ + tableName

              + ” Row “ + rowCnt.ToString()

              + ” of “ + totalRows);

        }

 

        try

        {

          // Here’s where all the action is

          // sports racers! This is what sends

          // our insert statement to our local table.

          cmd.ExecuteNonQuery();

        }

        catch (Exception ex)

        {

          // You’ll probably want to be a bit more

          // elegant here, but for an example it’ll do.

          throw ex;

        } 

      } 

    }

 

My code comments elaborate pretty well on what’s going on, a quick read should be all you need. Only two things to really point out. First, I create a delegate to send a progress message. I’ll go into more on delegates another day, if need be, there’s a ton of info on them out on the web. Let me show you the declarations I used for my delegates so you can repeat them in your class:

 

    private CacheMessageHandler _CacheMessage;

 

    public void Messenger(CacheMessageHandler messageRoutine)

    {

      _CacheMessage = messageRoutine;

    }

 

I created a method in the calling program called ShowStatus. Very simple, takes one string as a parameter and displays it somewhere to the user. (I chose a status bar, you might use a label). All I had to do was call the Messenger method like so: 

    myMethod.Messenger(

         new MyClass.CacheMessageHandler(ShowStatus))

 

In retrospect I could also have created a CacheMessage property, I just didn’t think of it at the time. If you paste in the declarations you should be able to use the method even though you never use the delegate (note I check to see if _CacheMessage is null) but if you have issues, just delete that small section, it’s not that important to the process of loading the table.

The other major thing, and a gold star to you if you already noticed this: in order for this method to work, the column names in your DataTable must match exactly with the column names in your SSCE table!

Personally this seems like a small price to pay, and frankly if you are replicating data it will make your debugging and programming life much easier as you work through bugs. I do this as a standard, which is why this kind of routine fits well into my environment.

This wraps up (for now) my exploration of SQL Server Compact Edition. If you have decided to leverage SSCE in your own apps, please leave a comment, let us all know what your experiences have been, problems, benefits, and the like.

Inserting Rows Into A SQL Server Compact Edition Table in C#

Now that we have some tables, you naturally want to put some data into them. As you might have guessed from my last post, you perform data manipulation (insert, update, delete) just like you do when creating the table and use the SqlCeCommand object. Only this time there’s a twist.

First, because I wanted to load several rows I created a method to load a single row and pass in the parameters to my “CoolPeople” table. Here’s the small bit of code that handles it:  

        LoadARow(“Carl”, “Franklin”, @”http:\\www.dnrtv.com”);

        LoadARow(“Richard”, “Campbell”, @”http:\\www.dotnetrocks.com”);

        LoadARow(“Leo”, “Laporte”, @”http:\\www.twit.tv”);

        LoadARow(“Steve”, “Gibson”, @”http:\\www.grc.com”);

        LoadARow(“Arcane”, “Code”, @”http:\\arcanecode.wordpress.com”);

 

I then wrote a routine that would take the passed in data and insert it into the database. As with my create table example yesterday, I’m using the command object. This time though, I am adding parameters to the command.

If you look in the SQL, you see I have three parameters, noted by the @ sign. @first, @last, and @url. When SSCE creates the insert statement for the database, it will then look for three parameters and replace these three @ placeholders with the values you put into the parameters.

Sure, you could concatenate it all together in a string, but then you have to worry about things like the “O’Malley Issue” and SQL Injection attacks. (See Bill Vaughn’s book “Hitchhiker’s Guide to Visual Studio and SQL Server, 7th edition for a complete discussion on these topics, or browse the web. There’s lots of info so I won’t take up further space now.)

Here’s the entire LoadARow routine. Note that my choosing to name the method parameters the same as the SSCE command parameters is entirely a coincidence, it simply makes it self documenting and is not a requirement.

 

    private void LoadARow(string first, string last, string url)

    {

      SqlCeConnection cn = new SqlCeConnection(ConnectString());

 

      if (cn.State == ConnectionState.Closed)

      {

        cn.Open();

      }

 

      SqlCeCommand cmd;

 

      string sql = “insert into CoolPeople “

        + “(LastName, FirstName, URL) “

        + “values (@lastname, @firstname, @url)”;

 

      try

      {

        cmd = new SqlCeCommand(sql, cn);

        cmd.Parameters.AddWithValue(“@lastname”, first);

        cmd.Parameters.AddWithValue(“@firstname”, last);

        cmd.Parameters.AddWithValue(“@url”, url);

        cmd.ExecuteNonQuery();

        lblResults.Text = “Row Added.”;

      }

      catch (SqlCeException sqlexception)

      {

        MessageBox.Show(sqlexception.Message, “Oh Crap.”

          , MessageBoxButtons.OK, MessageBoxIcon.Error);

      }

      catch (Exception ex)

      {

        MessageBox.Show(ex.Message, “Oh Crap.”

          , MessageBoxButtons.OK, MessageBoxIcon.Error);

      }

      finally

      {

        cn.Close();

      }

    }

 

After creating the SqlCeCommand object by passing in the sql string and the connection object, I can then add the parameters with a single line for each. By using the Parameters object of the command object, I can call the AddWithValue method, and simply pass in the parameter name as a string and the value for that parameter. Once you add all the parameters, simply call the ExecuteNonQuery method and the data is inserted!

This method can be a basis for all your future work with SSCE. Everything you need to do revolves around the command object and sending SQL commands to the database. Need to delete a record? Just change the SQL from an insert to a delete, pass the correct parameters and you are in business. Update? Same thing.

Using the code I’ve shown in this series you can create your own SSCE applications, and store data locally. Lest you think this is the wrap up, I have one more cool method to show you, but that’ll be for tomorrow!

Create a Table in SQL Server Compact Edition with C#

Before we create a table in SSCE, you need to understand that SSCE only supports a subset of the data types provided in full SQL Server. The specific list is: bigint, integer, smallint, tinyint, bit, numeric (p, s), money, float, real, datetime, national character(n) (Synonym:nchar(n)), national character varying(n) (Synonym:nvarchar(n)), ntext, nchar, binary(n), varbinary(n), image, uniqueidentifier, ROWGUIDCOL , and IDENTITY [(s, i)].

The last two, strictly speaking are not data types but properties of the data column. SSCE was designed to be small and lightweight (hence the Compact name). To keep it small, many of the datatypes were eliminated.

A full list, with detailed explanations can be found at on the MSDN site: http://msdn2.microsoft.com/en-us/library/ms172424.aspx or http://shrinkster.com/lij.

Now that you know what’s available, let’s talk about how to create a table. In keeping with it’s theme of compactness, SSCE doesn’t include classes for manipulating the structures inside a SSCE database directly. Instead, you have to do everything through SQL DDL (Data Definition Language) statements. You know, Create Table, Drop Table, etc.

The code below shows the fairly simple steps involved. First you need to open a connection to the database. You do this by creating a SqlCeConnection object and passing in the connection string. The connection string has the same format I described in my previous post on creating the database.

Next, we create a string to hold some SQL (sql), and create a SqlCeCommand object (cmd). Finally we call the ExecuteNonQuery method of the command object to kick off the SQL. Here’s the code, with some try / catch logic thrown in to trap for errors.

 

      SqlCeConnection cn = new SqlCeConnection(ConnectString());

 

      if (cn.State==ConnectionState.Closed)

      {

        cn.Open();

      }

 

      SqlCeCommand cmd;

 

      string sql = “create table CoolPeople (“

        + “LastName nvarchar (40) not null, “

        + “FirstName nvarchar (40), “

        + “URL nvarchar (256) )”;

 

      cmd = new SqlCeCommand(sql, cn);

 

      try

      {

        cmd.ExecuteNonQuery();

        lblResults.Text = “Table Created.”;

      }

      catch (SqlCeException sqlexception)

      {

        MessageBox.Show(sqlexception.Message, “Oh Fudge.”,

          MessageBoxButtons.OK, MessageBoxIcon.Error);

      }

      catch (Exception ex)

      {

        MessageBox.Show(ex.Message, “Fooey.”, MessageBoxButtons.OK,

          MessageBoxIcon.Error);

      }

      finally

      {

        cn.Close();

      }

 

There’s a few things I’d like to point out. First, the connection. In this simple example, I opened and closed the connection in the same routine. For talking to large databases, especially when doing so from a web app, this methodology makes a lot of sense. You keep the connection open for a brief time and reduce the load on your server.

With SSCE however that need does not exist. The database is local with one connection, so you save nothing by the brief open / close connection, and instead slow yourself down. With SSCE I would recommend you open the connection when your application launches, and close it when it exists. Pass it to the classes that need it. You will gain a lot of speed doing so, and minimal cost in terms of memory.

The other thing to note, within the line that instantiates the new SqlCeConnection I have a method called ConnectString(). To make life easy for my demo I encapsulated the connection string in a method that returns a string. You could place your connection in a string, method, or whatever is convenient for you. I already documented the code to create a connection string in yesterday’s post on creating a database, so I won’t bore you with it again.

Would you like to see the database? Cool. Open up SQL Server Management Studio (I’m guessing you have SQL Express or SQL Server Developer Edition installed). When it opens, pick SQL Server Mobile as the database type. Under Database File, click the Browse for more… option and navigate to the sdf file you created, and click on OK. Enter in the password, and click OK again to open the database.

Now you can do many of the normal things you’d do with a SQL Server database. Click on the Tables in the tree and you should see it expand and show our “CoolPeople” table. You can drill down further to see all the fields.

OK, we’ve created our table, the next step will be to load it. But that’s for the next post!

Create a SQL Server Compact Edition Database with C#

Before I start on the coding route, I found one other component you might want to download for SSCE. It’s not absolutely needed, but makes life in VS much easier. It’s the SSCE Developer SDK. It has the Northwind sample database, CAB files for installing SSCE on mobile devices, and MSI for installing the help files, and more. Grab it at the URL below and install all the goodies.

http://www.microsoft.com/downloads/details.aspx?FamilyId=E9AA3F8D-363D-49F3-AE89-64E1D149E09B&displaylang=en or http://shrinkster.com/lho

OK, let’s have a little fun. First thing I did was create a test app using a Windows Forms application. Next thing you need to do is create a reference to the SSCE. Click on Project, Add Reference. According to all directions I’ve read, all you have to do is be on the .Net tab, scroll down to System.Data.SqlServerCe, click on it and click OK. Found it?

Nah, I couldn’t find it either. So here’s the part the instructions don’t tell you. Go click on the Browse tab. Now navigate to C:\Program Files\Microsoft SQL Server Compact Edition\v3.1. Now pick the System.Dadta.SqlServerCe.dll, click on it, then click on OK to pull in the reference. (That little nugget will save you a lot of hair pulling, and if you did find it then good for you, just click on it and keep going.)

Now let’s write some code. First, go to the head of the class, whoops I mean header of your class and let’s create a using reference.  

using System.Data.SqlServerCe;

using System.IO;

While I was at it I also set a reference to the System.IO namespace since we’ll want to check for the databases existence. Now in a method somewhere, let’s setup a few variables. Then we’ll check to see if the database already exists. Since this is just a sample, I’m going to delete it. This would be a valid action if the database was just a temporary cache for reports. In your situation you might want to return an error, or just exit gracefully without causing any problems.

 

      string connectionString;

      string fileName = “ArcaneCode.sdf”;

      string password = “arcanecode”;

 

      if (File.Exists(fileName))

      {

        File.Delete(fileName);

      }

 

      connectionString = string.Format(

        “DataSource=\”{0}\”; Password='{1}'”, fileName, password);

 

You can see the last thing I did was establish the connection string. Some very important things you need to note. First, the DataSource must be surrounded with double quotes. The password, on the other hand, must be surrounded with single quotes.

In this example I’m just giving the database an SDF name, and letting it drop in the current directory. You could hard code a full path, or put it in a variable.

OK, we’ve got everything setup. Turns out it’s quite easy to create a new database. First, we create a new instance of a SqlCeEngine, and initialize it by passing the connection string into the constructor. Then, it’s as simple as calling the CreateDatabase method of the engine.  

      SqlCeEngine en = new SqlCeEngine(connectionString);

      en.CreateDatabase();

 

And that’s all there is to it. Looking in our bin debug folder we see the new database:

 

[Picture of ArcaneCode.sdf]

 

Of course an empty database doesn’t do us a lot of good, so in the next installment we’ll start creating some tables, and maybe even put a little data in them.

Ruler

While my life is busy being turned upside down by the SSIS scripts I’ve been testing, let me take this chance to fill you in on a cool new tool I found for Windows (I promise to get back to the SQL Server Compact Edition stuff soon!).

This tool is very cool, no matter what you happen to do for a living, be it a web designer, programmer, or even a housewife who loves graphics. It’s Ruler, it displays a simple ruler on the screen like so:

[Picture of Ruler]

Here you can see I’ve placed the ruler over yesterday’s blog entry. The ruler can be resized by simply holding the mouse cursor over the edge, clicking and dragging as you would resize a normal window.

Pressing the spacebar will flip the ruler from horizontal to vertical. Right click on the ruler to access the menu options. You can control the opacity, and set it to “Stay on Top” mode among other things.

It also supports “nudging” via the keyboard when you need to place or size it exactly. The arrow keys will move the ruler five pixels, CTRL+Arrow moves it one pixel, and CTRL+Shift+Arrow will resize the ruler.

This delightful little tool can be found for free at http://www.sliver.com/dotnet/Ruler/. You can even download the source, in case you want to add your own enhancements. It consists of 3 simple files that don’t even have to be installed, just unzip and place in a directory.

I love this thing, I have often wished I had a pixel ruler and this has found a permanent place in my toolbox. Kudos to Jeff Key (http://weblogs.asp.net/jkey/) for writing this gem.

SSIS Package Not Reading Environment Variables

My work today on SQL Server Compact Edition (see yesterday) has gotten interrupted by some issues with a SQL Server Integration Services (SSIS) package. We keep our connection strings inside environment variables, and had this one package that just would not read those environment variables correctly. To further compound our fun, our development platform is using a 32 bit version of SSIS but our test server is running 64 bit SSIS.

We finally corrected the issue by opening the package in BIDS (Business Intelligence Developer Studio, basically Visual Studio with some SQL Server components loaded into it). Once open, we deleted the connections, recreated them, and then redeployed the package.

And ta-da! It suddenly started working correctly. I put this out here for two reasons: first, to pass along our knowledge in case you are having the same issue. Second, if you’ve had this issue I’m curious to know about it, please leave a comment and let me know your experience.

Follow

Get every new post delivered to your Inbox.

Join 101 other followers