Tag Archives: SSIS

SQL Server SSIS SSDT Error – Method Not Found: Microsoft.SqlServer.Dts.Design.VisualStudio2012Utils.IsVisualStudio2012ProInstalled

Every so often, especially when setting up a new virtual machine, at some point I get this error when working in SSIS:

 

image

 

It happens often enough that, to be honest, I mostly wanted to make this as a record to myself and friends, but I am hopeful that you will be helped as well. And to give credit where it is due, I found the original answer at Stack Exchange:

http://stackoverflow.com/questions/24745396/isvisualstudio2012proinstalled-method-not-found-error-when-running-an-ssis-pac

The steps are pretty straight forward.

1.  First, if you have Visual Studio open, close it.

2. In the classic Windows menu go to Start, All Programs, Microsoft Visual Studio 2012, Visual Studio Tools, and right click on the “Developer Command Prompt for VS2012” and pick “Run as administrator”. If you are in Windows 8 or Server 2012 or later, probably easiest to just do a search for “Developer Command Prompt for VS2012”.

3. Navigate to Visual Studio’s Private Assemblies folder by entering “CD C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PrivateAssemblies”  (If you have installed VS to another drive or folder, adjust the path accordingly).

4. Enter the command “gacutil /if Microsoft.SqlServer.Dts.Design.dll” into the command prompt.

 

image

 

Now you should be able to open your SSIS projects.

The dll is the main set of libraries for SQL Server Integration Services. For some reason, after certain Windows updates, this becomes deregistered and you have to manually add it back. Not a huge deal, but annoying if it happens often enough. As I just had it happen today after a Windows update to the Hyper-V VM I use for some of my SQL Server development, I wanted to post this as a reminder on how to correctly fix the issue.

Advertisements

SQL Saturday Atlanta #285–Everything You Ever Knew About SSIS Is Null and Void

For the SQL Saturday #285 in Atlanta, May 3rd 2014, I am presenting “Everything You Ever Knew About SSIS Is Null and Void”. You’ll find the presentation itself at:

http://sqlsaturday.com/viewsession.aspx?sat=285&sessionid=19929

All of the SSIS samples including the simple SSIS packages, SQL Scripts, and Slides will be available there shortly.

The main pieces of code, the SQL scripts to  monitor SSIS, can also be found in my TechNet Gallery:

http://gallery.technet.microsoft.com/SQL-to-Monitor-and-Explore-e485fa84

Thanks for attending!

Importing MongoDB Data Using SSIS 2012

I have embarked on a little quest to learn other database platforms (especially NoSQL) as more and more of our clients at Pragmatic Works have them in their enterprise, and want to be able to import data from them into their SQL Server data warehouses using SQL Server Integration Services (SSIS). While I found several articles that showed how to do so, these were outdated due to changes in the MongDB C# driver. After quite a bit of effort figuring out how to get this working, I thought I’d pass along my hard fought knowledge.

First, I assume you are familiar with MongoDB (http://www.mongodb.org/) and SQL Server (https://www.microsoft.com/en-us/sqlserver/default.aspx). In my examples I am using SSIS 2012 and MongoDB 2.4.8, along with the C# driver version 1.7 for MongoDB available at http://docs.mongodb.org/ecosystem/drivers/csharp/ .

First, download and install the C# driver. This next step is important, as there was a change that occurred with version 1.5 of the driver: the DLLs are no longer installed in the GAC (Global Assembly Cache) automatically. They must be there, however, for SSIS to be able to use them.

By default, my drivers were installed to C:\Program Files (x86)\MongoDB\CSharpDriver 1.7. You’ll want to open a CMD window in Administrator mode, and navigate to this folder. Next you’ll need GACUTIL, on my computer I found the most recent version at:

C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\x64\

A simple trick to find yours: Since you are already in the CMD window, just move to the C:\Program Files (x86) folder, and do a “dir /s gacutil.exe”. It will list all occurrences of the program, just use the one with the most recent date. Register the dlls by entering these commands:

“C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\x64\gacutil” /i MongDB.Bson.dll

“C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\x64\gacutil” /i MongDB.Driver.dll

Note the “ quote marks around the path are important for the CMD window to correctly separate the gacutil program from the parameters.

Once that is done, create a new SQL Server Integration Services project in SQL Server Data Tools (SSDT), what used to be called BIDS in SQL Server 2008R2 (and previous). Put a Data Flow Task on the Control Flow design surface. Then open the Data Flow Task for editing.

Next, drag and drop a Script Component transformation onto the Data Flow design surface. When prompted, change the component type to Source.

image

Now edit the script transform by double clicking on it. Move to the Inputs and Outputs page. For my test, I am using the dbo.DimCurrency collection I created using the technique I documented in the previous post, Exporting Data from SQL Server to CSV Files for Import to MongoDB Using PowerShell ( https://arcanecode.com/2014/01/13/exporting-data-from-sql-server-to-csv-files-for-import-to-mongodb-using-powershell/ )

I renamed the output from “output” to “MongoDB_DimCurrency”. I then added four columns, CurrencyName, CurrencyAlternateKey, CurrencyKey, and ID.

image

Make sure to set CurrencyName, CurrencyAlternateKey, and ID to “Unicode string [DT_WSTR]” Data Type. Then change CurrencyKey to “four byte signed integer [DT_I4]”.

Now return to the Script page and click Edit Script. In the Solution Explorer pane, expand References, right click and pick Add Reference. Go to Browse, and navigate to the folder where the MongoDB C# drivers are installed. On my system it was in C:\Program Files (x86)\MongoDB\CSharpDriver 1.7\. Add both MongoDB.Driver.dll and MongoDB.Bson.dll.

image

Click OK when done, your Solution Explorer should now look something like:

image

Now in the script, expand the Namespaces region and add these lines:

using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Bson.Serialization;

Now scroll down to the CreateNewOutputRows() procedure. Here is a sample of the code I used:

public override void CreateNewOutputRows()
{
  string connectionString = "mongodb://localhost";
  string databaseName = "AdventureWorksDW2014";

  var client = new MongoClient(connectionString);
  var server = client.GetServer();
  var database = server.GetDatabase(databaseName);
  string CurrencyKey = "";

  foreach (BsonDocument document in database.GetCollection<BsonDocument>("dbo.DimCurrency").FindAll())
  {
    MongoDBDimCurrencyBuffer.AddRow();
    MongoDBDimCurrencyBuffer.CurrencyName = document["CurrencyName"] == null ? "" : document["CurrencyName"].ToString();
    MongoDBDimCurrencyBuffer.CurrencyAlternateKey = document["CurrencyAlternateKey"] == null ? "" : document["CurrencyAlternateKey"].ToString();
    CurrencyKey = document["CurrencyKey"] == null ? "" : document["CurrencyKey"].ToString();

    MongoDBDimCurrencyBuffer.CurrencyKey = Convert.ToInt32(CurrencyKey);
    MongoDBDimCurrencyBuffer.ID = document["_id"] == null ? "" : document["_id"].ToString();
  }

}

I start by defining a connection string to the MongoDB server, followed by the database name. I then create a MongoClient object. Note the MongoClient is the new way of connecting to the MongoDB server. In earlier versions of the C# driver, you used MongoServer objects.

I then cycle through each document in the collection “dbo.DimCurrency”, using the FindAll() method. For each item I use the AddRow() method to add a row to the buffer. In order to find the proper name for the buffer I went to the Solution Explorer and expanded the BufferWrapper.cs file. This is a class created by the script transform with the name of the output buffer.

image

For each column in my outputs, I map a column from the document. Note the use of the ternary operator ? : to strip out nulls and replace them with empty strings. String columns you can map directly from the document object to the output buffers columns.

The CurrencyKey column, being an integer, had to be converted from a string to an integer. To make it simple I created a string variable to hold the return value from the document, then used the Convert class to convert it to an INT 32.

Once you’ve done all the above, validate the code by building the code. If that all checks out save your work, close the code window, then close the Script Transformation Editor by clicking OK.

Now place a destination of some kind on the Data Flow. Since I have my company’s Task Factory tools I used a TF Terminator Destination, but you could also use a Row Count destination. On the precedence constraint between the two, right click and Enable Data Viewer. Execute the package, if all goes well you should see:

image

A few final notes. This test was done using a MongoDB document schema that was flat, i.e. it didn’t have any documents embedded in the documents I was testing with. (Hopefully I’ll be able to test that in the future, but it will be the subject of a future post.) Second, the key was the registering of the DLLs in the GAC. Until I did that, I couldn’t get the package to execute. Finally, by using the newer API for the MongoDB objects I’ve ensured compatibility for the future.

SSIS Training Resources

I’ve been asked to provide links to some useful resources for learning about SQL Server Integration Services. Below are a list of my favorite blogs, books, and other sites to learn from.

A quick disclaimer, some of the links below are by co-workers or other people I have an affiliation with, financial or otherwise. That’s because I’m lucky enough to work with some of the best people in the field. Also, in the case of the books I’ve linked to the Kindle version where possible, mostly because I’m a Kindle junkie. There are paper versions of the books, and you are free to buy from your favorite retailer.

 

Books

Microsoft SQL Server 2008 Integration Services: Problem, Design, Solution – I’m a big fan of problem / design / solution books. They do a great job of teaching you how to solve real world issues. The chapter by Jessica Moss on package configurations is especially useful, one I refer back to often.

Professional Microsoft SQL Server 2008 Integration Services – This is another of those big thick books (1000+ pages if you decide to lug around the paper version) that covers the gambit of everything you ever wanted to know about SSIS but were afraid to ask.

Knight’s 24-hour Trainer: SQL Server 2008 Integration Services – This is one time where I suggest getting the physical book, as it is accompanied by a DVD full of videos focused on SSIS. They will take you from an absolute beginner and walk you through each of the most used Tasks and Transformations used in SSIS. If you want to get up to speed fast on SSIS this is the book to get.

 

Blogs

Jamie Thomson – Better known as the SSIS Junkie, Jamie posts some really advanced topics on his blog.

Andy Leonard – Andy is a whiz at SSIS, and does a great job at both teaching and consulting.

Christopher Price – A very active blogger on SSIS.

 

Videos

Pragmatic Works Webinars – On our website we have a big catalog of past webinars (all of which are free to watch), many of which focus on SSIS. 

Pluralsight – Pluralsight has an extensive catalog of courses, including an SSIS course. It’s subscription bases so there is a modest fee (starts at $29 US per month last I checked) but well worth it for the training you can get. There’s also a free trial.

SQL Share – This site takes a new twist on videos, in that each video is very short and very focused on one specific task. Lots of good stuff on here on SSIS, much of it by Brian Knight. Free.

 

For a quick link to this post, you can use http://bit.ly/arcanessis.

The Great Pragmatic Works Task Factory Round Up

Over the last month or so, I’ve been blogging about my companies (Pragmatic Works) cool suite of SSIS (SQL Server Integration Services) add in tools called Task Factory. I did this mostly to learn about the various components, and decided to video the whole thing in order to a) help myself remember and b) share the knowledge with everyone.

I’m about to move onto some of our other tools, and as I explore and learn them will be adding videos as I go. I wanted to wrap up the Task Factory series with a summary of all the tools, and links to the blog post and the direct link to the video. Please note I’ve listed these alphabetically, to make them easy to find, instead of the order in which they were published. 

Task Factory Address Parse video | blog

Task Factory Advanced E-Mail and SMS Task video | blog

Task Factory Case Transform video | blog

Task Factory Compression video | blog

Task Factory Data Cleansing video | blog

Task Factory Data Validation video | blog

Task Factory Delete Batch Transform video | blog

Task Factory Dimension Merge Slowly Changing Dimension video | blog

Task Factory Email Source video | blog

Task Factory File Properties video | blog

Task Factory Null Handler blog

Task Factory Replace Unwanted Characters video | blog

Task Factory RegEx video | blog

Task Factory SalesForce.com Source and Destination blog

Task Factory SharePoint Destination video | blog

Task Factory SharePoint Source video | blog

Task Factory Surrogate Key transform video | blog

Task Factory Terminator Destination blog

Task Factory Trim Plus video | blog

Task Factory Update Batch transform video | blog

Task Factory Upsert Destination video | blog

Task Factory XML Destination video | blog

Task Factory–SalesForce.com Source and Destination

Over the course of the last month I have created video tutorials on the components that ship with Pragmatic Works Task Factory suite of SSIS components. There are two though, that unfortunately I’m not able to cover, but I did want to make sure everyone was aware of. Those are the SalesForce.com Source and Destination components.

These are quite similar to the SharePoint Source and Destination components I covered in previous posts. Here’s a screen shot of the Source properties window:

 

SNAGHTML6a6b6c3

 

As with most of the tools in Task Factory, you simply pick the connection manager, pick the object to connect to, supply and parameters, and pick what you need. Just that simple and straight forward.

The Destination component is just as easy, as you can see:

SNAGHTML6aafcf6

Again, simply create a connection, pick the target object, map the columns and away you go.

Since I don’t currently have a way to communicate with a SalesForce.com environment, as of this writing we’ll have to settle for a few screen shots. I believe though they will convey the simplicity of the product. Most importantly though you are now aware of these handy components, something you’ll find invaluable if you deal with SalesForce.com.

Task Factory–SharePoint Destination

In a previous video we looked at the SharePoint Source component, one of the many components in Pragmatic Works Task Factory suite of SSIS tools. We saw how easy it was to extract data out of a SharePoint list.

What though, do we do if we need to push data into a SharePoint list? It turns out it’s every bit as easy, using Task Factory’s SharePoint Destination component.