SSIS to Stored Procedure with Table Variable Parameter? Really? Can you do that?

There is a new 2008 feature that enables you to pass a table variable.  It’s a great feature with a few limitations.*  To take this feature further I wanted to pass in a table variable from within an SSIS package.

A.  BACKGROUND:
Table Variables have been around a while, but in 2008 you can use them as parameters.  No more comma separated strings and parsing them with functions.  No more staging tables, custom clrs, tricky uses of the file system and no more  bulk insert bloatedness.  Scratch all that heavy lifting.  You can now use a table variable as a parameter.  With ssis, touting its versatility I knew there had to be a way to do this.  My situation: I had an ‘upserting’ scenario (insert unless data is there, then update) and I didn’t want to evaluate and execute one row at a time into my tables.  I created a a stored procedure that was using the new MERGE keyword and it’s only parameter was a table variable.  It was perfect…all new technology to feed my ego.

For our example create a table called PersonTestMC

CREATE TABLE [dbo].[PersonTestMC](
    [PersonFirstName] [varchar](50) NULL,
    [PersonLastName] [varchar](50) NULL
)

B.  HOW TO SET UP A TABLE VARIABLE PARAMETER:
There are plenty of posts about this but here’s the short version.  You have to create a User Defined Table type that is the structure for your table variable.  So step one is to create this type:

CREATE TYPE [dbo].[PersonType] AS TABLE(
    [personfirstname] [varchar](50) NULL,
    [personlastname] [varchar](50) NULL
)

If you are looking for your object it can be found in the Programmability.Types.User-DefinedTableTypes.  There isn’t a pretty gui for it, so if you need to modify your changes drop and re-create.  Step two is to create a stored procedure to accept this type as a parameter.

CREATE PROCEDURE [insertPersons]
(@Persons PersonType READONLY )
AS
BEGIN
SET NOCOUNT ON;
insert into PersonTestMC
select * from @Persons
END

Note that your parameter is read only.  Don’t freak out, you’ll figure out how to get around this one.  If you want to read more on setting up/creating/using a table variable parameter try this or this. 

As an aside, if you want to call this from another stored procedure you have to use your new type in place of a table variable:

Declare @mytable PersonType
insert into @mytable select ‘firstnametest’, ‘lastnametest’
execute insertpersons @mytable

If you want to read more about this try this post from Andras.

C.  Calling it from SSIS:
You need a source.  I used an oledb connection to the Adventureworks Person.Contact table.  But any source with a first name and last name will do.  So follow these steps

1. Create a data flow component
2. Inside that component create your source
3. Create a Script Component (of destination type).
   1. Add your input columns
   2. Make sure you add a Connection Manager in your Script Component and name it something you will remember (adoAdventureWorks for our example). 
   3. You are almost ready to edit your script.  Before you click ‘Edit Script’ make sure to pick your Script Language (otherwise you will have to delete and re-create the script component like I have to do every time).  It defaults to C#.  Change it to ‘Microsoft Visual Basic 2008’.
   4. Click Edit Script.

Two things to be aware of in this VSTA environment. 

1. There is no need to save anything.  If you type something and hit the magic X button to close the app then it will save all your changes. 
2. You cannot stop/debug your code in the Script Component (unlike the Script Task object available in the Control Flow tab).  Just use the old fashioned msgbox method that we used rely on.
3. If you have variables that you need then you will have to take an extra step here (You can access data from our source object but variables are different).  You will have to place those in the ReadOnlyVariables or the ReadWriteVariables property of the script component.  Otherwise your attempt to access those variables will come up empty.  We don’t need them for our example.
4. There are three events to be aware of: pre-execute, post-execute, and a processInputRow.  The ProcessInputRow event sets us up for transformations and in-row processing. 

Import the system.data.sqlclient to support our laziness
     Imports System.Data.SqlClient

Create a private variable at the class level:
     Private dt As DataTable

PRE-EXECUTE EVENT:  Initialize your datatable and add your columns in the pre-execute event

     dt = New DataTable
     dt.Columns.Add("firstname")
     dt.Columns.Add("lastname")

PROCESSINPUT-ROW EVENT:  Ideally we wouldn’t have to do this but there isn’t a dt.load command that can load the data from the entire collection of data.  However, processing it here is much quicker than executing a stored procedure at the row level, which is one of our bad alternatives

Also, note below the line that checks for the null value.  An extra field is created that you can check to see if the value is null before attempting to use it.  You might not need it, but it is there just in case (for some reason setting the AllowDBNull on the column didn’t resolve this little issue)

     Dim row2 As DataRow
     row2 = dt.NewRow()
     row2("firstname") = If(Row.FirstName_IsNull,””,Row.FirstName)
     row2("lastname") = if(Row.LastName_IsNull,””,Row.LastName )
     dt.Rows.Add(row2)
     row2 = Nothing

POST-EXECUTE EVENT: That’s it you’ve made it.  This is the easy part.

     Dim connMgr As IDTSConnectionManager100
     Dim sqlConn As SqlConnection
     Dim sqlCmd As New SqlCommand

     connMgr = Me.Connections.adoAdventureWorks
     sqlConn = CType(connMgr.AcquireConnection(Nothing), SqlConnection)

     sqlCmd.CommandType = CommandType.StoredProcedure
     sqlCmd.CommandText = "insertPersons"

     sqlCmd.Parameters.AddWithValue("@Persons", dt)
     If sqlConn.State = ConnectionState.Closed Then sqlConn.Open()
     sqlCmd.Connection = sqlConn
     sqlCmd.ExecuteNonQuery()
     If sqlConn.State = ConnectionState.Open Then sqlConn.Close()

*Limitations of the new table parameters:

• It’s read only
• You cannot drop or alter it once it’s referenced, so get it right the first time or you will have to alter your procedure to not use it first.
• You can’t alter it (just drop and re-create)
• You cannot pass it to to user-defined functions.
• No stats are maintained.