Working as a BI Developer with the Microsoft stack of tools, I often handle complex business rules/functionality within stored procedures using temp tables and call it in SSIS within Execute SQL task (or) in OLE DB source components within the dataflow task depending on the requirement. As you know, most companies are slowly but surely upgrading from SQL Server 2008 to SQL Server 2012/2014 and this blog is for those who are facing some problems with reading column metadata when calling certain stored procedures in SSIS and discusses some possible solutions.
Scenario:
You may have come across the following errors in the OLE DB source component of SSIS 2008 & SSIS 2012 if a stored procedure contains Temporary tables (OR) Dynamic SQL which is driven by input parameters.
“The metadata could not be determined because statement ‘SELECT * FROM #TempTable’ in procedure ‘P_Procedure1′ uses a temp table.”.
Figure 1: OLE DB Source Component Error message when the source contains temp table
“The metadata could not be determined because statement ‘EXEC (XX)’ in procedure XX contains dynamic SQL. Consider using the WITH RESULT SETS clause to explicitly describe the result set.”
Figure 2: OLE DB Source Component Error message when the source contains dynamic SQL
SSIS 2008 Options:
Here are 3 possible solutions to this problem that I can think of while calling stored procedures within the Data flow task in BIDS 2008.
1. Expose the column metadata through a dummy SQL statement at the beginning of the stored procedure and call it in SSIS OLE DB source component without the SET FMTONLY OFF option as shown in Figure 3.
2. Use SET FMTONLY OFF option to retrieve the column metadata along with the data & set the DelayValidation to ‘TRUE’ on the data flow task & ValidateExternalMetadata to ‘False’ on OLE DB source component. Still, this approach will result in poor performance. See http://www.sqlservercentral.com/articles/Integration+Services+%28SSIS%29/61824/ for more information.
3. Use Table variables instead of Temporary tables as shown in Figure 3. This may perform slow.
SSIS 2012 Options:
So, all 3 approaches work fine SSIS 2008. Approaches 1 & 2 don’t work in SSIS 2012. You need to explicitily describe the result set metadata using WITH RESULT SETS clause in order for the Approaches 1 & 2 to work in SSIS 2012 as shown in Figures 3.
Summary:
On investigation, I found out that, SSIS 2008 by default uses the Native OLE DB\SQL Server Native Client 10.0 driver for connectivity to SQL Server 2008 database, & SSIS 2012 uses the Native OLE DB\SQL Server Native Client 11.0 version for connectivity to SQL Server 2012. Internally SSIS uses the sp_prepare stored procedure to get the column metadata and apparently sp_prepare returns (or does not return) metadata depending on the sql server version. In SQL Server 2008 R2 and earlier versions, SET FMTONLY ON is used in order to return the metadata to the calling application but this feature is deprecated in SQL Server 2012. Refer to this blog from Microsoft for more information.
http://blogs.msdn.com/b/psssql/archive/2013/07/23/when-does-sp-prepare-return-metadata.aspx
I hope this helps resolving column metadata detection issue you are facing while calling stored procedures within the data flow task in SSIS.
Please feel free to comment on the post with your experiences.
The post SSIS Data Flow Task not reading columns correctly from stored procs? 3 possible solutions appeared first on Concentra Blog.