ETL

SSIS 使用EventHandler处理错误 Error Handling using Event Handler

Categories: Database; Tagged with: ETL • SSIS; @ October 29th, 2012 12:02

需求：

某组件/package发生错误时进行发送邮件/通知等响应操作

解决：

1. 配置组件/package：必要时配置FailPackageOnFailure = true/ FailParemtnOnFailure=true；

2. 创建Handler：切换到EventHandlers 页面，选择Handler的执行范围，拖拉控件进行设计。

SSIS Difference between OLE DB Source & DataReader Source, OLE DB Destination & SQL Server Destination

Categories: Database; Tagged with: Database • ETL • SSIS; @ October 29th, 2012 11:41

OLE DB Source and DataReader Source (formerly known as ADO.NET source)

The OLE DB source and ADO .NET source are so similar in fact, it is forgivable to ask why they are even separate components. The simple(ish) reason for this is that the two components talk to their underlying data sources in very different ways – the OLE DB source will talk directly to relational OLE DB compliant sources, but the ADO .NET source goes through a layer of abstraction so it can talk to the source through a .NET provider. If all that sounds like gibberish, don’t worry. You only need to consider using the ADO .NET source in two cases. Firstly if no OLE DB provider is available – e.g. if you need to talk to a SAP BI instance. Secondly if you need to access the source through a Script Task.
from:http://www.bimonkey.com/2009/05/the-ado-net-source-and-sql-in-the-script-task/

And I think the difference between destinations should same as source.

OLE DB Destination v.s. SQL Server Destination

The SQL Server destination connects to a local SQL Server database and bulk loads data into SQL Server tables and views. You cannot use the SQL Server destination in packages that access a SQL Server database on a remote server. Instead, the packages should use the OLE DB destination. __http://msdn.microsoft.com/en-us/library/ms141095.aspx

SQL Server Destination

The SQL Server destination is the fastest way to bulk load data from an Integration Services data flow to SQL Server. This destination supports all the bulk load options of SQL Server – except ROWS_PER_BATCH.

Be aware that this destination requires shared memory connections to SQL Server. This means that it can only be used when Integration Services is running on the same physical computer as SQL Server.

OLE DB Destination

The OLE DB destination supports all of the bulk load options for SQL Server. However, to support ordered bulk load, some additional configuration is required. For more information, see “Sorted Input Data”. To use the bulk API, you have to configure this destination for “fast load”.

The OLE DB destinationcan use both TCP/IP and named pipes connections to SQL Server. This means that the OLE DB destination, unlike the SQL Server destination, can be run on a computer other than the bulk load target. Because Integration Services packages that use the OLE DB destination do not need to run on the SQL Server computer itself, you can scale out the ETL flow with workhorse servers.

— http://msdn.microsoft.com/en-us/library/dd425070(SQL.100).aspx

Using for loop in SSIS

Categories: Database; Tagged with: ETL • SSIS; @ October 25th, 2012 18:07

Like for loop in any language, the for loop component in SSIS has three for loop parameters:
InitExpression, EvalExpression, AssiognExpress.

Step1: create a new variable: Counter, int;
Step2: Drug and drop a for loop component, set the properties:

Step3: Drug a ‘Script’ component into the for loop, codes: we’ll add 1 every time.

Dts.Log("Counter: " + Dts.Variables("User::Counter").Value.ToString, 0, emptyBytes) Dts.Variables("User::Counter").Value = CInt(Dts.Variables("User::Counter").Value.ToString) + 1

run, the package, check the log:

User:ScriptTaskLogEntry,…,25/10/2012 6:02:51 PM,25/10/2012 6:02:51 PM,0,0×00,Counter: 0
User:ScriptTaskLogEntry,…,25/10/2012 6:02:52 PM,25/10/2012 6:02:52 PM,0,0×00,Counter: 2
User:ScriptTaskLogEntry,…,25/10/2012 6:02:52 PM,25/10/2012 6:02:52 PM,0,0×00,Counter: 4

Ref: http://www.julian-kuiters.id.au/article.php/ssis-how-to-configure-for-loop-control

BIDS: Sorting files in SSIS/DTSX projects

Categories: Database; Tagged with: BIDS • ETL • SQL Server • SSIS; @ October 17th, 2012 16:42

It’s really ridiculous that we cannot sort the files in SSIS project in BIDS.
fortunately, we got a plug-in named ‘BIDS Helper’, which has a feature called ‘Sort Project Files’:

http://bidshelper.codeplex.com/

Sort Project Files

This feature adds a “Sort by name” menu option to the “SSIS Packages” folder of an SSIS project in Visual Studio:

This feature is implemented natively in BIDS in SQL2008.