SQL Server Business Intelligence

SQL SERVER 2005 & 2008 IS THE MAIN THEME OF THIS BLOG

Thursday, January 26, 2006

Steps Involved in Loading Text File Into SQL Server Using SSIS

How many steps involved in loading a simple text file straight into a SQL Server table using SSIS?

6 steps involved to complete this process and they are listed below:
1. Read data from the text file – Disk speed is significant.
2. Load data from the text file into the transport (SSIS) – Network speed is significant.
3. Open a transaction on SQL Server – Server contention is significant.
4. Pass data to the SQL Server interface
5. Load the data into SQL Server – Memory resource is significant.
6. Commit the transaction – Disk performance is important.

When loading from text to typed SQL Server columns (int, datetime etc) the data conversion must take place somewhere either step 2 or 5 in this above example. Is it necessary to use SSIS for this? Are the operations simple enough for BCP / BULK Import? With low volume of data the overhead of starting up SSIS package process may offset any performance gain over BCP or Bulk Insert. Where should SSIS run (Server1 or Server2)? SSIS On Server1 – Competes for memory and processor resources with other applications. Will data conversion on server1 reduce or increase the volume of data transferred across the network? But can not use the fast SSIS SQL server Destination. Where should SSIS run (Server1 or Server2)? SSIS On Server2 – Competes for memory and processor resources with SQL Server. Will pulling text over conversion be expensive (also consider transferring the file unparsed to server2 and read it locally from there). You can use the fast SSIS SQL Server Destination. If you were going to run in two servers you would need SQL Server licensee in both servers.

Thanks
Sutha

0 Comments:

Post a Comment

<< Home