Chris Ware is a Principal Consultant at iOLAP, Inc., with a focus on ETL. He began his career providing analysis support and reporting in marketing and then insurance for a major retailer. Chris has had the opportunity to work within many different verticals throughout his data warehousing career using a variety of different integration tools. He feels that because data integration is ever changing that the opportunity to employ new processes is endless.

When you first read this title you probably think these are instructions for someone else—but not you—right? But read on—see where you’re at when it comes to the issues of talking, listening, moderating and overall meeting etiquette.

Within SSIS a FTP task exists which enables you to access a FTP server. However, it does not support Secure FTP (SFTP). There are many Secure FTP tools to choose from and each requires a unique set of commands and designs to be used within SSIS.

For this article, I am using Putty.org’s file transfer tool commonly known as PSFTP (a free open source tool). This tool performed as needed and we were able to build additional processes around it to enhance its use and satisfy our customers’ requirements.

So what exactly is Big Data?

In the real world view, Big Data is the culmination of several years’ worth of data that your company has stored in their data warehouse as instructed by their DBA since, well, forever. This data that has been archived in different locations for safe keeping, and possible later use, is extremely valuable for marketing, sales and other decision makers in your organization.

The official Wiki definition of Big Data is: “a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, transfer, analysis, and visualization.” You will see that definition used in a lot of places.