Migrating to the cloud is like moving to a new house. Where do you move to? What do you bring, what do you get rid of? Start fresh, or drag stuff you’re still using with you? Good questions. One question I get when moving a Microsoft Data platform to the cloud is: what tools do we use there? The classic Microsoft answer is: it depends. If starting completely fresh, you have a wide range of options and can target implementing a modern cloud architecture immediately. If you’re moving legacy environments, you’ll be constrained by the systems and business operations you have to support and move but will have the advantage of eventually moving to a modern cloud architecture down the road. If you’ve made a significant investment in, or possess the most knowledge of, a tool set and architecture—such your ETL architecture with SQL Server Integration Services (SSIS), there’s a good chance you will have to port the existing infrastructure to the cloud.
We’ll start focusing more on SSIS, but—as with looking to move to a new house—a lot of questions need to be asked first. These are common to most toolsets. Some key questions are as follows:
If you’re rearchitecting your infrastructure, you’ll be evaluating everything. As with moving, you’ll throw stuff out, give stuff away, and box up what you need or feel you can’t leave behind. You’ll look at systems, toolsets, and new capabilities across the board to determine what should go, what can be left behind, what has to be rewritten, and where re-platforming should occur. This is a good opportunity to review and remove code/processes not utilized or do some lightweight rewriting to reduce or streamline certain parts of your code base—or re-engineer, if justified.
Most of the time, legacy environments will need to be moved. Hardware/data center costs are often the key driver in the move. For example, my CIO said leases are coming due for our servers and I need to move before the leases are up. An uplift scenario, or a partial uplift, is the most common scenario I see for supporting existing business operations. Position ourselves in the cloud as painlessly as possible and place us in a position to phase-in cloud capabilities without disrupting (or minimizing the disruption of) the business. Sound familiar? If it’s a complete rewrite, then, options abound but there are still time pressures. At some point you will have to cut the new environment over to production with minimal disruption to the business. And sometimes, you will have to just start small and trial-out scenarios that might work best for you. You will determine these based on your needs, and on what you can support.
So, if we’ve decided to go with the scenario of uplifting our existing ETL infrastructure—or the ETL that I need turns out to be so small that it doesn’t justify coming up on a new toolset—I can still look to leveraging SSIS as a toolset. New tools and capabilities will be available and coming into the mix, but the impact on how you phase them in is minimized by leveraging existing toolset knowledge, initially. I have used SSIS for 16+ years and love using it but find myself chomping at the bit for the new cloud capabilities, such as Polybase and Kafka, and repurposing/limiting SSIS usage moving forward when the code is ported. Once we’re in the cloud, you can immediately start bringing in new capabilities in a composite architecture for your ETL infrastructure and start sunsetting portions of your SSIS usage.
SSIS in the cloud. What options do I have?
I would not be reluctant to migrate your SSIS to the cloud. However, with a change in architecture comes many questions. What is the long-term direction of my toolsets and how will the tools map into my strategic direction? What is the effort and cost? TCO is the final justification. If you’re re-platforming an existing ETL infrastructure to the cloud, I would recommend doing so in manageable efforts. Get the cloud infrastructure, and your presence in the cloud in place. Then move your SSIS there, make changes where appropriate (or required), and re-cast your business infrastructure when re-platformed. This way you can manage changes without significantly impacting your business. Disrupting your mission-critical business operations is the last thing we can justify. However, once you’re in the cloud, the tools and capabilities available to you are staggering. So, how are you planning to move your ETL/SSIS to the cloud? For help answering this question, please feel free to reach out to us at any time. We’d love to help you get started.
Business Intelligence Architect
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie | Duration | Description |
---|---|---|
cookielawinfo-checbox-analytics | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics". |
cookielawinfo-checbox-functional | 11 months | The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional". |
cookielawinfo-checbox-others | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other. |
cookielawinfo-checkbox-necessary | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary". |
cookielawinfo-checkbox-performance | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance". |
viewed_cookie_policy | 11 months | The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data. |
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.