info@techdevops.com | 437-991-3573 | Data Engineering Services
TechDevOps.com
Resources Tools
Experts in Microsoft SQL Server on Windows, Linux, Containers | Clusters, Always On, FCI | Migrations, Cloud, Performance



Migrating a Giant to the Cloud
by BF (Principal Consultant; Architecture; Engineering)
2018-11-20








Migrating a Giant to the Cloud


A popular, online, public-facing Website with millions of searches per day & a 24/7/365 up-time service-level. Aka – “The Giant”. The IT directive to us was to move away from an on-premise “source” infrastructure comprised of VMWare, EMC SAN, Web Servers (IIS) and Microsoft SQL Server Databases, that was being hosted in multiple, local data centers. My role in this project was Lead Database Operations in charge of migrating the complete data-tier infrastructure into Microsoft's Azure Public Cloud Platform. A major Cloud Migration project was the task.

The first step in this Cloud Migration project, for the core data-tier infrastructure, was to analyze the full list of SQL Server offerings that Microsoft Azure had for it's SQL Server Application. Following a few days of research it was clear there were two main “target” platforms: (1) SQL Server in Azure Virtual Machines (IAAS) or (2) Azure SQL Database (PAAS). Note - Today there are 3 choices w/ the addition of Azure SQL Managed Instances. The first offering - SQL in VM - is the full Microsoft SQL Server Application running in a Virtual Machine which allows for a any configuration option and, thus, a very attractive technical solution. When using SQL Server in VM, you can either pay-as you-go for a SQL Server license already included in a SQL Server image or easily use an existing license. The second offering - Azure SQL Database - is a Database As-A-Service (Managed Service) Platform which presents a database to manage, residing on a logical server, and Microsoft takes care of the remaining management items – i.e. High Availability, Disaster Recovery, Automated Patching, Automated Backups, Automated updates to SQL Server version. Azure SQL Database eliminates hardware costs, reduces administrative and is available in several service tiers with different prices for resources, all of which are billed hourly at a fixed rate based on the service tier and compute size you choose. Given those two different data platforms to choose from and even though the PAAS Platform offered many of the technical items we were interested in, we eventually landed on the SQL in VMs solution as the option to configure & tune the SQL Server Engine from Server-Level to Database-Level was something we needed to maintain on any platform. Decision made.

After a couple weeks of planning, the Cloud build started and an initial Hybrid approach was implemented using SQL Server 2016 Standard Transactional Replication Publisher (mission-critical system), hosted on-premise, updating data real-time to numerous SQL Server Replication Subscribers located in multiple, physical, separate regions within Microsoft Azure Cloud. The Hybrid Model (HM) worked extremely well and allowed us to obtain not only a feel for Cloud Computing, but also a huge benefit with the Disaster Recovery requirement being met with the Hybrid Model. With the HM in place, we were able to use Azure Traffic Manager to distribute a percentage of the Website traffic to on-premise servers and a percentage to Azure severs as we closely monitored the actual level of performance we were getting in Azure. We saw *immediate* performance gains with using the Azure Cloud IAAS Infrastructure. Azure also allowed us to quickly scale vertically with Virtual Machines, based on the load changes, and this was critical to us achieving a consistent, high-level of full application performance. The Cloud Migration was off to a good start.

A few months into using the Hybrid Model, we then planned for a migration of the main, mission-critical on-premise SQL Server Publisher Database Server to Azure – comprised of ~ 1TB of data in 15-16 databases total. A super-fast on-premise network infrastructure allowed us to use a Backup & Restore strategy, leveraging Azure Cloud Blob Storage as a staging data location, to migrate this final infrastructure piece over with no issues seen during this process. The Replication Subscribers remained online serving traffic throughout this process and thus zero downtime for the public Website. A couple hours of no replicated transactions were agreed upon and we managed to stay under that timeline.

Today, being 100% in the Azure Cloud for all application & data-tier infrastructure, the main SQL Server Database Publisher, configured with Windows Server Fail-over Clustering running Always-On Availability Groups, a Remote Replication Distributor replicating data to the same Azure Subscribers that were pre-migrated, is able to deliver the fastest data processing times for our data loading applications with flawless replication out to the search servers. The Website is running smoothly with the highest performance levels seen and up-time is at a new high with over a year(and counting) of no outages. The project was a huge success.


Below is a list of the main Azure Cloud services used for this project:

- Subscriptions for Development, Testing & Production Environments
- Resource Groups holding similar cloud-based resources (Web, DB, Network, Storage)
- Virtual Private Networks that bridged on-premise and cloud networks
- Traffic Managers
- Load Balancers (regional)
- App Services (Web)
- SQL Server in Virtual Machines
- Network Security Groups
- Storage Accounts

Azure Services List


Things I learned:

- The Hybrid Model is a great strategy to successfully move into Cloud Computing.
- Powershell is a great tool for automating overall Cloud Management.
- Microsoft Azure delivered strong performance, capacity, scale, security & the always important - availability.
- Detailed & consistent resource naming conventions is important for implementation & resource management
- Microsoft Azure led to an IT Budget savings as we no longer had to pay for any on-premise data center Infrastructure.


The Cloud is a good place to be. Save money. Less Outages. More sleep.







Resources:

Microsoft Azure

Choose a cloud SQL Server option

Azure Status