Blog
How to Realize Your File Requirements in Azure - Part 2
Enterprise Database Applications – Challenges in the Cloud
In order to meet mandates to get core application environments to the cloud, there comes the need to have a level of performance that is often difficult to achieve with today’s highly flexible cloud resources. Where compute scales with technology, having highly performant data access is a serious challenge.
Robust, high-performing, and scalable storage in the cloud is essential for deploying enterprise applications, which often have key performance requirements of some form of database. Reliable storage and retrieval of data across hundreds or thousands of concurrent client connections are paramount. Large organizations rely on their databases to be the permanent systems of record for all business transactions, which means that the data they contain must survive localized server and disk failures, as well as any site-wide failures that necessitate disaster recovery. As data volumes grow, database administrators require the flexibility to quickly scale up both the size and performance of database storage volumes to meet demand.
The following list summarizes the most crucial features that the storage environment must support:
Reliability. Because the storage environment is an integral part of the database platform, it must be available for sustained access by database servers, without fail. If access to the storage is interrupted, database operations might come to a halt, potentially causing a major disruption for all dependent applications and systems.
Durability. When a database system is ACID-(atomicity, consistency, isolation, and durability) compliant, it means the database system can guarantee that when a database transaction has been committed, the data is durable; that is, it will survive a failure. Storage environments must ensure adequate data redundancy to protect against failures.
Performance. Because almost all database operations involve reading or writing data, I/O performance determines the speed at which a database can operate. Ideally, the storage environment provides the ability to allocate storage of varying performance. This gives database administrators the flexibility to match storage capacity to performance requirements in order to stay cost effective.
Security. Organizations with strict requirements for data storage require features such as encrypted transport and data encryption at rest. These features are necessary to protect sensitive data, such as personal, financial, and healthcare information.
Data Protection. Due to the criticality of data for an enterprise application, there is a need to keep that data protected. Leveraging the capabilities of storage infrastructure to take a snapshot copy and then restore these copies is a requirement for backup and restore operations to prevent inadvertent data loss and data corruption.
Rapid Test/Development Access. A rapid copy, or clone, of data is often required to perform testing, such as for a database upgrade or an application update and deployment. Being able to clone existing volumes and quickly create temporary, writable, and up-to-date copies of large production databases, without any adverse effects to the live environment, is a big win for DevOps engineers and database administrators. Software developers with access to database test environments are able to accelerate the development and testing of new application features, ensuring faster time-to-market (TTM).
Enterprise Database Applications with Azure NetApp Files
Azure NetApp Files delivers flexible and ready-to-use file services in the cloud without administration overhead. With Azure NetApp Files, users can simply set up a data volume in seconds with the availability and performance their business-critical applications require. As well as providing a powerful solution for creating and managing cloud file services, Azure NetApp Files offers an additional set of storage management features that simplify many common database administrative tasks.
Enterprise Applications can realize the following benefits:
Snapshot copy and restore. Unlike any other cloud snapshot capabilities, NetApp Snapshot technology offers instant creation of a snapshot copy of a volume without impacting the performance of the application. As well, the snapshot itself is extremely efficient and takes no additional capacity. The volume can then be instantly restored back to the point in time when the copy was created, whenever it is needed. To create a consistent Snapshot copy, NetApp recommends that users first quiescence their database system to ensure that in-flight I/O operations have been completed.
Storage cloning. Azure NetApp Files can quickly create writable copies, or clones, of existing data volumes. This is especially useful for database administrators who need the flexibility to rapidly create database test environments without the overhead of manually copying large volumes of data.
Replication and data synchronization. Leverage NetApp replication and synchronization services that can read data from any dataset, whether cloud-based or on-premises, and incrementally synchronize them with your volume. Synchronization can also be performed in the opposite direction, out of the cloud volumes, to other locations.
“Azure NetApp Files delivers flexible and ready-to-use file services in the cloud without administration overhead.”
Using Azure NetApp Files makes it easier to manage cloud storage deployments by providing the following tools and support to cloud architects and database administrators:
Support for all major database platforms. Reliable database storage for all major database systems, including SAP, Oracle, PostgreSQL, MySQL, MongoDB, and Microsoft SQL Server (over SMB). Oracle Direct NFS is able to open multiple, parallel client sessions to NFS shares to further increase I/O performance and scalability.
Configurable Service Levels. Storage is allocated in accordance with the service level defined when a volume is created. The service level can then be changed on demand to best suit the user’s needs. This allows the performance of a volume to be controlled and storage pools of varying size and performance to be made available to a database system. Users can control cloud storage costs by allocating faster storage only where necessary.
Robust data protection. Azure NetApp Files is a cloud-native service for allocating storage in the cloud and therefore requires no user management of underlying resources. Because volumes are highly available, customers can be sure that their data will be durable and online when they need it. They can also use built-in replication capabilities to set up secondary regional copies of their data for enhanced protection.
Data encryption. All data is encrypted at-rest, and users can encrypt data both at-rest and in-transit by using a VPN. For NFS and SMB connections, users can encrypt data-in-transit from the database server to the storage volume. This provides transparent protection from malicious attempts to access the data.
Scalability. Volumes can be expanded on-the-fly, whenever more storage is required, without compromising performance or data protection. Database administrators can also reduce the size of the volume as needed to use their resources better. They can easily allocate storage for new databases or existing databases that are growing in size, without the need to manage any of the underlying physical infrastructures. This substantially reduces the administrative overhead of reacting to a change in database storage requirements.
“Using Azure NetApp Files makes it easier to manage cloud storage deployments...”
Conclusion and Next Steps
Database systems are complex enterprise applications that depend heavily on the I/O systems they use. For the best results, storage services must combine performance, data protection, scalability, security, and flexibility into a single solution.
High-performance, scalable, and highly available shared file storage is crucial to delivering a data analytics platform. The ability to effectively manage data from multiple source systems can be another major obstacle. Azure NetApp Files provides cloud-based file service solutions that address the major challenges in creating a repository for data analytics workloads and can be used with custom-built Apache Hadoop clusters or public cloud analytics services.
Azure NetApp Files has been purpose-built to deliver the highest levels of I/O performance and scalability. End-users simply input the size of storage volume they need, choose the appropriate service level for their performance requirements, and NetApp takes care of the rest. This removes the significant burden on organizations to manage in-house NAS solutions.
The synchronization capabilities of Azure NetApp Files allow data from multiple systems to be consolidated into a single storage volume. Data can also be synchronized out of Azure NetApp Files to provide integration with other external systems. Volume cloning adds to the ability to manage and work with large volumes of data.
Deciding whether Azure NetApp Files is right for you is simply because it is a service you can easily run from your Azure portal. For instructions on how to get started, contact us an email: [email protected]