Snowflake Cost Optimization
One of our clients had migrated to Snowflake from a legacy platform. While the migration led to decreased overall costs, as they migrated more workloads they realized that they costs were scaling linearly. DataGrokr was asked to evaluate and implement various cost optimization opportunities utilizing Snowflake best practices. We were able to implement changes to optimal warehouse sizing, data compression, archiving and retention policies, task scheduling, usage of transient tables, etc. and helped realized monthly saving of more than $50K.
About the Client
Our client is a prominent player in the US grocery industry, ranking among the top 5 grocery chains. With an extensive network of over 1500 stores, they have annual revenues exceeding $60 billion. They have 30+ million active customers and manage more 100,000+ items. The size of their data warehouse is 180TB, with an annual growth rate of 10TB, encompassing both customer and product information.
Client’s need and Problem statement
The client was looking for reducing their Snowflake data warehouse spend as their spend continued to increase as they began migrating more workloads into Snowflake. They were looking for resource right-sizing and automation to bring down resource usage and overall costs. We were given a mandate to evaluate, recommend and implement changes that will reduce spend but at the same time maintain similar levels of performance without degrading functionality or user experience.
Tech Stack
SnowFlake
SnowPipe
Tech Stack
Our solution and outcomes
- We analyzed the query workloads and the utilization of the different Snowflake virtual warehouses and were able to bring down size of the warehouses and minimize over-provisioning.
- Auto-suspend and auto-resume policies were implemented on warehouses that were being used for batch workloads and those that were mainly used during office hours by end users.
- Data compression was implemented on large transaction and log tables which helped decrease storage costs.
- Adjusting time travel fail safe settings was another lever that was used to strike a balance between data recovery needs and costs.
- Overall, through a combination of various strategies we were able to realize monthly cost savings of more than $50K.
