Modern analytics: Cost savings, speed to market and new capabilities.
The cloud has delivered significant value – cost savings, speed to market and new capabilities, but getting access to data for analysis and analytics is a challenge. Analysts are downloading reports and files, but it isn’t working and it feels like you are flying blind as you struggle to understand your business. Meanwhile, IT is tied up with software updates and support, managing the complexities of the traditional environment and they don’t have the budget, skillset or experience to transition to a modern platform and don’t know where to begin.
They’re limping along with a patchwork of cumbersome processes that take too much effort to support, fail too often to be dependable and don’t meet the standard of excellence you know is required. You can’t solve the challenges created with modern applications with old tools and approaches – It costs more, doesn’t work and creates as many problems as it solves. Can you bring the benefits of the cloud to your analytics environment?
Quantified Mechanix
This isn’t a question of inventing or innovating your way out of the mess, you just need an approach tailored to meet the challenge. That is what we do.
We help our clients build a sustained competitive advantage with a modern and sophisticated analytics platform that delivers faster time to insight, lower operating costs, increased revenue and profitability, with advanced capabilities. Once you see it, the solution is actually easier, costs less and delivers more. Imagine incorporating data from a brand-new application on a dashboard 20 minutes after you signed up and went live, where analysts have near real-time access to data from all of your SaaS applications, and IT was able to give it to them in hours or days, not weeks or months. It may sound too good to be true, but it isn’t. A reliable, secure and high-performance platform exists that is low cost, easy to administer, easy manage and support, and easy to start. It’s even easy to quit if necessary.
Guiding Principles
Users have very high demands and expectations for ease of use, reliability, performance and security. Traditionally for many organizations, this meant complex configurations, significant hardware and an array of niche applications, stitched together something like the diagram below.
In the old days, this meant months of planning, installing and configuring, and hundreds of thousands, maybe millions of dollars building the infrastructure and even more supporting it before the first reporting or analytics application was deployed. Meanwhile, everyone struggles to develop and retain the staff with the expertise demanded by these complex environments. If you consider the uncertainty and risk inherent to analytics projects, it’s a testament to the potential value of analytics that anybody assumed the risks to build them.
SaaS Revolution
First virtualization, then Software as a Service (SaaS) came along to simplify these environments. Key requirements like managing performance, reliability and failover have been ‘outsourced’ to application providers who have designed applications and infrastructure at a greater scale. The results have been revolutionary. Applications perform better, have more capabilities, they are more reliable and they cost less, a lot less. IT no longer spends weeks and months planning upgrades, patches and disaster recovery and applications are continuously improved, updated and augmented.
Big SaaS Benefits, New Challenges
But extracting data from SaaS applications and integrating it with data from other applications is challenging, cumbersome and fragile. Data is not readily available from some applications, while others provide clunky methods that require manual interventions. It’s not easy to create reliable data flows and provide access to trusted data to your analytics users. Some users are downloading files and incorporating that data into spreadsheets. Others download reports from various sources and combine and use data incorrectly. Some users are typing data into spreadsheets. If you are doing any of these things and asking yourself if this is how it should be done, the answer is No!
Analytics on a SaaS Platform
Quantified Mechanix has developed an approach and platform that makes all of this easy. This platform lives in the cloud, is easy to use, low cost to operate and provides a number of benefits that are important, including:
- Reliable and robust data extraction pipelines that deliver data from cloud and on-premise applications in near real time.
- A data lake that manages history, handles schema changes and isolates downstream applications from disruptions.
- Access to the data lake for near real time reporting for cloud and on-premise applications.
- A simple environment and approach for data engineers to transform and integrate data.
- An industry standard data layer that performs well with any reporting and analytics applications.
Approach
We use FiveTran to make connections to 150 popular cloud applications, 20 common dataset platforms and deliver data to a data lake on SnowFlake. We create access to data from other applications through API’s where there are no built-in connectors or standard database access. Fivetran creates and maintains a perfect replica of your data, with minimal user configuration, performs change data capture and automatically adjusts to schema and API changes, making interruptions and failures due to application changes a thing of the past. FiveTran seamlessly integrates with SnowFlake, where we have configured a mix of databases, schemas, warehouses to organize analytics workloads and effectively manage performance and cost.
Benefits of this Modern Analytics Platform
This approach is very easy to adopt. You do not need to provision hardware, or install and configure software and the costs are significantly lower than traditional on-premise software and support costs. Though the cost savings and simplicity are significant, the benefits are even more compelling:
- Reliability, performance, disaster recovery, security without additional staff, and testing.
- Ability to adopt new features, and tools without disruptive upgrades, patches or migrations.
- Manage the environment to meet performance expectations easily and quickly.
- Replaces slow and error prone manual processes of downloading data from cloud applications.
- Provides reliable pipelines and delivers data to a data lake for immediate use.
- Establishes a standard process for consolidating/integrating data, ETL, reporting and analytics.
These benefits allow you to change the way you work with the business. Instead of spending time and money managing the environment and struggling to get access to data, you can focus on building an analytics team that has a real impact on your business.
- Analytics that would normally take months can be done in days or even hours.
- Develop high impact analytics instead of managing and supporting software.
- Create a sustained competitive advantage for your business.
A Real World Example
Quantified Mechanix collaborated with a client to pilot this modern analytics platform. This included connectors for SalesForce.com, Marketo and Google Analytics delivered to SnowFlake. This approach immediately resolved frequent failures with SalesForce.com that had the analytics team tied up in a never ending loop of adjusting to schema changes, and data extraction failures and enabled near real time reporting, and integration with data from other on-prem applications. The Marketo pipeline gives them the ability to report on historical data, and enables year over year comparisons and the Google Analytics pipeline enables them to accumulate detailed historical data, avoid sampling and is a critical element in the development of a marketing attribution model. All of these connectors were set up and delivered data in a few minutes, into a database that was used by analysts within the day. The alternative approach had been API development, expected to take weeks and months, without certainty of success and constant care and maintenance to stay up to date or extend and enhance.
How to get Started
We can help you create a road map and a pilot to test and validate this approach for your team. This is an inexpensive and low-risk way to get started and understand how to move forward with a modern analytics environment. The pilot will help you understand the benefits, costs and skills needed to get started, and build a plan for the future, while testing assumptions with a real world project.
If you would like to hear more, drop me a note at elealos@quantifiedmechanix.com.
– EmL