Defining the approach to cost modeling data lakes - Cost Modeling Data Lakes for Beginners

Defining the approach to cost modeling data lakes

IT projects historically have well-defined milestones that make cost modeling a fairly simple process. Selected software is usually a commercial off-the-shelf (COTS) product, which can be costed (including licenses) and based on predictable metrics, such as number of users and number of CPUs.

The effort to implement the software is well within the standard skill sets of the IT department, with plenty of experience in similar deployment models to draw on to get an accurate picture of the implementation time. This will all feed into a large design document that can be used to calculate costs.

Therefore, you can expect to use the same cost modeling you have previously used for an analytics project. The challenge here is that an analytics project often doesn’t have a clear end. It’s a journey where you explore and prepare data in line, with some aspirational business outcomes. This makes it difficult to know the following:

  • How much data will be ingested

  • Where that data will come from

  • How much storage or compute capacity will be needed to make sense of that data

  • Which services are required to exploit the data to realize and deliver value throughout the business

However, analytics projects provide the radical growth and business opportunities to enable your organization grow and get a competitive edge. Customers who have realized business benefit include the following companies.

  • Siemens mobility, a leader in transport solutions, cut their energy costs by 1015% with analytics projects.

  • Kumon, an online talent market for freelancers, managed to increase purchasing conversion by 30% and decrease churn by 40%.

  • 3Victors, a leader in travel data analytics, built their business by collecting billions of shopper requests for flights and analyzing them on AWS. This provided actionable real-time insights to travel marketers and revenue managers. Those insights, in turn, enabled 3Victor to improve the customer conversion rate and improve their return on advertising spend.