A look at data validation and what it means for companies
In addition to being one of the fastest growing areas of the moment, big data can cost businesses a lot of money – especially if that data is of poor quality.
While understanding and using data can be a daunting task, it can help your company or department make important decisions. The problem, as mentioned above, is that if your records are poor quality or parts are missing, you leave an incomplete picture of the problem you are addressing and you may miss the mark overall.
It is for this reason that data validation continues to grow as companies who specialize in it can help ensure that your records are correct and that the information obtained from them is correct.
I had the chance to interview Elliot Shmukler from Anomalo, a data validation platform, to learn more about data validation, what it means to businesses and why it’s important, and to learn a little about what businesses can do with big data it to the fullest.
You can read the full review below.
Would you like to introduce yourself and your role at Anomalo?
Certainly. I’m Elliot Shmukler, Co-Founder and CEO of Anomalo. Based in the San Francisco Bay Area, we’ve been leading our small but growing team since our inception in 2018. Previously, I was the product and growth leader at technology companies such as Instacart, LinkedIn, and Wealthfront.
What is anomalo in a few sentences?
Anomalo is a platform that helps companies validate data – in other words, to ensure that the data companies use to make business decisions and develop products is accurate, complete, and in line with their expectations.
The premise here is that as organizations increasingly rely on large amounts (and often from multiple sources) of data to make product, marketing, and strategy decisions, they need to ensure that the data they are using is actually correct. Otherwise, they can easily make wrong decisions.
What inspired the founding of the company?
My co-founder Jeremy and I spent many years in Silicon Valley seeing firsthand the importance of good, high quality data in order to make informed business decisions and develop great products and services, including at Instacart where we met to have.
We have seen many situations there where Instacart customers have had bad experiences due to poor or inaccurate data, or where the company has made poor decisions due to poor data entry. When we left Instacart within 6 months and got together to come up with some startup ideas, we found that this data quality and data validation problem that we had seen for years was still not resolved. So we founded Anomalo to bring our best ideas to the problem.
Why has data validation become so important? Is it the growth of machine learning / algorithmic development or something else?
Quite simply, companies are much more likely to use data to make decisions and manage their products and services. They also import and aggregate larger amounts of data from a larger number of different sources.
This creates two problems: When data is used more frequently, the consequences of using inaccurate, corrupted, or out of date data are much more severe. And with a larger volume of data and a greater variety of data, the problem of actually validating data correctness has also become much more complicated. Anomalo makes the data validation process a lot easier and can help companies avoid the cost of making decisions based on bad data.
Machine learning and the adoption of AI are of course also driving this forward, as they are powerful new technologies that have increased the value and need for data in many organizations. However, machine learning models are also prone to the “garbage in, garbage out” problem, which means that ML models produce results based on the data they ingest. and when the data is bad, the results are bad too. This speaks to the importance of data validation tools like Anomalo, which use machine learning themselves to ensure that the content of your ML models is as high quality as you want them to be.
Who is Anomalo for? Big business, SMEs, or a little bit of both?
Anomalo is most useful for companies that have a lot of data and data from a variety of sources and that use data extensively to manage and grow their business. So far, we have had a lot of feedback from companies in areas such as e-commerce, fintech, social media and adtech.
How quickly could a company get their tools ready? Can you help with the setup?
Anomalo can be deployed in less than an hour. Sometimes we even book a one-hour zoom with new customers, where we can deploy and use the product for their data within the meeting.
Deployment is only the first step, of course. Now that Anomalo is available, organizations still need to point it to the right records and metrics to make sure the right things are being monitored and validated. We consistently help them throughout this process and are available on Slack and Zoom to provide as much assistance as needed.
Do you have any tips for companies looking to get more out of their data?
First of all, it is important to democratize access to data. As soon as more people in a company have access to high-quality data, it is used more and offers added value.
Second, every key business area should have a set of clear, reliable, accessible, and continuously updated dashboards to describe how the key metrics in that area are moving and what is going on in general. This is important to managing these companies and empowering teams to understand how their areas are performing and where they can have an impact.
Third, it is important to understand why a business metric changes unexpectedly and what happened when it changes unexpectedly. Very often such changes lead to insights that can reveal opportunities for business improvement or problems that must be resolved if the business is to be successful. Without taking the time to develop such an understanding (and tools like Anomalo and others can help here), it’s very easy to miss out on these opportunities for improvement.