Since we launched Compare a few years ago, people from around the world have used this simple tool to analyze hundreds of millions of records within their business applications to assess data quality and flag differences in data sets.
We are delighted to announce that Compare is now free to download and use on an unlimited basis!
Why Compare?
As a company that builds integration and automation solutions, we are always concerned about data quality.
If you have integrations, you have the potential for data quality issues
Companies today have dozens of mission-critical applications that share data amongst themselves and 3rd party services. Data quality tends to deteriorate over time, and often companies don’t really know how in sync their systems are.
Data quality issues can creep in for a variety of reasons:
- Manual data entry introduces keying mistakes.
- Non-standard data uploads into one system may not propagate to other systems.
- Data in “systems of truth” may be inadvertently overwritten by poorly conceived integrations pulling data in from other systems.
- Configuration changes in individual systems may introduce custom fields or data models that aren’t perpetuated to other systems.
- Applications may be updated or replaced, introducing data quality risks.
Most companies live with these problems but don’t really understand their magnitude. For instance, it’s common to see customer data in a CRM become de-synchronized with customer data in a billing system. That’s because, even if there is an integration to flow data between the systems, the sales team updates the CRM, and the finance team updates the billing system. This might be just an irritant until invoices end up unapproved and unpaid.
All parts of the organization struggle with this – sales, finance, marketing, HR, operations, IT, and more, all deal with data that they may not fully trust but also don’t really know how “bad” it is.
Compare Use Cases
We recommend you think of Compare as another simple tool in your toolbox. Especially if you are an analyst or developer that builds, configures, or manages enterprise data integrations. Use Compare for:
Assessing data alignment between systems
- Before you start a new data integration or workflow automation project, use Compare to assess the data alignment between the systems in scope. By doing this work upfront, you can uncover whether there are fundamental data quality issues that should be addressed before you integrate the systems.
- After you complete a data integration project, use Compare to assess the quality of the data being synchronized to ensure the integration is working as designed.
- If applications are being replaced, run a Compare on the data in the old system versus the new system to ensure data has been transferred correctly
Assessing data quality
- If functional teams complain about data quality, use Compare to run spot checks on the data in question – especially to compare the data in originating systems vs. downstream systems. If significant differences are found, this can trigger a data cleansing project and an analysis to understand the root causes of the differences.
Regression testing
- If you are reconfiguring or upgrading applications, run Compare on the data before and after the configuration changes to ensure data problems haven’t been introduced.
Auditing
- Since Compare is so simple and easy to use, you may want to run data comparisons across your critical business systems as part of a regular internal audit process.
Why did we Build Compare?
There are tools generally available to compare datasets, including Excel, but we found them clunky and somewhat difficult to use. They tend to be geared towards sophisticated data analysis problems, and the problem we are trying to solve is much simpler.
We only wanted a simple tool to compare the output of two systems and show us the differences.
We couldn’t find a suitable system, so we built one. We designed it to run locally on a Mac or PC because the business data being compared is often extremely sensitive and confidential. We also developed a custom “diff” algorithm that is extremely fast. This comparison engine is limited only by the amount of memory in your computer and can compare millions of records in minutes.
Finally, we know that Excel sits on just about everyone’s desktop, so Compare generates a .xlsx file containing the differences found between datasets that can be further analyzed and manipulated by anyone with Excel skills.
Automating Data Comparisons
In some cases, you may wish to fully automate dataset comparisons. This can be very helpful for your auditing processes, which you may require for internal controls or regulatory compliance, such as Sarbanes Oxley. By automating your data quality assessments, you can be confident that the data across all your applications is always in sync and accurate.
If you are interested in learning how you can automate these data analysis processes, please contact us. We can help you build and implement an automated auditing process to meet your exact needs.
How to Get Compare
Compare is free to download and use for the use cases above or any other situation where you need to compare datasets that can be represented as .csv files. Just click here to learn more about Compare and get your copy.
Cameron Hay is the CEO of Dispatch Integration, a data integration and workflow automation company with clients in Canada, US, Europe and Australia. He has over 30 years of leadership experience in various technology-oriented industries.