Developing a Scalable Integration Architecture for a High Growth Global Tech Company
Global computer security and anti-malware technology firm.
Our client is a global leader in malware and ransomware protection software. They are undergoing massive growth and tasked Dispatch to develop workflow automation solutions using Workato to enable scalable business processes.
Our client had experienced incredible growth over the past few years and recognized that many of their business processes were not as efficient as they could be. They had invested in Workato as a platform to build workflow automation solutions, developed significant expertise in this domain, and looked for Dispatch to accelerate the development of automations that spanned applications. Because of our finance and HR processes expertise, they asked that we focus on these functional areas.
While the client had a general sense of the opportunity, they had not yet completed a discovery process with each business unit to understand the current processes and automation options. As such, our first step was to hold a series of discovery workshops with stakeholder groups. The objectives of these sessions were to:
- Understand current state processes and look for opportunities to improve speed, reduce cycle times, reduce cost and errors, improve communication, and improve business process maintainability.
- Educate business leaders about the potential of automation to improve organizational results.
- Identify high potential use-cases for automation quantify business value.
These discovery sessions resulted in a prioritized portfolio of integrations that could help the organization achieve greater efficiency, quality, and scalability of core business processes.
After these sessions, we worked with our client’s internal team to develop an appropriate architectural framework to deliver these integrations. The company has a modern IT infrastructure and a sophisticated understanding of data architecture and so elected to build a publish-subscribe model that incorporates Snowflake as a data system of truth. This model separates data from applications and enables many different applications to feed Snowflake core data elements and “subscribe” to Snowflake for data required in key business processes.
The advantage of this model is scalability. Instead of building multiple point-to-point integrations with individual applications, integrations can be designed to move data to and from Snowflake using a standard model.
One of the critical decisions with this architectural approach is where data transforms occur (inside Snowflake or the integration platform). This decision is very dependent on existing competencies, maintainability, and long-term architectural design. In this case, the client elected to build data transformation inside Snowflake, which enabled each integration to be much simpler in design.
When using an intermediary data layer, it is essential to synchronize user access rights and permissions to the source system’s data system. This is mandatory when handling confidential data and is especially critical when the system handles personally identifiable information (PII), subject to regulatory compliance rules regarding access controls.
We immediately began building two key integrations that the organization prioritized – one that synchronized employee data between Workday and NetSuite, and another that sent demographics data to a 3rd party employee equity platform Shareworks. In both these cases, the existing data flows were manual, file-based, and very error-prone.
These integrations involved publishing data from Workday to Snowflake and creating subscriber integrations to NetSuite and Shareworks. In both cases, we developed fully automated processes, including error and exception handling.
The two integrations we developed had returns on investment measured in just a few months. The elimination of manual data manipulation resulted in a significant reduction in labor, which improved scalability. These business processes both contained sensitive employee data, and the reduction in errors helped enhance employee experience and improved compliance with data privacy requirements. The pub-sub architecture is robust and will enable many more integrations to be developed rapidly going forward.
Results that Matter
Dispatch Integration has built data integration and workflow automation systems for companies worldwide. Learn how we can help unlock the full potential of your IT infrastructure.