Data Engineering Solution Cuts Costs by 4x & Enables Real-Time Reporting for eClinical Company
Our client is an eClinical technology company that specializes in developing digital health solutions for the pharmaceutical, biotechnology, and healthcare industries. The company’s patient-facing platform collects clinical trial data and engages patients. It leverages mobile devices to collect and analyze patient data in real-time. The aim is to improve access to healthcare for all people everywhere.
Data Engineering, Data Infrastructure, Cloud Infrastructure Management, Software Architecture, Product Development
Business Challenges
Time- and cost-consuming data aggregation
The Client spent 4x more costs in the data pipeline with 70% effectiveness. Additionally, they received data with a delay of 2 hours.
Inability to generate reports
Data analysts on the client’s side knew how to build those reports for their needs and, yes, this process was uncomfortable as there was not one place for all data to be stored. The report generation process is used to overload the data and analytics team with report creation.
Lack of expertise in data engineering
The Client was looking for a skilled data engineering partner. likeAnd Sombra was a good fit, since it has an extensive experience, focus on what matters to Client’s business and get things done right. The results speak for themselves.
How we worked
1. Agreed on sharing responsibilities with the Client
Sombra team suggested a dedicated development team for the Client. This engagement model implies that both parties are sharing responsibilities as follows:
| Client | Sombra |
|---|---|
|
|
2. Strategic Alignment
The Client had previously developed a solution, which required Sombra to evaluate and define the most efficient way to improve so they could see fast ROI for their business. In order to deliver a solid product with minimum change requests, our team of professionals took the following steps:
- Assessed the existing solution.
- Analyzed the Client’s functional and non-functional requirements.
- Provided our recommendations for the new system.
- Approved this new vision with the Client.
3. Solution design
With approved vision, the team was able to move forward with developing a brand new architecture with new approaches and modern tech stack.
4. Approach to technical implementation of the solution
During the implementation phase, the Sombra Team of professionals used the following tools and technologies:
AWS stack, Data lake on S3, DWH on Amazon Redshift, Spark on AWS EMR, Apache airflow on AWS MWAA;
Clients: Sisense, Holistics
To deliver the solution, we:
- Implemented distributed data processing using Apache Spark on Amazon EMR, designed to handle data at a scale of hundreds of gigabytes.
- Established a three-layer data architecture (Raw → Trusted → Analytical) to ensure the delivery of high-quality, normalized analytical data to Data Clients for business intelligence (BI) and reporting purposes.
- Orchestrated complex data pipelines that integrate various data sources, including files, databases, and APIs, using Apache Airflow.
- Implemented robust data security measures, including encryption of sensitive data both in transit and at rest, along with strict data access policies and other best practices for data security.
Business Value
Applying proven methodologies, the Sombra team built a solution that aided the client’s initial pain points. Moreover, the project was delivered on time and within scope, helping the Client to achieve their business goals without unnecessary complications.
- The new approach reduced regular data infrastructure costs by 50%.
- Implementing modern technologies enabled near real-time data syncing with delays of no more than 2 minutes, improving data accuracy and timeliness.
- The client achieved 99.9% reliability in data pipeline workflows, ensuring uninterrupted operations.
Looking for the right team to accelerate your business growth?
Contact Us
Trusted consulting technology company
Money-back
guarantee325+
tech talents84
Net Promoter Score in 202310+
years on the marketCertifications & Reviews
