Databricks Data Platform for a Pharmaceutical Corporation in USA
Designed and Developed a data platform on Azure Databricks that provides self service capabilities with low code and configuration driven approach.
Capabilities include – cloud infrastructure automation, config driven data ingestions and transformation, standardized logging and monitoring, operational reporting, data catalogs, data sharing, sensitive data management etc.
Snowflake Data Platform for a Credit Union Bank in USA
Phase 1 – migrated approximately 5K+ plus tables with in a year from on-prem Cloudera environment to Snowflake platform.
Phase 2 – configuration based framework to build curated datasets in Snowflake with auditing, monitoring, logging and validation control capabilities.
Business Transformation for Insurance Corporation in Canada
Implemented real-time integration of customer master data to sync the updates between three systems (Memberships, Insurance Policies and Salesforce); data updates are captured and propagated to downstream with in SLA of 3 seconds.
Migrated data various on-prem sources into a new Data platform on AWS; this enabled client teams to use cross systems data on one platform and fuel business activities.
Built SQL based framework with build data marts on AWS S3 and Athena combination using PySpark, EMR, and Airflow tools.
ERP Data Pipeline for a Drug Distributor in USA
Designed and Developed a data pipeline on Azure platform to copy and store the data from Microsoft ERP and SAP ERP systems; built a new data warehouse and BI solutions on AZURE platform for client’s business teams.
Data consumption layer included – Power BI Dashboards, Email Subscriptions via Power Automate, Report Exports via Azure Data Factory
Database Monitoring for Financial Services Corporation in USA
Client had hard time monitoring various databases used by their 100s of IT applications; we developed data crawlers to collect the metrics for all types of DB systems and enabled client’s infrastructure team to monitor their resources in one dashboard.