Senior Data Engineer

Job Summary

We are seeking a highly skilled and experienced Senior Data Engineer to lead the design and architecture of enterprise-scale data integration solutions. In this role, you will spearhead complex cloud data migration strategies, oversee Informatica platform governance, and manage critical production incidents. You will be responsible for developing and optimizing data warehouse solutions, executing cloud SaaS integrations, and applying advanced performance optimization techniques. This position requires deep expertise in cross-platform databases and a strong focus on automation and data quality. You will play a pivotal role in driving our data engineering initiatives across multiple industry verticals, ensuring data integrity, performance, and business alignment.

Key Responsibilities

1. Lead ETL/ELT Architecture Design: Design and architect enterprise-scale data integration solutions across cloud (Snowflake, GCP, Azure) and on-premises environments (Oracle, SQL Server, Teradata) using Informatica IDMC/IICS and PowerCenter.
2. Cloud Data Migration Strategy: Spearhead complex data migration initiatives from on-premises databases to cloud platforms (Snowflake, BigQuery), ensuring data integrity, performance optimization, and minimal business disruption while achieving cost efficiency.
3. Informatica Platform Governance & Administration: Oversee Informatica infrastructure including installation, upgrades, server migrations, secure agent deployment, and repository management; establish standards and best practices for enterprise-wide adoption.
4. Production Support & Incident Management: Manage critical production incidents and escalations through structured management processes (ServiceNow); coordinate cross-functional stakeholder engagement to resolve issues and implement permanent solutions with minimal downtime.
5. Data Warehouse Integration Solutions: Develop and optimize data warehouse solutions implementing advanced techniques (Slowly Changing Dimensions, Change Data Capture, partition strategies) supporting real-time and batch processing requirements across diverse business domains (Retail, Pharma, Banking, Healthcare).
6. Cloud SaaS Integration: Execute integration projects connecting enterprise SaaS platforms (Salesforce, Veeva CRM) with data warehouses and analytical systems, enabling seamless data flow and business intelligence capabilities.
7. Performance Optimization & Tuning: Apply advanced optimization techniques including pushdown optimization, partitioning strategies, and CDC implementations to enhance ETL/ELT performance and reduce processing times for large-scale data operations.
8. Automation & Infrastructure as Code: Implement infrastructure automation using shell scripting and scheduling tools (Control-M, Autosys, UC4, Cron) to ensure reliable job execution, monitoring, and operational excellence across distributed systems.
9. Data Quality, Testing & Governance: Establish data validation frameworks, develop comprehensive test strategies, and oversee System/Integration Testing to ensure data accuracy, completeness, and business requirement alignment across all ETL/ELT pipelines.

Required Skills & Qualifications

1. Extensive experience with Informatica (IDMC/IICS, PowerCenter, Power Exchange).
2. Deep expertise in Snowflake, GCP, and Azure cloud platforms.
3. Proven experience in various data engineering methods including Data Integration, Data Warehousing, Data Migration, Application Integration, and Business Intelligence.
4. Hands-on experience creating Informatica Cloud IICS services (CDI, CDQ, CAI) and objects like Mappings, Tasks, Task Flows, and Processes.
5. Experience migrating data from Oracle and MS SQL Server to cloud databases like Snowflake and BigQuery (GBQ).
6. Proficiency in creating data flows to extract/load data into various databases (Oracle, SQL Server, Snowflake, Postgres, GBQ, Teradata, DB2).
7. Advanced SQL/PL/SQL skills across RDBMS systems (Oracle, Teradata, MS SQL Server, Postgres), including DDL creation, views, complex SQL constructs, stored procedures, indexing, and partitioning techniques.
8. Experience with DOS PowerShell scripting and UNIX Shell scripting.
9. Required experience extracting data from GCS, AWS, and Azure blob files into a data warehouse.
10. Strong experience in developing test plans and test scripts for data validation, and involvement in System and Integration Testing.

Preferred Qualifications

1. Experience with on-premises databases such as Oracle, SQL Server, Teradata, and DB2.
2. Familiarity with scheduling tools like Control-M, Autosys, UC4, or Cron.
3. Experience with incident management platforms like ServiceNow.
4. Knowledge of SaaS platforms such as Salesforce and Veeva CRM.
5. Experience in multiple industry verticals including Retail, Pharma, Banking, and Healthcare.

Soft Skills

1. Strong leadership and architectural design capabilities.
2. Excellent problem-solving and analytical skills.
3. Effective communication and cross-functional collaboration skills.
4. Ability to manage and prioritize multiple complex projects.
5. Detail-oriented with a focus on data quality and governance.