Solution Architect

This role requires strong technical knowledge of enterprise data platforms, excellent stakeholder management skills, and proven experience delivering complex projects within the banking and financial services industry.

• 10+ years of experience.

• 5+ years managing Enterprise Data Lake, Data Warehouse or Big Data delivery initiatives.

• Proven experience in Hadoop/Cloudera environments.

• Experience managing multiple concurrent project deliveries and BAU portfolios.

• Banking and Financial Services experience preferred.

Data Engineering & Integration

·        Design, develop, and maintain scalable data pipelines within the Enterprise Data Lake platform using Cloudera CDP, Hadoop, Spark, Hive, Java, Python, and SQL.

·        Develop high-performance batch and near real-time data ingestion frameworks for onboarding enterprise source systems into the Data Lake.

·        Develop complex transformation logic using Spark, HiveQL, SQL, and Java-based processing frameworks.

·        Design and implement data ingestion patterns supporting structured, semi-structured, and unstructured datasets.

·        Build and maintain batch and near real-time data integration solutions.

·        Implement source-to-target mappings and ingestion specifications.

Hadoop & Cloudera Platform Development

·        Develop and support solutions within the Cloudera Data Platform ecosystem.

·        Build and maintain data pipelines leveraging:

·        Hadoop HDFS

·        Hive

·        Impala

·        Spark

·        Kafka

·        YARN, Oozie

·        Perform deployment, troubleshooting, and performance tuning of applications running on the Hadoop ecosystem.

·        Partner with platform teams to ensure optimal cluster utilization and workload management.

Data Quality & Governance

·        Ensure adherence to Data Lake onboarding standards and governance requirements.

·        Investigate and resolve data quality issues and production incidents.

·        Ensure compliance with enterprise data governance standards.

DevOps & Platform Automation

·        Integrate data engineering solutions into CI/CD pipelines using Git, Jenkins, Azure DevOps, or equivalent tools.

·        Automate deployment, testing, monitoring, and operational support activities.

·        Develop reusable frameworks and utilities to improve engineering productivity and platform scalability.

Production Support & Incident Management

·        Provide L3 support for Data Lake application.

·        Perform root cause analysis for production incidents and implement permanent fixes.

·        Monitor pipeline execution, cluster health, and workload performance.

·        Participate in release implementation, deployment validation, and post-production support activities.

·        Strong hands on experience with Hadoop ecosystem and data pipeline orchestration.

·        Hadoop/HDFS

·        Hive

·        Spark

·        Unix/Linux scripting

·        Shell scripting

·        ETL/ELT development experience.

·        Data modelling fundamentals.

·        Performance tuning and optimization.

·        Experience in Data Warehousing and Data Lake environments.

·        Strong Java / Python competency with experiences.

·        Strong SQL development skills.