Location : Chennai /Hyderabad/ Bangalore
Experience: 5 to 12 Years
Notice Period : Immediate Joiners
Required Skills : Teradata Development experience (Mandatory) and GCP
No Teradata Migration experience please
Qualifications:
· Experience in ETL & Data Warehousing
· Should have excellent leadership & communication skills
· Should have strong working experience on Data Lakehouse architecture
· Should have experience in developing Data Engineering solutions using GCP BigQuery, Cloud Storage, AirFlow, Dataflow, Cloud Functions, Pub/Sub, Cloud Run, etc.
· Should have built solution automations in any of the above ETL tools
· Should have executed at least 2 GCP Cloud Data Warehousing projects
· Should have worked at least 2 projects using Agile/SAFe methodology
· Should Have experience in Pyspark and Teradata
· Should have working experience on any DevOps tools like GitHub, Jenkins and Cloud Native, etc & on semi-structured data formats like JSON, Parquet and/or XML files & written complex SQL queries for data analysis and extraction
· Depth understanding on Data Warehousing, Data Analysis, Data Profiling, Data Quality & Data Mapping
· Should have global experience and been part of a team with at least 15+ members in a global delivery model
· Should have experience in working with product managers, project managers, business users, applications development team members, DBA teams and Data Governance team daily to analyse requirements, design, development and deployment technical solutions
Job Description:
· Design and implementation of data pipelines using GCP services for one or more projects
· Manage Deployments of data applications and ensure efficient orchestration of services.
· Implement CI/CD pipelines using Jenkins or cloud native tools to automate data pipeline deployment, testing, and integration with other services, ensuring quick iterations and deployments.
· Guide a team of data engineers in building and maintaining scalable, high-performance data pipelines.
· Build data pipelines and ETL/ELT processes leveraging Python, Beam and SQL scripts.
· Willingness and ability to learn and adapt to new technologies as needed during client engagements.
· Continuously monitor and optimize data workflows for performance and cost-effectiveness.
· Design workflows to integrate data from various sources using GCP services and orchestrate complex tasks with Cloud Composer (Apache Airflow).
· Set up monitoring, logging, and alerting using Cloud Monitoring, Datadog, or other tools to ensure visibility into pipeline performance and quickly identify and resolve issues.
· Guide and mentor junior developers and data engineers, helping them overcome technical challenges and ensuring high-quality code and solutions.
· Work closely with application developers, data architects, and business stakeholders to define and deliver robust data-driven solutions.
· Work on data migration from various databases/legacy DW systems build on Oracle, Teradata, SQL server etc to GCP cloud data platform,
· Facilitate agile processes like sprint planning, daily scrums, and backlog grooming.
· Interact with client’s stakeholders on assigned data, BI and analytics programs
· Work closely with program leadership team on stakeholder management, governance and communication.