Data Engineering Team Lead Job Vacancy in BatchService Trichur, Kerala – Updated today
Are you looking for a New Job or Looking for better opportunities?
We got a New Job Opening for
Full Details :
Company Name : BatchService
Location : Trichur, Kerala
Position :
Job Description : Job Summary
We are looking for an experienced Databricks data engineer to lead our data engineering team. Our data engineering team has recently migrated from NodeJS/TypeScript to Databricks/Python (PySpark). We are looking for a data engineering leader to take our pipeline to the next level, improving performance, reliability, maintainability, and flexibility. You will work closely with the Data Architect and the Chief Data Officer. You will report to the department’s engineering manager. You will work with a project manager to manage the API team’s Jira board and lead the Jira ceremonies (daily standup, retrospective, backlog refinement and sprint planning).
Supervisory Responsibility
You will lead a team of four Databricks data engineers.
Duties and Responsibilities
Write efficient ETL pipelines to ingest data into Databricks Delta Tables.
Write OLAP to OLTP connection logic to transmit data from Delta to the appropriate backing service.
Architect the integration between different system components to share and transmit data.
Maintain ETL pipelines and adapt them to business requirements.
Fine tune the Databricks Spark clusters to achieve a good balance between performance and cost.
Facilitate any requirements of the Data Analytics/Science teams and enable them to work with the data.
Come up with creative and innovative solutions to integrate network-based 3rd party data vendors into the ETL pipelines.
Review pull requests raised by your team members. You will be responsible for ensuring that the code satisfies the documented requirements, includes the necessary automated tests and follows our documented coding standards and best practices.
Lead the onboarding of new team members.
You will help write the job descriptions for open positions and you will participate in the interviewing and candidate review process.
Skills and Qualifications
Exceptional analytical and problem-solving skills
Self-starter with strong self-management skills
Strong team leader: 2+ years management experience required
Databricks: 3+ years required
Python: 3+ years required
Big Data: 2+ years’ experience querying datasets with hundreds of millions of rows desired
Excellent verbal communication skills
Strong technical writing skills
BS or MS degree in Computer Science preferred
This post is listed Under Technology
Disclaimer : Hugeshout works to publish latest job info only and is no where responsible for any errors. Users must Research on their own before joining any company