[Job-32197] Tech Lead Data - Databricks
At a glance
Senior Data Engineering role at CI&T. Remote, Brazil.
Pay not stated Data Engineering salary
remoteroles summary, based on the employer's posting.
What you'll do
- Guide data engineering teams through design choices and reviews, stepping in to clear delivery blockers
- Set Lakehouse coding, modeling and technical practices with the Data Architect across Delta Lake and Medallion layers
- Shape Databricks pipelines in PySpark and Spark SQL, covering security, performance and governance across Bronze, Silver and Gold
- Build DataOps into delivery with Git, CI/CD, automated tests and documentation, while improving pipeline reliability and cost
- Turn business and analytics needs into delivery plans, and coach engineers using shared templates and playbooks
What you bring
- Strong data engineering experience, including technical and operational leadership of teams as a Tech Lead or Senior
- Hands-on Databricks skills in PySpark and Spark SQL, plus Delta Lake and practical Medallion architecture experience
- Experience with Azure data services, including Databricks, ADLS and security or identity integrations
- Strong Python and SQL, sound data modeling and schema practices, and experience applying Git, CI/CD, testing and documentation
- Advanced English for stakeholder work and technical documentation
Who this fits
This senior, hands-on technical leadership role suits someone who can guide engineers while still designing and improving data pipelines. The position is remote in Brazil.
From the employer
At CI&T, we help large enterprises transform the potential of AI into real business impact with AI Deployment, AI-native execution, and tech-integrated business solutions.
With 30 years of experience in technological transformation, we accelerate innovation with expertise in Agentic SDLC, Application modernization, Data & AI, Martech and Business strategy.
We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day.
About the Role
We are looking for a senior, hands-on Data Tech Lead to drive the technical leadership of our data engineering initiatives on Databricks in Azure.
You will partner directly with the Data Architect to define standards and references, remove roadblocks, and enable the Data Engineering team to deliver scalable, reliable pipelines aligned with the business within the Medallion architecture.
Key Responsibilities
-
Provide technical leadership to Data Engineering squads, supporting design, reviews, and decision-making to unblock delivery.
-
Co-define with the Data Architect the technical vision, coding standards, modeling conventions, and best practices for the Lakehouse (Delta Lake + Medallion).
-
Design and oversee pipelines in Databricks (PySpark/Spark SQL) across Bronze/Silver/Gold layers, ensuring security, performance, and governance.
-
Implement and reinforce DataOps practices: version control (Git), CI/CD, testing (unit/integration/data quality), and documentation.
-
Optimize jobs/pipelines: partitioning, Spark tuning, cost management, and reliability (baseline observability and alerting).
-
Ensure non-functional requirements: security, access control, and data policies in Azure/Databricks aligned with defined governance.
-
Collaborate with stakeholders (business/analytics) to translate requirements into scalable solutions with realistic delivery plans.
-
Mentor engineers, promote standards, and create references (templates, playbooks) to accelerate the team.
Required Skills and Qualifications
Must-have Skills
-
Strong experience in Data Engineering and projects, with experience as a Tech Lead/Senior leading teams technically and operationally.
-
Strong hands-on experience in Databricks: PySpark and Spark SQL; batch pipelines (and working knowledge of streaming when needed).
-
Mastery of Delta Lake and practical application of the Medallion architecture (Bronze/Silver/Gold).
-
Solid experience with Azure for data, including at least: Azure Databricks, Azure Data Lake Storage (ADLS), and integration with security/identity services (e.g., Azure Active Directory).
-
Strong SQL and Python skills; fundamentals in modeling (dimensional/consumption) and good practices for contracts/schemas.
-
Experience with DataOps: Git, CI/CD for data pipelines, automated testing, and clear documentation.
-
Proven ability to review design/code, resolve performance/scale issues, and guide teams.
-
Advanced English to work with stakeholders and produce technical documentation.
Nice-to-have Skills
-
Unity Catalog (governance/lineage/access control) and Databricks Workflows/Repos; Delta Live Tables.
-
Data quality frameworks (Great Expectations/Soda) and/or dbt on the Lakehouse.
-
Orchestration (Airflow/Databricks Workflows) and observability (metrics, logs, alerts).
-
Experience with CDC/streaming ingestion (Event Hubs/Kafka) when applicable.
-
Consulting experience and international client exposure; Databricks/Azure certifications.