New 
Databricks Tech Lead
 Tata Consultancy Services | |
$130,000 - $140,000
| |
sick time, 401(k)
| |
 United States, California, Pleasanton | |
 Oct 28, 2026 | |
|
Job Description
Must Have Technical/Functional Skills * 10+ years of experience in Data Engineering, Data Architecture, ETL Modernization, or Enterprise Data Platform Migration. * 6+ years of hands-on Databricks experience in large-scale enterprise environments with Data Modernization or migration to Databricks expertise. * Strong expertise in Databricks, Apache Spark, PySpark, SQL, Delta Lake, and Databricks Workflows/Jobs. * Proven experience assessing and migrating legacy data platforms, ETL solutions, and data warehouses to Databricks. * Experience with Alteryx workflow assessment, migration planning, and modernization. * Strong understanding of data modeling, data pipelines, source-to-target mappings, data quality, and performance optimization. * Experience designing scalable Lakehouse architectures, including Medallion Architecture and Delta Lake. * Knowledge of cloud data architecture, security, governance, monitoring, CI/CD, and DevOps practices. * Hands-on experience with Azure Databricks, ADLS, Azure Data Factory, Purview, Key Vault, and Azure DevOps. * Ability to lead technical discovery workshops, define migration strategies, prepare architecture documentation, and provide technical leadership throughout delivery. Roles & Responsibilities * Lead the assessment of current-state data platforms, including source systems, data warehouses, databases, ETL/ELT pipelines, reporting environments, and Alteryx workflows. * Define and drive the migration and modernization strategy for legacy data platforms to Databricks Lakehouse Architecture. * Analyze existing data assets, transformation logic, dependencies, source-to-target mappings, data quality requirements, and performance considerations. * Identify workloads suitable for lift-and-shift migration versus those requiring redesign, refactoring, consolidation, or retirement. * Design target-state data architecture leveraging Databricks, Delta Lake, Medallion Architecture, and cloud-native data services. * Lead the migration of legacy ETL, data integration, and analytics workloads to Databricks using PySpark, SQL, Databricks Workflows, and related technologies. * Assess and modernize Alteryx workflows and legacy data processing solutions into scalable Databricks-based implementations. * Define best practices for data ingestion, transformation, orchestration, security, governance, monitoring, metadata management, and lineage tracking. * Conduct technical discovery workshops and architecture reviews with business, data engineering, analytics, and platform stakeholders. * Create architecture documents, migration roadmaps, dependency assessments, implementation plans, effort estimates, and risk mitigation strategies. * Provide hands-on technical leadership during proof-of-concept, pilot, and large-scale migration initiatives. * Guide and mentor development teams on Databricks architecture, coding standards, performance tuning, and engineering best practices. * Collaborate with cross-functional teams to ensure migration and modernization initiatives align with business objectives and delivery priorities. * Establish CI/CD, DevOps, testing, and deployment standards fo r Databricks-based data solutions. * Ensure compliance with enterprise security, governance, data privacy, and operational standards throughout the migration lifecycle. * Act as the primary technical lead for architecture decisions, solution reviews, and stakeholder communications related to Databricks modernization programs. Generic Managerial Skills, If any * Strong stakeholder management and communication skills with the ability to lead technical discussions and influence architecture decisions. * Proven experience leading cross-functional teams in enterprise data migration and modernization programs. * Strong planning, problem-solving, and delivery management skills to drive complex initiatives from assessment through implementation Salary Range: $130,000-$140,000 a year TCS Employee Benefits Summary: Discretionary Annual Incentive. Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans. Family Support: Maternal & Parental Leaves. Insurance Options: Auto & Home Insurance, Identity Theft Protection. Convenience & Professional Growth: Commuter Benefits & Certification & amp; Training Reimbursement. Time Off: Vacation, Time Off, Sick Leave & Holidays. Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing. #LI-SP1
Location
Pleasanton, CA
Function
TECHNOLOGY
Role
Technical Lead
ID
436399
Skills
SQL | Apache Spark
Salary Range
$130,000 - $140,000 Desired Candidate Profile
Qualifications BACHELOR OF COMPUTER SCIENCE | |