Job Description
Posted on:
February 3, 2025
As the Tech Lead of Data Engineering you’ll be at the forefront of we shape it, store it, and use it. You’ll lead a team of highly skilled and mission driven engineers, committed to making the internet safer for the next generations. You’ll work together with the other leads across technology, product, and other departments to identify areas of improvement for the overall approach to data engineering, lay the groundwork to enable AI driven capabilities in our products, and enable everyone with an easy and accessible path to make data driven decisions. Together with our Chief Architect, Infosec and Legal functions you’ll ensure the overall data and information architecture is kept safe and secure. You’ll work with your team to refine the roadmap and break it down into deliverables of the highest quality and impact. As a lead, you’ll champion continuous improvement and always aim to improve the product your team owns and measure your impact with the appropriate tech, product, or delivery metrics.
Responsibilities
Be accountable for:- design and implementation of features and services for SuperAwesome’s data lake, ingestion and distribution systems,- availability and performance of our data lake, ingestion and distribution systems,- design and implementation of ingestion, enrichment and transformation pipelinesLead system design activities aimed to create clarity, alignment, and ascertain feasibilityWork across the full stack depending on where you can drive the highest impact: from ETL pipelines to data warehousing to visualisations, as well as testing and cloud infrastructure.Use your experience in data management and governance to define and implement best practices across our systems ensuring security, consistency and quality across all of our data.Develop and optimise Spark jobs, notebooks, and data pipelines within the Databricks environment.Work with the Chief Architect and ML engineers to define an information architecture capable of enabling an easy implementation of data models and catalogues within the data lake, ensuring data quality, integrity, and optimal performance.Train/mentor other engineers inside and outside of the team on Data engineering best practicesFrequently review system, delivery and quality metrics and drive the right initiatives to improve them, ensuring long-term quality, scalability and maintainability of our systemsSet the bar in what good quality data looks like, and use your expert knowledge to train and mentor other engineers to improve data qualityEnsure business continuity by keeping the design choices well documented and explained, from raw data analysis to defining and implementing data ingestion/enrichment pipelines to creating data visualisationsChampion the DataOps culture, support data systems in production, including participation in our out-of-hours on-call rotaClearly communicate our data platform strategy and plans to others in the companyJob Requirements
Technologies
Python
SQL
Databricks/Snowflake
ETL
AWS/GCP
Must technologies:
Expert understanding of Data pipeline design and implementation using Databricks, and Python (or Python derivative, like PySpark)Expert visualisation skills using Sisense and/or other visualisation toolsExpert with SQLExpert understanding of Data management and/or Data governance (making sure the data is of expected volume, schema, etc.)Experience with modern cloud-based Data Engineering on AWS or other cloud platforms
Nice to have:Experience with Kedro on DatabricksExperience in designing and implementing Data meshExposure to Airbyte reading from multiple different data sourcesGood understanding of microservices architecture principles.
Recruitment ProcessInitial screen (Alcor’s side) - 30 minutes;
Technical screen (video call with Engineering manager) - 45 minutes, 12-15 quick question (different for each position), to understand what the shape of technical knowledge of a candidate is;
Code test (completed at home and typically takes 1.5 to 2 hours, with no strict time limit), the scope is different for each role;
Systems design/Final interview/cultural fit interview
BenefitsPTO - 25 days per year
Summer Fridays - last Friday in June to last Friday in August is a half day
Winter Break - two week company closure over Christmas and New Year
Volunteer Days - two per year
Sick Leave - 10 days per year
Sabbatical - 30 days after 7 years of service
Referral Program for bringing us new team members
Annual company offsite in the UK
Team social events and treats (online and in person)
Laptop provided along with SA swag
.png)
