DESCRIPTION

The Senior Data Engineer role reports to the Manager, Data Engineering. The role is part of our Enterprise Data Platform team that manages the ingestion, storage, modeling, and consumption of data for Acosta and our clients. The ideal candidate will have extensive experience in data engineering, strong proficiency in Azure, Databricks, Snowflake knowledge, and a proven track record of working on large-scale data projects. Candidate must have strong communications skills and ability to collaborate with technical and nontechnical audience. The role is responsible for supporting workloads running on the Azure Big Data environment and associated technologies. This position will be responsible for designing, creating, and monitoring the data solutions, resolving incidents, optimizing workloads, tuning solutions, and making solution enhancements. The role will focus on development and production support and will be involved in R&D of emerging technologies with the application administrators and technical architects.

RESPONSIBILITIES

Essential Functions of this Position

• Building data ingestion and ETL pipelines at an enterprise level, following corporate standards and best practices

• Work with Tech Leads and Architects to understand and implement solutions based on high level designs and requirements

• Stay current with big data, ETL and general data technologies, with a strong focus on the Azure technology stack

• Investigate new and emerging technologies based on future needs of the Data Platform and architectural direction

• Monitoring, investigating, and maintaining ongoing production systems, specifically triggered and schedule ETL pipelines and data ingestion patterns

• Optimization and refactoring of existing platform and legacy solutions

• Migration of legacy systems into a modern data platform

• Mentor Data Engineers on the Data Platform team

• Other duties as assigned

QUALIFICATIONS

Minimum Education and Work Experience

• Bachelor’s degree in Technology Industry is preferred, or equivalent related work experience may be substituted for some of the educational requirements

• 8 years of related work experience

Knowledge, Skills, and Abilities Requirements

• 5+years developing and supporting applications leveraging a big data stack (such as Azure)

• 5+years of experience with data integration/ETL tools such as Azure Data Factory or alternatives

• 3+years of experience with SDLC knowledge in both waterfall and agile methodologies

• 2+years of experience on following tools: SQL, Scala, Python, Spark, Kafka, and other related languages

• Experience in scripting languages & automation such as bash and Python

• Hands-on experience with source code management system (SVN, Git) and continuous integration tools (Azure Devops or alternatives)

• Experience with Snowflake Technology Stack strongly preferred

• Experience with Data Modelling strongly preferred

• Experience in data warehousing (Data Vault methodology preferred)

• Experience in handling data processing, delivering distributed and highly scalable applications

• Experience with high-volume, transaction processing software applications

• Good understanding of workload management, schedulers, scalability, and distributed platform architectures

• Excellent communication skills (both written and oral) combined with strong interpersonal skills

• Strong analytical skills and thought processes combined with the ability to be flexible and work analytically in a problem-solving environment

ABOUT US

Acosta, and its subsidiaries, is an Equal Opportunity Employer

Job Category: Administration

Position Type: Full time

Business Unit: Corporate

Salary Range: $100,000.00 - $1,250,000.00

Company: Mosaic Sales Solutions Canada Operating Co

Req ID: 3901