Position Overview
The Data Engineer will design, build, and maintain data pipelines, semantic models, and notebooks for BradyPLUS Data Warehouses in the Azure Cloud as well as MS Fabric OneLake. The Data Engineer will also collaborate with Data Analysts, Data Scientists, and the Business Intelligence team to ensure data quality, reliability, and accessibility. Data will be both structured and unstructured in nature with structured data including output from multiple ERP platforms requiring the ability to create, edit and troubleshoot ETL design.
Responsibilities Include
- Develop and optimize data pipelines using Azure Data Factory, SQL Server Integration Services, and other tools.
- Integrate data from various sources, including internal and external databases, APIs, files, and web services.
- Implement data quality checks, validation rules, and error handling mechanisms.
- Create and maintain data models using SQL Server and Azure Analysis Services.
- Support data warehouse users and applications by providing data access, query optimization, and performance tuning.
- Work with data analysts and data scientists to understand their data needs and provide solutions.
- Document data pipelines, data models, data dictionaries, and data standards.
- Ensure data security and compliance with data governance policies and best practices.
The ideal candidate will have
- Azure SQL Server & Azure Data Factory
- SQL Server Integration Services (within Visual Studio)
- Data pipeline development and maintenance using ELT or ETL methodologies
- Data modeling using dimensional and relational techniques
- Data quality, validation, and testing skills
- Good communication and collaboration skills
- MS Fabric experience including OneLake build out and Power BI development
- Azure Blob Storage, Azure Key Vault, Azure Integration Runtime, and Azure Monitoring