Azure Cloud Data Engineer

Sumitomo Mitsui Financial Group Inc.; Ireland; Hybrid; EUR 60,000 - 80,000; Full time

Job summary

A leading global financial institution in Ireland seeks an Azure Cloud Data Engineer passionate about production support. The candidate will monitor and troubleshoot Azure Data Factory and Databricks jobs, ensuring high performance and uptime of critical data pipelines. The role requires 7+ years of experience in Azure and SQL, with familiarity in ETL processes. Employees enjoy a hybrid work model, allowing flexibility to work from home or the office.

Benefits

Hybrid work model

Reasonable accommodations for disabilities

Qualifications

• 7+ years of experience in Azure Data Factory and Azure Databricks required.
• Proficient in SQL for writing and debugging queries.
• Experience with cloud solutions and ETL processes.

Responsibilities

• Monitor and troubleshoot ADF pipelines and Databricks jobs in Production.
• Conduct root cause analysis for incidents and outages.
• Support CI/CD pipelines using Azure DevOps.

Skills

Azure Data Factory

Databricks

SQL

Python

PowerShell

Bash

Tools

Azure Monitor

ServiceNow

JIRA

Confluence

Job description

SMBC Group is a top-tier global financial group. Headquartered in Tokyo and with a 400‑year history, SMBC Group offers a diverse range of financial services, including banking, leasing, securities, credit cards, and consumer finance. The Group has more than 130 offices and 80,000 employees worldwide in nearly 40 countries. Sumitomo Mitsui Financial Group, Inc. (SMFG) is the holding company of SMBC Group, which is one of the three largest banking groups in Japan. SMFG’s shares trade on the Tokyo, Nagoya, and New York (NYSE: SMFG) stock exchanges.

In the Americas, SMBC Group has a presence in the US, Canada, Ireland, Mexico, Brazil, Chile, Colombia, and Peru. Backed by the capital strength of SMBC Group and the value of its relationships in Asia, the Group offers a range of commercial and investment banking services to its corporate, institutional, and municipal clients. It connects a diverse client base to local markets and the organization’s extensive global network. The Group’s operating companies in the Americas include Sumitomo Mitsui Banking Corp. (SMBC), SMBC Nikko Securities America, Inc., SMBC Capital Markets, Inc., SMBC MANUBANK, JRI America, Inc., SMBC Leasing and Finance, Inc., Banco Sumitomo Mitsui Brasileiro S.A., and Sumitomo Mitsui Finance and Leasing Co., Ltd.

Role Description

SMBC is seeking an Azure Cloud Data Engineer who has a strong passion for Production Support and is interested in building a career at a fast growing and reputable Bank. The Azure Cloud Data Engineer will have strong Azure Data Factory and Databricks knowledge. They will be given opportunities to participate in providing support for Azure based data integration and analytics pipelines built using Azure Data Factory and Azure Databricks ensuring the uptime and performance of critical pipelines and workflows. The successful candidate will have knowledge of complex interface process development and is required to troubleshoot issues. This role will report to the Executive Director of Production.

Role Objectives: Expertise

• Monitor, troubleshoot and support ADF pipelines and Databricks notebooks/jobs in Production.
• Extensive experience on Cloud solutions, specifically in Azure.
• Experience with Azure cloud services, Azure Data Factory, Gen 2, Azure Databases, Functions, Databricks, or similar technology.
• Good understanding of ETL/ELT.
• Analyze pipeline failures, spark job issues, data mismatches, cluster timeouts, resource unavailability and latency bottlenecks.
• Root cause analysis for incidents and outages.
• Understand the ADF Components and architecture.
• Knowledge of data integration techniques and best practices.
• Experience with connection to various data sources and destinations.
• Ability to orchestrate complex data workflows and transformation.
• Monitoring and troubleshooting data pipeline executions.
• Familiarity with ADF data flow activities for data transformations.
• Version control and deployment management using Azure DevOps or similar tools.
• Awareness of ADF Integration with Azure services like Azure Data Lake Storage, Azure Databricks, etc.
• Skills in implementing streaming and batch data ingestion using Delta Lake.
• Skills in implementing data pipelines and workflows in Databricks.
• Familiarity with Databricks notebooks for interactive data exploration and development.
• Integrating Databricks with Azure Services like ADLS Gen2, Azure SQLdb.
• Monitoring and optimizing Databricks jobs for cost efficiency.
• Proficiency in Git for managing code repositories, including branching, merging and pull requests.
• Support CI/CD pipelines for deployment using Azure DevOps.
• Participate in on‑call rotation and ensure business continuity via proper DR strategies.
• Experience with RDBS systems like Azure SQL DB, Oracle, and NoSQL Databases like MongoDB.
• Understanding of indexing, partitioning, and other optimization techniques.
• Experience with stored procedure, functions, and triggers.
• Ensure High Availability (HA) of ADF pipelines and auto‑scaling/failover readiness of Databricks clusters.
• Manage alerts, incidents and escalations using ServiceNow, Azure Monitor, Log Analytics, etc.
• Experience with Confluence, ServiceNow & JIRA.
• Review and provide feedback on core code changes and support production deployment.
• Knowledge on ETL DataStage Application would be a plus.
• Azure Monitor, Application Insights, Log Analytics.
• Cluster and pipeline‑level metrics and logs.

Qualifications & Skills

• Recommended years of experience: 7.
• Strong hands‑on experience with Azure Data Factory (ADF) – Pipeline orchestration, linked services, integration runtimes.
• Experience in Azure Databricks – running and debugging notebooks, clusters, spark job logs.
• Proficient in SQL – Writing/debugging queries, validating data.
• Good understanding of Azure Services: ADLS, Key Vault, Azure Monitor, Log Analytics.
• Familiarity with Azure DevOps pipelines and Git Integrations.
• Scripting knowledge: Python, PowerShell, or Bash.
• Understanding of Spark concepts and Delta Lake (Preferred).
• Ability to work on weekends for maintenance, production implementations, recovery tests and system verifications/validations.
• Ability to address production issues from home outside of normal business hours.

Additional Requirements

SMBC’s employees participate in a hybrid workforce model that provides employees with an opportunity to work from home, as well as from an SMBC office. SMBC requires that employees live within a reasonable commuting distance of their office location. Prospective candidates will learn more about their specific hybrid work schedule during their interview process.

SMBC provides reasonable accommodations during candidacy for applicants with disabilities consistent with applicable federal, state, and local law. If you need a reasonable accommodation during the application process, please let us know at [e-mail].