Big Data Engineer

Navisoftech; Chester; On-site; GBP 60,000 - 90,000; Full time

Job summary

A technology company is seeking a seasoned Big Data Engineer to architect and implement complex ETL pipelines. Candidates should have significant experience with Java and AWS technologies, as well as strong skills in data modeling and analytics. The ideal candidate will be proficient in big data tools, including Spark, Hadoop, and SQL, and have a robust analytical mindset. This is a full-time position based in Chester, and offers a competitive salary and opportunities for career development.

Qualifications

• At least 8 years of experience in architecting and implementing ETL pipelines.
• Strong background in Java (4+ years) and knowledge of AWS technologies.
• Experience working with big data tools and Agile methodologies.

Responsibilities

• Architect and implement complex ETL pipelines using Spark.
• Process, transform, and transport large-scale data.
• Collaborate in an Agile environment to build reusable frameworks.

Skills

ETL

Spark

Java

Data Modeling

Database Design

Data Mining

SQL

Scala

Python

AWS

Agile

Job description

We are seeking a seasoned Big Data Engineer with at least 8 years of experience architecting and implementing complex ETL pipelines, preferably using Spark. The ideal candidate will have a strong background in Java (4+ years) within the data space, expertise in data modeling, database design and development, data mining, and segmentation techniques. Proficiency in writing complex SQL and ETL processes is essential, along with excellent coding and design skills in Java/Scala and Python. Experience working with large data volumes, including processing, transforming, and transporting large-scale data, is crucial. A strong understanding of AWS technologies such as EC2, Redshift, CloudFormation, EMR, S3, and AWS Analytics is required, along with big data related AWS technologies like Hive, Presto, and Hadoop. Experience with Apache Hadoop, Apache Spark, Kafka, Scala, Python, and strong analytical skills are essential. Knowledge of algorithms and data structures and experience building reusable frameworks are highly valued. Experience working in an Agile environment is preferred. AWS certification (Developer/Architect/DevOps/Big Data) is a plus.

Big Data Engineer

We are seeking a seasoned Big Data Engineer with at least 8 years of experience architecting and implementing complex ETL pipelines, preferably using Spark. The ideal candidate will have a strong background in Java (4+ years) within the data space, expertise in data modeling, database design and development, data mining, and segmentation techniques. Proficiency in writing complex SQL and ETL processes is essential, along with excellent coding and design skills in Java/Scala and Python. Experience working with large data volumes, including processing, transforming, and transporting large-scale data, is crucial. A strong understanding of AWS technologies such as EC2, Redshift, CloudFormation, EMR, S3, and AWS Analytics is required, along with big data related AWS technologies like Hive, Presto, and Hadoop. Experience with Apache Hadoop, Apache Spark, Kafka, Scala, Python, and strong analytical skills are essential. Knowledge of algorithms and data structures and experience building reusable frameworks are highly valued. Experience working in an Agile environment is preferred. AWS certification (Developer/Architect/DevOps/Big Data) is a plus.

Please note: F1 OPT candidates are not eligible for this role
Job ID: KK11DR10314

Required Skills

ETL, Spark, Java, Data Modeling, Database Design, Data Mining, SQL, Scala, Python, AWS, EC2, Redshift, CloudFormation, EMR, S3, AWS Analytics, Hive, Presto, Hadoop, Kafka, Agile

ETL Spark Java Data Modeling Database Design Data Mining SQL Scala Python AWS EC2 Redshift CloudFormation EMR S3 AWS Analytics Hive Presto Hadoop Kafka Agile