- Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters.
- Converted code is causing failures/performance issues.
- Spark Architecture – component understanding around Spark Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans.
- Spark SME – Be able to analyse Spark code failures through Spark Plans and make correcting recommendations.
- Spark SME – Be able to review PySpark and Spark SQL jobs and make performance improvement recommendations.
- Spark – SME Be able to understand Data Frames / Resilient Distributed Data Sets and understand any memory related problems and make corrective recommendations.
- Monitoring – Be able to monitor Spark jobs using wider tools such as Grafana to see whether there are Cluster level failures.
- Cloudera (CDP) – Knowledge of understanding how Cloudera Spark is set up and how the run time libraries are used by PySpark code.
- Prophecy – High level understanding of Low-Code No-Code prophecy set up and its use to generate PySpark code.
-
Spark Architect
Found in: Ziprecruiter UK C2 - 11 hours ago
PRACYVA Sheffield, United KingdomJob Description · Spark Architect / SME · Contract Role- 6 months to begin with & its extendable · Location: Sheffield, UK (min 3 days onsite) · Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converter ...
-
SQL Architect
Found in: Jooble UK O L C2 - 3 hours ago
PRACYVA Sheffield, United KingdomSpark Architect / SMEContract Role- 6 months to begin with & its extendableLocation: Sheffield, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causi ...
-
Cloud Platform Engineering Manager
Found in: Appcast UK C C2 - 1 day ago
Twinkl Sheffield, United Kingdom Full timeAt Twinkl, we encourage diversity, and our doors are open to everyone. We're committed to creating an inclusive workplace for all. If you need any adjustments during the application process to showcase your abilities, please let us know. We're here to support you on your journey. ...
-
Cloud Platform Engineering Manager
Found in: Ziprecruiter UK C2 - 11 hours ago
Twinkl Educational Publishing Sheffield, United KingdomJob Description · At Twinkl, we encourage diversity, and our doors are open to everyone. We're committed to creating an inclusive workplace for all. If you need any adjustments during the application process to showcase your abilities, please let us know. We're here to support yo ...
-
Cloud Platform Engineering Manager
Found in: Jooble UK O C2 - 1 day ago
Twinkl Educational Publishing Sheffield, United KingdomAt Twinkl, we encourage diversity, and our doors are open to everyone. We understand that great ideas aren't confined to an office. That's why we embrace a remote-friendly culture, giving you the flexibility to choose where you work best. Whether it's from our amazing HQ in Sheff ...
-
Cloud Platform Engineering Manager
Found in: Jooble UK O C2 - 1 day ago
Twinkl Education Company Sheffield, United KingdomAt Twinkl, we encourage diversity, and our doors are open to everyone. We understand that great ideas aren't confined to an office. That's why we embrace a remote-friendly culture, giving you the flexibility to choose where you work best. Whether it's from our amazing HQ in Sheff ...
-
Cloud Platform Engineering Manager
Found in: Jooble UK O L C2 - 1 day ago
Twinkl Educational Publishing Sheffield, United KingdomAt Twinkl, we encourage diversity, and our doors are open to everyone. We're committed to creating an inclusive workplace for all. If you need any adjustments during the application process to showcase your abilities, please let us know. We're here to support you on your journey. ...
-
Lead Engineer/Data Engineer
Found in: Appcast UK C C2 - 4 days ago
HCLTech Leeds, United KingdomLead Data Engineer: · We need some strong Data engineer profiles... they need good experience with · Pyspark, Python, SQL, ADF and preferably Databricks experience · Job description: · Building new data pipelines and optimizing data flows using the Azure cloud stack. · Build ...
-
Lead Engineer/Data Engineer
Found in: Jooble UK O L C2 - 4 days ago
HCLTech Leeds, United KingdomLead Data Engineer:We need some strong Data engineer profiles... they need good experience withPyspark, Python, SQL, ADF and preferably Databricks experience Job description:Building new data pipelines and optimizing data flows using the Azure cloud stack.Building data products f ...
-
Python Developer Job
Found in: Jooble UK O C2 - 4 days ago
HCLTech Bradford, United KingdomHCLTech is a global technology company, home to 219,000+ people across 54 countries, delivering industry-leading capabilities centered around digital, engineering and cloud, powered by a broad portfolio of technology services and products. We work with clients across all major ve ...
-
Python Developer Job
Found in: Jooble UK O L C2 - 4 days ago
HCLTech Bradford, United KingdomHCLTech is a global technology company, home to 219,000+ people across 54 countries, delivering industry-leading capabilities centered around digital, engineering and cloud, powered by a broad portfolio of technology services and products. We work with clients across all major ve ...
Spark Architect - Sheffield, United Kingdom - PRACYVA
Description
Spark Architect / SME
Contract Role- 6 months to begin with & its extendable
Location: Sheffield, UK (min 3 days onsite)
Skills: