Lead Big Data Developer

Reading Time: 2 minutes

Job title: Lead Big Data Developer

Company: Software Guidance & Assistance

Job description: Software Guidance & Assistance, Inc., (SGA), is searching for a Lead Big Data Developer for a Contract engagement with a financial services firm in Rockville, MD.This position is 100% Remote.Responsibilities :

Understand complex business requirements
Design and develop ETL pipeline for collecting, validating and transforming data according to the specification
Develop automated unit tests, functional tests and performance tests.
Maintain optimal data pipeline architecture
Design ETL jobs for optimal execution in AWS cloud environment
Reduce processing time and cost of ETL workloads
Lead peer reviews and design/code review meetings
Provide support for production support operations team
Implement data quality checks.
Identify areas where machine learning can be used to identify data anomalies

Required Skills:

Strong experience in Java or Scala
Strong experience in big data technologies like AWS EMR, AWS EKS, Apache Spark
Strong experience with serverless technologies like AWS Dynamo DB, AWS Lambda
Strong experience in processing with JSON and csv files
Must be able to write complex SQL queries
Experience in performance tuning and optimization
Familiar with columnar storage formats (ORC, Parquet) and various compression techniques
Experience in writing Unix shell scripts
Unit testing using JUnit or ScalaTest
Git/Maven/Gradle
Code Reviews
Experience with CI/CD pipelines
Agile

Experience & Qualifications :

7+ years of experience in programming language Java or Scala
7+ years of experience in ETL projects
5+ years of experience in big data projects
3+ years of experience with API development (REST API’s)
Believes in Scrum/Agile, and has deep experience delivering software when working on teams that use Scrum/Agile methodology
Strong and creative analytical and problem-solving skills

Preferred Skills:

AWS Cloud
BPM/ AWS Step Functions
Python scripting
Performance testing tools like Gatling or JMeter

SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. Each year, we match consultants like you to more than 1,000 engagements. When we say let’s work better together, we mean it. You’ll join a diverse team built on these core values: customer service, employee development, and quality and integrity in everything we do. Be yourself, love what you do and find your passion at work. Please find us at .SGA is an Equal Opportunity Employer and does not discriminate on the basis of Race, Color, Sex, Sexual Orientation, Gender Identity, Religion, National Origin, Disability, Veteran Status, Age, Marital Status, Pregnancy, Genetic Information, or Other Legally Protected Status. We are committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, and our services, programs, and activities. Please visit our company to request an accommodation or assistance regarding our policy.

Expected salary:

Location: Rockville, MD

Job date: Fri, 07 Jun 2024 02:46:56 GMT

Apply for the job now!