​Develop, maintain, and support data pipelines and systems to meet business SLAs using Scala, Apache Spark, and cloud technologies.

Ensure code quality, performance, and maintainability by supporting version control and deployment practices.

Perform unit testing, support integration and performance testing, and ensure adequate test coverage for developed components.

Research emerging technologies, recommend solutions, and contribute to the continuous improvement of architecture and engineering practices.

Own data quality, integrity, and consistency for assigned datasets and ensure robust monitoring and alerting are in place.

Collaborate with cross-functional teams to design scalable data solutions and models.

Analyze large datasets to identify trends, gaps, and improvement opportunities.

Support DataOps best practices including orchestration (Airflow, Oozie), workflow automation, and real-time streaming (Kafka, NIFI).

​

Required Qualifications:

5+ years of hands-on experience in Scala or Java development.

5+ years of experience with SQL and NoSQL databases, working with both structured and unstructured data.

5+ years of experience with Apache Spark and distributed data processing.

Strong experience with big data ecosystem tools such as Hive, Impala, OOZIE, Airflow, Kafka, and NIFI.

Experience in Data Modeling and designing large-scale data architectures.

Proficient in working on cloud-based or on-prem Big Data platforms like AWS Redshift, Google BigQuery, Azure Data Warehouse, Netezza, or Teradata.

Strong problem-solving skills and the ability to analyze complex data sets for insights and optimization.

Nice to Have:

Programming experience in Python.

Experience with data governance, security, and compliance in cloud environments

Familiarity with CI/CD pipelines and DevOps tools.