Job Details

Senior Software Engineer

Get AI-powered advice on this job and more exclusive features.

Direct message the job poster from Vernovis

Location/Work Structure: Columbus, OH / Hybrid / 12–24 Month Contract

Who We Are

Vernovis is a Total Talent Solutions company specializing in Technology, Cybersecurity, Finance & Accounting functions. At Vernovis, we help professionals achieve their career goals by matching them with innovative projects and dynamic contract opportunities across Ohio and the Midwest.

Client Overview

Vernovis is partnering with a leading organization in scientific data management and innovation to modernize its big data platform. This initiative involves transitioning legacy systems, such as Cascading, Hadoop, and MapReduce, to Spark 3, optimizing for scalability and efficiency. As part of this well-established organization, your work will contribute to transforming how big data environments are managed and processed.

What You'll Do

• Legacy Workflow Migration: Lead the conversion of existing Cascading, Hadoop, and MapReduce workflows to Spark 3, ensuring seamless transitions.
• Performance Optimization: Utilize Spark 3 features like Adaptive Query Execution (AQE) and Dynamic Partition Pruning to optimize data pipelines.
• Collaboration: Work closely with infrastructure teams and stakeholders to ensure alignment with modernization initiatives.
• Big Data Ecosystem Integration: Develop solutions that integrate with platforms like Hadoop, Hive, Kafka, and cloud environments (AWS, Azure).
• Support Modernization Goals: Contribute to key organizational initiatives focused on next-generation data optimization and modernization.

What Experience You'll Have

• Spark 3 Expertise: 3+ years of experience with Apache Spark, including Spark 3.x development and optimization.
• Migration Experience: Proven experience transitioning from Cascading, Hadoop, or MapReduce to Spark 3.
• Programming Skills: Proficiency in Scala, Python, or Java.
• Big Data Ecosystem: Strong knowledge of Hadoop, Hive, and Kafka.
• Performance Tuning: Advanced skills in profiling, troubleshooting, and optimizing Spark jobs.
• Cloud Platforms: Familiarity with AWS (EMR, Glue, S3) or Azure (Databricks, Data Lake).

Vernovis offers Health, Dental, Vision, Voluntary Short & Long-Term Disability, Voluntary Life Insurance, and 401K.

Vernovis does not accept inquiries from Corp to Corp recruiting companies. Applicants must be currently authorized to work in the United States on a full-time basis and not violate any immigration or discrimination laws.

Vernovis provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.

This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.