Job Details

Big Data Software Engineer - Intelligent DataOps Software Leader

Big Data Software Engineer — Batch & OLAP

Overview

Build robust batch pipelines and analytical stores for large-scale reporting, experimentation, and ML feature generation.

Responsibilities

• Develop cost-efficient batch processing (Spark/Dataproc/EMR) and orchestration (Airflow/Dagster).
• Design lakehouse tables (Iceberg/Delta/Hudi) with compaction, partitioning, and Z-ordering.
• Implement semantic/metrics layers and OLAP acceleration (BigQuery/Snowflake/ClickHouse).
• Enforce governance: lineage, access controls, PII handling, and retention policies.

Requirements

• 4+ years in big data; expert SQL and performance tuning.
• Strong understanding of storage formats, file layout, and query engines.

About Andiamo

Andiamo is a globally recognized staffing and consulting firm specializing in placing the top 2% of technology and go-to-market professionals with the world's largest and most well-known companies.

For over 20 years, we have maintained the status of tier-one vendor for firms such as Amazon, Bloomberg, Palantir, MasterCard, Visa, Two Sigma, Citadel, as well as other major financial services firms, elite hedge funds, Google-backed tech start-ups, and major software firms.

Our talent solutions include Permanent Placement, Contract Staffing, Executive Search, and Dedicated Recruiting Services (RPO). Find out more at www.andiamogo.com

Details

• Seniority level: Mid-Senior level
• Employment type: Full-time
• Job function: Engineering and Information Technology
• Industries: Staffing and Recruiting

Referrals increase your chances of interviewing at Andiamo by 2x

New York, NY