Software Engineer

Poshmark's Big Data team is central to the organization, dedicated to building a world-class big data platform that extracts value from data for both internal stakeholders and customers. Their mission is to democratize data, support business expansion, and develop data and ML pipelines to drive critical business initiatives.

The Data Engineering team is seeking an experienced Software Engineer to manage Poshmark's growth data, ensuring real-time access to high-quality data for all stakeholders. This role demands a strong understanding of software engineering best practices and excellent development skills for constructing and maintaining scalable, optimized real-time and batch data pipelines. Collaboration with Data Science, Analytics, and other Engineering teams is essential for building new ETL processes and analyzing terabytes of data.

The engineer will write clean, scalable code to extract datasets from diverse sources, including external APIs, S3 transfers, and Kafka. You will utilize cutting-edge technologies such as Scala, Ruby, Apache Spark, Airflow, Redshift, and Databricks, and manage growth data infrastructure including ETL pipelines, Hive tables, Redshift tables, and BI tools. The ideal candidate will shape the future of Poshmark's growth data systems, focusing on scalability and stability.

Responsibilities:

• Design, develop, and maintain growth data pipelines, integrating paid media sources like Facebook and Google to generate business insights.
• Build highly scalable, available, and fault-tolerant data processing systems using AWS technologies, Kafka, Spark, and other big data technologies. These systems will handle batch and real-time data processing for hundreds of terabytes of daily ingested data and a petabyte-sized data warehouse.
• Architect, design, and develop critical data pipelines at Poshmark.
• Maintain and support existing platforms, evolving them to newer technology stacks and architectures.
• Contribute to the continuous improvement of development best practices.

Desired Skills:

• 1+ years of overall software development experience.
• Strong software development experience with Big Data technologies and hands-on coding proficiency.
• Expertise in architecting and building large-scale data processing systems using Big Data technologies such as Spark, Hadoop, EMR, Kafka/Kinesis, Flink, or Druid.
• Proven ability to deliver complex projects end-to-end.
• SQL expertise with knowledge of data warehouse technologies like Redshift.
• Self-driven with a commitment to taking ownership, making pragmatic technical decisions, and collaborating effectively with cross-functional teams.
• Excellent communication skills and the ability to work cross-functionally.
• Experience with Google Apps Script, Databricks, or API integrations is a plus.

Technologies Used:

• Languages: Ruby, Scala, Python, Google Apps Script
• Databases: MongoDB, Redshift, Druid
• Orchestration: Airflow, Jenkins
• Big Data: Spark, Hadoop