Senior Data Engineer

Addressing the significant volume of machine log data offers organizations unparalleled real-time insight into their infrastructure and operations. This opportunity presents substantial technical challenges in ingesting, managing, and interpreting high-throughput streams of diverse data.

As a Senior Data Engineer at Sumo Logic in Noida, Uttar Pradesh, India, you will actively participate in, execute, and lead IT in the design and development of sophisticated data engineering solutions. You will play a crucial role in resolving complex issues related to data integrity, security, processes, and sanitization across Sumo Logic's comprehensive suite of cloud services, data tools, and business applications. Collaborating with a team of skilled professionals, you will contribute to high-impact projects, reporting to the Senior Manager of Data Engineering and ultimately to the Global Head of IT.

We are seeking a proficient data engineer who is dedicated to building scalable, resilient, secure, and cost-effective data solutions. A commitment to producing clean, elegant, maintainable, robust, and well-tested code is essential, working collaboratively to achieve superior solutions and providing expert guidance. Experience with the performance, scalability, and reliability considerations of 24/7 operational systems and solutions is highly desirable.

Responsibilities

• Design and implement complex, well-architected enterprise data solutions suitable for mid-to-large organizations.
• Enhance existing data warehouse systems, solutions, and processes through architectural reviews and design improvements.
• Apply DevOps methodologies and practices to infrastructure, cloud, and business application data.
• Improve systems to ensure strict data compliance, robust security measures, and cost optimization.
• Demonstrate strong understanding and hands-on experience with AWS data infrastructure and compute services supporting our data platform.
• Design and manage data schemas and data flows across corporate systems and applications to maintain compliance, integrity, and security.
• Deliver well-architected, end-to-end data solutions for a growing enterprise across diverse infrastructure, data sources, and business applications.
• Foster strong partnerships and collaborate with other Sumo Logic teams to develop mutually beneficial solutions that increase derived value.
• Ensure global delivery and alignment of all data initiatives, maintaining 24/7 system uptime.

Requirements

• Extensive experience with Databricks, Spark, and AWS services (EC2, RDS, Aurora, DynamoDB, S3, Kinesis).
• Proven experience developing scalable, secure, and resilient data architectures and implementations.
• A minimum of 5 years of industry experience with a demonstrated history of ownership and successful delivery.
• Proficiency in Python scripting, PySpark, and other relevant data frameworks and tools.
• Solid experience with SQL and data schemas.
• Experience with API calls and API-based data ingestion.
• Hands-on and in-depth experience with Git and GitHub.
• Proven experience in AWS infrastructure management and deployment for data platforms, including EC2, S3, RDS, VPC, basic network adjustments, KMS, and PrivateLink (a bonus).
• Experience working across the data ingestion, data storage, and data consumption layers.
• Familiarity with both structured and unstructured data.
• Experience with Agile development methodologies, including Jira, sprints, and story pointing.
• Experience building robust, well-architected designs for enterprise-scale data architectures and workflows.
• A passion for continuous learning and deep technological curiosity.
• Excellent verbal and written communication skills.
• Experience and comfort with an on-call schedule for enterprise systems.

Desirable Skills

• Experience in AI/ML, Data Science, LLMs, Contextualization, Amazon Bedrock, Amazon SageMaker, and Iceberg.
• Experience with Terraform is highly advantageous.
• Experience with the following technologies is a plus: Tableau, Looker, AWS Quicksight.
• Experience with Big Data services such as HDFS, Spark, Hive, HBase, Yarn, and Oozie.