Senior Member

D. E. Shaw India Private Limited is seeking exceptional Data Engineers for its product development teams in Hyderabad, Bengaluru, and Gurugram. As a Data Engineer specializing in Data Extraction, you will develop sophisticated web robots (spiders) to crawl and retrieve data in various formats, including HTML, plain text, PDFs, and Excel, from structured and unstructured sources. Your responsibilities will include efficiently scraping website data into structured formats and building automated custom reports for business knowledge. You will also focus on automating end-to-end data pipelines.

Your daily responsibilities will include:

• Understanding business data requirements and reverse-engineering website technologies and data retrieval processes.
• Developing web robots to automate data extraction and building monitoring systems to ensure data integrity and quality.
• Managing dynamic website changes and layout modifications to ensure clean data downloads.
• Constructing scraping and parsing systems to transform raw data into structured formats.
• Providing operational support for high availability and zero data loss.
• Storing extracted data in appropriate databases and building high-performing, scalable data extraction systems.
• Automating data pipelines.

We are looking for candidates with:

• 2 to 4 years of experience in website data extraction and scraping.
• Strong knowledge of relational databases, SQL for complex queries, and ETL operations.
• Proficiency in Python and its data manipulation capabilities.
• Expertise in Python frameworks such as Requests, UrlLib2, Selenium, Beautiful Soup, and Scrapy.
• A solid understanding of HTTP requests/responses, HTML, CSS, XML, JSON, and JavaScript.
• Experience with Chrome debugging tools for website reverse-engineering.
• A strong academic background with a BCA/MCA/BS/MS degree and a firm grasp of data structures and algorithms.
• Excellent problem-solving, analytical, and debugging skills.

This is a permanent position with D. E. Shaw India Private Limited, offering excellent benefits, a collegial work environment, and an attractive compensation package. We encourage candidates with relevant experience looking to restart their careers after a break to apply through our website: https://www.deshawindia.com/recruit/jobs/Ads/Link/SrDataExtEnggDec25.

Learn more about our Recommence, gender-neutral return-to-work initiative. For details on our recruitment process and applicant data handling, visit https://www.deshawindia.com/careers.