Senior Member
Full Job Description
D. E. Shaw is seeking resourceful and exceptional individuals for Data Engineer roles within its product development teams located in Hyderabad, Bengaluru, and Gurugram. At D. E. Shaw India Private Limited (DESIS), Data Engineers are instrumental in developing web robots, or web spiders, designed to traverse the web and extract data in various formats, including HTML, plain text, PDFs, and Excel, encompassing both structured and unstructured data. Key responsibilities include transforming scraped website data into a structured format and constructing automated, custom reports to serve business intelligence needs. The team also focuses on automating end-to-end data pipelines.
What you will do day-to-day:
As a Data Engineer, you will be responsible for critical aspects of data extraction, including understanding business data requirements, reverse-engineering website technologies and data retrieval processes, and re-engineering these processes by developing web robots for automated data extraction. You will also build monitoring systems to ensure data integrity and quality, manage website dynamic changes and layout adjustments for reliable data downloads, and develop scraping and parsing systems to convert raw data into structured formats. Operational support for high availability and zero data loss will also be part of your role. Additional tasks involve storing extracted data in recommended databases, building high-performance, scalable data extraction systems, and automating data pipelines.
Who we are looking for:
Basic Qualifications:
- 2 to 4 years of experience in website data extraction and scraping.
- Strong knowledge of relational databases, including writing complex SQL queries and performing ETL operations.
- Proficiency in Python for data manipulation and processing.
- Expertise in Python frameworks such as Requests, UrlLib2, Selenium, Beautiful Soup, and Scrapy.
- Solid understanding of HTTP requests and responses, HTML, CSS, XML, JSON, and JavaScript.
- Proficiency with Chrome debugging tools for reverse-engineering website dynamics.
- A strong academic record with relevant achievements.
- A BCA/MCA/BS/MS degree demonstrating a robust foundation and practical application of data structures and algorithms.
- Excellent problem-solving and analytical skills.
- Strong debugging capabilities.
Interested candidates are encouraged to apply via our careers website: https://www.deshawindia.com/recruit/jobs/Ads/Link/SrDataExtEnggDec25
D. E. Shaw India Private Limited is an equal opportunity employer and values diversity at our company. We do not discriminate on the basis of sex, race, colour, caste, creed, religion, pregnancy, national origin, age, military service eligibility, veteran status, sexual orientation, marital status, disability, or any other protected class.
Company
D. E. Shaw India Private Limited
The D. E. Shaw group is a globally recognized investment management firm, entrusted by investors worldwide to optimize the balance between risk and reward in asset management. While renowned as a pion...