Data Engineer II/ Senior Data Engin...

About The Role

The DataDev team builds and operates Cvent's platform services that power secure data deletion and analytics across all Cvent products.

• Implement metadata-driven deletion, onboarding processes for new products, and ensure compliant removal of customer data across multiple data stores in line with regulations like GDPR/NSA.
• Manage a centralized lakehouse to standardize data handling, reduce manual effort, and improve trust among customers regarding their data privacy.
• Develop robust data services and pipelines (ETL/ELT/CDC) using technologies such as Snowflake, Redshift, AWS S3/Kinesis, and Apache Spark. Focus on capturing, cleansing, transforming, and routing customer data to destination systems for near real-time processing.
• Maintain strict adherence to regulatory frameworks by removing customer data from underlying stores based on standardized metadata policies (Right-to-be Forgotten).
• Onboard Cvent products onto established data pipelines through configuration, validation, and tight integration with product teams.
• Investigate and remediate pipeline issues including missed tables, incorrect metadata mapping, performance bottlenecks, and drive preventive architectural improvements.
• Ensure connectivity and compatibility between application services and underlying data stores across various Cvent products.
• Create technical design documents for major modules; propose approaches, review code, test solutions, and implement designs while adhering to non-functional requirements (scalability, reliability).
• Participate in Agile ceremonies: standups, grooming, retrospectives, PI planning. Deliver work items with clear estimates and status updates.
• Provide on-call operational support for DataDev services including managing observability dashboards, alerts, logs, and metrics to monitor service health.
• Maintain runbooks, write onboarding guides, document best practices, and share knowledge effectively with peers.
• Collaborate closely with analytics teams to support downstream reporting needs, BI integration, and compliance reporting requirements.
• Utilize approved AI tools (LLMs) responsibly for tasks like log analysis, impact assessment, documentation generation, and troubleshooting—always validating output against code/logs before deployment.
  Note:Avoid pasting sensitive production data into unapproved AI tools. Treat all AI suggestions as proposals requiring human validation.
• Contribute feedback to improve how DataDev services can be augmented by AI (e.g., metadata validation helpers, failure summarization).

Requirements

• Bachelor's degree in Computer Science or related technical discipline.
  Experience: 2-6 years of hands-on programming with Java and strong object-oriented design skills.
• Rigorous experience building RESTful Web Services using Dropwizard or Spring Boot frameworks.
• Strong knowledge of at least one relational (RDBMS) and non-relational database system. Proficient in complex SQL queries across major platforms.
• 2+ years developing applications with AWS services: IAM, S3, CloudFormation, Kinesis, Redshift, CDK.
  Architecture:Demonstrated experience contributing to architecture design including patterns and non-functional requirements (performance, security).
• Ethical AI proficiency: Ability to prompt effectively while adhering to data privacy guidelines. Basic understanding of LLM mechanics (prompts, context limits) is essential.

Preferred Skills

• Experience with GDPR/NSA compliance frameworks and metadata-driven deletion/masking solutions.
  Technical proficiency in Node.js, Python, Apache Spark, TypeScript for auxiliary tooling.
  Expertise with Java Spring Boot specifically paired with Snowflake data platforms. Knowledge of CDC tools like Debezium or Fivetran is a plus.