Software Engineer

Cognite is seeking a Senior Data Platform Engineer to join its team in revolutionizing industrial data management using Cognite Data Fusion. This role involves building high-performance distributed systems for Fortune 500 companies within a fast-paced startup environment.

Key Responsibilities:

• Data Systems Design: Build robust pipelines using Apache Spark, Flink, and Kafka; optimize storage/retrieval for time-series data; implement Redis caching strategies.
• Distributed Processing: Engineer Spark applications with deep Catalyst optimizer knowledge; develop real-time streaming solutions (millions of events/sec); design S3/GCS-based data lakes using Parquet/ORC formats.
• Scalability & Performance: Scale systems to 10K+ QPS while ensuring high availability; optimize JVM performance and memory management; implement fault-tolerant architectures with circuit breakers.
• Innovation: Contribute to open-source projects (Spark, Kafka); prototype new technologies for industrial challenges; collaborate on architectural reviews.

Tech Stack Highlights:

Languages: Kotlin, Scala, Python, Java | Big Data: Spark, Flink, Kafka | Storage: PostgreSQL, ClickHouse, Elasticsearch | Infrastructure: Kubernetes, Docker, Terraform