Senior Data Engineer
— Tallinn, EstoniaBy agreement
Tallinn, Estonia (Remote)RemoteFull-timeSeniorNEW
Description (EN)
Join Veriff's Data Engineering team to build and operate the analytical data platform powering machine learning, data science, analytics, and reporting. You will own and evolve the data lake and data warehouse infrastructure, driving platform-level data management, governance, and reliability at scale.
Key Responsibilities:
- Own and evolve data lake and data warehouse infrastructure using Spark, Apache Iceberg, S3, Trino/Athena, and Redshift
- Design and maintain platform-level data transformation pipelines in Python and SQL
- Implement optimized storage formats (Parquet, Avro, ORC), partitioning strategies, and indexing
- Drive data governance initiatives: PII detection, access control, data cataloging, lineage tracking, and data quality frameworks
- Ensure availability, reliability, and cost efficiency with observability and monitoring
- Collaborate with ML, analytics, product, and engineering teams on data contracts and schema consistency
- Contribute to disaster recovery and multi-region reliability strategy
Required Qualifications:
- Strong experience with Python, SQL, and Apache Spark/PySpark
- Deep knowledge of modern analytics platform architecture and columnar/row-based data formats
- Experience with data governance and PII handling at scale
- Experience designing and operating data lake and warehouse infrastructure
- Solid understanding of storage optimization
- Experience building observability and monitoring for data platforms
- Strong problem-solving and collaborative mindset
Preferred Qualifications:
- Infrastructure as Code (IaC) and Terraform experience
- Docker and Kubernetes familiarity
- CI/CD pipeline experience
- Knowledge of Delta Lake, Hudi, or other table formats
- Data catalog tools (DataHub, Amundsen, AWS Glue Catalog)
- GDPR and data privacy regulation knowledge
- Streaming data pipeline experience
- AWS data stack experience
Benefits:
- Flexibility to work from home
- Stock options
- Extra recharge days
- Comprehensive relocation support to Estonia or Spain
- Medical, dental, and vision insurance
- Learning and Development budget
- Health and Sports budget
- Four weeks paid sabbatical after 5 years
Published 4 days ago
Apply →