For a long-term assignment, an organization is looking for an experienced Big Data Engineer. You will join a technically challenging data environment where you will be responsible for realizing, integrating, and optimizing large-scale data streams. You will work closely with data scientists and translate complex analytical questions into concrete, high-performance technical solutions.
What you do
- Design and develop data pipelines
- Perform data cleansing, preparation & integration
- Work with large-scale data processing and Spark ETL
- Data wrangling, joining
- Provide component orchestration
- Contribute to real-time architectures and API integrations
- Collaborate with data scientists to translate analytical models into technical solutions
- Analyze and resolve issues within data processing and orchestration
What they ask
- Experience within a modern Big Data environment
- Knowledge of Hadoop, Pig & Hive
- Knowledge of Spark / PySpark
- Knowledge of Kafka & Confluent
- Knowledge of Apache Airflow
- Knowledge of Python
- Knowledge of Java / JavaScript
- Knowledge of Docker & Kubernetes
- Experience with API development
- Knowledge of database technologies such as Oracle, NoSQL, Graph
- Experience with GIT and CI/CD
- Experience with real-time architectures
- Experience with cloud environments such as AWS or Azure, AWS S3, data monitoring & quality, MLOps/LLMops, and front-end development are additional assets.
Profile
You are an experienced Big Data Engineer who enjoys hands-on work and can translate complex data challenges into scalable technical solutions. You work analytically and structured, and you are also strong in collaboration with data scientists and other technical stakeholders.
How to apply
View the full assignment text and application details once your tailored application is ready.
Order a tailored application to view the full assignment and application details.
More context, less searching.
You get enough context to judge whether this job is relevant. The full brief, client details and next steps stay available inside the app.