Introduction:
We are seeking an experienced and motivated Senior MLOps Engineer to join our dynamic team. As a Senior MLOps Engineer, you will be a key player in designing, building, and maintaining robust and scalable machine learning operational pipelines. Your contribution will significantly impact our customer-facing tools and innovative projects.
Role and Responsibilities:
- Collaborate closely with cross-functional teams to develop and maintain end-to-end machine learning operational pipelines.
- Design and implement scalable solutions for deploying, monitoring, and maintaining machine learning models in production.
- Leverage orchestration tools like Dagster, our single source of truth for asset management.
Contribute to pipeline prototyping using innovative tools like gathr.one.
- Implement event-driven architectures using Kafka, hosted on Aiven, as our messaging broker.
- Develop data engineering pipelines involving EL (Extract, Load) processes, harnessing Dagster, Spark Structured Streaming, and potentially Delta-Lake.
- Collaborate with teams working on transformation (T) pipelines, leveraging Dagster, Spark, and DBT.
- Utilize S3 as our warehouse storage solution, with potential integration with Databricks as the query engine.
- Contribute to the exploration of hot store options such as Druid, Pinot, and Clickhouse, enhancing real-time data availability.
- Participate in code reviews, mentor junior team members, and drive the adoption of best practices within the MLOps team.
Qualifications:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Proven experience as an MLOps Engineer, focusing on deploying and maintaining machine learning models in production environments.
- Strong proficiency in programming languages such as Python, and experience with relevant libraries and frameworks.
-Solid understanding of cloud platforms like AWS, Azure, or GCP, and familiarity with infrastructure as code (IaC) tools.
- Experience with containerization technologies like Docker and orchestration tools like Kubernetes.
Familiarity with continuous integration, continuous deployment, and version control systems (e.g., Git).
Excellent problem-solving skills and ability to troubleshoot complex issues in production environments.
Strong communication skills and ability to collaborate effectively with cross-functional teams.
This job is already closed and no longer accepting applicants, sorry.
