Lead Data Engineer
Indecomm Global Services
hace 5 días

Currently seeking a Lead Software Developer in our Data & Intelligence (DxI) team. We are developing future data platform to empower the Company’s business growth.

We develop central data models, data resources, and metrics that are critical to our business. As part of this team, you will work as tech leader and subject matter expert to lead development of scalable and robust data workflows.

Responsibilities :

  • Partner with technical and non-technical coworkers across the company
  • Build and manage data workflows in the cloud using Apache Airflow Optimize workflows and ETL / ELT jobs
  • Integrate existing data platforms with AWS services like Amazon S3, Amazon Redshift, Amazon EMR, AWS Batch, and Amazon SageMaker
  • Closely collaborate with stakeholders to ensure complete end-to-end high-quality data solutions to drive business growth Design and develop data resources to enable self-serve data consumption.
  • Build tools for testing workflow DAGs and validating data.
  • Define and share best practices on data modeling, workflow management, and ETL tuning Maintain workflows and troubleshoot complex data, deployment, security issues.
  • Being a trusted tech leader and subject matter expert by providing technical guidance and consultation to team members and stakeholders across the company
  • Coach and mentor team members
  • Requirements :

  • 6+ years of experiences as a Data Engineer
  • Excellent English communication skills (B2+ minimum)
  • Have a solid track record of driving complex projects to success by leading designs, justifying technical trade-offs, and solving hard challenges persistently.
  • Experience with workflow management solutions like Airflow Strong knowledge of AWS Cloud services
  • Experience in schema design and dimensional data modeling Strong skills in SQL and Python coding
  • Strong skills in distributed system optimization (e.g. Spark, Presto, Hadoop, Hive)
  • Strong track record of building large scale data sets, preferable petabyte scale and beyond Ability to perform basic statistical analysis to inform business decisions.
  • Experience with Docker and with engineering development tools like Git, Jenkins, Artifactory Excellent communication skills, both written and verbal
  • Preferred Qualifications :

  • Knowledge of Terraform
  • Experience with AWS Lake Formation
  • Experience with notebook-based Data
  • Science workflows Familiarity with A / B testing and machine learning
  • Reportar esta oferta

    Thank you for reporting this job!

    Your feedback will help us improve the quality of our services.

    Mi Correo Electrónico
    Al hacer clic en la opción "Continuar", doy mi consentimiento para que neuvoo procese mis datos de conformidad con lo establecido en su Política de privacidad . Puedo darme de baja o retirar mi autorización en cualquier momento.
    Formulario de postulación