Data Engineer - Big Data

The customer is an international player in the health and energy sectors. Its products can be found all over the world, particularly in the industrial, healthcare and environmental sectors. It places people and the planet at the heart of its activities, driving sustainable, responsible growth.

‍

You will work within the customer's "Global Data Operator" structure. Within this structure, the Data Engineer industrializes data processing with a view to adding value to data within application contexts ranging from finance to the Internet of Things, security, customer experience, etc. He or she works in close collaboration with Data Scientists, Data Architects and Data Managers to design and implement solutions based on data. He or she works closely with Data Scientists, Data Architects and Data Managers to design and implement solutions based on a performance/security/operation triptych, in a cloud environment and with a DevOps mindset.

‍

Long-term assignment. The context is international (French- and English-speaking collaborators) and team-based.

Working within the Global Data Operator structure, you will be involved in current and early stage projects. The scope of the mission includes :

• Design, develop, test, and put into production "Big Data" processing in various business and application contexts
• Together with Data Architects and DevOps, design, implement and optimize data processing pipelines to feed data applications from the Group's data sources.
• Data processing support
• Design and implement visualization and analysis tools, and automations, to facilitate the work of Data Scientists and other Analysts
• Ensure that the Group's data processing security and confidentiality rules are properly applied.
• In the case of a senior profile, act as Technical Advisor to the Junior Data Engineers in the team.
  

• Successful Big Data application development experience, ideally in an AWS or Azure cloud environment
• Excellent level in Data and cloud technologies: Spark, AWS (EMR, Lambda, Kinesis, DynamoDB, Athena), Airflow, Elasticsearch.
• Knowledge of Azure BI tools (Azure Data Warehouse, Azure SQL database, Power BI) or a time series database (InfluxDB, TimescaleDB, Prometheus, ...) would be a plus.
• Python
• Spark (PySpark, Scala), Hadoop
• SQL
• Excellent command of data management functions: collection, analysis, distribution, etc.
• Ability to select and apply certain standard Machine Learning libraries and algorithms, and to set up appropriate metrics would be a plus.