CV Data : Recevez rapidement les profils qui vous intéressent

Résumé des emplois de Mehdi,
freelance DATA résidant dans Paris (75)

Experience

❖ Société générale – DSI CFT - March 2021 - Today - Paris

Project « Finance District » :

Mission: Finance District supports the business areas of the CFT IT department in the design and
implementation of solutions related to data
Position: Technical Lead Big Data Engineer.

Lead tasks:
❖ Daily technical support for data engineers.
❖ Implementation of a monitoring module for the execution plan of Quartier Finance.
❖ Design of a data quality management module.
❖ Design of a production job monitoring module.
❖ Supervision of the technical and functional migration from Talend Big data to Scala/Spark.
❖ Ensuring the migration of Spark 2.1 projects to 2.4.
❖ Implementation of a data anonymization solution.
❖ Implementation of the Scala/Spark CI/CD pipeline.
❖ Ensuring the migration from Hortonworks to CDP.
Technical tasks:
• Analyzing and implementing "hot fixes" in production.
• Implementation of a Scala/Spark framework to facilitate and standardize Scala/Spark
developments.
• Developing a solution for sending files via WebHdfs.
• Tuning the performance of Scala/Spark applications for batch interval, parallelism, and memory.
• Optimization of existing algorithms in Hadoop using SparkSession, Spark-SQL, Data Frames,
and Pair RDDs.
• Manipulating large datasets using Partitions, Spark memory capabilities, Spark Broadcasts, and
efficient Joins.
• Developing audit logic to optimize the load in append mode.
• Developing solutions for pre-processing large sets of structured and semi-structured data with
different formats (text files, Avro, sequence files, JSON records).
• Use of Parquet, ORC, Avro files according to the technical need.
• Study of the choice of Spark partitions for "write HDFS" and the calculation of the coalesce to
avoid the shuffle
• Implementation of performance improvements based on job monitoring via SparkUI
• Adding "persist" via Spark and "serialization" of data as needed
• Use of RDD, DataFrame, DataSet according to the technical need.
• Development of some UDF on Spark 2.4
• Deployment and Orchestration of the project via ControlM
• Use of scalaTest to ensure unit tests and code coverage at the SonarQube level
• Development of test cases, test scenarios with TDD logic using Cucumber.
• Log recovery via Scala/Kafka & Spark Streaming from YARN for analysis and monitoring of
the application.

Project « PRORATA VAT »:
Mission: Equip accounting and fiscal business with a digital solution for calculating the "PRORATA
TVA" tax base
Position: Technical Lead Big Data Engineer.
Lead tasks:
❖ Facilitate or lead agility ceremonies.
❖ Implementation of the Data solution architecture using "Scala/Spark" and establishing a link
with visualization tools.
❖ Optimization of the performance and scalability of "Scala/Spark" data systems for jobs.
❖ Ensuring the quality and reliability of data.
❖ Synchronize with web restitution and PowerBI teams.
❖ Collaborate with security teams to ensure the security of data.
❖ Definition of standards and best practices for data projects.
Technical tasks:
• Writing technical specifications.
• Definition of big data architecture.
• Scala/Spark development of calculations for the "prorata tva" bases.
• Implementation of Scala/Spark jobs.
• Development of data extraction jobs from APIs.
• Analyze, design and build Modern data solutions using Azure PaaS service to support visualization
of data. Understand current Production state of application and determine the impact of new
implementation on existing business processes.
• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a
combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake
Analytics. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage,
Azure SQL, Azure DW) and processing the data in In Azure Databricks.
• Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and
load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, writeback tool and backwards.
• Developed Spark applications using Pyspark and Spark-SQL for data extraction, transformation
and aggregation from multiple file formats for analyzing & transforming the data to uncover
insights into the customer usage patterns.
• Responsible for estimating the cluster size, monitoring and troubleshooting of the Spark
databricks cluster.
• Experienced in performance tuning of Spark Applications for setting right Batch Interval time,
correct level of Parallelism and memory tuning

Project « C3S TAX »:
Mission: Proposal for a technical data/big data and Azure Cloud architecture to meet the need for
industrialization of the C3S tax calculation.
Position: Data Architect
Architect tasks:
❖ Define an architectural roadmap and technological standards for the development of the Big Data
solution and link it with the web components and PowerBI.
❖ Define the strategy for acquiring external data and their exploitation in the datalake.
❖ Define the data storage architecture in Azure Synapse for the use of PowerBI.
❖ Implement Trino Cluster to accelerate data reading from the web to the datalake.
❖ Participate in the project's ARB to present the Technical architecture of the project to the head of
the IT department, the RSI, and the project sponsor.
❖ Anticipate security issues and work jointly with security consultants to obtainn “GoLive By
Design”.

❖ Bouygues Telecom – DSI/EWD - March 2018 – February 2021 - Paris
Project « eSIM »:
Missions: Project « eSIM »: Embedded SIM (eSIM, or eUICC) technology allows mobile users to
download a carrier profile and activate a carrier's service without having a physical SIM card.
Position: Lead Data Engineer.
• Excel in guiding the work of technical teams. Articulated project goals and scope, translated
business needs into technical terms
• Use continuous improvement approach to review and improve the existing processes, always with
an aim of improving the cycle time, reduce churn and reduced unit costs, to ensure our business
and department’s objectives are met (productivity metrics and key customer service indicators)
• Establish and promote database management principles, models, best practices, standards and
ensure their practical adoption. Collaborates with the development teams to establish data quality
baselines.
• Lead the effort to build, implement and support the data infrastructure; ingest and transform data
(ETL/ELT process) & programming/scripting languages such as Pyspark, Scala.
• Define and maintain the testing strategy and various test plans. This should cover both automation
and manual testing.
• Build fault tolerant, adaptive and highly accurate data computational pipelines. Tune queries
running over billion of rows of data running in a distributed query engine.
• Performed end-to-end Architecture & implementation assessment of various AWS services like
Amazon EMR, Redshift, S3. Implemented the machine learning algorithms using python to
predict the quantity a user might want to order for a specific services so we can automatically
suggest using kinesis firehose and S3 datalake.
• Used AWS EMR to transform and move large amounts of data into and out of other AWS data
stores and databases, such as Amazon Simple Storage Service (Amazon S3) and Amazon
DynamoDB.
• Used Spark SQL for Scala, Python interface that automatically converts RDD case classes to
schema RDD. Import the data from different sources like HDFS/HBase into Spark RDD and
perform computations using PySpark to generate the output response.
• Creating Lambda functions with Boto3 to deregister unused AMIs in all application regions to
reduce the cost for EC2 resources.

Project « Asterix »:
Missions: The project - code-named project « Asterix » - aims at accelerating the roll out
of FTTH across medium-dense regions in France. SDAIF will “co-invest” by acquiring
long-term access rights (“IRU”) from Orange (in charge of physical roll-out in these areas),
and rent them to retail operators, of which Bouygues Telecom as anchor tenant.
Position: Lead Data Engineer.
• Answering and predicitng business needs with data analytics
• Work as a key member of an agile development team utilising Scrum based methodologies and
tools
• Analyzed the business requirements and translate them into technical specifications that can be
used by developers to implement new features or enhancements.
• Influence the direction of development in order to assist feature enrichment and platform growth
• Developed and implemented data pipelines using AWS services such as Kinesis, S3, EMR,
Athena, Redshift to process petabyte-scale data in real time.
• Designed and developed scalable AWS solutions using Scala/Spark for storing and processing
large amounts of data across multiple regions.
• Provided support during all phases of development including design, implementation, testing,
deployment and maintenance of applications/services.
• Participated in cross-functional teams (e.g., infrastructure engineering, Web Team) when required
to ensure effective communication between groups with overlapping functionality or shared
resources.
• Developing new features and extending existing data platform using Python and Scala/spark a
range of deployment automation and monitoring tools
• Support and coaching of software developers and data engineers through advice, guidance and
mentoring
• Review the code of others for accuracy and functionality and to offer guidance for improvement if
needed
• Monitor and assist with the deployment of code through test environments towards production and
the handling of any issues that arise using CloudWatc...