Marwa - Développeur APACHE KAFKA

PROFESSIONAL EXPERIENCE

Teaching missions
November 2019 - Today: Trainor at ORSYS (Big data, Kafka, Spark, machine learning,
python, Talend for Big Data ..)
July 2020 - Today: Trainor at WevOps
July 2020 - Today: Trainor at NobelProg
January - June 2020: Teacher at the USME Sudria engineering school (data analysis
module)
Speaking missions
October 2020: Speaker at the Confluent Streaming Event, Paris- France

February 2019 - Today: Big Data consultant and trainer at Zenika, Paris- France
I work as a consultant with my clients and I give trainings (Kafka, machine learning,
rabbitMQ)
I was a speaker at Confluent Streaming event - 2nd edition Paris 2019
Missions:
October 2019 - Today: Kafka developerat ITCE ( Banking)
Working with a team of developers and architectures.
Developing kafka topologies to create business values; Converting Json to Avro using
generic avro record, calculating scores for client authentication..
Security: ACLs, Encryption, Authentication
Spring Cloud Stream, Unittests, Integration tests..
February 2019 - September 2019: Kafka developer at SFR (Telecom)
Kafka project from scratch with the collaboration of the OPS team
- Develop kafka topologies (Java) to meet the businees requirements in a real time mode.
- Deal with kstreams and ktables, state stores and statefull/stateless transformation using KSQL,
transformers and Kafka streams DSL.
- Tune different kafka parameteres to get higher performance (I/O, throughput, handling
exception, buffer size, linger.ms, ...)
- Testing topologies with json and avro schemas
- Data ingestion using flume, spark (python) and custom kafka producers. (users' events, logs'
monitoring...).
- Deploy flume agents and develop their interceptors to process events before writing to kafka
(headers' topics, keys, timestamps..)
- Analyse data and detect common behaviors for tuning estimations of different business metrics.
- Python custom producers and consumers : producers watching new events, consumers with
filters and select statements, unittests..
- SpringBoot, Gitlab CI , Jenkins, Nexus
November 2018-January 2019: Solo traveller in Europe.

October 2017-October 2018: Data Engineer at Runtastic GmbH, Linz, Austria
- Flume: Introduce new flume agents (Cloudera manager), configure sinks (Jdbc, hdfs,
elasticsearch) and sources (RMQ, kafka), write interceptors to set event topic and headerKey
(Java), dynamic redirection of events based on topics using memory channels, kafka sources and
hdfs sinks
- Oozie: Implement daily jobs (Pyspark), Modify running coordinators, Implement Hadoop
applications (processing, sanity checks, anomaly detection, archiving)
- Cloudera Manager: monitor daily load (charts), manage security (kerberos), monitor services'
health
- SSIS package :Google Spreadsheet service (Automatically loading data from google
spreadsheet to DWH, c#)
- Manage data between Mysql database and Hadoop (sqoop, python)
- Manage VirtualEnvs with pySpark: Script to create new virtual environment on hadoop cluster
and install new python packages
- Data Quality : Sanity checks, Anomaly Detection, Alerting
- Alerta: install and fully manage Alerta, push alerts to Alerta server, write python plugins to meet
specific requirements(automatically create Jira tasks, include attachments, send alerts based on
customers)
- Data Security (GDPR, Hadoop): authorization, authentication (kerberos), Anonymization
- Ruby: Getting connection to Hadoop and fetch data for GDPR requirements (user-export
service)
- Git
November 2016-August 2017: Data scientist: Graduation Project at Austrian
Institute of Technology, Vienna, Austria
- Analyzing human mobility.
- Fingerprinting using Support Vector Machine method.
- Estimation and Prediction of crowd dynamics based on Hidden Markov Model (Python)
The project was validated in real circumstances within one of the biggest festivals in central
europe: Frequency festival.
July 2016-August 2016: Data Engineer Internship at Axe Finance, Tunis, TUNISIA
- Analyzing the existing provisioning engine offered as part of Axe Finance banking solutions
- Working on the database and modifying it in order to comply with the new standard (sql server)
- Applying the IFRS9 Standard in order to implement the provisioning module (c#)
February 2016-April 2016: Data science Internship at The Tunisian Central Bank,
Tunis, TUNISIA
- Analyzing economic data (Prices Indexes, Export/Import Prices, EUR/USD..)
- forecasting inflation using Dynamic Factor Model (R and Matlab)
- Forecasting Inflation for the next 6 months