- Data Engineer with about 1 year of experience in building data pipelines and offline financial reports for merchants and finance teams for a mobile payment platform in China.
- Performed ETL on payment, orders history data using SQL, Hive, Python.
- Experience on major components in Hadoop Ecosystem including Hive, Sqoop, Flume & knowledge of Map Reduce/HDFS Framework.
- Created Oozie workflows to run multiple MR, Hive and pig jobs.
- Experience with Exploratory Data Analysis using Python Matplotlib and Seaborn to identify underlying patterns and correlation between features.
- Experience with NoSQL databases like HBase in creating tables to load large sets of semi structured data coming from source systems
- 7 years of total IT experience in the field of User Interface development.
Big data: Hive, Pig, Spark, Hadoop, MapReduce,Oozie, Sqoop
Cloud: AliCloud (Aliyun), Amazon Web Services (AWS), ETL Pipelines
Version control: SVN, Agile development methodology
Strong Work Ethics - Problem solving - Initiative-taking - Adaptive - Collaborative - Eager to learn
- Built scalable and fault - tolerant systems to create accurate offline reports for the business in SQL.
- Worked on Sqoop jobs for ingesting data from MySQL to HDFS.
- Involved in creating Hive tables, loading data, and writing hive queries.
- Oversaw the complete ETL process and developed data pipelines.
- Ensured architecture met business requirements.
- Created conceptual data models based on business data requirements.
- Collaborated closely with team members, solution architect and product teams.
- Documented work in team wiki to help new team members.
- Functioned as a bridge between Chinese teammates and Indian counterparts at Paytm, India (Alibaba is an investor in Paytm) and ensured alignment across all stakeholders.
User Interface Developer
- Independently led the development of fully functional features in multiple releases by collaborating with product and testing teams.
- Coordinated the work of five team members and facilitated cross-functional collaboration by organizing and leading teamwide meetings
- Joined as a trainee and learned .Net framework and programming in C#
- Worked as a UI developer for an international bank.