Hadoop Developer Resume
Sfo, CA
SUMMARY:
- 10 years of IT experience in diversified fields of Big Data Hadoop/Spark, ETL, Teradata and Mainframe Development.
- Worked in Applications Design, Development, Testing, and Maintenance & Production Support projects.
- 3 years of extensive experience in Big Data and Big Data Analysis tools.
- Hands on experience with Big Data Analysis tools like Hadoop, HDFS, Hive, Pig, Sqoop, Spark Streaming and Python.
- Expertise in writing Hadoop Jobs for analyzing data using HiveQL and Pig scripts.
- Good knowledge in using job scheduling and monitoring tools like Oozie.
- Good Knowledge in Kafka.
- Experience in developing solutions to analyse large data sets efficiently
- In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, Mapreduce concepts
- 7 Years of Strong hands on experience as ETL developer.
- Excellent Experience in Designing, Developing of Extraction, Transformation and Loading (ETL) processes from various sources using IBM InfoSphere Information Server V8.5 & V9.1.
- Good experience in dimension modelling.
- Involved in reviewing business requirements and analysing data sources form Excel/Oracle SQL Server
- Good experience in Evaluating Data Sources and strong understanding of Data Warehouse / Data Mart Design.
- Excellent in writing Complex queries, stored procedures, functions, optimization and fine tuning.
- Experienced in developing UNIX shell scripts. Developed UNIX shell scripts to automate file manipulation and data loading procedures.
- Having excellent track record as a strong team player with effective communication, analytical and multi - tasking skills, resourceful, result driven and self-motivated.
- Ability to identify and analyze complex problems, review related information, and evaluate options to implement solutions.
TECHNICAL SKILLS:
Hadoop/Big Data Technologies: HDFS, Hive, Pig, Sqoop, Oozie, Spark, Yarn, Spark Streaming, Kafka
Hadoop Distributions: Horton works and Cloudera Hadoop
Languages: C, C++, Java, Python, Scala, UNIX Shell Script, SQL and PL/SQL
Tools: Toad, Teradata SQL Assistant, Pycharm, Autosys
Operating Systems: Linux, Unix, ZOS and Windows
Databases: Teradata, Oracle 9i/10g, DB2, SQL Server, MySQL 4.x/5.x
ETL Tools: IBM InfoSphere Information Server V8, V8.5 & V9.1
Reporting: Tableau
PROFESSIONAL EXPERIENCE:
Confidential, SFO, CA
Hadoop Developer
Responsibilities:
- Designed and developed the multiple batch data workflows for demand forecast track as part of PETE analytics.
- Interacted with product owners for understanding the requirements in terms BI Data lake development.
- Involved in brainstorming sessions for any requirement change to define the scope of requirements.
- Involved in Data pipeline validations for different internal data warehouse.
- Created in source to target mapping documents from staging area to Data Lake.
- Created and executed SQL queries to validate data whether all the requirements are met.
- Created Spark streaming job to receive website data.
Environment: Teradata, Hadoop, HDFS, Hive, Python, Spark, Spark Streaming SQL, Talend and Pycharm.
ConfidentialHadoop Consultant
Responsibilities:
- Developed Simple Spark Streaming Jobs using Python to pull data from Kafka server.
- Developed Hive scripts to transform data and load into Hive tables.
- Handled importing of data from various data sources, performed transformations using Hive and Extracted the data from RDBMS into HDFS using Sqoop.
- Analyzed the data by performing Hive queries to study customer behavior
- Continuous monitoring and managing the Hadoop cluster using Cloudera Manager
- Using Oozie workflow engine to run multiple Hive jobs
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team
Environment: UNIX, Hadoop, MapReduce, HDFS, Hive, Hbase, SQL, Cloudera Manager, Sqoop, Oozie, Python, Pycharm
Confidential, Smyrna
Hadoop Developer, ETL (Datastage) Developer and Mainframes Developer
Responsibilities:
- Developed Hive scripts to transform data and load into Hive tables.
- Worked in migration of the Warranty Services, Claims, Warranty Parts Replacement system to DataStage from Mainframes.
- Data Modelling, Evaluating Data Sources and designed Data Warehouse/Data Mart Design, ETL
- Designed and developed the new jobs using DataStage 8.5 & 9.1 Designer.
- Strong interaction with clients and other teams
- Involved in developing UNIX scripts to call Data stage jobs.
- Developed UNIX shell scripts to automate file manipulation and data loading procedures.
- Created mapping specification to Cleansing, Transforming the data
- Created PLSQL stored procedures and execution of Stored Procedures.
- Worked in application fixes for production issues.
Environment: Hadoop, HDFS, Hive, Pig, DataStage 8, 8.5 & 9.1, DB2, Oracle, UNIX, SQL, PL/SQL.