Hadoop Engineer Resume
Los Angeles, CA
SUMMARY:
- Nine years of IT experience in Software Development & Implementation, Financial Analytics and Corporate Actions.
- Four plus years of experience in Big Data / Hadoop which includes HDFS, Map Reduce, Hive, Pig, Spark, HBase, ZooKeeper, Flume, Sqoop, Scala, Oozie, Java/J2EE, SQL and PL/SQL. Good in R and SAS/SQL, Oracle RDBMS, Oracle 11i.
- Four years of experience in financial events like Profit & Loss Accounting (Revenue, Gross Income, Operating Income, Net income, EPS, Shares Outstanding), Cash Flow Statements and Balance Sheet of the corporate companies listed in NYSE and NASDAQ. Led and managed projects worth of over 10 Million Dollars.
- Very good in SDLC and implementation methodology like gathering business requirements, publishing gaps, AS IS & TO BE, documents.
- Good team player, quick learner, self - motivator and ability to work in challenging environments with good communication.
TECHNICAL SKILLS:
Big Data Technologies: Hadoop HDFS, Map Reduce, Hive, Pig, HBase, Flume, Kylin, Kafka, Sqoop, Spark, Oozie, Zookeeper.
Languages: Java, J2EE, SQL, PL/SQL, C, R Programming
Hadoop Distributions: Apache, Cloudera and Hortonworks
Operating Systems: Linux, Unix, Ubuntu, Windows
ETL Tools: Informatica
RDBMS: Oracle 9i, 10g, 11i
Applications: SAS, SAS/ BASE, SAS/SQL & SAS/MACROS and Oracle Apps 11i/R12 FIN & SCM
PROFESSIONAL EXPERIENCE:
Confidential, Los Angeles, CA
Hadoop Engineer
Responsibilities:
- Created Hive queries that helped market analysts spot emerging trends by comparing fresh data with EDV reference tables and historical metrics using Hue SQL and Impala
- Created Sqoop reusable frameworks for data Import and Export and tables creation in both Hive and Oracle
- Provided design recommendations and thought leadership to sponsors/stakeholders that improved review processes and resolved technical problems
- Architecture design of Frameworks/Workflows for loading and preprocessing of data
- Created some scripts in Spark to process the data quickly.
- Created Data Lake Enabler to send the data to the particular systems.
- Used Data warehousing tools to analyze the customer data.
- Build the data by using data warehousing tools such as ETL and script execution tools such as Informatica.
- Automated the scripts for production/SQA environment using Autosys and Linux Cron jobs.
- Helping Admin team to setup all environment and clusters
- Adding the CR’s to existing code based on new business requirements and fixing the old bugs.
- Drives the knowledge management sessions and mentors the new entrants into the team
- Coordinating offshore team as onsite lead.
- Drive the status calls for Daily status with offshore team.
Environment: HADOOP 2.0, CDH5.4, Beeline, Hue, Hive, Sqoop, Impala, DataTorrent, Data lake, Oracle10g, 11g, Autosys, Unix shell Scripts, ALM (Application lifecycle management).
Confidential
Hadoop Developer
Responsibilities:
- Installed and configured Hadoop Map Reduce developed multiple Map Reduce jobs in Java for data cleaning and preprocessing.
- Performance optimization solutions.
- Experienced in importing and exporting data into HDFS using Sqoop and Flume.
- Experienced working with a variety of data formats and protocols such as JSON, AVRO.
- Used HBase Java API to create tables in HBase.
- Installed and configured Hive and also written Hive UDFs.
- Created Hive tables to store the processed results in a tabular format.
- Used Oozie to manage Hive and Pig jobs.
- Used Flume to collect and aggregate weblog data from different sources and pushed to HDFS.
- Developed big data analytics system based on Apache Spark.
- Developed configurable batch processing system based on Spring Batch.
- Given various analysis charts using R studio
- Practicing machine learning Algorithms.
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
Environment: Hadoop, HDFS, Map Reduce, Core Java/J2EE, Hive, Pig, Hbase, JSON, AVRO, Hue, Oozie, Eclipse, Sqoop, Flume, Putty, Linux shell scripting.
Confidential
Senior Financial Analyst / Hadoop Developer
Responsibilities:
- Imported large sets of structured, semi - structured, unstructured data from web-servers to HDFS using Sqoop, Flume and FTP.
- Involved in analysis, design, development and architecture of Hadoop systems.
- Developed multiple Map Reduce jobs in java to parse the raw data and for distributed data processing.
- Created Hive tables to store the processed data in a tabular format.
- Implemented data transformation and processing using Pig.
- Used Oozie to automate data loading into the HDFS and Pig to pre-process the data.
- Experience with NoSQL databases such as Hbase.
- Created tables using Java API in HBase.
- Also analyzed the regular financial statements for key metrics and drivers of revenues; and companies’ performance in view of estimates/guidance and industry peers as per GAAP, Non-GAAP and IFRS.
- Ensuring that the corporate events get captured into the system in a proper, timely and consistent manner, performing checks on consistency of database and taking the necessary remedial actions.
Environment: Hadoop, HDFS, Map Reduce, Java/J2EE, Hive, Pig, Hbase, Oozie, Zookeeper, Eclipse, Sqoop, Flume, Putty, Linux shell scripting, Ubuntu
Confidential
Financial Analyst / Developer
Responsibilities:
- Developed tools for investment data analysis and data management system using MySQL
- Responsible for writing SQL functions and stored procedures in MY SQL server.
- Responsible to read and write different messages to different JMS and MQ queues.
- Involved in writing different JMX agents to process on demand requirements from business users.
- Used spring dependency injection and Spring-Hibernate integration.
- Wrote python scripts to parse XML documents and load the data in database
- Involved in entire lifecycle of the project including design, development, deployment, testing, implementation and support.
Environment: Java/J2EE, JDK, JDBC, JMS, MQ Series, JSP, Java-beans, SOAP, XML, Python, MYSQL Server, SQL, Hibernate, Linux Shell Scripting.