We provide IT Staff Augmentation Services!

Hadoop Developer Resume

4.00/5 (Submit Your Rating)

Dallas, TX

SUMMARY:

  • Around 6+ years of IT experience in a variety of industries, which includes hands on experience of 4 years in Big Data Analytics and development.
  • Good Knowledge and experience of System Development Life Cycle (SDLC), product development methodologies, database design concepts and system integration strategies.
  • Expertise with the tools in Hadoop (HDFS, YARN) Ecosystem including Pig, Hive, Map Reduce, Sqoop, Storm, Spark, Flume, Kafka, Oozie, Impala and Zookeeper and Machine Learning.
  • Strong experience in Spark SQL UDFs, Spark SQL Performance, Spark Streaming, Performance Tuning . Experienced in working with input file formats like orc, parquet, Sequence, Json, Avro .
  • Experienced in working with Amazon Web Services (AWS) using EMR for computing and S3 as storage.
  • Excellent Java development skills using Java, J2EE, Servlets, JSP, EJB, JDBC, SOAP and RESTful web services.
  • Worked in large and small teams for Systems requirement, design & development.
  • Experienced in using agile approaches, including Extreme Programming, Test - Driven Development and Agile Scrum.

TECHNOLOGIES:

Big Data/Hadoop Technologies: HDFS, YARN, Map Reduce, Hive, Pig, Impala, Sqoop, Flume, Spark, Kafka, Zookeeper, Oozie, Elastic Search

Hadoop Distribution/Monitoring: Cloudera, Horton works, Ambari, Cloudera Manager

NO SQL Databases: HBase, Cassandra, MongoDB

Relational Databases: Microsoft SQL Server, MySQL, Oracle, DB2

Languages : Java, Scala, SQL, PL/SQL, C, C++, Shell Scripting, Python

Java & J2EE Technologies: Core Java, JSP, Servlets, JDBC, JNDI, Hibernate, Spring, Struts, JMS, EJB, Restful, SOAP

Web Technologies: HTML, CSS, XML, Java Script, JQuery

Application Servers: Web Logic, Web Sphere, JBoss, Tomcat

Cloud Computing Tools: Amazon AWS: EC2, S3, IAM, Glacier, CloudFront, EMR

Operating Systems: UNIX, Windows, LINUX

Build Tools: Jenkins, Maven, ANT

ETL Tools: Informatica, Talend

Visualization: Tableau

Development Tools: Eclipse, NetBeans, IntelliJ

Development Methodologies: Agile/Scrum, Waterfall

Version Tools and Testing API: Git, SVN and JUNIT

PROFESSIONAL EXPERIENCE:

Confidential, Dallas, TX

Hadoop Developer

Environment: Scala, Spark, Cloudera, HDFS, Hive, Impala, GitHub, Eclipse, Bamboo, Sqoop, TDCH, Teradata SQL assistant

Responsibilities:

  • Worked with Care Evolution to parse clinical documents that come from various data sources in the form of CCDA’s (XML), HL7, ADT messages into SDO (Standard Data Output) files.
  • Have written code in Scala using spark and ingested data across different layers in HDFS.
  • Have used AVRO tables in Warehouse layer which enables to add/remove column names without affecting the history data.
  • Have used parquet format and higher order functions to load the latest and greatest data into outbound layer for reporting purpose.
  • Used Akka framework that enables concurrent processing while loading the data lake.
  • Ingested data from Teradata to Hadoop using TDCH jobs for information.
  • Developed spark code to generate monthly Eligibility extract for Anthem Eligible members.
  • Developed code in Scala that identifies Line of Business for every patient having clinical record.
  • Have improved job’s performance by using appropriate caching methodology and repartitioning principle.
  • Modified all the Scala projects and shell to migrate from unencrypted to encrypted zone. Ensured that the jar/shell is compatible to run in any environment without any modification.
  • Have used Control M for job set automation and GIT Version control.
  • Developed shell scripts to notify IT team when there is a failure in spark job and maintained logs for job completion.
  • Co-ordinated with Hadoop Admin Team, Control-M and ensured that changes have been installed successfully for every release. Responsible for doing post production validation and communicated stake holders.

Confidential, Littleton, CO

Senior Hadoop Developer

Environment: Spark core, Spark SQL, Spark streaming, Spark machine learning, Scala, S3, EMR, Kafka, Hive, Sqoop, GitHub, Web flow, Talend

Responsibilities:

  • Load and transform large sets of Structured, Semi Structured and Unstructured data coming from different source systems and a variety of portfolios.
  • Wrote Shell Scripts to move batch files from various sources to HDFS, S3 and scheduled using cron tab.
  • Worked on incremental Batch Ingestion using Sqoop into HDFS.
  • Implemented Streaming data Ingestion using Kafka.
  • Used Spark Streaming as a consumer in Kafka for preprocessing real time data.
  • Implemented partitions and buckets in Hive for optimizing query performance.
  • Used Spark core API and Spark SQL in Scala as per business analysis needs.
  • Used AWS S3 for data storage and EMR cluster for processing various jobs.
  • Worked on Talend with Hadoop. Worked in migrating from Talend jobs.
  • Using Spark Machine learning library to clean data and feed the input stream data to Trained Model.
  • Migrated python scikit learn machine learning to data frame-based spark machine learning algorithms
  • Worked with Oozie Workflow manager to schedule Hadoop jobs and highly intensive jobs
  • Worked with team members in data visualization using Tableau.

Confidential, IL

Hadoop with Spark Developer

Environment: Hadoop-HDFS, Hive, Sqoop, Kafka, Spark, Shell Scripting, HBase, Python, Zookeeper, Maven, Hortonworks, MySQL, Tableau

Responsibilities:

  • Experienced in running Hadoop streaming jobs to process terabytes of xml format data.
  • Importing and exporting data into HDFS and Hive using Sqoop
  • Created Linux shell Scripts to automate the daily ingestion of IVR data
  • Good experience in handling data manipulation using python Scripts.
  • Consumed the data from Kafka using Apache spark.
  • Worked on POC’s with Apache Spark using Scala to implement spark in project.
  • Implemented Partitioning, Dynamic Partitions, Buckets in Hive
  • Extending Hive core functionality by using custom User Defined Function’s (UDF).
  • Executed Hive queries on Parquet tables stored in Hive to perform data analysis to meet the business requirements.
  • Responsible for loading data files from various external sources like Oracle, MySQL into staging area in MySQL databases.
  • Worked in creating HBase tables to load large sets of semi structured data coming from various sources.
  • Exported the aggregated data onto RDBMS using Sqoop for creating dashboards in the Tableau .
  • Worked on Installation and configuring of Zookeeper to co-ordinate and monitor the cluster resources.

Confidential

Java Developer

Environment: Java/J2EE, Struts, Hibernate, Web Logic Server, Oracle, HTML, JavaScript, Ant, svn, JUnit

Responsibilities:

  • Coordinating with Project Manager for getting the requirements and developing the code to support new applications.
  • Developing Web applications using Java, J2EE, Struts and Hibernate.
  • Developing Action Form classes, Form beans, and Action classes using struts.
  • Using Hibernate for the backend persistence.
  • Involved in writing Spring Configuration XML file that contains object declarations and dependencies.
  • Implementing MVC, DAO J2EE design patterns as a part of application development.
  • Designing the front end using JSP, HTML, CSS and JavaScript as per the requirements that are provided.
  • Coding and maintaining Oracle packages, stored procedures and tables.
  • Working on Web technologies including Tomcat, Apache, Http, Web service architectures.
  • Using SVN for software configuration management and version control.
  • Preparing test cases and strategies for unit testing and integration testing.
  • Using the LOG4j to log regular Debug and Exception statements.

We'd love your feedback!