Hadoop Developer Resume
TX
PROFESSIONAL SUMMARY
- Over 8 years of IT experience as a Developer, Designer & quality reviewer with cross platform integration experience using Hadoop, Java andJ2EE.
- Good Understanding of the Hadoop Distributed File System and Ecosystem.
- Hands on experience in installing, configuring and using Apache Hadoop ecosystems such as MapReduce, HIVE, PIG, SQOOP, FLUME and OOZIE.
- Hands on experience on Hortonworks and Cloudera Hadoop environments.
- Strong understanding of Hadoop daemons and MapReduce concepts.
- Worked in importing - exporting data into HDFS format.
- Worked in analyzing big data using Hadoop environment.
- Worked in handling Hadoop Ecosystem Projects such as Hive, Pig and Sqoop.
- Worked in developing UDFs for Hive using Java.
- Strong understanding of NoSQL databases like HBase, MongoDB.
- Extensive experience in design, development and support Model View Controller using Struts and Spring framework.
- Develop reusable solution to maintain proper coding standard across different java project.
- Proficiency with the application servers like WebSphere, WebLogic, JBOSS and Tomcat
- Developed core modules in large cross-platform applications using JAVA, J2EE, Spring, Web Services, and JMS.
- Expertise in debugging and optimizing Oracle and java performance tuning with strong knowledge in Oracle 11g and SQL
- Effective leadership quality with good skills in strategy, business development, client management and project management
- Excellent global exposure to various work cultures and client interaction with diverse teams
- Ability to work effectively in cross-functional team environments and experience of providing training to business users.
TECHNICAL SKILL SET
Hadoop/Big Data: HDFS, MapReduce, Hive, Pig, Sqoop, Flume, Oozie, and ZooKeeper
No SQL Databases: Hbase, mongoDB
Languages: C, C++, Java, J2EE, PL/SQL, Pig Latin, HiveQL, Unix shell scripts
Java/J2EE Technologies: Applets, Swing, JDBC, JNDI, JSON, JSTL, RMI, JMS, Java Script, JSP, Servlets, EJB, JSF, JQuery
Frameworks: MVC, Struts, Spring, Hibernate
Operating Systems: Sun Solaris, HP-UNIX, RedHat Linux, Ubuntu Linux and Windows XP/Vista/7/8
Web Technologies: HTML, DHTML, XML, AJAX, WSDL, SOAP
Web/Application servers: Apache Tomcat,WebLogic, JBoss
Databases: Oracle 9i/10g/11g, DB2, SQL Server, MySQL, Teradata
Tools and IDE: Eclipse, NetBeans, Toad, Maven, ANT, Hudson, Sonar, JDeveloper, Assent PMD, DB Visualizer
Version control: SVN, CVS
Network Protocols: TCP/IP, UDP, HTTP, DNS, DHCP
PROFESSIONAL EXPERIENCE
Confidential, TX
Hadoop Developer
Responsibilities:
- Installed and configured Apache Hadoop, Hive and Pig environment on Amazon EC2
- Extensively involved in Installation and configuration of Cloudera distribution Hadoop, Name Node, JobTracker, Task Trackers and Data Nodes
- Configured MySQL Database to store Hive metadata.
- Responsible for loading unstructured data into Hadoop File System (HDFS).
- Created MapReduce jobs using Pig Latin and Hive Queries.
- Used Sqoop tool to load data from RDBMS into HDFS.
- Cloudera and Hortonwork POCs.
- Installed and configured Hadoop ecosystem like HBase, Flume, Pig and Sqoop
- Involved in Hadoop cluster task like Adding and Removing Nodes without any effect to running jobs and data
- Managed and reviewed Hadoop Log files
- Load log data into HDFS using Flume. Worked extensively in creating MapReduce jobs to power data for search and aggregation
- Worked extensively with Sqoop for importing metadata from Oracle.
- Responsible for smooth error-free configuration of DWH-ETL solution and Integration with Hadoop.
- Designed a data warehouse using Hive
- Designing and implementing semi-structured data analytics platform leveragingHadoop, with Solr
- Created partitioned tables in Hive
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS
- Developed workflow inOozieto automate the tasks of loading the data into HDFS and pre-processing with Pig
Environment: Hadoop, MapReduce, HDFS, Pig, Hive, HBase, Java, Oracle 10g, MySQL, Ubuntu
Confidential, Florham Park, NJ
Hadoop Developer
Responsibilities:
- Developed shell scripts to automate the cluster installation.
- Played a major role in choosing the right configurations for Hadoop.
- Developed Pig Latin scripts to extract and filter relevant data from the web server output files to load into HDFS.
- Involved in start to end process of hadoop cluster installation, configuration and monitoring.
- Responsible for building scalable distributed data solutions using Hadoop
- Installed and configured Hive, Pig, Sqoop, Flume and Oozie on the Hadoop cluster
- Setup and benchmarked Hadoop/HBase clusters for internal use
- Developed Simple to complex Map/reduce Jobs using Hive and Pig
- Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms
- Handled importing of data from various data sources, performed transformations using Hive,MapReduce, loaded data into HDFS and extracted the data from MySQL into HDFS using Sqoop
- Analyzed the data by performing Hive queries and running Pig scripts to study customer behavior
- Used UDF's to implement business logic in Hadoop
- Implemented business logic by writing UDFs in Java and used various UDFs from Piggybanks and otherSources.
- Continuous monitoring and managing the Hadoop cluster using Cloudera Manager
- Worked with application teams to install operating system, Hadoop updates, patches, version upgrades asRequired
- Installed Oozieworkflow engine to run multiple Hive and Pig jobs
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team
Environment: Hadoop, MapReduce, HDFS, Hive, Pig, Java, SQL, Cloudera Manager, Sqoop, Flume, Oozie, Java (jdk 1.6), Eclipse
Confidential, Warren, NJ
Hadoop Developer
Responsibilities:
- Installed and configured Hadoop MapReduce, HDFS and developed multiple MapReduce jobs in Java for dataCleansing and preprocessing.
- Involved in loading data from UNIX file system to HDFS.
- Installed and configured Hive and also written Hive UDFs.
- Evaluated business requirements and prepared detailed specifications that follow project guidelines required toDevelop written programs.
- Devised procedures that solve complex business problems with due considerations for hardware/software capacity and limitations, operating times and desired results.
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
- Provided quick response to ad hoc internal and external client requests for data and Worked in creating ad hoc reports.
- Responsible for building scalable distributed data solutions using Hadoop.
- Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting,Manage and review data backups, manage and review Hadoop log files.
- Worked hands on with ETL process.
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, andLoaded data into HDFS.
- Extracted the data from Teradata into HDFS using Sqoop.
- Analyzed the data by performing Hive queries and running Pig scripts to know user behavior like shoppingEnthusiasts, travelers, music lovers etc.
- Exported the patterns analyzed back into Teradata using Sqoop.
- Continuous monitoring and managing the Hadoop cluster through Cloudera Manager.
- Installed Oozie workflow engine to run multiple Hive.
- Developed Hive queries to process the data and generate the data cubes for visualizing.
Environment: Hadoop, MapReduce, HDFS, Hive, Ooozie, Java (jdk1.6), Cloudera, NoSQL, Oracle 11g, 10g, PL SQL, SQL*PLUS, Toad 9.6, Windows NT, UNIX Shell Scripting.
Confidential, Omaha, NE
Cognos Developer
Responsibilities:
- Worked on Cognos 8 Framework Manager to import metadata from multiple data sources and to create subject oriented business models (Relational Data Modeling and Dimensional Data Modeling) create and publish to cognos server.
- Creating list reports, cross-tab reports and chart reports using multiple queries in a single report.
- Created drill-through reports and conditional formatting to highlight the business needs.
- Created reports with prompts.
- Created dis-continuous cross-tab reports with Analysis Studio.
- Created Reports with Event Studio.
- Scheduling the multiple reports.
- Used bursting of the reports to analyze the country-wise performance.
- Involved in design & development of operational data source and data marts in Oracle
- Reviewed source data and recommend data acquisition and transformation strategy
- Involved in conceptual, logical and physical data modeling and used star schema in designing the data warehouse
- Designed ETL process using Informatica Designer to load the data from various source databases and flat files to target data warehouse in Oracle
- Used Power mart Workflow Manager to design sessions, event wait/raise, and assignment, e-mail, and command to execute mappings
- Created parameter based mappings, Router and lookup transformations
- Created mapplets to reuse the transformation in several mappings
- Used Power mart Workflow Monitor to monitor the workflows
- Optimized mappings using transformation features like Aggregator, filter, Joiner, Expression and Lookups
- Created daily and weekly workflows and scheduled to run based on business needs
Environment: Cognos Reportnet 1.1, Cognos 8, Data modeling,Informatica Power Centre 9.0, SQL Server SSIS, SSRS, Oracle 10g, Teradata 6, XML, TOAD, SQL, PL/SQL, IBM AIX, UNIX Shell Scripts, Web Intelligence, DSBASIC, Erwin, STAR team, Remedy, Maestro job scheduler, Mercury Quality Center, Control-M
Confidential
Java/J2EE Developer
Responsibilities:
- Involved in analysis and design of the application.
- Involved in preparing the detailed design document for the project.
- Developed the application using J2EE architecture.
- Involved in developing JSP forms.
- Designed and developed web pages using HTML and JSP.
- Designed various applets using JBuilder.
- Designed and developed Servlets to communicate between presentation and business layer.
- Used EJB as a middleware in developing a three-tier distributed application.
- Developed Session Beans and Entity beans to business and data process.
- Used JMS in the project for sending and receiving the messages on the queue.
- Developed the Servlets for processing the data on the server.
- The processed data is transferred to the database through Entity Bean.
- Used JDBC for database connectivity with MySQL Server.
- Used CVS for version control.
- Involved in unit testing using Junit.
Environment: Core Java, J2EE, JSP, Servlets, XML, XSLT, EJB, JDBC, JBuilder 8.0, JBoss, Swing, JavaScript, JMS, HTML, CSS, MySQL Server, CVS, Windows 2000