Hadoop Developer Resume
PROFESSIONAL SUMMARY:
- 7+ years of overall professional experience with strong emphasis on Design, Development, Implementation and Deployment of software Applications.More than 3 Years hands on experience in Big Data, Hadoop Development and Ecosystem Analytics, 4 years of development experience using Java,J2EE,JSP and Servlets.
- Well versed in installation, configuration, supporting and managing of Big Data and underlying infrastructure of Hadoop Cluster.
- Hands on experience on major components in Hadoop Ecosystem like Hadoop Map Reduce, HDFS, HIVE, PIG, Hbase, Zookeeper,Sqoop, Oozie, Cassandra and Flume.
- Experience in managing and reviewing Hadoop Log files.
- Experience with Oozie Workflow Engine in running workflow jobs with actions that run Hadoop Map/Reduce and Pig jobs.
- Worked on Importing and Exportingdatainto HDFSandHIVE using Sqoop.
- Worked on Import and Export of data using ETL tool Sqoop from MySQL to HDFS.
- Developed PIGLatin scripts using operators such as LOAD,STORE,DUMP,FILTER, DISTINCT,FOREACH,GENERATE,GROUP,COGROUP,ORDER,LIMIT,UNION,SPLIT to extract data from datafiles to load into HDFS.
- WroteHive queries for data analysis to meet the requirements.
- Created Hive tables to store data into HDFSand processed data usingHiveQL
- Loaded data using to database using ETL tools like SQL loader and external tables to load data from data warehouse and various other database like SQL Server, DB2.
- Experience using middleware architecture using Sun Java technologies like J2EE, JSP, Servlets, and application servers like Web Sphere and Web logic.
- Familiarity working with popular frameworks likes Struts, Hibernate, SpringMVC and AJAX.
- Ability to blend technical expertise with strong Conceptual, Business and Analytical skills to provide quality solutions and result - oriented problem solving technique and leadership skills.
TECHNICAL SKILLS:
Hadoop Ecosystem: HDFS, Map Reduce Hive, Pig, Hbase, Zookeeper, Sqoop, Oozie, Cassandra and Flume.
Web Technologies: CoreJava,J2EE,Servlets,JSP,javabeans,JDBC,XML,VMWare,Swing,JSF
Methodologies: Agile, UML, Design Patterns (Core Java and J2EE)
Frameworks: MVC, Struts 2/1, Hibernate3, Spring3/2.5/2
Programming Languages: Java, XML, Unix Shell scripting, HTML.
Data Bases: Oracle 11g/10g, DB2, MS: SQL Server, MySQL, MS-Access
Web Services: Web Logic, Web Sphere, Apache Tomcat
Operating Systems: Windows 8,Windows 7,UNIX,Linux,CentOS,Ubuntu
PROFESSIONAL EXPERIENCE:
Responsibilities:
- Proactively monitored systems and services, architecture design and implementation of Hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
- Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
- Documented the systems processes and procedures for future s.
- Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters.
- Worked on Talend to run ETL jobs on the data inHDFS.
- Used Hbase in accordance with Hive/Pigas per the requirement.
- Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
- Installed and configured Flume, Hive, Pig, Sqoop and Oozie on the Hadoop cluster.
- Used HiveQL to do analysis on the data and identify different correlations.
- Extensively used core Java technologies to create Hive/Pig UDFs to use in the project.
- Developed and maintained HiveQL, PigLatin Scripts and MapReduce.
- Facilitating testing in different dimensions.
- Worked on the RDBMS system using PL/SQL to create Packages, Procedures, functions,Triggers as per the business requirements.
- Also used Sqoop to transfer data between Rdbms and HDFS.
- Used Shell scripting for automation of scripts.
- Wrote and modified Stored Procedures to load and modifying of data according to business rule changes.
Environment: Apache Hadoop, Pig, Hive, SQOOP, PL/SQL, Oracle 11G, Linux OS, Windows OS,etc.
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop.
- Responsible for Cluster maintenance, adding and removing cluster nodes, Cluster Monitoring and Troubleshooting, Manage and review data backups and log files.
- Analyzed data using Hadoop components Hive and Pig.
- Worked hands on with ETL process.
- Responsible for running Hadoop streaming jobs to process terabytes of xml's data.
- Load and transform large sets of structured, semi structured and unstructured data using Hadoop/Big Data concepts.
- Involved in loading data from UNIX file system to HDFS.
- Responsible for creating Hive tables, loading data and writing hive queries.
- Handled importing data from various data sources, performed transformations using Hive, Map Reduce, and loaded data into HDFS.
- Extracted the data from Teradata into HDFS using the Sqoop.
- Exported the patterns analyzed back to Teradata using Sqoop.
- Installed Oozie workflow engine to run multiple Hive and Pig jobs which run independently with time and data availability.
Environment: Hadoop Cluster, HDFS, Hive, Pig, Sqoop, Linux, Hadoop Map Reduce, Hbase, UNIX Shell Scripting.
Responsibilities:
- Experience in administration, installing, upgrading and managing CDH3, Pig, Hive &Hbase
- Architecture and implementation of the Product Platform as well as all data transfer, storage and Processing from Data Center and to Hadoop File Systems
- Experienced in defining job flows.
- Implemented CDH3Hadoopcluster on CentOS.
- Worked on installing cluster, commissioning & decommissioning of datanode, namenode recovery, capacity planning, and slots configuration.
- Wrote Custom Map Reduce Scripts for Data Processing in Java
- Importing and exporting data into HDFS and Hive using Sqoop.
- Responsible to manage data coming from different sources.
- Supported MapReducePrograms those are running on the cluster.
- Involved in loading data from UNIX file system to HDFS.
- Created Hive tables to store data into HDFS, loading data and writing hivequeries which will run internally in map reduce way.
- Used Flume to Channel data from different sources to HDFS
- Created HBase tables to store variable data formats of PII data coming from different portfolios
- Implemented best income logic using Pigscripts.Wrote custom PigUDF to analyze data
- Load and transform large sets of structured, semistructured and unstructureddata
- Cluster coordination services through Zookeeper
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
Environment: Hadoop 1.0.0, MapReduce, Hive, HBase, Flume, Pig, Zookeeper, Java, ETL,SQL, CentOS
Responsibilities:
- Study and analysis of business and functional requirements
- Worked as a Component Developerto code inJava and J2EE technologies.
- Created UIusing JSPs and JSFbased framework.
- Used XMLBeansto buildJava objectsfrom XML Schema.
- Designed and developed the Application based onJ2EEArchitecture for server side on SpringMVC Framework
- Developed XML files for Object Relation Mapping (ORM) andHibernateConfiguration.
- UsedJNDIlookup for getting the Connection Objects.
- Used SQL Queriesand Stored Procedures extensively in retrieving the contents fromMysql
- Maintained documentation usingJavadoc.
- Provided extensive pre-delivery support usingBug FixingandCode Reviews.
- Unit testing was done usingJunit.
- Extensive usage ofXMLthroughout modules.
- Extensively used thelog4jfor logging
- Used CVS for code repository and as version control mechanism.
- Used Oracle 10gas a DB Serverand involved in writing stored procedures and SQL statements for data retrieve. Created test casesand Involved in Unit Testing.
Environment: Java, J2EE (JSP,Java beans, DAO), Eclipse, Apache Tomcat Server, Spring MVC, MySql.
Responsibilities:
- Involved in the analysis, Design, Coding, Modification and implementation of User Requirements in the Electronic Credit File Management system
- Designed the application usingFront Controller, Service Controller, MVC, Session Facade Design Patterns
- The application is designed usingMVC Architecture
- Developed business components usingStateless Session Beans(SLS) using Session Bean as aSession Facade
- Implemented the required functionality using Hibernate for persistence & Spring Frame work
- Used Spring Framework for Dependency Injection.
- Designed and implemented the Hibernate Domain Model for the services
- Designed and integrated the full scale Hibernate / Spring / EJB persistence solution with the application darchitectures
- Designs and develops cross-plane service components with Spring Framework
- Involved in tracking and resolvingProduction issues
- Implemented new releases to add more functionality as per the requirements
- Implemented search for documents in credit files and content in documents
- Involved in unit & integration testing to find and fix the bugs.
Environment: J2EE, Eclipse,Web logicApplication Server,JavaScript, HTML, DHTML, JSP, Servlets, JDBC, swing, XML, XSLT,CVS, Oracle 8i,CSS, UML designing, Springframework, JUnit,EJB2.0, JSTL, Solaris and Windows 2000 platform.
Responsibilities:
- Involved in coding of JSP pages for the presentation of data on the View layer in MVC architecture
- Used J2EE design patterns like Factory Methods, MVC, and Singleton Pattern that made modules and code more organized, flexible and readable for future upgrades
- Worked with JavaScript to perform client side form validations.
- Used Struts tag libraries as well as Struts tile framework.
- Used JDBC to access Database with Oracle thin driver of Type-3 for application optimization and efficiency.
- Client side validation done using JavaScript.
- Used Data Access Object to make application more flexible to future and legacy databases.
- Actively involved in tuning SQL queries for better performance.
- Worked with XML to store and read exception messages through DOM.
- Wrote generic functions to call Oracle stored procedures, triggers, functions.
Environment: JDK, J2EE, UML, Servlet, JSP, JDBC, Struts, XHTML, JavaScript, MVC, XML, XML,Schema, Tomcat, Eclipse.
