Hadoop Developer Resume
4.00/5 (Submit Your Rating)
Chicago, IL
SUMMARY:
- Around 7 Years of IT experience in Application Development domain of Hadoop and Java.
- Experienced in defining job flows Experienced in managing and reviewing Hadoop log files.
- Experienced in running Hadoop streaming jobs to process terabytes of XML and JSON format data.
- Developed a scalable application using NoSQL databases including Hbase.
- Experimented with Installation and configuration of Hadoop clusters in Single node, Pseudo - distribution and Multi-node clusters using Apache &Cloud era.
- Strong experience in architecting real time streaming applications and batch style large scale distributed computing applications using tools like Spark Streaming, Spark SQL, Map reduce, Hive etc.
- Hands on experience in various Hadoop infrastructures such as Map Reduce, Hive, PIG, SQOOP, Flume, HBase and DB2.
- Skilled in managing and reviewing Hadoop log files.
- Experienced in Application Development using Java, Hadoop, RDBMS and Linux shell scripting and performance tuning.
- Excellent knowledge in Java and SQL in application development and deployment.
- Versatile experience in utilizing Java tools in business, web and client server environments including Java platform, JSP, Servlets, Java beans and JDBC.
- Working knowledge on HTML5 and expert level proficiency in markup and scripting languages such as HTML, DHTML, XML, XSLT, CSS, Javascript, JQuery .
- Extensively worked on J2EE technologies like JSP, Servlets, Java, JDBC,, HTML, JavaScript and XML.
PROFESSIONAL EXPERIENCE:
Confidential,Chicago, IL
Hadoop Developer
Responsibilities:- Create mappings using different sources of Flat file by using Expression, Aggregator, Sequence Generator, Look-up, Joiner, Filter and update strategy transformations are extensively used in the project.
- Created and configured Sessions in Informatica workflow Manager for loading data into Data Warehouse tables from various heterogeneous database sources like Flat files and CSV files.
- Extensively used Source Qualifier Transformation to filter data Confidential Source level rather than Confidential Transformation level. Created different transformations such as Source Qualifier, Joiner, Expression, Aggregator, Rank, Lookups, Filters, Stored Procedures, Update Strategy and Sequence Generator.
- Worked on analyzing Hadoop cluster using different big data analytic tools including Pig, Hive, and Map Reduce.
- Worked on debugging, performance tuning of Hive & Pig Jobs.
- Implemented test scripts to support test driven development and continuous integration.
- Worked on tuning the performance Pig queries.
- Involved in loading data from LINUX file system to HDFS.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Experience working on processing unstructured data using Pig and Hive.
- Supported Map Reduce Programs those are running on the cluster.
- Gained experience in managing and reviewing Hadoop log files.
- Maintained and tracked defects in the defect tracker ALM.
Environment: Cloud Era Hadoop, MS SQL Server, Hadoop PIG, Hive, HDFS, HBase, Sqoop, Java, UNIX Shell Scripting.
Hadoop Developer
Confidential,Atlanta, GA
Responsibilities:- Evaluated business requirements and prepared detailed specifications that follow project guidelines required to develop written programs.
- Responsible for building scalable distributed data solutions using Hadoop .
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
- Developed Simple to complex Map reduce Jobs using Hive and Pig.
- Optimized Map Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
- Handled importing of data from various data sources, performed transformations using Hive, Map Reduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop.
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
- Extensively used Pig for data cleansing.
- Created partitioned tables in Hive.
- Managed and reviewed Hadoop log files.
- Involved in creating Hive tables, loading with data and writing hive queries which will run internally in Map Reduce way.
- Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
- Installed and configured Pig and also written Pig Latin scripts.
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
- Load and transform large sets of structured, semi structured and unstructured data.
- Responsible to manage data coming from different sources .
- Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required,
Environment: Hadoop, HDFS, Pig, Hive, Map Reduce, Sqoop, LINUX, and Big Data.
Confidential, Phoenix, AZ
Java/Hadoop Developer
Responsibilities:- Analyze Big Data patterns using Hive, Pig, Hbase and custom Map Reduce programs.
- Involved in Configuring Hadoop cluster and load balancing across the nodes.
- Importing data from various sources, transforming the data into recognizable format and load the data into HDFS using Hive and Map Reduce.
- Responsible for implementing Map/Reduce jobs to support distributed data processing and process large data.
- Used Sqoop to Import and Export Hugh data from different databases into HDFS and HIVE.
- Monitor Hadoop cluster connectivity and security.
- Manage and review Hadoop log files.
- Implemented ETL process with Hive, Pig, Map Reduce and Hbase.
- Wrote UNIX Scripts to automate the application configuration.
- Used Spring JDBC to write some DAO classes to interact with the database to access account information.
- Used Oracle as Database and used Toad for queries execution and also Involved in writing SQL scripts, PL/SQL code for procedures and functions.
- Developed PIG Latin scripts to transform the log data files and load into HDFS.
- Used Pig as ETL tool to do transformations, event joins and some pre-aggregations before storing the data onto HDFS.
- Developed hive UDF for functions that were not pre existing in Hive like the rank etc.
- Created External Hive tables and involved in data loading and writing Hive UDFs.
Environment: Java, Spring, Hibernate, JSP, JSF, HTML, CSS3, XML, JavaScript, JUnit, Agile.
Confidential
Java Developer
Responsibilities:- Understanding of System Requirements both functional as well as Technical.
- Involved in design and development of presentation layer using HTML, JSP.
- Involved in designing the screens and client/Server side validations using JavaScript and validation frame work.
- Involved in the complete SDLC software development life cycle of the application from requirement analysis to testing.
- Writing programs to full fill the project requirements using Java, Servlets, Struts, JSP, XML, J2EE, EJB and Advanced JavaScript, AJAX, HTML5, CSS.Create reusable components using Java, Servlets, Struts, JSP, XML, J2EE,DB2,JavaScript, jQuery, prototype, Spring and Design patterns.
- Working closely with the technical Lead to review the coding and to make sure all the coding standards are followed to be compliant with the Confidential processes.
- Deploying the project in the Server that is accessible to Confidential Customers.
- Developed different Action Classes for different modules in the application.
- Developed presentation layer using tag libraries in JSP pages.
- Implemented web page layout using struts tiles libraries, and performed validations using Struts validation framework.
- Developed front-end using JSP, HTML, JavaScript and CSS.
- Participated in UAT and also incorporating the feedback/changes received from system users.
- Responsible for deployment of this application in test serve.
- Testing and debugging of the code.
Environment:: Java, JSP, Servlets,DB2, JavaScript, CSS, HTML, Eclipse.
