Bigdata/hadoop Developer Resume
NC
SUMMARY:
- Over 7 years of Experience on Expertising in developing web applications using Servlets, JSP, jQuery, EJB 3.0, spring, Hibernate, SOAP, WSDL.
- Specializes in Java/J2EE technologies with JBoss 5.1, and Tomcat 5.5.
- Solid experience in Designing, implementing, and improving analytic solutions for Big Data on Apache Hadoop.
- Excellent knowledge of Software Development Methodology and OOAD, UMLs, Use Cases, Sequence Diagrams and Class Diagrams
- Good knowledge of developing applications based on architectures like Hibernate, Spring Framework and MVC architecture
- Good knowledge on JAXB.
- Excellent experience working with Oracle9i and DB2 9.0 databases.
- Good Knowledge on Design Patterns such as MVC, Singleton, Session Facade, Business Delegate, Factory Pattern and Data Access Object.
- Excellent experience with Version Control Systems like Git, CVS, SubVersion and IBM Rational Clear Case
- Good Experience on XML related technologies such as XML, XSL, DOM, SAX etc.
- Experienced with various IDEs like My Eclipse, Eclipse.
- In - depth understanding of Data Structure and Algorithms
- Expertise in Agile Software Development methodology.
- Working experience in Map Reduce programming model Big data and Hadoop Distributed File Systems.
- Good experience in writing Sqoop queries for transferring bulk data between ApacheHadoop and structured data stores.
- Good knowledge on Oozie, a workflow scheduler system to manage apache Hadoop jobs.
- Good experience in Pig Latin, a Scripting Language for Hadoop Distributed File system.
- Good Knowledge in Flume, Avro and Zoo Keeper Architecture.
- Working knowledge in Hadoop HDFS Admin Shell commands.
- Good Knowledge in NoSQL databases HBASE (Column family DB).
- Good knowledge on Hadoop MRV1 and Hadoop MRV2 (or) YARN Architecture.
- Extending HIVE and PIG core functionality by using custom User Defined Function's (UDF), User Defined Table-Generating Functions (UDTF) and User Defined Aggregating Functions (UDAF) for Hive and Pig.
- Hands on experience in installing, configuring cloudera Apache Hadoop ecosystem components like Flume, Hbase, Zoo Keeper, Oozie, Hive, Sqoop and Pig.
TECHNICAL SKILLS:
Big Data: Hadoop, HDFS, MapReduce, Hive, Sqoop, Pig, HBase, MongoDB, Flume, Zookeeper, Oozie.
Operating Systems: Windows, Ubuntu, Red Hat Linux, Linux, UNIX
Java Technologies: JDBC, JAVA, SQL, JavaScript, J2EE, C, JDBC, SQL, PL/SQL
Programming or Scripting Languages: Java, SQL, Unix Shell Scripting, C.
Database: MS-SQL, MySQL, Oracle, MS-Access
Middleware: Web Sphere, TIBCO
IDE’s & Utilities: Eclipse and JCreator, NetBeans
Protocols: TCP/IP, HTTP and HTTPS.
Testing: Quality Center, Win Runner, Load Runner, QTP
Frameworks: Hibernate 3.0, Spring 3.x, Servlets, JSP,XML, Struts,EJB 2.x/3.x, JDBC,MVC
PROFESSIONAL EXPERIENCE
Confidential, NC
BigData/Hadoop developer
Responsibilities:
- Developed and Supported Map Reduce Programs those are running on the cluster. Gatheird the business requirements from the Business Partners and Subject Matter Experts.
- Setting up Hadoop cluster and Hadoop eco system environment for different big data analytics project in data centers
- Created Hive tables and working on them using Hive QL.
- Involved in installing Hadoop Ecosystem components.
- Validated Namenode, Data node status in a HDFS cluster.
- Handled 2 TB of data volume and implemented the same in Production.
- Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers.
- Used to manage and review the Hadoop log files.
- Responsible to manage data coming from different sources.
- Supported Map Reduce Programs those are running on the cluster.
- Involved in HDFS maintenance and loading of structured and unstructured data.
- Wrote Map Reduce job using Java API.
- Wrote Hive queries for data analysis to meet the business requirements.
- Installed and configured Pig and also written PigLatin scripts.
- Developed UDFs for Pig Data Analysis
- Involved in managing and reviewing Hadoop log files.
- Developed Scripts and Batch Job to schedule various Hadoop Program.
- Utilized Agile Scrum Methodology to halp manage and organize a team of 4 developers with regular code review sessions.
- Used JUnit for unit testing and Continuum for integration testing.
- Worked hands on with ETL process.
- Upgrading the Hadoop Cluster from CDH3 to CDH4 and setup High availability Cluster Integrate the HIVE with existing applications
- Configured Ethernet bonding for all Nodes to double the network bandwidth
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce,
- Analyzed the data by performing Hive queries and running Pig scripts to know user behavior.
- Continuous monitoring and managing the Hadoop cluster through Cloudera Manager.
- Installed Oozie workflow engine to run multiple Hive and Pig jobs.
- Developed Hive queries to process the data and generate the data cubes for visualizing.
Environment: Java Hadoop, MapReduce, HDFS, Hive, Pig, Linux, Big data, XML,Eclipse, Cloudera CDH3/4 Distribution, Informatica 9.1
Confidential, Eden Prairie, MN
Hadoop Developer
Responsibilities:
- Replaced default Derby metadata storage system for Hive with MySQL system.
- Executed queries using Hive and developed Map-Reduce jobs to analyze data.
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
- Developed the Pig UDF’s to preprocess the data for analysis.
- Developed Hive queries for the analysts.
- Utilized Apache Hadoop environment by Hortonworks.
- Developed presentation layer using JSP, HTML, DHTML, CSS, AJAX, JavaScript, and JSTL Taglibs.
- Involved in loading data from LINUX and UNIX file system to HDFS.
- Supported in setting up QA environment and updating configurations for implementing scripts with Pig.
- Involved in loading and storing objects using Hibernate.
- Involved in configuring Hibernate mapping file.
- Used version one to work on agile development.
Environment: Core Java, Apache Hadoop (Horton works), HDFS, Pig, Hive, Cassandra, Shell Scripting, My Sql, LINUX, UNIX
Confidential, Vienna, VA
Hadoop Developer
Responsibilities:
- Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase NoSQL database and Sqoop.
- Importing and exporting data in HDFS and Hive using Sqoop.
- Extracted files from MongoDB through Sqoop and placed in HDFS and processed.
- Experience with NoSQL databases.
- Written Hive UDFS to extract data from staging tables.
- Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
- Familiarized with job scheduling using Fair Scheduler so that CPU time is well distributed amongst all the jobs.
- Synchronized different configuration files of hadoop cluster nodes across the system.
- Configured the MapReduce property to make sure local temporary storage is using large disk partitions.
- Involved in the regular Hadoop Cluster maintenance such as patching security holes and updating system packages.
- Managed Hadoop log files.
- Analyzed the web log data using the HiveQL.
Environment: Java 6, Eclipse, Hadoop, Hive, Hbase, MangoDB,Linux, Map Reduce, HDFS, Shell Scripting, Mysql
Confidential
Software Developer
Responsibilities:
- Created browser-based application that provides flexibility for sales force to create and edit presentations while maintaining control over the appearance.
- Design and integration of Struts, Flash, XML, XSL, Tomcat, Web Services, JSP, JUnit, and MySql, Oracle 9i, PostgreSQL 7.1
- Development of J2EE EJBs, Servlets, JSP, Applets, and Java Beans for financial internet/intranet application running on Weblogic 7.1 application server and Oracle 9i.
- Project initiated with Enterprise Architect for UML object design.
- Oracle 9i schema design, creation of table spaces, accounts, synonyms and data import/exports.
- Developed Web services for financial extranet, that generate real time portfolio reports, using Perl, Java, XSL and WAP.
- Development of Swing Jlist, Jtable, and Jtree beans for data manipulation and presentation.
- Created hierarchical listener for all Swing gui business objects to minimize database roundtrip synchronization.
- Weblogic 7.1 (Integration) workflow applications design and development.
- System administration programs written in Python - Web site migration, backups.
Environment: Java, Weblogic, Struts, JSP, JSTL, Linux, Servlets, Applets, Apache Axis, Python, Perl,Tomcat, Apache, MySQL, PostgreSQL 7.1, Oracle 9i, CVS
