PROFESSIONAL SUMMARY:

• 6+ years of experience in a various IT related technologies, which includes 4 years of hands - on experience in Big Data technologies.
• Proficient in installing, configuring and using Apache Hadoop ecosystems such as MapReduce, Hive, Pig, Flume, Yarn, HBase, Sqoop, Spark, Storm, Kafka, Oozie, and Zookeeper.
• Strong comprehension of Hadoop daemons and Map-Reduce topics.
• Used informaticaPower Center for Extraction, Transformation, and Loading (ETL) of information from numerous sources like Flat files, XML documents, and Databases.
• Experienced in developing UDFs for Pig and Hive using Java.
• Strong knowledge of Spark for handling large data processing in streaming process along wif Scala.
• Hands On experience on developing UDF, DATA Frames and SQLQueries in Spark SQL.
• Highly skilled in integrating Kafkawif Spark streaming for high speed data processing.
• Worked wif NoSQL databases like HBase, Cassandra and MongoDB for information extraction and place huge amount of data.
• Understanding of data storage and retrieval techniques, ETL, and databases, to include graph stores, relational databases, tuple stores
• Experienced in writing Storm topology to accept teh events fromKafkaproducer and emit into Cassandra DB.
• Ability to develop Map Reduceprogram using Java and Python.
• Good understanding and exposure to Python programming.
• Exporting and importing data to and from Oracleusing SQLdeveloper for analysis.
• Developed PL/SQL programs (Functions, Procedures, Packages and Triggers).
• Good experience in using Sqoop for traditional RDBMS data pulls.
• Worked wif different distributions of Hadoop like Hortonworks and Cloudera.
• Strong database skills in IBM- DB2, Oracle and Proficient in database development, including Constraints, Indexes, Views, Stored Procedures, Triggers and Cursors.
• Extensive experience in Shell scripting.
• Extensive use of Open Source Software and Web/Application Servers like Eclipse 3.x IDE and Apache Tomcat 6.0.
• Experience in designing a component using UML Design-Use Case, Class, Sequence, and Development, Component diagrams for teh requirements.
• Aptitude abilities in J2EE, J2SE, Servlets, Spring, Hibernate, JUnit, JSP, JDBC, Java Multithreading, Object Oriented Design Patterns, Exception Handling, Garbage Collection, HTML, Struts, Hibernate, Enterprise Java Beans, RMI, JNDI and XML-related innovations.
• Involved in reports development using reporting tools like Tableau. Used excel sheet, flat files, CSV files to generated Tableauadhoc reports.
• Broad design, development and testing experience wif Talend Integration Suite and knowledge in Performance tuningof mappings.
• Experience in understanding teh security requirements forHadoopand integrate wif Kerberos authentication and authorization infrastructure.
• Experience in cluster monitoring tools like Ambari& Apache hue.
• Solid Technical foundation, great investigative capacity, cooperative person, and objective arranged, wif a promise toward incredibleness.
• Outstanding communication and presentation skills, willing to learn, adapt to new technologies and third-party products.

TECHNICAL SKILLS:

Hadoop/Big Data: HDFS, MapReduce, Hive, Pig, Sqoop, Flume, Oozie, Spark, Kafka, Storm and ZooKeeper.

Languages: C, Java, Python, Scala, J2EE, PL/SQL, Pig Latin, HiveQL, Unix shell scripts

Java/J2EE Technologies: Applets, Swing, JDBC, JNDI, JSON, JSTL, RMI, JMS, Java Script, JSP, Servlets, EJB, JSF, JQuery

Frameworks: MVC, Struts, Spring, Hibernate

No SQL Databases: HBase,Cassandra, MongoDB

Operating Systems: HP-UNIX, RedHat Linux, Ubuntu Linux and Windows XP/Vista/7/8

Web Technologies: HTML, DHTML, XML, AJAX, WSDL.

Web/Application servers: Apache Tomcat, WebLogic, JBoss.

Databases: Oracle 9i/10g/11g, DB2, SQL Server, MySQL, Teradata

Tools and IDE: Eclipse, NetBeans, Toad, Maven, ANT, Hudson, Sonar, JDeveloper, Assent PMD, DB Visualizer

Version control: SVN, CVS, GIT

Web Services: REST, SOAP

PROFESSIONAL WORK EXPERIENCE:

Confidential, CA

Big Data Engineer

Environment: Hadoop, Cloudera, Talend, Scala, Spark, HDFS, Hive, Pig, Sqoop, DB2, SQL, Linux, Yarn, NDM, Quality Center 9.2, Informatica, Windows & Microsoft Office.

Responsibilities:

• Used Spark API over Cloudera Hadoop YARN to perform analytics on data.
• Exploring wif teh Spark improving teh performance and optimization of teh existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, Pair RDD's, Spark YARN.
• Worked on batch processing of data sources using Apache Spark, Elastic search.
• Involved in converting Hive/SQL queries intoSparktransformations usingSparkRDDs, Scala.
• Worked on migrating PIG scripts and MapReduce programs to Spark Data frames API and Spark SQL to improve performance.
• Able to assess business rules, collaborate wif stakeholders and perform source-to-target data mapping, design and review.
• Created scripts for importing data into HDFS/Hive using Sqoop from DB2.
• Loading data from different source(database & files) into Hive using Talendtool.
• Conducted POC’s for ingesting data using Flume.
• Used all major ETL transformations to load teh tables through Informaticamappings.
• Created Hive queries and tables dat halped line of business identify trends by applying strategies on historical data before promoting them to production.
• Worked on Sequence files, RC files, Map side joins, bucketing, Partitioningfor Hiveperformance enhancement and storage improvement.
• Developed Pig scripts to parse teh raw data, populate staging tables and store teh refined data in partitioned DB2 tables for Business analysis.
• Worked on managing and reviewing Hadoop log files. Tested and reported defects in an Agile Methodology perspective.
• Conduct/Participate in project team meetings to gather status, discuss issues & action items
• Provide support for research and resolution of testing issues.
• Coordinating wif Business for UAT sign off.

Confidential

Big Data Developer

Environment: Map Reduce, HDFS, Sqoop, Pig, Hive, Kafka, Flume, Shell Scripts, Oozie, ETL, Agile, SQL

Responsibilities:

• Developed in writing MapReduce jobs.
• Ingested data into HDFS using SQOOP and scheduled an incremental load to HDFS.
• Developed scripts in Pig for transforming data and extensively used event joins, filtered and did pre- aggregations.
• Analyzed data using Hive teh partitioned and bucketed data and compute various metrics for reporting.
• Good experience in developing Hive DDLs to create, alter and drop Hive TABLES.
• Worked on HiveQL.
• Implemented Kafka for streaming data and filtered, processed teh data.
• Developed data pipeline using flume, Sqoop and pig to extract teh data from weblogs and store in HDFS
• Developed Shell scripts for scheduling and automating teh job flow.
• Developed a workflow using Oozie to automate teh tasks of loading teh data into HDFS.
• Developed MapReduce jobs to calculate teh total usage of data by commercial routers in different locations, developed Map reduce programs for data sorting in HDFS
• Load balancing of ETL processes, database performance tuning ETL processing tools.
• Loaded teh data from Teradata to HDFS using Teradata Hadoop connectors.
• Optimized Hive queries to extract teh customer information from HDFS.
• Performed Data scrubbing and processing wif Oozie and for workflow automation and coordination.
• Involved in various phases of development analyzed and developed teh system going through Agile Scrum methodology.