SUMMARY

• An experienced Big Data - Hadoop developer with varying level of expertise around different Big Data/Hadoop ecosystem projects which include HDFS, MapReduce, HIVE, HBase, Storm, Kafka, Flume, Sqoop, ZooKeeper, Oozie, Pig etc.
• In depth and extensive knowledge of Hadoop platform.
• Experience in Hadoop cluster sizing and cluster design.
• Around 17 years of Software Design, Development, Programming experience on Middleware and Client/Server, Multitier, Web based front end applications in Java, J2EE & web services.
• Expertise in application development using Java, Servlet, JSP, Struts, Spring, Hibernate, JUnit, Log4J, XML, XSD, XSLT, Web Services, ANT, JavaBeans, EJB, JMS, Maven etc.
• Experience working with different file formats like Apache Avro, Sequence file, JSON, XML, and Flat text file.
• Worked closely with business clients and partners to understand and business requirements, develop use case and designed solutions.
• Extensive experience with successful projects execution in On-site and Off-shore models, leading and managing projects and teams independently.
• Import data from database to HDFS and vice versa using Sqoop.
• Knowledge of Real-time streaming technologies Spark, Kafka.
• Extensive 10+ years of experience in large scale developments, deployments and maintenance of middleware applications in Telecommunications industry with Confidential &T and British Telecom.

TECHNICAL SKILLS

Big Data/Hadoop Technologies: HDFS, MapReduce, Spark, NiFi, Hive, HBase, ZooKeeper, Oozie, Pig, Sqoop, Flume, Apache Avro, Storm, Kafka

Languages: JAVA, PL/SQL, XML, Groovy, Scala

Java & Web Technologies: JSP, Servlets, JDBC, JNDI, JMS, EJB, SOA, Web Services, XML Schema, XSLT

Reports: Jasper

Frame works: Struts, Spring, Hibernate

Scripting: JavaScript

Design Technologies: OOAD with UML (Rational Rose, Microsoft Visio).

Server Technologies: Web Logic, Web Sphere, Apache Tomcat

Operating Systems: Windows, Unix, Linux

Databases: Oracle 10g, MySQL, SQL Server, MS Access. DB2

Tools: Rational Rose (Clear Quest, Clear Case), Eclipse, XML Spy, TOAD, ANT, Jenkins, Maven, MS Office, JUnit, CVS, SVN, SOAP UI, Putty, WinSCP, File Zilla, Splunk, Nexus

Methodologies: Waterfall, Agile and DevOps

PROFESSIONAL EXPERIENCE

Confidential, Dallas, TX

Big Data Engineer

Responsibilities:

• Develop Sqoop job to move data from Oracle to Hadoop.
• Develop spark job to validate and cleansing data and then move to Hive table.
• Designing and development of HIVE tables and hql scripts for hive tables.
• Develop shell script and schedule a cron job to move delta csv file from ftp location to staging/HDFS after every day.
• Nifi - to replace FTP job for better performance and maintenance.
• Develop spark job to read delta csv file and move that data to Hive table.
• Design and development Oozie job to run spark job on daily bases.
• Oozie job uses Status checker to check the hive status table after every one minute to see if data transfer is completed from FTP location to HDFS via Nifi.
• Develop Oozie job to send email after Spark job completion with status as success or failure with details.
• Participate in Agile daily scrum calls, interact and work with QC and Hadoop admin team.

Environment: Hadoop, Hive, Oracle, Spark 1.6, Zookeeper, Scala 2.11.8 Sqoop, Oozie, Shell scripting, WinSCP, HUE, Agile, git, HP ALMQC, Maven, Splunk, Tableau for reports, CDH, HDF (Nifi)

Confidential

Big Data Engineer

Responsibilities:

• Involved in Cluster sizing discussions and cluster design for various component installation on cluster
• Develop map reduce programs for schema validator and IVP validation to validate different versions of xml request & responses.
• Created Sqoop integration scripts to ingest data from HDFS to database and database to HDFS.
• Sqoop ingest data in hive stage environment. Move to Master data set after cleansing using pig and load key meta data information in database.
• Writing HIVE script to load the payload in external table from HDFS.
• Writing scripts for creating, truncating, dropping, altering HBase tables to store the data after execution of map reduce job and to use that for later analytics.
• Cluster Monitoring and Troubleshooting using Ambari. Helping with troubleshooting issues.
• Develop shell scripts and invoke through program to process operations on HDFS file system.
• Design and develop controller to invoke splunk and Nexus adapters for log extraction and kit download to extract schema respectively.
• Schedule nightly batch jobs using Oozie to perform schema validation and IVP transformation Confidential larger scale to take the advantage of the power of Hadoop.

Environment: Windows 2000, Java, Spring, Angular JS, HDFS, Map Reduce, Storm, Kafka, Hive, HBase, Pig, ZooKeeper, Sqoop, Flume, Oozie, HCatalog, Ambari, Shell scripting.

Confidential

Big Data Engineer

Responsibilities:

• Implement storm topology job to create data streams to send data to bolts and then to solr.
• Designed and developed storm spout for handling file data stream which then distributed across all bolts for file comparison.
• Implement Hazelcast distributed queue messaging which is useful for all cluster members interaction.
• Integration of Hazelcast with storm spout to handle the incoming data to spout.
• Implement file level comparison during comparison of two Kits.
• Develop storm spout to handle file stream data coming from SVN, and direct the data streams to various bolts for processing.
• Design and develop bolt to analyze and compare files coming from data streams from Strom spout.
• Developer storm bolt to create Solr document with all metadata information and add that on Solr server.
• Read the tuples, serialize it, perform data processing for comparison and then store the result into the solr.
• Fetch the comparison result from solr after job completion.

Environment: Windows 2000, Java, HDFS, MapReduce, Storm, Hazelcast, Kafka, Hive, HBase, ZooKeeper, Hive, Sqoop, Flume, Ambari, Angular JS, Spring, Shell scripting etc..

Confidential 

Java Middleware Technical Lead

Responsibilities:

• Extensive coordination with offshore team and deliveries on schedule time with quality.
• Communicated project status, including project issues up and down the management chain, including senior management.
• Communicate the project design to other application designers and team members
• Ensure Process & SLA Adherence of the projects and deliverables based on Confidential &T standards (ITUP) and agile methodology.
• Managed change requests, including assisting the sponsor and stakeholders to understand the impact of the change on schedule or features
• Involved in planning, creating application designs, validating high level designs (HLDs) to ensure accuracy and completeness against the business requirements and programming the solutions and complete unit testing with unit test plans.
• Responsible to resolve design issues and develop strategies to make ongoing improvements that support system flexibility, performance and metrics reporting.
• Extensively worked on Java, J2EE, XML, Schema designs and on Weblogic server.
• Responsible for release level tasks and involvements in ‘Lessons Learned’ meetings and applied these ideas to improve processes for new projects
• Design validation strategies and review Test Cases, Test Plans prepared by testing team
• Handled multiple projects simultaneously and communicate requirements and status effectively
• Delivered results in assigned timeframes that were sometimes quite short
• Active involvement in the entire Software Development Life Cycle including Design, Development, Testing, Deployment and Support for Software Systems

Environment: Windows XP, Java 1.6, Tibco Business studio, Contivo, Weblogic 10.3, CVS, SVN, XML, XSD, xml spy, Eclipse, Ant, Maven, Nexus, Jenkins, MS Office 2007 products, Putty (UNIX), J2EE, Web Services, Eclipse, JMS, SVN

Confidential

Java Middleware Technical Lead

Responsibilities:

• Design for new functionality. Lead the team of developer for both the applications.
• Provided the solution architecture with different systems involved.
• Involved in the entire development, integration and validation for other components.
• Responsible for providing technical requirements based on business scope and vision.
• Interfacing with business team and providing overall design directions.
• Responsible for Managing, tracking of project.
• Served as an escalation point for resolving issues with Java/J2EE.
• Provide support to QA while testing
• Provide support during deployment and warranty period.
• Development of features as per the requirements.
• Integrate/Unit testing.
• Help test team to run through regress testing and performance testing