SUMMARY

• PMP certified wif 20+ years of experience in professional application development. Extensive experience in architecting and implementing Spark eco system applications, Hadoop eco system applications, Java applications, PL/SQL, SQL design and implementations to meet key client requirements and support business operations. Worked on all phases of SDLC focusing in multi - tier and web-based systems using Agile & Continuous Integration methodologies.
• 9+ years of experience in Hadoop and Spark.
• 3 years of experience in azure and databricks.
• Strong experience working wif azure data factory pipelines, adf triggers, azure blob and datalake Storage accounts, azure key vaults, azure virtual servers, Kibana, linked services,databricks notebooks, delta tables, event hub, pyspark, Power BI etc
• Strong experience on Hadoop framework, YARN, HDFS, MapReduce API, Oozie, MRUnit, Pig, Hive, Flume, Sqoop, HBase, Cassandra, NoSQL, ZooKeeper, Hue, Avro, Impala, Scala, Spark, Kafka, Ubuntu (Linux)
• Well versed wif developing and implementing MapReduce jobs using Hadoop to work wif Big Data.
• Experience on cloudera and Horton works environments.
• Extensive experience on kafka implementation, kafka-HDFS Pipeline, kafka-storm integration.
• Strong experience wif Spark Processing Framework Spark Streaming and SparkSQL.
• Strong experience in developing Spark jobs in Scala wif SBT tool.
• Have experience wif Storm.
• Good noledge on Impala.
• Experience in NoSQL databases like HBase, and Cassandra.
• Procedural noledge in cleansing and analyzing data using HiveQL, Pig Latin, and custom MapReduce programs in Java.
• Experienced in custom UDFs(User Defined Functions), UDTFs(User Defined Table-Generating Functions and UDAFs(User Defined Aggregating Functions) for extending Hive and Pig core functionalities.
• Experience in importing and exporting data using Sqoop from HDFS to structured data stores..
• Experience on Oozie, a workflow scheduler system to manage apache Hadoop jobs by Directed Acyclic Graph of actions and control flows.
• Good experience Flume, Avro and ZooKeeper.
• Good experience of R, Python.
• Good experience of AWS.
• Hands-on experience wif message brokers such as Apache Kafka.
• Experience in developing Python scripts for Hadoop streaming.
• Experience in UNIX, shell scripting, PERL scripts.
• Experience in developing the Spring Restful Web Services.
• Experience in testing the application using MRUnit, Junit.
• Extensive experience using JAVA/J2EE Technologies, Core Java, JDK, J2EE and Swing, Servlets, JSP, JDBC, RMI, EJB.
• Expertise in designing, writing and deploying Struts, Spring, Hibernate and EJB to empower Java based web solutions.
• Involved in front end validations using Java Script, Spring and Struts validation frame works.
• Proficient in RDBMS using SQL Server 7-2005/2008, Oracle 10/11g, and Sybase
• Worked on Weblogic, Websphere, Apache-tomcat, JBoss application servers.
• Good experience in Java Messaging Service (JMS) for Message-Oriented Middleware
• Good experience in web services such as JAX-WS, SOAP, WSDL, UDDI
• Experienced in web programming using HTML, CSS, JavaScript, JQuery, XML, XML Spy, JSON, Web Services.
• Extensive experience in implementing J2EE Design patterns
• Good experience in web services such as JAX-WS, SOAP, WSDL, UDDI.
• Expertise in TSQL, PL/SQL, TOAD, SQL Developer
• Expertise in Cognos Reports, SSRS, and Crystal Reports
• Expert on Agile Environment, Scrum procedures(Sprinting, planning, demo, retrospective), stories.
• Expertise in UML concepts and designing Use cases, Sequence, Activity and Class diagrams using Enterprise Architect, Visio and Rational Rose

TECHNICAL SKILLS

Languages: JAVA, JDK 1.6/1.7, PL/SQL, SQL, Scala

J2EE Technologies: J2EE (EJB 2.0/3.0, JSP2.0 and SERVLETS 2.3) J2EE Design Patterns, UML, JMS and JDBC

Frameworks: Struts 2/1.2, Hibernate, Spring, JQuery

Web/App Server: Web sphere 6.1/5.0/4.0, BEA Web logic 10.3/9.2/8.1 , JBOSS 4.0, Tomcat 6/4.0.4

Hadoop/Big Data/Spark: Hadoop, HDFS, Map Reduce, Hive, Pig, Sqoop, Kafka, Storm, HBase, Flume, Oozie, Hcatalog, Avro, Hue, YARN, HBase,Cassandra,Zookeeper, Spark, SparkSQL

Version Tools: SVN, CVS, VSS, TFS

Web Languages: XML, XSL, XSLT, HTML, DHTML, DOM, JavaScript, AJAX, JQuery, JSON, CSS and CSS3.

Messaging Systems: JMS, Kafka

IDE (Integrated Development Environment).: Eclipse, TOAD, SQL Developer, XML SPY, Weblogic Workshop 8.1/10.3,IBM Web Sphere Studio 4/5/5.1, JBuilder

RDBMS: Oracle8i/9i/10g/11g(PL/SQL), MS-SQL Server 7/200/2005/2008 , and Sybase

Operating System: Linux, MS Windows NT, Windows XP, Windows 7

Reporting Tools: Crystal Reports 10, Cognos Reports, SSRS

Others: Unix Shell Scripts, PERL, Rational Rose, MS-Visio, putty, SBT, Junit, Swing, Struts,EJB

PROFESSIONAL EXPERIENCE

Confidential, Plano,TX

Senior Data Engineer

Responsibilities:

• Worked on retrieving MongoDB change streams, publishing data to event hubs and consuming the data from event hub and flattening the data and save data to blob storage.
• Developed multiple ADF pipelines for different functionality and developed triggers to schedules the pipelines
• Worked on daily batch loaders to load the data to azure redis.
• Worked wif digital model, fraud orchestration, simswap and port out models.
• Worked on kafka publishing for Aiopx personalization.
• Worked wif azure data ingestion porting out all the onpremise hive data to azure blobs.
• Wif FMLA data, worked on different projects like Sec Stats, and Auto Encoder.
• Worked on the kafka publisher and consumer for FMLA data wif spark streaming.
• TMLA Analytics include application generates a series of tables containing descriptive stats. calculation of zscore, rzscore and holt winters.
• Worked wif Secstats Streaming using kafka and structured streaming
• For insider threat data cleaning and collect different applications data from different data sources and stitch them according to the business logic and create files to fed into Anomaly Reporting Model.
• Collect the data by taking max time records and applying joins, filters, groups etc for Single source, Telegency clarify and Opus data retrievals. Data retrieval is based on many formulas and calculations.
• Then features of all applications is calculated wif aggregations.

Environment: azure data factory pipelines, adf triggers, azure blob and datalake Storage accounts, azure key vaults, azure virtual servers, linked services, databricks notebooks, delta tables, pyspark, Databricks, Azure SQL, Power BI, Kibana, Linux, Java 8, Eclipse, IntelliJ IDEA, HDFS, Pig 0.11, Hive 0.11, MySQL, Spark 1.6/2.1.1, Spark SQL, SBT, Scala 2.10.5/2.11.8 , Kafka 0.8.0, Python 3.2, GitHub, CodeCloud, Sqoop, MongoDB, Vertica, Teradata, Terraform

Confidential, Austin,TX

Senior Data Engineer

Responsibilities:

• Involved in moving all log files generated from various sources to HDFS for further processing through Flume.
• Involved in loading and transforming large sets of structured data from relational databases into HDFS using Sqoop imports.
• Developed Sqoop scripts to import export data from relational sources and handled incremental loading on the customer, transaction data by date.
• Developed multiple simple and complex MapReduce programs in Java for Data Analysis on different data formats like sequence files, xml files for data cleaning and preprocessing.
• Developed multiple python scripts for Hadoop streaming for business requirements.
• Created Hive tables(Internal and External) wif partitions and buckets, loaded data and wrote Hive queries that run wifin the map
• Developed Spark scripts by using Scala as per the requirement.
• Developed and implemented core API services using Scala and Spark wif Scala Build Tool.
• Implemented Spark, Spark SQL for near Real-time analytics and for faster data aggregations.
• Worked on HIVE scripts and running them in parallel to reduce or optimize run-time of the scripts.
• Worked on Data Serialization formats for converting Complex objects into sequence bits by using AVRO, JSON, CSV formats.
• Responsible for analyzing and cleansing raw data by performing Hive queries and running Pig scripts on data.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Developed custom UDFs, UDAFs and implemented Pig scripts.
• Implemented business logic by writing Pig UDF's in Java and used various UDFs from Piggybanks and other sources.
• Used Oozie Operational Services for batch processing and scheduling workflows dynamically.
• Extensively worked on creating End-End data pipeline orchestration using Oozie.
• Created Cron job for automatic cache cleanup resulting in decreased support issues
• Populated HDFS and HBase wif huge amounts of data using Apache Kafka.
• Processed huge amount of incoming feeds through Apache Kafka.
• Implemented kafka implementation, kafka-HDFS Pipeline, kafka-spark streaming integration.
• Processed the source data to structured data and store in NoSQL database HBase.
• Used Subversion as the version control mechanism.
• Reading the delimited file and applying transformations and creating the output file of required formats
• Reading the data directly from table and loading the data directly to the target table
• Wrote MySQL scripts for business requirements.
• Used Zookeeper coordination system
• Responsible for loading customer's data and event logs into HBase using Java API.
• Created HBase tables to store variable data formats of input data coming from different portfolios
• Involved in adding huge volumes of data in rows and columns to store data in HBase.
• Developed Spring restful web services for HBase tables data as JSON.
• Developed MRUnit tests to test the mapreduce programs.

Environment: Linux, Java 7/8, Eclipse Luna, Web Services, HDFS, Pig 0.11, Hive 0.11, MapReduce, Sqoop 1.99.1, Cloudera Manager, CDH5.5, Flume 1.3.0, HBase 0.94, Zookeeper 3.4.5, MySQL, Hue, Oozie 3.2.0, Storm, Spark 1.4, Spark SQL, SBT, Scala 2.10.1, Kafka 0.8.0, Python 3.2, SVN 1.7

Confidential, Austin,TX

Senior Developer

Responsibilities:

• Developed design diagrams using UML like use cases, sequence diagrams, collaboration diagrams, class diagram and interaction diagrams.
• Developed User Interface using JSP, Struts.
• Implemented Enterprise Logging service using JMS
• Involved in development (generating and modifying) of Hibernate mapping files.
• Worked on JSP validations using spring validation frame work.
• Involved in developing transaction management using spring.
• Worked to implement accessibility standards for application JSPs.
• Involved in designing user screens and validations using HTML, CSS, XML, JSP, JQuery, Java script.
• Involved in HQL preparation and DAO implementations using Hibernate.
• Developed Triggers and Stored Procedures using PL/SQL.
• Used AJAX controls to minimize post backs to the server and enhance user experience.
• Extensively used spring IOC for dependency Injection to promote loose coupling between components.
• Involved in writing unit test cases using JUnit.
• Performed Unit Integration testing wif some test cases.
• Implemented Web Services using Spring, for Remote Application.
• Worked wif the quality assurance team for detecting bugs in the design of the system.
• Used Web Services using SOAP.
• Used XML Parser for Building the XML Documents.
• Created restful web services and consumed them in the application.
• Provided Technical and Functional support.
• Developed many PERL scripts to import the data from remote location and process xml save etc.

Environment: Java 6, JSP, Servlets, JSTL, Hibernate3.0, Struts 2, Spring2.5, Ajax, JUnit, Web Services, Oracle 10g, CVS, Eclipse, Weblogic, JAX-WS, SOAP, WSDL, PERL, HTML, CSS, XML, Java Script, JQuery, Linux, Windows 7.

Confidential, Austin, TX

Senior Developer

Responsibilities:

• Worked in an Agile Team environment using Continuous Integration development
• Performed analysis and design of the application and database
• Developed the application using Struts Framework that leverages classical Model View Controller (MVC) Layer
• Worked on UML design diagrams like use cases, class diagrams and sequence diagrams.
• Designed and developed NHibernate classes, entity, mapping, data access and repository classes.
• Developed User Interface using JSP, JSF, JSTL, JavaScript, and AJAX
• Worked on version control management using CVS.
• Developed various crystal reports for different business logics.
• Developed Unit test cases in JUNIT for testing the application.
• Developed PL/SQL stored procedures, views and triggers.

Environment: Java, JSP, Servlets, JSTL, NHibernate, Struts, Java Script, JQuery, CSS, XML, Spring, Ajax, JUnit, Web Services, Oracle, CVS, Eclipse, Weblogic, Linux, Windows NT.

Confidential, Waltham, MA

Lead Consultant

Responsibilities:

• Gathered/analyzed requirements, created functional specs, prepared project plans, developed use cases and prototypes
• Performed analysis, design, and application development of business layer, data layer, UI modules
• Created stored procedures, UDFs, and triggers for the application.
• Developed the UI using HTML, CSS, JSP and JavaScript.
• Developed Unit Test Cases. Used JUnit for unit testing of the application.
• Used Core Java to develop Business Logic.
• Created tables & records as specified by the client. JDBC was used to connect to the SQL Database and validate transaction entries.
• Configured Web Services, Authentication wif ACTIVE DIRECTORY, and User Profile Web Service
• Tested and deployed application
• Created user and technical manuals

Environment: Java, EJB, JSP, Servlets, Struts, JSTL, Junit, XML, HTML, JDBC, SQL Server Weblogic 8.1

Confidential

Project Leader

• Gathered/analyzed requirements, created functional specs, prepared project plans, developed use cases and prototypes
• Performed analysis, design, and application development of business layer, data layer, UI modules
• Created stored procedures, UDFs, and triggers for the application.
• Designed and developed EJB Components.
• Developed the UI using HTML, CSS, JSP and JavaScript.
• Developed Unit Test Cases. Used JUnit for unit testing of the application.
• Used Core Java to develop Business Logic.
• Created tables & records as specified by the client. JDBC was used to connect to the SQL Database and validate transaction entries.
• Tested and deployed application
• Created user and technical manuals