SUMMARY:

• Around 8 years of experience with 5+ years in Hadoop and Spark Development along with 2+ years of experience in Java/J2EE enterprise application design, development, and maintenance.
• Experience includes application development in Java (client/server), JSP, Servlet programming, Enterprise Java Beans, Struts, JSF, JDBC, spring, Spring Integration, Hibernate.
• Experience in designing and implementing complete end - to-end Hadoop based data analytical solutions using HDFS, MapReduce, Spark, Yarn, Kafka, PIG, HIVE, Sqoop, Storm, Flume, Oozie, Impala, HBase etc.
• Experience in using version control tools like Bit-Bucket, SVN etc.
• Good experience in working with cloud environment like Amazon Web Services () EC2 and S3.
• Extensive experience implementing Big Data solutions using various distributions of Hadoop and its ecosystem tools.
• Hands-on experience in installing, configuring, and monitoring Hadoop clusters (on premise & cloud AWS).
• Good experience in creating data ingestion pipelines, data transformations, data management, data governance and real time streaming at an enterprise level.
• Profound experience in creating real time data streaming solutions using Apache Spark/Spark Streaming, Kafka.
• Hands on experience using Spark framework with Scala.
• Worked with various formats of files like delimited text files, click stream log files, Apache log files, Avro files, JSON files, XML Files.
• Has good understanding of various compression techniques used in Hadoop processing like Gzip, SNAPPY, LZO etc.
• Good experience on NoSQL: HBase, MongoDB and Cassandra.
• Worked on maintaining Hadoop cluster such as Troubleshooting, Managing and reviewing the performance related configuration fine tuning.
• Good knowledge on AWS data services including Kinesis, Athena, Redshift and EMR.
• Worked in supporting and managing Cloudera's Hadoop distributions platform.
• Experience developing PigLatin and HiveQL scripts for Data Analysis and ETL purposes and also extended the default functionality by writing User Defined Functions (UDFs), (UDAFs) for custom data specific processing.
• Hands-on experience in using Flume to transfer log data files to Hadoop Distributed File System.
• Mastered in using the using the different columnar file formats like RCFile, ORC and Parquet formats.
• Familiar with installation of Oozie workflow engine to run multiple Hive and Pig jobs that run independently with time and data availability.
• Good experience in optimizing Map-Reduce algorithms by using Combiners and Custom practitioners.
• Strong front end user development skills using HTML, CSS, JavaScript, JQuery, Angular-Js.
• Very good understanding and Working knowledge of Object Oriented Programming concepts.
• Hands on experience on working with Amazon EMR framework transferring data to EC2 server
• Possess working knowledge on SQL, Stored Procedures, Functions, packages, DB Triggers, and Indexes.
• Strong working knowledge in using IDE tools like Eclipse, Intellij.

TECHNICAL SKILLS:

Big Data Technologies: HDFS, Hive, Map Reduce, Cassandra, Pig, Hcatalog, Phoenix, Falcon, Scoop, Zookeeper, Mahout, Flume, Oozie, Avro, HBase, MapReduce, HDFS, Storm, CDH 3-4

Scripting & Programming Languages: Java, Shell Scripting, Puppet, Scripting, Python, Bash, CSH, Ruby, PHP

Databases: RDS, Dynamo DB, Redshift Management, Oracle 11g, MySQL, MS SQL Server, HBase, Cassandra, MongoDB

Networks: HTTP, HTTPS, FTP, UDP, TCP/TP, SNMP, SMTP

Monitoring Tools: Cloudera Manager, Solr, Ambari, Nagios, Ganglia

Application Servers: Apache Tomcat, WebLogic Server, WebSphere

Reporting Tools: Kerberos

Cognos, Hyperion Analyzer, OBIEE & BI+,Cloud Formation.

ElasticSearchLogstash: Kibana,Athena, EMR, Kinesis Apache Storm, Spark streaming, AWS Ecosystem

Compute: EC2, Lambda. Hadoop MapReduce, Apache Spark Run

TECHNICAL SKILLS:

Hadoop Developer

Confidential - Los Angeles, CA

Responsibilities:

• Responsible for Managing, Analyzing and Transforming petabyte's of data and also quick validation check on FTP file arrival from S3 Bucket to HDFS.
• Responsible for analyzing large data sets and derive customer usage patterns by developing new MapReduce programs.
• Experienced in creation of Hive tables and loading data incrementally into the tables using Dynamic Partitioning and Worked on Avro Files, JSON Records.
• Experienced in using Pig for data cleansing and developed Pig Latin scripts to extract the data from web server output files to load into HDFS.
• Worked on Hive by creating external and internal tables, loading it with data and writing Hive queries.
• Involved in development and usage of UDTF's and UDAF's for decoding Log Record Fields and Conversion's, Generating Minute Buckets for the specified Time Interval's and JSON Field Extractor.
• Developed Pig and Hive UDF's to analyze the complex data to find specific user behavior.
• Responsible for Debug, Optimization of Hive Scripts and also implementing Deduplication Logic in Hive using a Rank Key Function (UDF).
• Experienced in writing Hive Validation Scripts which are used in validation framework (for daily analysis through graphs and presented to business users).
• Developed workflow in Oozie to automate the tasks of loading data into HDFS and pre-processing with Pig and Hive.
• Involved for Cassandra Database Schema design.
• Using BULK LOAD Utility data pushed to Cassandra databases.
• Responsible for creating Dashboards on Tableau Server.
• Generated reports for hive tables in different scenarios using Tableau
• Responsible for Scheduling using Active Batchjobs and Cron jobs.
• Experienced in Jar builds that can be triggered by commits to Github using Jenkins.
• Exploring new tools for data tagging like Tealium (POC Report).
• Actively updated the upper management with daily updates on the progress of project that include the classification levels that were achieved on the data.

Environment: Hadoop, HDFS, Map Reduce, Hive, HBase, Zookeeper, Impala, Java(jdk1.6), Cloudera, Oracle, SQL Server, UNIX Shell Scripting, Flume, Oozie, Scala, Spark, Sqoop, Python, kafka, PySpark.

AWS Hadoop Developer

Confidential - Malvern, PA

Responsibilities:

• Configure virtual data center by considering security requirements in the Amazon Web Services (AWS) cloud to support enterprise data warehouse hosting including VPC, public and provide subnets, security groups and assign to EMR clusters.
• Design, build and maintain Big Data work flows/pipelines to process billions of records into and out of our data lake (DB2 to AWS) and processing work flows required to support data science, machine learning, BI and reporting in AWS.
• Provided senior level system analysis, design, development, and implementation of applications, technical infrastructure, tools, modeling, external interfaces, and other technical areas. Complete complex development, design, implementation, architecture design specification, and maintenance activities.
• Design data processing pipelines by Creating Scala/Spark jobs for data transformation and aggregation with unit tests for Spark Scala code and write Scaladoc-style documentation with all code.
• Scheduling and Running the Oozie work flow upon EMR Cluster spin up to Process data validation and operation of Acxiom and Clients score Data to Hive/Presto Data Lake.
• Ensure the viability of IT deliverables and recommend development options, including design, build. Approve the team’s technical deliverable. Conducts testing, including functionality, technical limitations, and security.
• Assessing Client business functions and technology needs. Applying Vanguard’s technologies, tools, and applications, including those that interface with business area and system.
• Translating technical specifications, and/or logical and physical design into code for new or enhancement projects for internal clients. Developing code that reuses subroutines or objects, is well structured, includes sufficient comments, and is easy to maintain. Writing programs, and ad hoc queries, and reports.
• Design predictive models and implement end to end solution to use case and creating distributed data processing pipelines using Spark, Hive, Python, and other tools and languages prevalent in the AWS Hadoop ecosystem.
• Develops and applies domain knowledge of Vanguard’s businesses and data to shape the analytic road map. Data Integrity Monitoring (20%)
• Use complex SQL querying techniques to troubleshoot any anomalies reported by end users and Business users in reports.
• Support the Analytics team and assist members to connect to and successfully use data.
• Design and develop solutions to secure sensitive data and also model an access based solution to avoid data theft. Release Change Management (15%)
• Utilized Atlassian Bitbucket and Bamboo for Code repository, reviews, builds, and deployments to AWS.
• Participating in design, code, and test inspections throughout life cycle to identify issues. Explaining technical considerations at related meetings, including those with internal clients.
• Coordinate with change management team to deploy code in AWS environment to automate the models in cloud.
• Elevate code into the development, test, and Production environments on schedule. Provides follow up Production support.

Hadoop Developer

Confidential

Responsibilities:

• Installed and configured Hadoop clusters for application development and Hadoop tools like Hive, Pig, Sqoop, HBase, Flume and Zookeeper.
• Worked on developing ETL processes to load data into HDFS using Sqoop and export the results back to RDBMS.
• Used Flume to collect, aggregate and store the web log data from different sources like web servers, mobile and network devices and pushed to HDFS.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS
• Handled importing of data from various data sources, performed transformations using Hive, MapReduce, and loaded data into HDFS.
• Involved in creating hive tables, loading the data and write hive queries that will run internally in a map reduce way.
• Analyzed business requirements and cross-verified with functionality and features of NOSQL databases like HBase, Cassandra to determine the optimal DB.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades of Cloudera Hadoop distribution(CDH3 and CDH4) as required
• Moved data from Third party system to Hadoop File System (HDFS) vice versa using shell commands hosted on an AWS cluster.
• Configured Sqoop and developed scripts to extract data from MySQL into HDFS.
• Involved in POC to import TIFF, Text and JSON files from Rabbit MQ server to HBase using Spark Streaming.
• Hands-on experience with productionalizing Hadoop applications viz. administration, configuration management, monitoring, debugging and performance tuning.
• Involved in implementing High Availability & automatic failover infrastructure to overcome single point of failure for Name node.
• Supports and assist QA Engineers in understanding, testing and troubleshooting.
• Written build scripts using ant and participated in the deployment of one or more production systems
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig.

Environment: Java 7, Eclipse, Hadoop, Pig, Hive, MapReduce, HDFS, Sqoop, Flume, Unix Shell Scripting, Oozie, AWS, Rabbit MQ, CDH, Linux.

Java/J2EE Developer

Confidential

Responsibilities:

• Involved in designing the Project Structure, System Design, and every phase in the project.
• Responsible for developing platform related logic and resource classes, controller classes to access the domain and service classes.
• Involved in Technical Discussions, Design, and Workflow.
• Participate in the Requirement Gathering and Analysis.
• Employed JAXB to un marshal XML into Java Objects.
• Developed Unit Testing cases using JUnit Framework.
• Implemented the data access using Hibernate and wrote the domain classes to generate the Database Tables.
• Involved in implementation of view pages based on XML attributes using normal Java classes.
• Involved in integration of App Builder and UI modules with the platform.

Environment: Hibernate, Java, JAXB, JUnit, XML, UML, Oracle11g, Eclipse, Windows XP.