WORK EXPERIENCE

Lead Hadoop Developer DIRECTV Inc, El Segundo, CA 06/2013 t Present

• Defined job flows and orchestrated Hadoop jobs using Spring Data after upgrade t CDH4 t seamlessly integrate with present HCSC framework for better maintenance and security.
• Helped develop a REST full API t provide access t data in HBase and HDFS
• Devised procedures that solve complex business problems with due considerations for hardware/software capacity and limitations, operating times and desired results.
• Architect, develop, deploy, debug and maintain BigData applications
• Formulated procedures for installation of Hadoop patches, updates and version upgrades.
• Assisted in designing, development and architecture of Hadoop and HBase systems.
• Supported technical team members for automation, installation and configuration tasks.
• Suggested improvement processes for all process automation scripts and tasks.
• Provided technical assistance for configuration, administration and monitoring of Hadoop clusters.
• Participated in evaluation and selection of new technologies t support system efficiency.
• Designed and developed scalable and custom Hadoop solutions as per dynamic data needs.
• Designed and implemented a Cassandra NoSQL based database and associated RESTful web service that can persist high-volume user profile data as proof of concept.
• Involved in migration process t CDH4 from CDH3 and came up with proof of concept.
• Proficiency with mentoring and on-boarding new engineers wh are not proficient in Hadoop and getting them up t speed quickly.

Environment: Java jdk1.6 , Toad 9.6, Linux, Shell Scripting, WAS 6 and 7, Spring Data, Maven, Jenkins, git, Serena, Teradata, DB2, SQL Server, Oracle, Mong DB, Python, Cloudera Hadoop, YARN, Sqoop, Flume, Pig, Hive, Zookeeper, HBase, Cassandra

Hadoop Developer Confidential

• 1Load and transform large sets of structured, semi structured and unstructured data using Pig, Hive, MapReduce, and loaded data int HDFS.
• Automated all the jobs, for pulling data from FTP server t load data int Hive tables, using Oozie workflows and Flume Reviewed the HBase NameNode design for failover
• Implemented proof of concept on Hadoop stack and different big data analytic tools, export and imports from different databases i.e Teradata, Oracle, SQL Server, Mong DB t Hadoop.
• Implemented proof of concept of R based analytics on hadoop for rating component.
• Provided operational support services relating t Hadoop infrastructure and application installation.
• Supported technical team members in management and review of Hadoop log files and data backups.
• Used Python script t achieve most tasks related t map reduce, administration and data transformation, etc.
• Used Kerberos authentication for Hbase/HDFS security purpose.
• Assisted in creation of ETL processes for transformation of data sources from existing RDBMS systems.
• Led initiatives t automated application builds and deployments using Hudson/Jenkins

Environment Java jdk1.6 , Toad 9.6, Linux, Shell Scripting, WAS 6 and 7, Spring Data, Maven, Jenkins, git, Serena, Teradata, DB2, SQL Server, Oracle, Mong DB, Python, Cloudera Hadoop, Sqoop, Flume, Pig, Hive, Zookeeper, HBase, Cassandra

Big Data Confidential

• Kick started their Hadoop project. Used Maven, Mahout, HIVE, Java and Shell Scirpting at various stages of the project lifecycle, I deployed a 80 Node cluster t manage 20 years of weather and electricity usage data for all the customers of the company.
• Setup and established VPN tunnel between the Company's VPN and AWS VPC t allow creating an internal cloud for security purposes.
• Worked with CDH4 as well as CDH5 applications. Performed Data transfer of large data back and forth from development and production clusters.
• Worked in deploying R-3.1.1/2.14 and Rstudi 0.98 for Three Segment Regressor analysis.
• Performed Incremental Dataload on Oracle Tables int HDFS using Shell scripts and Java.
• Setup user accounts for Developers on the cluster and configured linux apps for individual users

Big Data Engineer Confidential

• Developed Big Data platform using R programming language and Python t perform Descriptive Statistical Analysis on Social Media data Twitter , als performed Sentiment Analysis on the Twitter data t provide analytics by geography and identify patterns in Brand Sentiment.
• Deployed and analyzed large chunks of Healthcare data using HIVE as well as HBase.
• Installed both CloudEra CDH4 and Hortonworks HDP1.3-2.1 hadoop clusters on EC2, Ubuntu 12.04, CentOS 6.5 on platforms ranging from 10-100 nodes
• Worked on full lifecycle for the Big Data solution t requirements analysis, technical architecture design, Development t testing deployment
• Worked with one of the country's biggest electricity providers t kick start their Hadoop project Used Maven, Mahout, HIVE, Python, Java at various stages of the project lifecycle. Managed 20 years of weather and electricity usage data for all the customers of the company
• Setup and established VPN tunnel between the Company's VPN and AWS VPC t allow creating an internal cloud for security purposes.
• Manage deployed services across AWS, focused on EMR/Hadoop data analysis and processing, including
• Setting up of EC2 clusters in a secure Virtual Private Cloud environment AWS .

Environment: Hadoop, EMR, Hive, MapReduce, Amazon Web Services AWS , Cassandra, NoSQL, HDFS,

Java, R Analytics, EC2, S3, Python, Unix, Redhat, CentOS

Java Architect Confidential

• Conceptualization and definition of the tool suite for migration t Service Oriented Architecture SOA and alignment with future state goals of Organization. Detailed design of the components for integration with external services.
• Involved in development and re-platform of the legacy Integration platform t the new SOA based platform.
• Building architecture design documents from business requirements, and overseeing the development, code quality, and security of the application throughout the project life-cycle.
• Developed coding standards project life-cycles and software development life-cycles t increase quality and reduce maintenance costs while minimizing schedule impact.
• Using process choreographer, scripted enterprise beans t manipulate processes als created Web services from business processes that choreograph other Web services processes
• Implemented the Adapter for Web Services t enable true bidirectional support for event and request processing from within the adapter.
• Configured the adapter for Event processing that can be synchronous or asynchronous, and listeners within the adapter t provide support for SOAP over HTTP, HTTPS, and JMS transports.
• Worked with application development teams t define, design and implement the solution architecture, with the responsibility of cross project technology and architecture vision.
• Researched extensive existing infrastructure components, and development requirements and UML artifacts.
• Led a remote development team through a fast paced development effort and coordinated with technical and business contacts on-site at the client.
• Captured functional requirements by collaborating with business partners and stakeholders within other departments.
• Designed portal application architecture and work flow Designed, defined and provided guidance relating t developing Web services around business functionality
• As a member of a SOA architecture team, developed SOA Strategy and standards.
• Researched and prototyped usage of different Web services related specifications including WS-Atomic Transactions, WS-Attachments, WS-Security, BPEL and SAML.
• Been a go-t person for technical solutions and for troubleshooting complex technical issues.
• Used the STRUTS web application framework t implement presentation layer.
• Involved in Design, Implementation and coding in Java, Java Servlets, J2EE, EJB, and JSP etc.
• Used Chordiant framework for service and persistence of legacy system.

Environment: Java jdk1.6 , MongoDB, SQL Server, Oracle, Toad 9.6, Linux, WAS 6 and 7, Spring, Hibernate, Maven, git, Serena, WPS, Prototype, SAML 2.0, CXF, EJB2.0, DB2 8.0, WebSphere 5.1.2 Application Server, Chordiant framework, JUnit, Design Patterns, Tag Libs, Star Team, Windows-2000, WSAD, RAD, Axis